I recently joined Goodreads, partly because I hope it’ll give me the chance to slap more people in the face with my lit-peen and partly because I find it useful to have a place to look up books I’ve read in the past — going through the bibliographies of past work is a time-consuming and haphazard method. I’d include a link to my profile, but I don’t want to, so I won’t.
Anyway, Goodreads users can use a five star rating system to rank what they’ve read. Each star rating is glossed by a tooltip. The tooltips run, from one star to five, like this: ‘didn’t like it’, ‘it was okay’, ‘liked it’, ‘really liked it’ and ‘it was amazing’. I thought that this sounded rather different from the glosses on MAL’s out-of-ten rating system, which go like this:
Much has been written about ratings (I suggest you start here and follow the pingbacks) and favourites. What interests me here isn’t the way we choose ratings, if we choose them, but the after-spin, so to speak, put on these ratings systems by their glosses.
Three of Goodread’s glosses are direct reports of the reader’s emotional reaction. The other two, ‘[i]t was okay’ and ‘it was amazing’, are ostensibly (and, I suppose, grammatically) about the book’s qualities rather than the reader’s emotional reaction, but ‘amazing’ at least requires the presence of a reader to be mazed: rating books as an affective, emotional judgement. Whether Goodreaders follow their site’s lead, I don’t know; personally I stick to the letter of its glosses just to be perverse, and I haven’t given a book five stars unless it genuinely mazed me when I finished it.
Very nearly all of MAL’s glosses, by contrast, relate to the inherent qualities of the title being rated, though the glosses for one, five and ten out of ten are unusual. ‘Average’ is a relative term, or at least a more obviously relative one. ‘Unwatchable’ doesn’t have to mean ‘bad’: I find Sayonara Zetsubou Sensei unwatchable, but that’s because I don’t have enough knowledge about Japan, or the right kind of funnybone for its style of humour. ‘Masterpiece’ brings in the most interesting baggage of meaning: can you have a masterpiece without a master (hello studio- and director-worship)? If you have a master, can he, she, or it only produce one magnum opus in his, her, its lifetime? And ‘piece’ is buried in there, too. That’s not a word I’d apply to anime or manga unless it were something like Angel’s Egg — something too clever for me — as though its creators are musicians.
(I’m going to be tramping around the Isle of Purbeck — which is not an island, much like how the New Forest was created in the eleventh century — for a few days, so I won’t be able to reply to any comments immediately.)
Animanachronism 
In this Goodreads thingie it’s striking that there’s but one bland flavour of negative response, rather than 4 delightful ways to detail The Suck Level. To me at least that chimes with the way people talk about books (at least literary ones) in everyday conversation. I suppose that’d tie in with the idea of people thinking of books they have read in emotional terms, or just people being less confident in their judgemental side re. good writing.
So I’d be inclined to identify the Goodreads “after-spin” as an attempt to unify a ratings do-dah with the manners of day to day bookish discourse, a concern seemingly irrelevant to the hyperbolic hoards of MAL.
Good point about Goodreads’s lack of a way to distinguish between different kinds of negative reactions — ‘all distasteful reactions resemble one another, but every good read is good in its own way’, perhaps?
Jokes about my lit-peen aside, I don’t spend as much time as I should chatting about ‘literary’ books with people, but your identification sounds plausible to me, for what that’s worth. MAL’s hyperbolic hordes are certainly quite a contrast to Goodreads’s userbase.
I’ve had concerns over score labels myself, but now I’m perplexed by my own actions. For reviews, I use a rating system that is purposefully suggestive to my taste by giving points in categories like ‘bonus’ and by hiking the importance of different categories to my standards, and yet for my final scores, I’ve employed a system of objective terms. Perhaps I should adjust this.
My current system –
10/10 = Absolute Perfection, 9.5-10 = The Best of the Best, 9-9.5 = Completely Amazing, 8.5-9 = Excellent, 8-8.5 = Superb, 7.5-8 = Great, 7-7.5 = Very Good, 6.5-7 = Good, 6-6.5 = Above Average, 5-6 = Average, 4.5-5 = Decent, 4-4.5 = Halfway Decent, 3.5-4 = Slightly Redeemable, 3-3.5 = Bad, 2.5-3 = Crap, 2-2.5 = Horrible, 1.5-1 = Irredeemably Shitty, 0-1 = Un-watchable Unless For Unintentional Hilarity
Perhaps I merely borrowed too much from MAL’s style, seeing as I did have their score titles in mind when I made it.
‘Irredeemably Shitty’, heh. To be honest, I admire your ability to list out a set of precisely-defined set of ranges.
This provokes a further thought, that both Goodreads and MAL are basically serving a spread of people, some of whom (like yourself) can handle the precision of numbers and some of whom (like myself) instinctively respond more to the glosses. That’s obvious, and probably the real reason for the mixture of the two systems, but I hadn’t considered it.
My head hurts just thinking about ratings. I end up doing a very instinctive knee-jerk numerical thing on MAL — but these aren’t really reliable I think.
What they are, are what I would call “If a gun were pointed at my head, how would you rate this show?” kind of values.
I do the same. They’re very much knee-jerk ratings, but at the same time I aim them, however wildly, at what I genuinely think the title in question’s quality is.
That’s one of the reasons my lists on MAL are set to private: they’re an attempt at aesthetic judgement, but I know sweet F.A. about how to properly go about that judgement, so I don’t like the idea of possibly influencing others by revealing my scores.
Rating systems meant for mass usage are always near impossible to use for every situation. I mean some people do not want the hassle of saying exactly how the book/anime/show felt to them if they were just casual about it.
Then again I also need to use a revised and suitable system of scoring for my anime series, and as for books, since I stopped reading them regularly already, I suppose I can no longer really comment on them.
It’s true that some people don’t want the hassle, and I’ve seen lists sans ratings on MAL before — and I’m sure they exist on Goodreads, though I haven’t been exploring that long enough to find any.
Garururu :3
MAL’s scores are directly tied to AniDB I believe (there may be more info on scoring in the AniDB wiki, who knows).
The issue I have with all this is that everyone’s scores are different, and usually it’s about forcing users to “work with what they got.” Perhaps it wouldn’t be so bad if there wasn’t a level like “Unwatchable” … there is absolutely no logic there; I can rate a show because I watched it, I rated it unwatchable, but I watched it…. -_- wtf
A second, perhaps more useful look would be at levels of recommendation, say for the most basic system: highly recommended, recommended, and not recommended.
The to ad a bit of relevance between titles, comparing or recommending one over the other.
If the environment of a recommendation is a single title and some numbers, the system is flawed. A rating is only relevant per-title, and not between titles. One way around this is if every user considered their previous ratings in making a new rating (assessing between items).
Nothing new, I’ve been spitting this same rating stuff for a while. It’s good to see some simplicity though in the 5-star system ^^
Recommendation is an interesting idea, though I suspect it’d work independently of ratings — I can think of good titles I wouldn’t recommend and bad ones that I would, and MAL’s recommendations board is actually quite a good tool for getting recommendations tailored to specific tastes.
Goodreads’s five stars do have a certain cheery simplicity, don’t they? Thinking further on the fact that they are stars, I think that gives them a certain lightness. Like . . . maybe this varies around the world, but when I was a little kid, we used to be given stars for good behaviour or work at school. Using stars recalls some of that informality, and suggests that it doesn’t really matter that much.
Oh ratings, when will we ever be rid of thee?
Since the idea of ratings are based on making short and quick quips on the work in question, I value the systems that equate single word descriptors or short phrases with each rating, like MAL. Of course, the problems with MAL’s rating vocabulary occur like you said (the biggest offenders being ‘Masterpiece’, ‘Average’, and ‘Unwatchable’), but at least it offers a mutual point of reference (even if they are huge generalizations) rather than a blank numerical system, or even worse, a letter grade.
Letter grades irk me the most, because they imply objective rightness and wrongness. Same with percentage scores. It dubiously creates strict standards and expectations on fiction, like some anime or book getting marked up or down depending on how complete their multiplication tables are.
Coburn hits on another peculiar behavior people have with ratings: they vastly under use the bad ratings and clump all their scores in the upper half of the scale. Take a random MAL user and look at his/her ratings (if they rate things). They would all fall between 10 and 6, despite the fact that there are the scores 1-5 beneath that. People don’t really care about describing the level of suck, it just sucks, but do care to detail exactly how good ‘good’ is.
Percentage scores are an interesting topic. I’ve seen them used in magazines which have to rate thousands of titles over their lifetime, and need that kind of discriminating detail. There’s a neat piece from Kieron Gillen on some music-reviewing website somewhere (my Google-fu is weak) in which he compares music reviewing and game reviewing, and says that games reviewers usually think of 70% (I think) as an important line: games that function usually get 70 or more, and games that have bugs or serious problems get less.
I think under-using the bad ratings can be justified, in some cases at least: if someone’s convinced that they only watch decent titles, and they drop bad ones and don’t rate them, then they’ve no real use for the negative ratings.
To touch on the subject of emotions influencing ratings (as that’s what your post is most about), they’re always going to have major sway on how one decides. Like I said before, a work of fiction isn’t taking a test; there isn’t some universal checklist to measure its completeness or incompleteness. We are all judges making arbitrary decisions, but hopefully we can clearly justify those judgments, at the very least to ourselves.
Unpicking emotional experience from a work’s objective quality is beyond me, ultimately. I’m particularly suspicious of evaluations that claim to dissect the story and assess things like character development and pacing, but then I’ve never really studied the creative side of writing.
The justification element is perhaps what I find most interesting, because in justifying I think we learn about what’s important to us — it’s more self-examination than evaluation, sometimes.
Pingback: a fall sci-fi reading list « Haibane.info
Pingback: Rating an Experience - aloe, dream
Damn you on two counts – first of all, I was about to write some kind of obnoxious post/attempt at public (?) humiliation over your lack of posts as of late… and then I click on your blog and find you posted TWICE in the past week! Second of all, great, there’s a place to log all the books we’ve read now? So not only do I need to continue my attempts to remember every scrap of anime I’ve seen for MAL, now I have to remember all the books I’ve read?!?! Holy shit, I’m toast before its even started.
Hah! A dastardly plot foiled! Though ideally I suppose I would’ve posted just as you posted.
Goodreads is a massive timesink, even if you just record what you read after you join it. But hey, it’s lit-peen.
IKnight: I thought you died or something after you didn’t respond to my email to you. I will be adding my new Tokyopop translation of Welcome to the NHK to my re-reading list since I got myself a remarkably fresh copy at Borders today.
I am already a member of LibraryThing.com, but GoodReads seems to be a bit more like MAL, which I didn’t care for much. But maybe GoodReads is different, do enlighten me about the similarities and differences.
Sorry about the lack of response — as I said in my eventual reply, I was in the wilds of Dorset.
I suppose the most obvious difference between Goodreads and MAL is that MAL gives its users more scope to customise (I’m tempted to say ‘pimp’) their profiles. I suspect the different age ranges of the two sites’ target audiences may have something to do with that.
Pingback: Better well read than dead — SNAG Vs. World
Interesting meta-post. I don’t know much about MAL, except that I tried it and didn’t have the patience, but I know it’s a rather big deal with fans these days.
The funny thing about MAL’s ratings is that they resemble conversations with real-life anime fans. Lain? “Oh my god, it’s a masterpiece, you have to watch it.” Musashi GUN-dou? “It’s totally unwatchable, I turned it off.” And so on.
MAL’s an interesting place. I find that, in the long run, the list and tagging system save me quite a lot of effort in remembering things: in a few clicks I can pull up a list of all the titles I’ve seen or read which were set in space, or featured child soldiers, or had noir elements &c &c. Some users obviously get a lot more out of its social networking functions than I do, and it’s interesting to see the effort that people will put in to prettifying and customising their profile pages (which probably takes us back to Ghostlightning’s post about self-branding through choosing favourites).
“…I’ve seen lists sans ratings on MAL before…”
I gladly declare that I’m one of those! I wasn’t always this way. I used to rate things… and then re-rate them… again and again. I’d try to spread out the scores, realign scores as new series entered the fold, assign my own meanings to each score (which I explained on my profile page), etc etc.
But it was all a waste of time and effort. Never mind being externally consistent. I couldn’t even be internally consistent. Because scoring is more than personally subjective; it’s time and place subjective.
Sometimes I would feel much different after time had passed since seeing the show. Sometimes I would re-watch a show and have a wholly different reaction to it. I could never pin down a number (or place, ratings-wise) for the majority of shows.
Don’t even get me started on the difference between a nine and a ten, and what constitutes a favorite. That seemed to change with each hour of the day. As bad as that was, it might not even qualify as my biggest headache while trying to arrive at a “comparatively correct” scoring spread.
There was a long running (and maybe illogical) desire of mine to have two separate lists: one for more “serious” works, and one for lighter, more comedic affairs. Because trying to rank Azumanga Daioh against Neon Genesis Evangelion is seemingly ludicrous. They don’t speak in the same comparitive language!
Eventually, after enough undue agonizing about it all (I have enough stress in my life, being a working adult and all), I said “to hell with it.” Each individual show makes me feel a unique set of emotions that can’t be quantified and compared using numbers.
And the more I think about it, the more that makes sense. In fact, the whole number system now no longer makes any sense to me at all! Thus, I arrive at the point where I feel that written reviews are the only real truth, and that the numbers are a “mistranslation” of sorts.
TL;DR: While we attempt to be honest with numbers, I feel there’s no actual way to make them wholly honest.
I’d agree that we can’t have ‘wholly honest’ numbers, but I think we can say the same of words too — they too have connotations and a slipperiness that the writer can’t control. Words can alter dramatically in weeks or days. (Nowadays when someone says ‘rendition’ I think ‘extraordinary’, and it only took one news story to effect that change.) Reviews also have a wider, more contradictory set of goals than ratings: description, evaluation and recommendation don’t always align themselves, and I can think of titles which I would still recommend despite being unable to properly describe them or thinking that they’re particularly good.
So I suppose I’d say that numbers and words are very inadequate tools, but that doesn’t mean we can’t make a stab at truth with both, even though we know before we try that we’ll fail.
More practically, I agree that if assigning numbers to what you’ve watched and read feels like work, it’s probably not worth the effort!
One of the major issues I have with numbers is that they are so definitive in their ranking. For example, on certain occasions or moods, anime X (rated lower than Y) may actually exceed Y in my enjoyment. Or anime X may have more issues than Y, but at the same time do a few things far better.
This is hard to express in one single number applied to a series. A collection of numbers for different properties of a series would help here, but that would become an exponential pain in the ass as you further broke the ratings down. Written words — while not perfect — allow me to better express how the series and its various aspects make me feel.
yay commenting on a three-week-old post on ratings
What bugs me about the MAL rating system is that all the glosses are “objective” words which therefore bear “subjective” interpretation, often involving placing a shamanistic hypothetical ideal at the top of the scale and working downwards. This causes me no end of frustration when I run across someone who uses the “full spectrum” of numbers because oftentimes their heuristic methods make no sense to my intellectually addled mind.
The problem with Goodreads’ ratings system is that there are not half-stars. This drives me NUTS when I feel I fall between the (gloriously subjective) glosses.
I’m probably too self-absorbed, but I must admit I don’t normally read others’ explanations of their rating habits. And perhaps I’m a little insulated from your experience of frustration because too many pieces of abstruse literary theory have broken the part of my mind that worries when things don’t make sense.
I would find it easier to sort things by gut feeling as I usually do if Goodreads had more degrees, half-stars of some kind, in its ratings, though I think it would lose some of its cheery informality if it had them.
I hate you… I just caved and created an account.