Why star ratings fail for movies

Star ratings ask you to invent a number on a scale nobody agrees on, including you a year from now. The result is that almost everything lands on the same two or three values, dozens of films end up marooned at an identical score, and the rating cannot tell you the one thing you wanted it for: which of these did I actually like more.

Everything piles onto the same few values

A five-star scale with half stars offers ten values, and in practice most people use about four of them. Ratings cluster hard around three and a half and four stars. On Letterboxd, roughly a quarter of all ratings given are four stars. Half stars get skipped by plenty of people entirely, which quietly halves the scale again.

The consequence shows up the moment you look at your own profile. A regular filmgoer with four hundred logged films has a hundred of them sitting at four stars, in no order, with no way to say that one of them changed your life and another was a pleasant Tuesday. The rating did its job as a rough signal and then quit exactly where the interesting question starts.

This is not a flaw in anyone's judgement. It is what happens when you press a continuous feeling through a four-value gate. The compression is built into the instrument.

Your scale drifts, and it was never anyone else's scale

Two separate problems hide inside one number.

• It drifts. The 7 you gave in 2020 was calibrated against what you had seen by 2020. You have watched three hundred films since. Nothing went back and adjusted the old number, so half your history is scored in a currency you no longer use.
• It does not transfer. Your 8 and a friend's 8 are different claims. Some people rate generously, some reserve five stars for a handful of films a decade, and both are reasonable. Averaging them produces a number that is nobody's opinion.
• It moves with your mood. Rate a film on the drive home and rate it the following weekend and you can easily land half a star apart, which is a large move on a scale where most of your films already sit within a single star of each other.

A comparison has none of these properties. "I liked this one more" is a statement about two specific films, and it stays true next year.

The false precision trap

Scoring systems that offer decimals make the problem worse in a new direction. An 8.2 and an 8.3 look like a distinction, and no one alive can defend one. If a score has more resolution than the judgement behind it, the extra digits are decoration, and people can tell.

The honest version is to let precision come from somewhere it can be earned. A number that is derived from where a film sits in your own ordered list can carry one decimal, because the position underneath it is exact. A number you typed cannot.

What comparison fixes

Head to head is a different instrument, not a nicer coat of paint on the same one.

• Every answer is a real preference. You are never asked to translate a feeling into a scale. You are asked which of two films you liked more, which is a question you can answer in a second.
• The pile-up disappears. An ordered list has one film per position, so the hundred films that were all four stars end up in an order instead of in a heap.
• The list is the rating. If you want a number, it can be read off the position rather than invented: in Reelist, a band sets the range (Liked it runs 6.7 to 10.0) and your order inside the band sets the decimal.
• It is measurably the quieter channel. When researchers put the two methods side by side on quality judgements, rating an item took around 6.1 seconds against 1.2 seconds for a comparison, and the rating channel came out roughly 50% noisier. Their finding was that comparisons produce better estimates for the same human effort.

Where comparisons have their own problems

Comparison is better, not perfect, and pretending otherwise would be the same mistake in a new hat. Some pairs really are too close to call, which is why a tie answer has to exist. Some pairs are unfair questions (a documentary against a musical), which is why skipping one has to be free. People are not perfectly consistent either: over enough answers, small contradictions turn up, and a ranking system has to treat them as noise rather than as errors to be corrected out of you.

What comparison removes is the invented number, and that is the part of a rating that was never carrying weight.

Stars are the weak part of a very good thing

None of this is an argument against film diaries. Letterboxd is a wonderful record of a watching life: the diary, the reviews, the lists, the community, and the running joke of a rating culture that produced some of the best writing on the internet. Millions of people are right to love it. The star is just the one component being asked to do a job it cannot do, which is to tell you the order of the films you love.

That is the job a ranked list does. Keep the diary. Get the order from somewhere it can actually come from.