What is pairwise movie ranking?

Pairwise movie ranking orders films by asking you to choose between two of them at a time. Instead of scoring a film out of five or ten, you answer which of two you liked more, a handful of times, and those answers place the film in one ordered list. The list is the rating, so no number ever has to be invented.

How pairwise ranking works

There are two passes: a coarse one that costs a single tap, and a fine one made of comparisons.

The coarse pass is a gut call. In Reelist it puts a film into one of three bands: Liked it, It was fine, or Didn't like it. Almost nobody hesitates over that question, and answering it removes two thirds of the list before a single comparison is asked.

The fine pass is where the order gets made. The app puts the new film beside one you have already ranked in the same band, near the middle of it, and asks which you liked more. Your answer rules out half the positions in that band. The next comparison comes from the middle of the half that survived, and rules out half of that. It is the same move as opening a dictionary in the middle rather than starting at page one.

The halving is what makes the method cheap. Placing a film in a list of n by comparison takes roughly log2 of n questions: about seven for a hundred films, about ten for a thousand. Reelist asks at most ten, which is enough to place a film exactly within a band of up to a thousand titles, and the band has already done most of the sorting, so most films land in far fewer. You can also say two films are too close to call, or skip a matchup you would rather not make.

The position is where the number comes from. Each band owns part of the 0 to 10 scale (Liked it runs 6.7 to 10.0, It was fine 3.4 to 6.6, Didn't like it 0.0 to 3.3), and the films in a band are spread across its range in the order your comparisons put them in. Your best film in a band lands near the top of it. One decimal, never two, because nothing can honestly tell an 8.42 from an 8.43.

Why two at a time beats a ten point scale

People are reliable at comparison and unreliable at absolute judgement. Ask someone whether a film is an 8 or a 9 and the honest answer depends on what else they have watched that month. Ask which of two films they liked more and the answer arrives at once, with no scale to hold in their head.

The difference has been measured. Researchers building quality scales for images and video found that rating an item took about 6.1 seconds against 1.2 seconds for a head-to-head choice, and that the noise in the rating channel ran roughly 50% higher than in the comparison channel. Their conclusion was that comparisons give better estimates for the same amount of human effort. Comparison also skips the calibration problem: there is no shared scale to agree on, so there is nothing to disagree about.

A comparison also keeps its meaning. Your 7 in 2020 is not your 7 today, and it was never anyone else's 7 either. "I liked this one more" is a fact about two specific films, and it is still true a year later.

Where the idea comes from

The statistics are old. The Bradley-Terry model, published in 1952, works out a strength for every competitor from nothing but the record of who beat whom, and Arpad Elo's chess rating system made the same idea famous a decade later. Chess players never rate themselves out of ten. They play, and the ordering falls out.

The idea keeps getting rediscovered wherever taste is hard to score. LMArena ranks AI models by showing people two anonymous answers and asking which is better, then fitting a Bradley-Terry model to the votes. Beli does it for restaurants: a quick liked it, fine, or did not like it, then a few comparisons, then a 0 to 10 score. Flickchart has been asking film fans which of two movies is better since 2009 and turning the answers into ordered charts.

There is a folk version too, and it is enormous. Character sorters, which ask a few hundred questions and hand back a ranked list, are hand-driven merge sorts. Tier lists on TierMaker are the same instinct with the comparisons done in your head. People reach for ordering because ordering is what they actually mean.

What it feels like once your list is long

The cost of a comparison method grows with the logarithm of the list, not with the list, so a long list is barely more work than a short one. A film joining a list of 500 finds its place in the same handful of taps it took when the list held 50. You do not review your top 20 to see where the new one goes, and you never renumber anything.

That is the whole appeal of tools that rank movies by comparing two at a time: the questions stay easy while the list gets serious. Reelist adds one tap for how you felt, at most ten comparisons, and a score that comes out of the position rather than out of you.

What pairwise ranking does not fix

It is not magic, and the honest limits are worth stating. Some pairs are genuinely too close to call, which is why a tie option matters. People are not perfectly consistent: prefer A to B, B to C, and C to A on a bad day, and any ranking system has to absorb that as noise rather than pretend it did not happen. And a comparison-built list is a record of your answers, not an objective verdict. It has no opinion about whether you are right.

What it removes is the part that was never real anyway: the number you were asked to invent before you had anything to compare it to.