OR instrumental in Netflix prize

One of the leaders in the Netflix prize is operations researcher (and psychologist) Gavin Potter. Wired has an article about Potter that summarizes some of his ideas.

Most of the contestants use k-nearest-neighbor algorithms and singular value decomposition (SVD)

that determines the best dimensions along which to rate movies. These dimensions aren’t human-generated scales like “highbrow” versus “lowbrow”; typically they’re baroque mathematical combinations of many ratings that can’t be described in words, only in pages-long lists of numbers. At the end, SVD often finds relationships between movies that no film critic could ever have thought of but that do help predict future ratings.

The differences between most of the contestants is quantified by how well they resist overfitting their algorithm.

Potter’s approach links movie predictions to human behavior, and he uses ideas by Amos Tversky and Daniel Kahneman to account for human biases in how they rate movies.

One such phenomenon is the anchoring effect, a problem endemic to any numerical rating scheme. If a customer watches three movies in a row that merit four stars — say, the Star Wars trilogy — and then sees one that’s a bit better — say, Blade Runner — they’ll likely give the last movie five stars. But if they started the week with one-star stinkers like the Star Wars prequels, Blade Runner might get only a 4 or even a 3. Anchoring suggests that rating systems need to take account of inertia — a user who has recently given a lot of above-average ratings is likely to continue to do so. Potter finds precisely this phenomenon in the Netflix data; and by being aware of it, he’s able to account for its biasing effects and thus more accurately pin down users’ true tastes.

You can read more in Tversky and Kahneman’s seminal 1974 and 1981 papers in Science.