Yahoo! labs has unveiled Predictalot for combining user rules that don’t explicitly rely on math for predicting tournament winners, although there is a lot of complexity in the game. Predictalot is a #P-Hard game that uses combinatorial prediction market methodologies for combining human input and high-performance computing for making better tournament predictions.
Predictalot limits users to a pretty restricted set of rules. The rules focus on more aggregate or simple outcomes that are easy to count (like the sum of the seeds in a given round, rather than the mix of individual seeds). This isn’t too bad for beta version 1.0, but I still found it frustrating. For example, when predicting which seed range advances to what round, I am unable to create a rule that indicates that a single five seed or worse will make it to the final four. I can only create a rule about all final four team seeds. I also wanted to create a rule about how many Final Four teams would be from the Big Ten conference. The only two conference rules allowed: (1) predicting the winner and (2) predicting if a conference will have more or fewer wins than another conference.
It is also not clear that “better than a 4 seed” is strictly-better-than or better-than-or-equal-to. Accuracy is important to me. For example, I was unable to create a rule that a one seed would win the tournament (see image below). This is something that can easily be fixed.
Still, Predictalot looks pretty good for a beta version, and it will be interesting to see how it works, both in terms of predicting a winner and harnessing the power of social networking.
Links:
- Predictalot
- NY Times article about Predictalot
Punk Rock Operations Research 
March 18th, 2010 at 10:33 am
Hi Laura, thanks for playing with Predictalot and for the great writeup. Sorry for any difficulties: Reports like yours will help us improve in the future.
A couple comments for now:
* Better than is strictly better than
* I think if you choose the proposition type “what seed will win tournament” you can make the prediction you want that a #1 seed will win
* When entering ranges, put the numerically smaller seed on the left: “Seeds between 1 and 5”
* In the future we’d love to allow users to more flexibly create their own propositions. The back end can handle it, but it’s the front end (user interface) that is hard.
March 18th, 2010 at 10:49 am
Thanks, Dave! These types of things are typically worked out during Beta testing. If the Netflix prize has taught us anything, it’s that combining several approaches from different people works better than any one approach. I am interested in post-tournament analysis about what new insights you found.