One of the most popular features of our site this season was our Expert Consensus Rankings (ECR). Whether it was a straight look at the composite rankings of all 40+ experts we tracked or one of the many custom combinations available (e.g. Top 5 experts only), we’ve taken note that readers loved the ability to combine rankings of multiple experts.
Now that the season is over, we thought it would be interesting to see how well several of the common ECR combinations stack up against individual experts. In other words, does the “Wisdom of the Crowd” or the “Wisdom of the Few” prevail when it comes to the accuracy of fantasy football rankings? Let’s take a look at how the top consensus cuts did to find out!
Consensus of All Experts
| Overall Accuracy Scores | ||
|---|---|---|
| Position | Expert | PAY™ |
| 1. | John Paulsen – The Scores Report | 61.3% |
| 2. | Consensus of All Experts | 60.5% |
| 3. | David Dodds – FootballGuys | 60.4% |
| 4. | Andy Behrens – Yahoo! Sports | 60.2% |
| 5. | Josh Moore – 4for4.com | 59.9% |
The Consensus of all Experts (40+) was the most popular cut that our readers looked at. It’s a good thing too considering that it produced the second highest accuracy score amongst the experts. What jumps out the most when looking at this set of rankings is that it finished in the Top 10 for each of the 4 main positions that we used to generate our overall ratings: RB (#4), WR (#3), QB (#10), and TE (#9). This type of consistency across the positions is extremely important since most owners have start/sit questions for multiple positions. It also shows the power of consensus rankings because most individual experts have a set of strengths and weaknesses that prevent them from nailing each position. These weaknesses are generally reduced when a collective opinion is looked at since the majority outlook prevails.
2010 Rolling Top 5 Consensus
| Overall Accuracy Scores | ||
|---|---|---|
| Position | Expert | PAY™ |
| 1. | John Paulsen – The Scores Report | 61.3% |
| 2. | David Dodds – FootballGuys | 60.4% |
| 3. | 2010 Rolling Top 5 Consensus | 60.4% |
| 4. | Andy Behrens – Yahoo! Sports | 60.2% |
| 5. | Josh Moore – 4for4.com | 59.9% |
If you used our Cheat Sheet Wizard each week to get a consensus of the then top 5 experts, you would have benefited from a set of rankings that only 2 individual experts beat out. Something important to point out with this cut is that it was much more reliable as the season went on since a larger set of data was available to determine the most accurate experts. For example, these consensus rankings were more accurate than any individual expert over the final 4 weeks of the season whereas they were rated just 10th best through the first 4 weeks of the season.
2009 Top 5 Consensus
| Overall Accuracy Scores | ||
|---|---|---|
| Position | Expert | PAY™ |
| 1. | John Paulsen – The Scores Report | 61.3% |
| 2. | 2009 Top 5 Consensus | 60.6% |
| 3. | David Dodds – FootballGuys | 60.4% |
| 4. | Andy Behrens – Yahoo! Sports | 60.2% |
| 5. | Josh Moore – 4for4.com | 59.9% |
If there’s one thing we learned this past year it’s that the experts who scored well in our accuracy competition didn’t do so as a result of blind luck. A great example of this is the fact that the consensus of 2009’s top 5 experts generated this season’s second best accuracy rating across all of the individual experts. Specific credit goes to David Dodds (FootballGuys) and Andy Behrens (Yahoo! Sports) as they finished in the top 5 for both years.
2010 Top 5 Consensus
| Overall Accuracy Scores | ||
|---|---|---|
| Position | Expert | PAY™ |
| 1. | 2010 Top 5 Consensus | 61.6% |
| 2. | John Paulsen – The Scores Report | 61.3% |
| 3. | David Dodds – FootballGuys | 60.4% |
| 4. | Andy Behrens – Yahoo! Sports | 60.2% |
| 5. | Josh Moore – 4for4.com | 59.9% |
Yes, you’re reading that right: Something finally edged out John Paulsen (The Scores Report)! This consensus group specifically looks at the rankings from our final top 5 experts in 2010. Of course, Paulsen is a member of that top 5 so he’s a huge reason why this particular set of consolidated rankings did so well. It’s worthwhile to note that it would have been tough to predict these experts were going to come out on top so this was not a common combination that users input into our Cheat Sheet Wizard; at least not early in the season. It does, however, show that once again there was some value at a high level in looking at a combined list of rankings.
So what does all of this data tell us regarding our initial question of whether the “Wisdom of the Crowd” prevails over the “Wisdom of the Few”? By and large, the data suggests that there is tremendous value in looking at consensus rankings. In many ways, this makes sense because consolidated recommendations help to reduce the negative impact of outlier predictions. Simply put: On average, going with the majority opinion that is represented by the crowd is more likely to yield a positive fantasy football outcome.
At the same time, following the “Wisdom of the Crowd” does not necessarily guarantee you of getting the best results. As John Paulsen proved in 2010, an individual expert can beat out the consensus over the long haul and it’s certainly possible that the best advice for certain start/sit decisions will come from specific experts.
More than anything, it’s important to find the expert or combination of experts you trust the most given all of the data points and use their picks as a guideline to aid your decisions. We hope to help you further with this process in the future as we’re working to provide deeper accuracy analysis that is specifically tailored to your team. Stay tuned!
Image Credit: Link
