With our accuracy study results recently announced, we thought it would be interesting to take a look at how well two of the most popular sites in the fantasy industry, Yahoo! & ESPN, performed with their 2009 weekly rankings. Our previous findings revealed how the individual experts at each site fared with their predictions so the next obvious question is: Which site gave more accurate advice with their composite staff rankings?
The answer: Yahoo! and it was a clean sweep across all of the positions in our study. In the table below we take a look at the accuracy results for each site using our PAY™ (Prediction Accuracy Yield) scoring methodology.
Yahoo! vs. ESPN Accuracy Results
| Site | Running Back | Wide Receiver | Quarterback | Tight End | Kicker | DST |
|---|---|---|---|---|---|---|
| Yahoo! Staff | 63.1% | 60.2% | 61.0% | 59.3% | 59.7% | 61.2% |
| ESPN Staff | 60.9% | 56.9% | 58.1% | 54.7% | 57.4% | 56.3% |
Of course, these composite results aren’t too surprising considering that all of the Yahoo! experts finished inside the top 10 of our overall results while each ESPN expert was outside the top 10. What obviously stands out from this though is how consistently Yahoo! had the edge over ESPN regardless of the position in question. Given this, we half expect that if the sites had predicted each team’s water boy performance that Yahoo! would have had the advantage there as well.
In all seriousness, if we take a deeper look at the numbers, we can start to see some interesting data emerge about how the sites differed with their predictions and what gave Yahoo! its edge. As an example, let’s isolate the WR position for this site faceoff.
This chart shows how often the two sites disagreed with their picks. For those unfamiliar with our methodology, we ran a round robin of head-to-head player match-ups and noted which player each site predicted would score more fantasy points, based on how the sites ranked each player in their published rankings. We also scrubbed out predictions for match-ups like “Andre Johnson vs. Domenik Hixon” where all of the experts in our study were in agreement. The rationale for removal being that these picks don’t provide much value to advice seekers (at least, we hope this is the case for the advice seekers’ sake).
So, you’re probably looking at this chart and thinking “Wow, these guys agreed with each other a lot!” It’s true but the real key is the small bucket where the sites disagreed because this largely represents the most difficult player predictions. In many ways, these are the player decisions that fantasy owners struggle with regarding who to start or sit because they’re such close calls. Ultimately, this is where Yahoo! gets its advantage over ESPN as the following chart shows:
This chart looks at the Win % for each site and shows that Yahoo! had a decisive edge over ESPN for this category of picks. Using Win % as a success metric doesn’t paint a full picture of how good each site’s advice is, however, because not all predictions are worth the same in fantasy football. For example, when you make a great start/sit call, you are rewarded with the difference in fantasy points between the guy you started and the guy you benched. In our assessment, we factor in this value of benefit for each correct prediction using our PAY™ scoring methodology. When we bake this element into the results we can see that Yahoo!’s advantage over ESPN increases even further:
This tells us that not only did Yahoo! pick the right player more often than ESPN but they also did a better job of correctly predicting the outcomes that provided high value to fantasy owners. As mentioned, this wide gap in accuracy when the two sites disagreed is ultimately what gave Yahoo! the overall advantage with its WR predictions.
So what does all of this tell us? Basically, we now know that Yahoo! was more accurate than ESPN with their composite 2009 weekly player rankings across the RB, WR, QB, TE, K, and DST positions. And when we isolate the WR position we can see that Yahoo! gained its advantage by outperforming ESPN with their predictions for the difficult, high value start/sit decisions that fantasy owners labor over each week.
Will this hold up in 2010? That’s anyone’s guess and we’ll certainly be tracking the accuracy results again and sharing them with you as the season progresses. For now, Yahoo! has the edge but the chance for redemption starts next week for ESPN when the new season begins. As always, may the best site win!
