A Heads-up comparison of two heavyweights from ESPN and FootballGuys

Well, if you’ve been studying our accuracy scores, the answer is obvious. After slugging it out over 16 weeks in 2009, David Dodds beat out Matthew Berry in the key categories of RB, WR, QB, and TE accuracy. They split on DST and K, but this is the last time I’ll ever mention the word kicker in the first paragraph of a fantasy football post.

Here’s how high their PAY™ (Prediction Accuracy Yield) was for the four key positions in 2009:

So why am I bringing up something that can be easily found on our Accuracy Scores page? Am I just taking a cheap shot at Mr. Berry because he’s the most well know fantasy football expert? Or am I secretly on the FootballGuys payroll? Neither.  OK, a little of the first (people like reading about Matthew Berry), and none of the second (although I’m open to free subscription offers).

I’m bringing up a specific example of two experts because I think it helps illustrate the foundation of how we do our accuracy assessments. A surprising number of people have actually taken the time to read about our methodology, so this seems to be an area of interest. Using the RB category as an example, let’s take a closer look at this expert faceoff:

This chart shows how often Matthew Berry and David Dodds agreed and disagreed with each other’s RB predictions. We basically ran a round robin of head-to-head player match-ups and noted which player each expert predicted would win, based on how they ranked them in their published rankings. Note that we scrubbed out predictions that all the experts in our pool agreed on (e.g. Chris Johnson vs. Mr. No Name Desperation Start), so this data only represents predictions where there was at least one expert backing each player being compared.

For RBs, these two experts disagreed with each other about 26% of the time.  The reason why David Dodds performed better than Matthew Berry was because when they disagreed, he was more likely to be right. Here’s how often each was right when they argued:

That doesn’t tell the whole story though. Not all predictions are worth the same amount of value in Fantasy Football. When you make a great start/sit call, you’re rewarded handsomely with the difference in fantasy points between the guy you started and the guy you benched. We believe the magnitude of this benefit must be included in an accuracy assessment because this is the true benefit that you’re receiving from following the expert’s advice. Once we bake in the net fantasy points from each expert’s correct predictions, the spread between Berry and Dodds becomes a bit more pronounced. Basically, not only did Dodds get more of the disagreements correct, he also tended to get the more valuable predictions right. Here’s the difference once we bake in fantasy points:

The metric we use to measure accuracy is called PAY™ because it tells you what benefit you receive for following the expert’s advice. A 100% PAY™ would mean that following the expert’s advice would have given you the optimal lineup each week of the season. Note that the PAY™ amount in the chart above is different from what’s reported in the very first table because it only looks at match-ups where Berry and Dodds disagreed.

Also, keep in mind again that we only score predictions that are not unanimous among the experts. So these ratings represent how the experts are doing with the relatively harder predictions in fantasy football. No one’s looking for advice on whether to start Chris Johnson over Mr. No Name. We believe we should base our accuracy assessment on the predictions that actually matter to fantasy players. A score in the 50’s and 60’s may look poor to you, but we all know how tough it is to get the close calls right. Similar to sports gambling, a small relative edge can mean all the difference between whether you make a trip to the cashier window or the ATM.

I hope this post helps to illustrate the approach we take for rating fantasy advice. Obviously, accuracy is only one benefit we get from listening to true experts. I’m still going to read what TMR has to say, but perhaps with a little less blind faith. The good news for Mr. Berry is that he gets to get in the ring with Dodds again this year for a rematch. May the best fantasy expert win!

Note: We’re planning on following up this post with a look at how these two guys did when they agreed and disagreed with the expert consensus opinion. So, not compared to each other but compared to the wisdom of the consensus rankings. Stay tuned for that insightful analysis…

Image Credit: Link