Baseball is clearly the best sport for projections. A long history of hundreds of games per year and long careers means we have a wealth of data to project talent. The low rate of injury (compared to some sports) and consistent nature of batting orders take much of the guesswork out of projecting opportunity. There is a well-established culture of approaching baseball analytics scientifically. All this adds up to a sport where projections are quite good and getting better every year.

At FantasyPros, our ultimate goal is to help you win your fantasy league. An important part of winning is deciding which projections are worth listening to, and that’s what this article aims to do. In preparation for fantasy drafts in 2024, let’s take a look at which projection systems were the most accurate in 2023.

Accuracy Methodology

There are many possible reasons to care about projections, and many possible ways to judge their accuracy. Since we are primarily concerned with fantasy, we have developed an accuracy methodology that evaluates projections in terms of their effectiveness as fantasy advice. This has 3 steps:

1. Select the right data: Because we’re here to win fantasy leagues, we mainly care about the standard 5×5 roto categories. For hitters, that means run scored (R), home runs (HR), runs batted in (RBI), stolen bases (SB) and batting average (AVG). For pitchers, we look at wins (W), saves (SV), strikeouts (K), earned run average (ERA) and walks + runs per inning pitched (WHIP). Many of these are counting stats, so we also care about playing time, and thus include at-bats (AB) for hitters, and innings pitched (IP) for pitchers for a total of 12 categories. We then evaluate counting stats on a per-opportunity bases, rather than raw totals, so that we can separate playing time projections from quality-of-play projections. For example, last year Shohei Ohtani hit 44 home runs on 497 at-bats. If you projected him to hit 49 homers on 550 at-bats, your at-bat projection missed by 53, but your home run projection was right on the money, correctly predicting his 8.9% home run rate.
2. Evaluate each projection: We ultimately care about how the projection systems stack up against each other, so we measure projection error in context. That 53 at-bat miss for Ohtani might sound like a lot, but it’s only a slightly larger miss than the average projection. To measure this precisely, we transform each projection error into a z-score, in the context of projections from all sources for the same player and stat. To continue the Ohtani example, last year all sources had an average error of 47.5 for Ohtani’s at-bats, with a standard deviation of 13.6. Thus, your projection that missed by 53 at-bats was (53-47.5)/13.6 = 0.4 standard deviations worse than the average.
3. Aggregate the projection errors: Another advantage of measuring projection error with z-scores is that it puts every category on the same scale. For a given projection source and stat we will take the average of all their projection error z-scores across the player pool, to get an accuracy score for that stat. Then to judge the best overall projection systems, we add together the scores from each stat.

2023 Projection Accuracy Results

Note that since our z-scores represent projection error, lower is better. Without further ado, here are the best projection sources from 2023!

Ariel Cohen‘s Average Total Cost (ATC) projection system is an institution. Incredibly, this is ATC’s fifth consecutive year as the #1 projection system! This is a testament to the incredible power of ATC’s consensus methodology. Some projection systems, like FantasyPros’ own Zeile, are a simple average of other projection sources. ATC takes that idea a step further and combines several sources smartly, weighting them differently for different stats to achieve an optimal mix, which combines the strengths of the component systems. If you’re putting together a cheat sheet for your fantasy draft, there’s no better set of projections to use than ATC. The consistent track record of accuracy means we can be confident that ATC’s projections will be excellent again this year.

Our #2 system is an impressive showing from a newcomer to the competition, Draft Buddy by Mike MacGregor! Draft Buddy uses a consensus of multiple sources for playing time projections and it’s clearly working. Such a strong showing for Draft Buddy’s first year is promising – will they be able to challenge ATC’s throne in the future?

Taking a consensus of original projection sources like ATC does is clearly the path to the best possible projections, but that could not exist without the projection systems it’s made of. Nobody does original projections better than Derek Carty with THE BAT. Carty has been publishing THE BAT since 2013, and introduced THE BAT X for hitters in 2020. THE BAT X is cutting edge, layering statcast data on top of an already solid foundation of projections. To learn more, check out the announcement article. The results speak for themselves – since it’s introduction, THE BAT X has been the most accurate original projection system for four straight years.

Category Results

And now, for the breakdown of how well each system projected each of the hitter stats we care about for fantasy:

And Pitchers:

—

Jacob Herlin is a Data Scientist for FantasyPros. You can find him on twitter @jacoblawherlin.