How accurate are Moneyline picks?

For Confidence Pool Picks, I use the implied win probability that Moneylines give us to create the “base picks” each week. The idea is to assign higher win probabilities higher confidence values. But is Moneyline even accurate?

I wanted to put this to the test. I analyzed data from the 2013 season, and compared what Moneyline values predicted to what actually happened. To review, the Moneyline values are in the form of +150 or -300. Positive values mean you bet the value, in this case $150, to win $100. Negative values mean you bet $100 to win the value, in this case $300. You can then calculate an implied win probability based off these values, in other words “there’s a X% chance the favored team will win the game.”

I took all the Moneyline win probabilities, rounded it to the nearest 5%, and then compared it with what percentage of picked teams that actually won. Here is what it looked like:

2014-10-18_1326

And graphed here, with the Moneyline predicted win probability on the x-axis and the actual win % on the y-axis:

chart 1

A perfect model would have all the dots on the line. The Moneyline predictions look pretty good.

Last year, I also used an Advanced NFL Stats (now Advanced Football Analytics) model, thinking it would do a better job of predicting the games because it took the mood swings of the crowd out of it. So I also ran the same analysis on it. However, there were less data points because the model doesn’t start until Week 5 because it uses previous data in its calculations. I rounded each bucket to the nearest 10% instead of nearest 5%, and got the following:

chart 2

Not as accurate as the Moneyline picks, which is surprising to me because this model is constantly being reviewed and updated for accuracy. It may be that the Moneyline picks take into account last minute injury situations, or the model still needs more refining.

For the final confidence points, I used an average of the ANS and Moneyline probabilities. Here’s what the accuracy of using this method looks like:

chart 3Not too bad, but at best only as good as the Moneyline picks alone. So for 2014, I’m going to stick with just the Moneyline values. It simplifies things and may be just as good.

Just for fun, I figure I’d check to see how good the people are in predicting win probability. So I did the same plot, taking the percentage of the people who picked that team to win as the predicted win probability. Something interesting came out:

chart 5 More people picked a team to win than the team actually won. Most people were overoptimistic or risk averse; not many brave souls would go against the crowd. What this means to our pick strategy I’m not sure yet, but it’s an interesting phenomenon.

Running a “hindsight is 20/20” analysis, I figured out how many total points you would’ve won if you went with the straight Moneyline implied probabilities and didn’t pick any upsets. And the answer is:

1456.

Would that have won your league?

[See 2014 retro analysis]

 

  • Rick

    This is beyond fascinating. I’m curious to see how this could help me in my league.
    Looking forward to more insights like this!

    • ddting

      Thanks Rick. It’s comforting to know the probabilities are relatively accurate. I’m working on some simple methods to maximize your chances of winning your weekly pool, check back here soon!

  • Victor

    Awesome analysis. It’s amazing how accurate the moneyline really is. There are always going to be outliers and upsets that come out of nowhere (and I’ve yet to win a week because of it), but staying the course has kept me in the top 20% each week, which has been good enough for first place overall.

    • ddting

      Agree, Vegas knows their stuff. They literally have millions riding on it.

  • AM

    I have no prior years to look at, but looking at the point totals after 8 weeks, I believe 1456 would be just enough to win the season.

    However, I’ve been told our league is determining the winner by overall win-loss record instead of total points. The points are being used to break ties. I question how popular this approach is (if it all) since Yahoo is actually organizing the standings by points and not win-loss records. Seems like a lot of extra work for the commish to figure it all out at seasons end.

    On a long enough timeline one would think most points would also equate to best win-loss, but I don’t think it’s out of the realm of possibility that you could score the most points yet finish out of the money on win-loss record. I’m not sure yet how this really changes my approach, but it should be interesting to see how things fall going forward.

    • ddting

      Wow, that is interesting. That actually makes the all in strategy more interesting, because if you lose you only lose that one game and not the 15 points or what not.

  • KM

    I’ve always based my confidence picks on vegas spreads, but using moneyline values is very interesting to me. How exactly are you calculating implied win probability based on moneyline values? Do you have any info that might allow for comparison between the success of spreads vs moneyline?
    Thanks!

    • I haven’t done any correlations with spreads vs moneyline… I originally looked at spreads, and found some research paper that said each point was about 3% worth of win probability. Check out http://www.bettingexpert.com/blog/how-to-convert-odds for how to convert Moneyline to win prob. I figured Moneyline gave me more quantitative values vs. just multiples of 3.

  • Frank

    1456 would have been 4th in my league, out of the money. I won it with 1512. I have a pretty good idea for why my total is so much bigger, but I’ll keep that secret sauce to myself.

    • Frank

      BTW, I’m not surprised by the ANS model sucking. The first year I played a pool I won it by using straight probabilities. The 2nd and 3rd years, I thought I could improve on it by using a model. I finished out of the money both years. I went back to using the probabilities, and I haven’t finished worse than 2nd since.

    • gggg

      Just to be sure…we are talking just the regular season (17 weeks) with 1456 pts & not including playoffs? Assume your 1512 pts was just regular season totals correct?

      • You are correct, regular season score.

      • Frank

        No playoffs.