2014 US Open Men’s Draw Simulation

The U.S. Open main draw begins this morning and for the fourth year in a row, I will not be able to attend. Gone are the good ol’ days of working for the USTA and getting to take the trip up to New York to take it all in.

Since I cannot go, I decided to utilize Markov Chain models and Monte Carlo simulations to predict who will win.

Markov Models for tennis are essentially placing some initial inputs into a model and allow it to simulate an entire match, giving you the probabilities player A wins over player B. A Monte Carlo simulation is when you run an entire tournament over and over like this. Even if you can do the math, one of the most difficult parts is creating the initial inputs to run the Markov Model.

MY METHODOLOGY
I decided to experiment with an idea that begins with something I read in Dr. Kamran Aslam’s PhD dissertation he wrote at USC. Dr. Aslam and his advisor, Dr. Paul K. Newton published portions of this paper several times, including in the Journal of Quantitative Analysis in Sport back in 2009.

Dr. Aslam took the idea that you start by finding the overall mean probability to win a point while returning. This is defined as the returning average of ‘the field’. Let’s say this is 0.330. Then, if Roger Federer is playing Novak Djokovic and Roger’s average ability to win a point returning is 0.40, then he is 0.07 better than ‘the field’. If Novak’s average is 0.41, then he is 0.08 better than ‘the field’.

Then, if Roger’s percentage he wins serve is 0.7, you subtract Novak’s ability ‘above the field’ (0.08), making Roger’s effective serving percentage, 0.62.

Likewise, if Novak’s serving percentage is 0.68, then his effective serving percentage is 0.61. Therefore, the input to the program would be 0.62 and 0.39 for Roger (one minus Novak’s effective serving percentage). If you ran this for Novak, the inputs would be 0.61 and 0.38.

Modifications
To get the data, I scraped all serving and receiving stats from the ATP website for each player in the draw. I also decided to scale the data.

Scaling
Using only hard court results for the 2014 season, I scaled the data based on the level of competition. This allowed me to include all Challenger data as well as ATP-level, which is available on the ATP site. If an opponent was inside the top-64, no scaling was done. If the opponent was ranked between 65 and 128, then I scaled it down by 1.5%. If the opponent was in the top-192, I scaled it another 1.5%. I scaled it another 1.5% for between 193-256 and another 1.5% for those over a 256 ranking.

For some matches, the opponent’s ranking is listed at N/A. In those cases, the scaling was done based on the player’s own ranking, which seemed to be close enough to the actual ranking, except in a few instances.

Scaling this way may not be the best solution, but this is a solid starting point.
I then found ‘the field’ by averaging the scaled percentages of all players in the tournament. Five players have not played on the hard courts yet this season, so I removed them when calculating the field. Also, rather than placing zeroes in the data for them, I substituted numbers slightly below the averages for both serving and receiving.

In future versions, I may substitute scaled, full season statistics, irrelevant of surface for these players.

Noah Rubin
Then there was the case of Noah Rubin, who only had one hard court match, where he had some pretty good numbers, despite losing last week in Winston-Salem. In this case, I decided to manually modify his percentages down closer to the five I had to manually enter who had not played a single hard court match.

Shortcomings
Most of the problems come from too little data on some players. Some of this can be handled by using more stringent scaling for Challenger-level matches. Two of the most noticeable are Gilles Muller and Jared Donaldson. Muller won the Guadalajara Challenger and none of his opponents’ rankings are listed in the data, so they were only scaled per his No.68-ranking. Donaldson also had a lot of Challenger results that were not scaled sufficiently.

Coding
Jeff Sackman at tennisabstract.com published some python code to run the Markov Models a few years ago (here’s a link to his 2014 predictions, which you may like more than mine). He uses similar inputs and generates a probability player A wins the match. I modified Jeff’s code for my purposes, then wrapped it within a Monte Carlo Simulation and ran it 50,000 times.

I am not posting my entire code just yet on github, but hope to soon. I need to refine my entire process, soup-to-nuts, before I feel comfortable with that.

THE RESULTS
The table below shows howe far a player advances. For instance, Roger Federer lost 1802 out of 50,000 trials in the first round, but won the tournament 16895 times.

Federer seems to be the biggest winner here with Rafael Nadal out. I know this isn’t perfect, but it is a good start and something to work with moving forward. There are some basic assumptions I make and some data that needs refining, but overall I am satisfied with the outcome.