Live by the Variance, Die by the Variance (and why I hate Duke [and Mercer] for that matter

The first weekend of the NCAA Tournament was a wild one. In our competition, we chose models with high variance, knowing full well we could be in a world of hurt if a game or two did not go our way. Being scored on a log loss scale was new to us, and we knew of the risks, but did not really think things could get too bad.

We knew Wichita State was rated too high, but we thought, “hey, maybe they are just that good,” and rolled with it. Anyhow, we had figured they would eventually lose, but one game would not kill us since we could make it up somewhere else.

That was of course until Mercer took it to the Dukies. Most models had Duke as a very strong 3-seed. I believe the fivethirtyeight blog even had Duke as a 93-94% favorite. Those kinds of numbers are for wimps. We had them as a prohibitive 98.4% favorite in one model and 98.9% in another. When you do something like this in a competition where the scoring is based on log loss, you better hope that they win, or else you are dead in the water.

The average score using this model (at least your goal) should be in the 0.4 to 0.5 range. The Duke loss was a 4.1 to 4.5 point penalty that just destroyed us. Unfortunately, no one else was dumb enough to match our probability.

By comparison, last year’s Cinderella, Florida Gulf Coast, had only a 4% chance to beat Georgetown. Using the same formula, you penalty is in the 3′s.

We could have easily solved all of this and handled the huge variance by increasing the variance when calculating our probabilities. But we were in win or go home mode… oh well.

Here is a quick updated simulation of the remaining teams. I ran a somewhat newer updated model 50,000 times.

UPDATED MONTE CARLO SIMULATION WITH NEW PROBABILITIES