Statistics, Start Times, and Snowstorms: The Women’s 10 k in Falun

Note: we owe a thank you to John Estle, who sent us an Excel spreadsheet looking at results from the 10 k freestyle race grouped by starting times, in bins of ten starters. We thought he was on to something, but that FIS points and the structure of the start list might play in as well. Thanks, John, for sending us down this rabbit hole.

The results of Tuesday’s 10 k freestyle race at FIS Cross Country World Championships in Falun, Sweden, were surprising.

It wasn’t strange that Charlotte Kalla won by 41 seconds. A winner the previous weekend in Östersund, the Swede was among the favorites to win, and certainly the favorite among her countrymen watching from the grandstand and lining the trails.

But just who joined her on the podium was surprising. Jessie Diggins and Caitlin Gregg achieved their first World Championships individual podiums in finishing second and third. For Diggins, who has a team sprint gold medal from 2013 World Championships, it was not shocking; for Gregg, who hasn’t scored a World Cup point since finishing 30^th in this event at the 2010 Olympics, it was.

Besides the Americans’ strong showing – every U.S. woman placed in the top 15 – the other surprise was Norway’s weakness. The top Norwegian finisher was Heidi Weng in 22^nd place. It was in sharp contast to World Cup competitions all season, where Norwegians have usually won and not infrequently swept the podium.

A snowstorm which rolled in as the individual-start race continued certainly complicated the results. Late starters were at a disadvantage, so it is easy to say that early starters like Gregg, who wore bib 3, got lucky. But was start time the main explanation for results? We set out to use statistics to answer this question.

First, we asked what provides a better correlation for race time: FIS Distance points, or start time (in minutes after the first starter)? When we do correlation tests, both factors provide significant correlations to race time, but FIS points provide a far better explanation, with 76% of the variance in race time explained as opposed to 41% by the start time.

So, the snowstorm did not disrupt the race so much that the historically “better” skiers could not beat the historically “worse” skiers. At almost every point along the FIS points axis, there were skiers who had a wide variety of race times, but in general, the racers with lower points had better results.

And in fact, the racers with the lowest points had a smaller variance in their finishing times. Therese Johaug and Marit Bjørgen of Norway, with the lowest FIS points (zero!), finished just over two minutes behind Kalla. But racers with 80 to 90 FIS points had range of four or five minutes between their various finishing times, and that’s in relation to each other, not even in relation to Kalla.

So why, if FIS points could predict pretty well where racers would finish, is our first reaction to the results to be so surprised? It’s individual results, like Gregg finishing third or Bjørgen finishing 31^st, that got us. Let’s look a bit more at those results in context, by putting some names onto the points on the graph.

To understand the results better, it’s important to look at the start list. It can be divided into three blocks: an initial unseeded block of five; then, a long block where unseeded skiers alternate with seeded skiers, each of these groups being seeded in reverse order with the highest-ranked skiers wearing the highest bib numbers; and finally, a block of six skiers with very unfavorable FIS points who were selected in a special qualification race on day one of World Championships.

To look at how seeded versus unseeded skiers raced, we split the dataset into two and made simple linear models including both start time and FIS points. We’ll set the model up with an interaction factor: is there some emergent result of combining FIS points and start time? Do skiers with high FIS points struggle more as start time advances, or vice versa? This sort of interaction might explain some of why we find the results so interesting.

(A statistical note: this isn’t actually statistically a good model, because FIS points and start time are highly correlated: the skiers’ start times are seeded according to ranking. However, the variation in the two factors is very different. While start time increases by half a minute with every bib, the distribution of FIS points is much more irregular, with clumps and gaps – thus if it does explain finishing time, it might do so in a different way. This is interesting, so we’ll break a few rules and put both factors in a model together, which will also allow us to see whether they interact.)

For the unseeded skiers, including Gregg, we find that both FIS points and start time are significant factors explaining finishing time.

First, look at the relationship to start time.

It’s pretty good, although there are definitely plenty of skiers with later start times who still had good results: Alena Prochazkova of Slovakia, Jennie Öberg, and Perianne Jones and, in particular, Emily Nishikawa of Canada, pop out.

Then, look at the relationship to FIS points.

The left end of the regression line – the one driven by Caitlin Gregg, Kikkan Randall, Anouk Faivre Picon (who earned a best-ever finish by a French woman at a World Championships) and Yuki Kobayashi (who contributed to an amazing day by Japan with two women in the top 11) – is drawn by the early starters.

And it reveals something important: yes, the first group of five unseeded skiers turned in strong results for being unseeded. However, they also had the lowest FIS points of any of the other unseeded skiers. Among all the unseeded athletes, they would still be expected to do the best on a day where start time didn’t matter, and if anyone were to shock the field by getting on the podium, they would be the best candidates.

There’s no significant interaction in our model between FIS points and start time. So a skier’s points profile predicts their performance in one way, their start time predicts their performance in another way, and there is no effect of one factor on the other.

That’s also true in the seeded group. Interestingly, however, not only is there no interaction, but start time is a much better predictor of a skier’s finish, the opposite of what we found from the unseeded group. The correlation between FIS points and race time is not significant, and its slope is close to zero: basically, a horizontal line through a cloud of data. Start time is not nearly as good a predictor as either factor was for the unseeded skiers, but it does explain something.

The most interesting thing to look at in this graph is how performances fall by teams. If you consider the black regression line to be the mean effect of start time on performance – the amount that the snow slowed skiers down as the race went on – then some teams fall mostly below the line (better than expected race times) and some fall mostly above the line (worse than expected race times).

Sweden and the U.S. have the best, most unexpected outcomes, with Kalla and Rydqvist for Sweden and Diggins and Stephen for the U.S. racing incredibly well regardless of their start times.

And while Norway made much of their waxing disaster, it’s actually Germany who perhaps did the worst: Nystad, Boehler, and Fessel all had very poor results, the slowest times of the entire seeded group.

The Russians must have had about average skis, because Zhukova, Tanygina, and Tchekalova all fall almost exactly on the line where their performance would be predicted based on their start time.

On the contrary, the Finnish women were all over the place: they had similar start times, but Mononen had a poor race while Niskanen excelled, and Parmakoski and Roponen fell somewhere in the middle.

The takeaway? Two big teams missed the wax, but aside from that, skiers did pretty much as would have been expected by their previous results and the deteriorating conditions. While the U.S. is expected to thank luck for putting Gregg and Randall in bibs three and four, they should thank their wax team even more, because American skiers excelled throughout the field and did even better than should have been expected based on their start times.

They can still covet Sweden’s wax truck if they want, though.