How Useful Is A Single Good Result?

Following up from last time, we’re still discussing the recent USSA rule changes for qualifying for WJC/U23s. Now, it’s not like USSA has a ton of options here, but I would like to point out a few difficulties with pre-qualifying an athlete using only a single result.

Being a stats guy, I always think about things in terms of variability. So when I think about a skier’s performance, I visualize it as having a distribution. An athlete’s best races come from one extreme end of that distribution, but they are necessarily fairly rare. So to my way of thinking, a skiers best race isn’t a good estimate of how well they’re likely to ski. Rather, it’s a good (-ish) estimate of how well they might ski, if we’re really lucky.

To give you some context on this, I took all the FIS point results for Americans since 2006. Then I counted up how often each skier matched (or bettered) their best FIS point race from the early season (11-01 to 12-31) during the rest of the season. I counted this separately for men/women and sprint/distance and only kept folks who had at least 4 early season results, and at least one result from the remainder of the season. Here’s what we have:

I’m not quite so interested in the sprint numbers, since FIS points are such a dicey way to measure performance for sprinting. But if you look at the men’s distance panel, what this is saying is that they ski as well (or better) than their best early season result at most once more that season around 60% of the time. The drop-off is a bit less dramatic for the women, but still it is much more likely that you aren’t going to see a race with FIS points that low for the rest of the season.

And this trend is even more stark when we focus in on just the group of skiers who’ve managed a sub-50 point race during the early season:

Granted, this simple method of counting the number of results at a certain level may obscure some things. For instance, maybe folks don’t race quite that fast ever again, but they come pretty close fairly often. That will require a different sort of analysis that we’ll delve into next week.

Related posts:

1. <Participation Rates
2. <Thanksgiving Turkeys
3. <Week In Review: Friday Jan 7th