Don't Miss

Elo Rankings: Changes in Bid Allocation

By Joshua Mansfield
- Tweet
- Pin It
Updated: October 13, 2017

Credit: Jenna Bollweg

Early last week, USQ announced the bid allocation distributions for USQ Cup 11. While the announcement of this system was accompanied by much fanfare heralding the splitting of college and community divisions, the return to the Swiss format for collegiate nationals and the expansion of the field to 88 teams, many noted another detail—the elimination of the auto-bid in the allocation process.

While it has existed in several permutations, the auto-bid has been a feature of the USQ bid process since the bid program started, and has generally sought to give regions an extra bid for each team they placed in the round of 16 at the previous year’s nationals. The rationale behind this was that teams who were able to make the Sweet 16 were more or less guaranteed a bid the next year, barring unlikely situations, and so by giving their regions extra bids, more opportunity for qualification was given to teams that were perennially stuck behind these national-caliber opponents.

The decision to remove the auto-bid in determining community team bids is fairly easy to rationalize. Nine community teams made the Sweet 16 last year. USQ either had to reduce or eliminate auto-bids, or increase the minimal number of teams required for a regional—a decision that would’ve forced USQ to renege on their commitment to host a regional in Olympia, Wash. in Spring of 2018. In addition to that, every registered team in the Northeast would have earned a guaranteed bid, thereby eliminating any need for a Northeast community regional. Up against logistical nightmares like those, it is not surprising that the community auto-bid was given the hatchet.

What is harder to understand is why the auto-bid was also removed from the college team bid process. With 48 bids being distributed at a regional level, the seven auto-bids to college teams that made the round of 16 would have easily conformed to the regional bid allocation with little perturbation. Had this system of auto-bids been taken into account, the Great Lakes, Midwest and Southwest would have each gained an extra bid, while the West would have lost one bid, and the Mid-Atlantic would have lost two. Whether this would be a better format would likely vary widely depending on who you asked, but using our Elo rating system, we can at least take an objective look at the differences in these systems.

To use our Elo system to evaluate how optimal the bid allocation is, we need to make a couple assumptions. The first assumption is that each team’s Elo is currently an accurate representation of their team’s strength and that these ratings will not move significantly throughout the season in a way that changes team seeding. The second assumption we make is that only the best teams from a region will earn that region’s bids, and there will be no upsets in the bid process. While neither of these assumptions are very likely to hold true for every team, they still allow us to do our best to estimate the relative strength of teams within a region, and how many bids of which that region is deserving.

Credit: Jenna Bollweg

To compare these two systems, we can start by comparing the average Elo of a predicted regional-qualifying team for both. Under USQ’s system, the average Elo would be 1574, while under our hypothetical auto-bid system the average Elo would be 1577—a relatively small differential. However, because we are averaging 48 teams and only six teams lay on the margin, the significance for those borderline teams is hidden by looking at this simple average. Instead, if we look at the teams with the highest Elos who do not earn bids under either of these systems, we can see a clearer difference. Under USQ’s system, the top five teams who are not predicted to earn a bid at their regional have an average Elo rating of 1501. Using an auto-bid system, this number drops to 1468—a much greater jump than just the average of all teams competing. Fortunately for these teams on the margin, there are 16 at-large bids they should be able to earn, should everything go accordingly. However, since under both of these systems four of the top five teams missing out on regional bids are in the Southwest, there is a small but present chance that one or more of these teams may end up missing nationals due to a fluke of scheduling or record.

The current allocation of guaranteed bids by region, including a hypothetical allocation had auto-bids been used and a purely hypothetical allocation based on bids being assigned to the highest Elo-rated teams.

One could argue that these hypothetical changes under an auto-bid system are more or less arbitrary—even if USQ did an auto-bid system it may not look exactly like our hypothetical one, and most of these bids will go to deserving teams through at large bids should they miss a regional bid. Because of this, why not make a fully hypothetical system where regional bids are distributed in such a way to guarantee that the top 48 Elo-rated teams earn bids? Under this situation, things change significantly. Compared to the USQ provided system, the Great Lakes would gain one bid and the Southwest would gain five bids, while the West would lose one bid, the South would lose two bids and the Midwest would lose three. Under this system, the average Elo of a regional-qualifying team would jump 13 points to 1587, and the average rating of the top five teams who miss out on a bid would plummet 75 points to 1425. While this hypothetical system may produce the most competitive Nationals, it certainly wouldn’t create the most fair. While 77 percent of Southwest college teams would receive guaranteed bids, only 31 percent of Midwest college teams and 22 percent of South teams would have a guaranteed spot at Cup.

Based on current Elo ratings, we can further extrapolate a fully idealized bid distribution, including at large bids. This situation would distribute four more bids to the Great Lakes and Northeast regions, three more to the West, two more to the South and Southwest, and one more to the Midwest. Under this system, the Midwest would still only have five qualifying teams – two less than they are guaranteed in actuality. This means that all things held equal, at least two of the top 64 college teams in the country will not be earning a bid to Nationals. While season play and shifting Elo ratings may ultimately prove this wrong, it further underlines the importance of these at large bids for teams on the margin. While most teams in yellow are currently expected to qualify, any team shown below in orange could conceivably gain a bid through regional qualification or at-large bid or no bid at all.

A depiction of what an idealized US Quidditch Cup 11 bid allocation would look like if bids were determined only by Elo ratings. Teams in yellow note those earning bids at their regional championship while teams in orange note those earning bids through the at large bidding process.

Overall, this season will likely feature a grouping of teams similar to every season past; nearly every worthy team will earn a bid to nationals, while near the margins a few teams will upset their way in and a few teams will be barely nudged out. Thanks to the at-large bid procedure, the likelihood of that happening this year are even less likely than in years past, but odd schedule; inflated or deflated win percentages; or regional isolation could still lead to unpredictable outcomes in who earns an at-large bid. Using USQ’s ranking system, it is incredibly likely the top 64 college teams will earn bids to Cup this year. But only time (and Elo ratings) will tell if those teams are truly the best in the country.

All Elo ratings are accurate as of scores submitted by Oct. 10, 2017. These Elo ratings reflect the 1/3 seasonal readjustment toward the mean from their ending rating from their last active season.