Out of the Box, Part I: Another Look at NCAA Selections
by Adam Wodon/Managing Editor
Two aspects of the NCAA tournament selections and seeding remain controversial. We touched upon it in our analysis articles leading up to the selections, and right after, but they are worth addressing now in greater detail after having some time to mull them over.
We refer, of course, to the selection of Wisconsin over Minnesota State, and to the "protection" of top seeds Michigan and Miami while sending New Hampshire to the West.
The first time I wrote about the Selection Process was 1997. I made a fool of myself on HOCKEY-L — the online college hockey discussion area of its day — trying to argue about the selections and seedings. When I was corrected by some people in the know, I went about righting the wrongs. I wrote an extensive article about the process, talking to then-chair Joe Marsh, and have spent the last 11 years writing all sorts of educational and opinionated pieces.
In that time, I became an evangelist for "the numbers," and have enjoyed shooting down many conspiracy theories — but also an advocate to keep using better numbers. In other words, I support the transparency of the objective system, but want the committee to improve the numbers it uses.
The point here is this ... the committee uses an objective process, but has become more beholden to these imperfect numbers than was originally intended to be. This is better than pulling them out of a hat, but it also boxes the committee in unnecessarily.
In the Wisconsin-MSU case, I believe it was a mistake — albeit a mistake the committee genuinely believed it had no choice but to make. In the case of New Hampshire, I believe it was the right move.
This heading should be more appropriately called Wisconsin-Minnesota State-Notre Dame. But to discuss this, we need to again back up.
Over the years, I've tried to repeatedly explain how the committee has shifted its approach to looking at the Pairwise. But as the years go on, many fans — even those who follow this closely — have forgotten this, and the current committee members probably have no idea it was ever done differently.
It's only in the last five, six years or so that the committee actually sees the "Pairwise Standings," so to speak — i.e. the simple listing of teams, in order, of most "Comparison Wins." This was, by and large, a creation of the online community. Prior to this, the committee only ever saw a large printout of all the different individual comparisons. If you are unfamiliar with these individual comparisons, you can see them by going to the "grid" from our Pairwise page, and click on the links.
From the discussions with committee members over the years, it was clear that the committee looked at this pile of individual comparisons, and quickly determined a line in the sand where certain teams clearly deserved to be in. But then it came down to a "bubble" that involved an arbitrary amount of teams, and the committee then compared the teams within that bubble to each other, using the head-to-head comparisons.
None of the other comparisons came into play at that point.
(I promise this pertains to Wisconsin — bear with me)
Back in the day, I argued that the committee should just use the "standings" and make life easier on themselves. But this comment must also be put into historical context.
That opinion came at a time when many people in the hockey community, particularly committee members themselves, criticized the online community for an incorrect understanding of the process. These people would argue that the "Pairwise" was not really what the committee used — even though the only difference was that the online community went the extra step of compiling all of the individual comparisons into a handy chart/standings. That was the only difference — but the NCAA and the committee would deny this, whether out of ignorance to what the online community was doing, ignorance to their own process, or intentional obfuscation.
Breaking the Cycle
Eventually, the committee learned to stop being scared of the online community, and realized it was all the same thing. I believe this was around the time that Jack McDonald and Ian McCaw were the committee chairs, in 2002-03.
As a result, and particularly as it coincided with the transition to a 16-team tournament, the committee did eventually start using the "standings" as complete gospel. This became abundantly clear in 2003, even though we didn't realize it at first. The 2003 seedings made no sense on first glance because it went against many past practices ... until we realized that it followed a 1-16 gospel ordering of the Pairwise, without any regard for individual comparisons.
It's about that time that the committee also started breaking ties in overall comparisons won by using the RPI — not the head-to-head comparison itself. Despite us pointing this out repeatedly, many very knowledgable people still make this mistake over and over. It happened in 2006, and CHN got it right while others missed it. Last year, it happened again in a tie between Maine, St. Lawrence and Massachusetts. This year, it was the tie between Denver and Boston College at No. 6. I still see the mistake all over.
So ... finally ... to the point. How does this pertain to this year?
Because Minnesota State wins the head-to-head comparison with Wisconsin. But in the "standings," Wisconsin won 11 comparisons and Minnesota State only won 10. That's because Minnesota State lost comparisons to two teams down the chart — Princeton and Northern Michigan — that the Badgers otherwise won. That's because those two teams did well against Nebraska-Omaha in a couple of games, while Minnesota State lost two games to UNO.
So that's it.
By What Numbers?
Committee chair Joel Maturi — a person I respect greatly — repeatedly insisted that his committee would go strictly "by the numbers," and lo and behold, they did.
By and large, I applaud this perspective, and I would much prefer the committee continue doing it this way than blow things up and go too far in the other direction.
But the question begs to be asked ... WHICH NUMBERS?
The committee went "by the numbers," HOWEVER, if it went by numbers it used to use, it could very easily have justified putting Minnesota State in the tournament.
In fact, were it six years ago, Minnesota State would've been in the tournament over Wisconsin. Case closed. Minnesota State had a better league record, better head-to-head record, better overall record, and won the head-to-head Pairwise Comparison.
Thing is, with great respect to Maturi and his colleagues — all of whom I firmly believe are men of tremendous integrity who care about the game — I don't believe any of them even realize this is an option, or that it used to be done this way. So when they say they went by the numbers, they are being completely genuine.
But nowhere in the championships manual does it say that the committee must select teams via the Pairwise Comparison Standings. It outlines the criteria, and how to determine the list of teams under consideration, and how to apply the criteria towards each of the individual comparisons between two teams. But that's it.
The committee has put itself into a box that it doesn't know how to get out of — even though the option is there for them to do so.
And here's the biggest point, perhaps: Minnesota State losing the comparisons to NMU and Princeton, and that costing it a berth, points out the Pairwise's most significant flaw ... the TUC Cliff. The definition of "Team Under Consideration" is completely arbitrary. If the TUC Cliff were at 20 teams instead of 25 — or if it was the old definition of any team over a .500 RPI — then Minnesota State would've made the NCAA tournament anyway.
So ... let me be clear ... I completely support an objective system of picking the NCAA tournament. But that doesn't mean the methodology is concrete. I suppose you could say I'm bringing in subjectivity here — in how to apply the methodology — and maybe I should be careful what I wish for. Maybe the real argument is against the Pairwise criteria itself, and this arbitrary TUC line.
But I've sort of given up trying to argue in favor of KRACH, because the committee has shown no interest in going to that system.
KRACH, by the way, has Minnesota State ahead of Wisconsin.
We could easily see the rule put in place — for real this time — that a sub-.500 team cannot get an at large bid.
Part II, about New Hampshire, later.