March 24, 2008
Out of the Box, Part I: Another Look at NCAA Selections
by Adam Wodon/Managing Editor
Two aspects of the NCAA tournament selections and seeding remain controversial. We touched upon it in our analysis articles leading up to the selections, and right after, but they are worth addressing now in greater detail after having some time to mull them over.
We refer, of course, to the selection of Wisconsin over Minnesota State, and to the "protection" of top seeds Michigan and Miami while sending New Hampshire to the West.
The first time I wrote about the Selection Process was 1997. I made a fool of myself on HOCKEY-L — the online college hockey discussion area of its day — trying to argue about the selections and seedings. When I was corrected by some people in the know, I went about righting the wrongs. I wrote an extensive article about the process, talking to then-chair Joe Marsh, and have spent the last 11 years writing all sorts of educational and opinionated pieces.
In that time, I became an evangelist for "the numbers," and have enjoyed shooting down many conspiracy theories — but also an advocate to keep using better numbers. In other words, I support the transparency of the objective system, but want the committee to improve the numbers it uses.
The point here is this ... the committee uses an objective process, but has become more beholden to these imperfect numbers than was originally intended to be. This is better than pulling them out of a hat, but it also boxes the committee in unnecessarily.
In the Wisconsin-MSU case, I believe it was a mistake — albeit a mistake the committee genuinely believed it had no choice but to make. In the case of New Hampshire, I believe it was the right move.
Wisconsin-Minnesota State
This heading should be more appropriately called Wisconsin-Minnesota State-Notre Dame. But to discuss this, we need to again back up.
Over the years, I've tried to repeatedly explain how the committee has shifted its approach to looking at the Pairwise. But as the years go on, many fans — even those who follow this closely — have forgotten this, and the current committee members probably have no idea it was ever done differently.
It's only in the last five, six years or so that the committee actually sees the "Pairwise Standings," so to speak — i.e. the simple listing of teams, in order, of most "Comparison Wins." This was, by and large, a creation of the online community. Prior to this, the committee only ever saw a large printout of all the different individual comparisons. If you are unfamiliar with these individual comparisons, you can see them by going to the "grid" from our Pairwise page, and click on the links.
From the discussions with committee members over the years, it was clear that the committee looked at this pile of individual comparisons, and quickly determined a line in the sand where certain teams clearly deserved to be in. But then it came down to a "bubble" that involved an arbitrary amount of teams, and the committee then compared the teams within that bubble to each other, using the head-to-head comparisons.
None of the other comparisons came into play at that point.
(I promise this pertains to Wisconsin — bear with me)
Back in the day, I argued that the committee should just use the "standings" and make life easier on themselves. But this comment must also be put into historical context.
That opinion came at a time when many people in the hockey community, particularly committee members themselves, criticized the online community for an incorrect understanding of the process. These people would argue that the "Pairwise" was not really what the committee used — even though the only difference was that the online community went the extra step of compiling all of the individual comparisons into a handy chart/standings. That was the only difference — but the NCAA and the committee would deny this, whether out of ignorance to what the online community was doing, ignorance to their own process, or intentional obfuscation.
Breaking the Cycle
Eventually, the committee learned to stop being scared of the online community, and realized it was all the same thing. I believe this was around the time that Jack McDonald and Ian McCaw were the committee chairs, in 2002-03.
As a result, and particularly as it coincided with the transition to a 16-team tournament, the committee did eventually start using the "standings" as complete gospel. This became abundantly clear in 2003, even though we didn't realize it at first. The 2003 seedings made no sense on first glance because it went against many past practices ... until we realized that it followed a 1-16 gospel ordering of the Pairwise, without any regard for individual comparisons.
It's about that time that the committee also started breaking ties in overall comparisons won by using the RPI — not the head-to-head comparison itself. Despite us pointing this out repeatedly, many very knowledgable people still make this mistake over and over. It happened in 2006, and CHN got it right while others missed it. Last year, it happened again in a tie between Maine, St. Lawrence and Massachusetts. This year, it was the tie between Denver and Boston College at No. 6. I still see the mistake all over.
So ... finally ... to the point. How does this pertain to this year?
Because Minnesota State wins the head-to-head comparison with Wisconsin. But in the "standings," Wisconsin won 11 comparisons and Minnesota State only won 10. That's because Minnesota State lost comparisons to two teams down the chart — Princeton and Northern Michigan — that the Badgers otherwise won. That's because those two teams did well against Nebraska-Omaha in a couple of games, while Minnesota State lost two games to UNO.
So that's it.
By What Numbers?
Committee chair Joel Maturi — a person I respect greatly — repeatedly insisted that his committee would go strictly "by the numbers," and lo and behold, they did.
By and large, I applaud this perspective, and I would much prefer the committee continue doing it this way than blow things up and go too far in the other direction.
But the question begs to be asked ... WHICH NUMBERS?
The committee went "by the numbers," HOWEVER, if it went by numbers it used to use, it could very easily have justified putting Minnesota State in the tournament.
In fact, were it six years ago, Minnesota State would've been in the tournament over Wisconsin. Case closed. Minnesota State had a better league record, better head-to-head record, better overall record, and won the head-to-head Pairwise Comparison.
Thing is, with great respect to Maturi and his colleagues — all of whom I firmly believe are men of tremendous integrity who care about the game — I don't believe any of them even realize this is an option, or that it used to be done this way. So when they say they went by the numbers, they are being completely genuine.
But nowhere in the championships manual does it say that the committee must select teams via the Pairwise Comparison Standings. It outlines the criteria, and how to determine the list of teams under consideration, and how to apply the criteria towards each of the individual comparisons between two teams. But that's it.
The committee has put itself into a box that it doesn't know how to get out of — even though the option is there for them to do so.
And here's the biggest point, perhaps: Minnesota State losing the comparisons to NMU and Princeton, and that costing it a berth, points out the Pairwise's most significant flaw ... the TUC Cliff. The definition of "Team Under Consideration" is completely arbitrary. If the TUC Cliff were at 20 teams instead of 25 — or if it was the old definition of any team over a .500 RPI — then Minnesota State would've made the NCAA tournament anyway.
So ... let me be clear ... I completely support an objective system of picking the NCAA tournament. But that doesn't mean the methodology is concrete. I suppose you could say I'm bringing in subjectivity here — in how to apply the methodology — and maybe I should be careful what I wish for. Maybe the real argument is against the Pairwise criteria itself, and this arbitrary TUC line.
But I've sort of given up trying to argue in favor of KRACH, because the committee has shown no interest in going to that system.
KRACH, by the way, has Minnesota State ahead of Wisconsin.
We could easily see the rule put in place — for real this time — that a sub-.500 team cannot get an at large bid.
Part II, about New Hampshire, later.
Adam - This is a great 1st step in educating (or perhaps "reminding") the college hockey community of what so often happens when the lessons of the past are forgotten in the present. I've always felt that the earlier tie-breaking protocol (evaluating comparisons exclusively between the tied teams) of the Marsh-led IH Committee(s) had an elegant & common sense simplicity about them that has been lost in more recent iterations. AFAIK the ECACHL still breaks league ties this way, and as an ECAC coach Marsh apparently just carried that philosophy over to NCAA tie-breaking when he became Committee Chair. The decision to scrap individual team comparisons in favor of just RPI to break ties was an unfortunate one, IMHO, and was likely driven by the ''Not Invented Here" mantra that so often guides organizational thought: what's good for hoops is good for hockey too. Maturi should give Marsh a call; I'm sure Joe would be happy to bend his ear re: historical precedent just waiting to be used ...
Mar 24 2008, 8:11 pm by JL, upstate NY
A further point ... I wonder if the Committee had any notion of the outsize impact a SINGLE game had on this year's team selection / seeding. By winning the ECACHL Tourney Title, Princeton in effect dragged Wisconsin into the NCAA Tournament with them, thus setting in motion the subsequent cascade of geographic shifting that was necessitated by the Committee's ad hoc decision to protect the top seeds. If instead *Harvard* had won the ECACHL Title, the Crimson would've brought MSU-Mankato into the NCAAs with them, bumping the Badgers out, and thereby removing the stated obstacle to placing overall #1 Michigan in Madison. That frees up Miami to be slotted in Albany, UNH in Worcester, and NoDak in CO. Flights would be reduced, gate receipts increased, and we'd all be convinced we know just how the NC$$ operates. But once the Tigers triumphed, there was rejoicing in Mad Town, disbelief in Durham,and another half-dozen planes warming up to cross-cross the country. One game ...
Mar 24 2008, 8:33 pm by JL, upstate NY
Adam - Thanks for the in depth review, it helps me understand (because I've been scratching my head over this) just how WI managed to find a berth over Minnesota State.
Your analysis makes some sense out of an otherwise confusing issue. I was really hoping the committee had something other in mind than selling tickets in Madison.
Mar 24 2008, 9:56 pm by Siouxfan, Duluth
Going by the KRACH, UMD would have made it this year. I'm a UMD fan first, but even I don't think that should have happened.
Mar 25 2008, 12:15 am by Luke, Duluth
Ironically, had Wisconsin gotten that waved-off tying goal in Denver, both Wisconsin AND Minnesota State would be in.
Mar 25 2008, 10:31 am by Josh, Madison
This is an excellent explanation, and including the history makes it both interesting and informative. Thank you very much.
Mar 25 2008, 11:41 am by Chris Haake, Grand Forks ND
I can understand that Mankato needed one more win (preferably against Minnesota or UN-O) to clinch a spot in the tournament. Fair enough. But to lose out because a team beat a team that beat a team, that lost to a team that tied a team that beat a team, that beat my team just doesn't make a lot of sense when its diluted down that far. How about just a head to head comparison when it comes down to two teams for one spot?
Secondly, to award the last spot to a team with a losing record? And, give them a #3 seed on their home ice? Against Denver? Like many others, I thought Wisconsin got shafted in their game against Denver earlier this year. But this whole thing smells of overcompensation, not objectivity.
It seems to me that the Selection Committee should exist for applying common sense when the computers spit out something as illogical as this. They didn't do that as quoted by Mr. Maturi, above. Why then have a committee?
Mar 25 2008, 12:28 pm by Marco, Mankato, MN
It should not be forgotten that Notre Dame was in fact the last team in, not Wisconsin. The Badgers won the head-to-head with Notre Dame, so the Irish may be every bit as deserving of the heat that Wisconsin is getting. With that said, I am a Badger fan and know how irritating it must be to have your team knocked out by one that was under .500. When it comes down to it, it is a bunch of mediocre teams at best vying for those last spots. Everyone can look back and blame themselves for missing out, just like I would be doing if the Badgers were out.
Mar 25 2008, 12:55 pm by Jason, Woo-town
Jason good point. If your team is on the outside looking in just remember that playing better all year would have put them in the tournament.
Mar 25 2008, 1:03 pm by Joe, Deerfield NH
bubble play in game??????? like basketball or wcha
Mar 25 2008, 1:29 pm by Lake,ND
This should be required reading for newbies, speculators, conspiracy theorists, and the close-minded.
Speaking of which, the impression to the uninformed/less-informed is the tournament selection and seeding processes are fraudulent and stacked to work to favor host teams and revenue. (This was the opinion of the afternoon-drive-time local Minneapolis sports-talk host.)
Mar 25 2008, 2:25 pm by Gijs Voortvlek, WCHA Epicenter
Get me his name and contact info, we'll set him straight :)
Mar 25 2008, 2:28 pm by CHN Editor
First off I am a UW grad and like the team/program. That said, please other Badger fans, enough about the disallowed goal in Denver. These types of things happen all the time to all teams, why this is such a special case is beyond me.
Mar 25 2008, 2:32 pm by c lake, wi
Mankato got screwed, plain and simple. Common sense needs to dictate here instead of letting some formula, that no one knows entirely how it works, make the decisions.
Mar 25 2008, 3:27 pm by Ryan, Shakopee, MN
Actually, we know exactly how it works. That is how we publish it on our site.
Mar 25 2008, 3:31 pm by CHN Editor
I agree with those who express dissatisfaction, as it doesn't seem right. However, as CHN (and other sites) have consistently pointed out, it can NOT be a case of adjustment/favoritism/conspiracy/etc. simply BECAUSE they're following the set formula. Following the mathematical criteria is the way to PREVENT such mishandlings in the first palce. Perhaps the criteria should be tweaked -- I'd agree with that -- but to accuse the committee of some sort of wrongdoing, especially after the PWR system has been so well explained, is simply ignorant.
Mar 25 2008, 4:16 pm by Chris Haake, Grand Forks ND
There's no question Wisconsin got screwed on the disallowed goal in Denver, but I'm not sure if that goal counts it changes the PairWise ratings in favor of Wisconsin. For one thing, the game would have then gone to OT and maybe the Badgers would have lost in OT. Second, the injustice of the call clearly sparked WIsconsin the following night as they jumped all over DU. If the goal on Friday had counted and Wisconsin then went on to win or tie DU in OT, who knows how that would have affected the Saturday night game? Maybe DU would have beaten a less fired up Wisconsin team? Maybe DU would have been the fired up team after letting one slip away. My point is, it goes too far to suggest that getting that Friday night call right would have eliminated any controversy about Wisconsin getting in. There is no question about it: any system that allows a team like UW into the tournament over MSU is flawed and broken.
Mar 25 2008, 4:29 pm by Steve, Denver
Adam- You eluded to the “rule changes” that were mistakenly added to the NCAA Division I Men's Ice Hockey Championships Handbook just a few weeks ago, and addressed in your article titled “Not So Fast ... Or, How to Mess With Brackets Without Really Trying”. While the team has changed from UMD to Wisconsin in the time since the article was written I cannot help but find it ironic that the articles focus was about a rule, that if implemented could disqualify a team from the national tournament, yet by not implementing the rule, it can and did leave an above .500 team out. While changing the rules midseason is not something we would want, had that “rule change” been real, this entire debate would be moot.
Mar 25 2008, 8:49 pm by Carl, Mankato MN
Aren't you all glad the main controversy is about the last teams selected and not who's in the championship, like football? At least in a couple weeks, we'll have an undisputed hockey champion determined by a playoff. I'm glad the Badgers made it, but I agree that MSU-Mankato has performed better this year and is more deserving. Of all the major ranking systems (PWR, KRACH, RPI, USCHO, USA Today) only PWR has the Badgers ahead of MSU-Mankato Personally, like many on this site, I favor the KRACH, but then we'd have MSU-Mankato and UMD in and Notre Dame and Clarkson out - two under .500 teams and 8 from the WCHA would probably be even more controversial. Maybe the committee needs to use a BCS-style combination ranking using multiple ranking systems, since I don't think any one of them is perfect. But I do like the transparency in the selection so if the committee makes a change in the future, I hope they make their criteria public and keep the selection objective.
Mar 26 2008, 2:24 pm by BF, Madison
BF/Madison said" At least in a couple weeks, we'll have an undisputed hockey champion determined by a playoff." Great sentiment with which I totally agree, but it seemed surprisingly scarce not so long ago. Slightly more than a year from present, there were fans of some tournament-underperforming schools discounting or discrediting Michigan State's cinderella run, decrying the brackets and clamoring for re-seeding.
Mar 26 2008, 3:27 pm by Gijs Voortvlek, WCHA Epicenter
I to agree that a system is needed to choose the teams. However, careful what you wish for, though you state that Minnesota State is higher than Wisconsin in the KRACH, Wisconsin still would get in as an at large. Guess the Badgers are a pretty good sub-.500 team then. I'd rather see a good sub-.500 team than a mediocore .600 team any day of the week.
Mar 27 2008, 10:30 am by KH, Minneapolis
Hurray for Bucky and sorry to the purple Bull. The Committee did the right thing in putting Bucky in after the horrendous call in the Denver Game.
As for the Pairwise & KRACH, they both need major refinement in the methods they use to compare intra-conference games, and the implications on the strength of schedule. They do not do anything relative to where those games are played and those teams of the AHA, CHA and ECAC typically have to play CCHA or WCHA teams on the road or in so-called "neutral tournaments" - yeah, like in Detroit, Minneapolis and Madison. Not much neutrality there.
It will be nice to see Meat-chicken play in a state other than Michigan and Ohio. They better watch out for the other purple team in the tournament....who could be another spoiler....especially if Sauer plays like swiss cheese as in the past.
Mar 27 2008, 1:13 pm by FTG, Rochacha
Perhaps they should use a word like "relatively neutral;" no matter where one goes, I don't think there will be TRUE neutrality.
Mar 27 2008, 3:56 pm by Chris Haake, Grand Forks ND
What are you talking about Rochacha?
Michigan's non-league games this year were BC and Minnesota in a preseason tourney in Minnesota, a home series against BU, the Showcase against Minnesota and Wisconsin that rotates annually and happened to be at home this year, and the GLI, the one local tournament.
Mar 27 2008, 5:46 pm by Stephon, Ann Arbor
KH both a sub .500 team (Wisc) and a mediocre .600 team (Notre Dame) are in the tournament, so what exactly are you wishing for?
Mar 28 2008, 12:59 am by CW, Minnesota
Thanks for the break down, I can't say I agree with the final result, but I understand it. Before the tournament started I felt the last couple of teams out have nothing to complain about, because it was only a few years ago it was a 12 team tourney. Then Notre Dame (one of the last teams in) had pull off two upsets and make it to Denver. I still feel they should go back to the 12 team 2 region tournament. With the CHA losing their automatic bid, have the 5 conference champs and 7 at large bids. That is plenty. Also, with 6 teams at the regionals, attendance would be better. ( Less than 6,000 for the regional final in Worcester, that is lame.)
Apr 1 2008, 9:43 am by Joe, Augusta
Comment on this Article
