BU and College Hockey's Score Effects Problem
by Ryan Lambert/Columnist
One of the things that has come to be understood about hockey at any level is that you can measure the likelihood of future success by examining the ways in which a team controls the game.
While coaches often have their own methods for determining this upon reviewing video, we can largely understand "control" through the lens of the percentage of shot attempts taken at 5-on-5, because this is the event in the game that happens the most often. In most college hockey games, you're likely to see anywhere from 60 to 80 attempts, depending upon how much time is played at full strength, so you can really gain a good understanding of who has the puck, who doesn't, and how often. After all, you can't shoot if you don't have possession in the attacking zone.
But another thing that has come to be understood is that if you're trailing, you're more likely to have the puck more often than if you have a lead. The reason for this is simple and intuitive: If you have a lead, you go into something of a defensive shell in an effort to protect it, and if you're trailing, you try to shoot more often in hopes of creating the goal (or goals) needed to tie the game. This phenomenon is known as “score effects,” and score effects have a very real impact on possession numbers.
Further, teams with one-goal leads tend not to do as much to protect them as they do with two-goal leads, at least until the third period. This is because they keep looking for an insurance goal. Therefore, the score effects for a two-goal lead are greater than those for a one-goal lead. Anything beyond two or three goals, though, just tends to flatten out, meaning teams do as much to protect a three-goal lead as they do a six-goal lead (not that six-goal leads happen very often, but you see the point). Again, this much is indisputable and has been proven out years ago by people who could really dig into the math at the NHL level.
The problem with shot attempt data at the college level, though, is that it's not sorted by game state. That is, we can look at a box score and see that North Dakota took 50 shot attempts at 5-on-5, but what that doesn't tell you is how many of those are taken when the game is tied, when the Fighting Hawks lead, or when they trail. The ability to take 50 shot attempts when you lead by two for half the game is more impressive than the ability to take them when you trail by two for that long, and this information is critical to our understanding of a team's overall quality.
It's not always easy to get that information, though. Now, we could get it pretty easily if we want to comb through every box score and record the “score state” (tied, one-goal lead, one-goal deficit, etc.) under which each 5-on-5 shot attempt is taken. This is obviously quite tedious, and probably not worth the effort on a national basis (1,000-plus games every year), but might be informative team-by-team if you're so inclined.
In theory, once you have all that data, you could monitor how much each attempt is “worth” in terms of its relative value. These are just numbers I'm making up, but suppose a shot attempt with the score tied is worth “1.” If you can run the numbers for every college game in the country over a period of a year or two, you might find that an attempt with your team up two goals is worth 1.2 score-tied attempts, and that one with your team down two goals is worth 0.8. This kind of math would show us which teams are having their numbers inflated by the amount of time they spend trailing, which is something bad teams do, and which are having theirs depressed because they lead a lot of the time.
Until we can automate that work though — and I'm told that CHN is working on it — the next-best thing we have is to examine shot attempts when the score is “close” (defined as one team being within a goal in the first two periods, or tied in the third). This helps to smooth out a lot of the score effects, but people also rightly criticize the “close” methodology for necessarily shrinking the data set. It may produce numbers that are slightly more accurate in terms of the ability to predict future outcomes, but it's still not ideal.
Here's a real-world example of the current problem: Boston University is currently 9-6-3, but many observers would say that doesn't necessarily tell the story of the Terriers' quality. CHN's shot attempts data suggests BU is one of the very best possession teams in the country, whether you're going by all attempts or just those with the score close.
However, there's a pesky stat that has plagued the team all year but hasn't really done much to hurt their record. In their 18 games, they've entered the third period with a deficit of at least one goal 10 times. And yet, they've managed a 2-5-3 record in those games, rescuing seven points from the jaws of defeat. It should come as no surprise, then, that the Terriers have outscored opponents 31-16 in all situations in the third period.
Of course, they did a similar thing last year, but the big difference between last year's BU team and this one is, obviously, the lack of Jack Eichel. Last season, you could see in any number of games that Eichel decided he was going to win the game by himself, and he had the talent to go out and do it. BU has plenty of talent on the roster (10 NHL draft picks) but no one even approaches the touching-the-face-of-god, game-changing ability at this level Eichel did. That's not a slight, it's reality.
So you have to ask: How much of BU's really-good possession numbers come as a result of them trailing a lot? Well, it's tough to say for sure without putting in hours of work, but given the amount of time they spend trailing, the answer is, “Probably a lot.”
To this point, the Terriers have spent almost 40 percent of their total minutes trailing their opponents, as opposed to less than 30 percent leading, and exactly 31 percent tied. That's a massive difference that is going to pay dividends in the ability to work one's way back into games, and they largely have the talent to do it. However, BU has played nine games in which they've led for 10 minutes or less, and they're only 2-4-3.
It's not a recipe for success. You'd have to say the 2-5-3 record is luck to some extent; they're shooting 14.6 percent in all situations in the third period, which isn't going to last regardless of your skill level.
On the other hand, you look at a conference rival like UMass Lowell, which has not yet played the Terriers. Lowell has long been a dominant possession team, but this year isn't hovering much above 50 percent.
The reason why might have something to do with the fact that they've led for 42.9 percent of their games, and trailed for just 9.8 percent. Indeed, out of the more than 1,041 minutes they've played in 17 games this season, they've only trailed for 102:32. That's only about five periods of hockey in which they were behind out of more than 52.
Obviously other factors come into play here, like where your games are played (you're going to have better possession stats at home for a variety of reasons) and the quality of your opponent (games against Providence aren't going to go as well for you as games against Bentley).
To that end, BU has played 10 of 18 at home, but against what is currently the eighth-toughest schedule in the nation. Meanwhile, Lowell has played 8 of 17 at home but against much weaker competition in general. So those are leveling factors as well.
But at the end of the day, I'd imagine that if we were able to score-adjust these numbers, BU isn't quite as good as the current ones indicate, nor that Lowell is quite as mediocre. You have to win the games and all that, of course, but having as much mathematically sound data as possible would probably go a long way toward helping create a fuller understanding of team quality in college hockey.