Do The Thrashers Have Large Talons?

Tuesday, July 15, 2008

How Much Does Coaching Matter?

My last post where I attempted to model the upcoming season resulted in two posters (Tony and anonymous) asking about the effect of new head coach John Anderson. This is a great question, but also one that is hard to answer so let me take a crack at it.

Coaching is one of the great mysteries of performance analysis. Let me start by talking about baseball where the statistics allow us to break down the game much more precisely. Most baseball analysts find little evidence that managers make a big difference. Their conclusions is that if coaching matters it tends to show up a) bullpen usage b) strategic choices (bunting, pinch hitting, etc) in close ball games.

For a real world example consider the case of famed Braves pitching coach Leo Mazzone who moved to Baltimore. A number of baseball analysts were curious to see if he the Baltimore staff would show evidence of "unexpected" improvement with Mazzone's arrival, but that turned out not to be the case. There was no dramatic improvement in Baltimore's pitching beyond what you might expect based on their talent level and margin of error.

What about hockey? You could make the case that in team sports like football and hockey coaching matters more than in baseball where it is a one-on-one confrontation between a single batter and a single pitcher. In hockey and football the guys may play well together as a unit to achieve success.

My own opinion is that coaching in hockey probably most strongly manifests itself in special teams. Why? Because pre-designed "plays" are most likely to show up in PP and SH situations on a regular basis. When you're on the PP you can count on maintaining possession of the puck in the offensive zone for an extended period of time. In contrast, at even strength it is much more difficult to keep puck possession and run specific strategies or plays--yes you can do it, but player raw skill and improvisation abilities also matter a great deal at ES.

If you look around the NHL every year you see some skilled teams that struggle with special teams and you also see weak teams with great special teams. That suggests to me that more than raw player skill level is at work here and these oddities are caused by either coaching or luck.

You also have to consider coaching within a competitive environment. Certainly John Anderson will be doing everything he can to get the most out of his roster--but the same thing can be said for every opposition coach. So when you're attempting to forecast all the Eastern conference teams I have a hard time giving the Thrashers any extra points over say Tampa who will also have a new coach as well as new players.

Is there any empirical evidence that coaching makes a big difference? Perhaps. Take a look at the list of winners of the Jack Adams Award (Coach of the Year) and you will see that that vast majority of them are coaches who took over bad teams and the team showed a big improvement. So there is some anecdotal evidence of a "new coach bump" out there in the data.

But if you look hard at some of these "Coach of the Year" cases what you see at times is the new coach getting credit for things he might not deserve to get credit for. Let me try and explain. In some cases the "new coach" bump is just random luck--for example if a team was riddled with injuries the season before and they return to being healthy the following season--that team should be better--but almost invariable the hockey media is going to credit that improvement to the new coach when in fact it is health (a luck factor) at work. Likewise players have a certain skill level but they also have variation around that fundamental skill level. If a team had several players with down years (i.e. Kozlov, Exelby, Havelid) and those players return to their previous level of performance the coach usually gets the credit--when it is could be that the player had a hidden injury or off-ice distraction that depressed their performance the previous season. Again the coach will usually be credited for this rebound. In many cases when people say "coaching" produced this, they simply capturing luck or rebound performances with the coaching variable.

Does every coach produce a "new coach bump"? No. Some new bench bosses see their teams struggle and get fired after just one year. If you're a new head coach taking over a club without any major injuries or down years by key players the year before, look out because you may be on the short road to a pink slip.

So if you're trying to forecast how do you handle this? Should I assume that both John Anderson and Barry Melrose will get that "new coach" kick, or will it just be Anderson? Because I don't have any feel for this right now I'm just assuming that it will be within my margin of error for the forecast.

So that's a long explanation but until I see empirical evidence to the contrary I tend to think that most of the "new coach bump" can be at the player level--you project an injured guy to be healthy or a slumping guy to rebound some. Special teams are the key area where Anderson might be able to squeeze out some extra points not predicted by the team talent level.

Let me conclude by saying this. I'll be more than happy to write a post at the end of the 2008-09 season titled "why my model/prediction was wrong" if that is the case. But last year my model had us around 10th in the conference and we finished 14th (mostly because Zhitnik was much worse than I expected). If we make the playoffs I'll be more than happy to say I was wrong, but right now I'm not sweating it.


  • Falconer,

    Thanks for the retort and thoughtful explanation of the coach factor.

    That's about the best you can do with a model I suppose. You can't factor in the coaches motivational ability or his ability to make Atlanta a place players want to play by building trusting relationships (among other things) - as he's mentioned in several reports I've read.

    I admit I'm trying to be optimistic (hoping) about the upcoming season. I hope at the very least we can witness exciting, never-give-up, hockey this year (as you alluded to in your previous post).

    I guess with all else being equal, if the Thrashers end up with 93 points and sneak into the playoffs, we can attribute that last 10 points to the Coach's and team's intangibles.


    By Blogger Tony, at 1:21 PM  

  • Falconer,
    First, I apologize for not signing up a a Goggle Blogger so you received an earlier Anonymous response from me.

    Second, your modeling approach is interesting, I believe you said 75% to 80% accurate. BUT, I believe coaching has a significant impact on player and TEAM performance; maybe minus 20% to plus 20%. I believe that accounts for Washington's strong finish last year and will account for a Thrasher improvement this year.

    If not, then why is Scotty Bowman so revered in hockey just as Lombardi was in football.

    KEEP UP THE GREAT ANALYSIS and hope I am right on the coaching which a 90 plus point & playoff finish by the Thrashers might confirm.


    By Blogger Denny, at 3:50 PM  

  • I think you are making a bit too light of a coach's potential impact on his team. Sure it is very difficult to quantify, but it has substantial value. Coaches like Toe Blake and Scotty Bowman had HUGE impacts on their teams that extended long beyond a first year bump. Particularly where I see consistent improvement over time and/or sustained competitiveness over time with a basic same core roster in place, I look to coaching as a partial (not whole) answer.

    I also believe coaching has implications beyond just special teams. It is in the development of players (see Hartley, Bob and Kovalchuk, Ilya). It is in the attitude and atmosphere that permeates the organization. It is a coach's ability to assess which players compliment others the best in order to gain optimal combinations (and also match-ups against other teams).

    But it is certainly hard to predict and each situation is usually going to be different. Our situation is certainly far from the situation in Ottawa or San Jose.

    Personally I think Anderson will have an enormous impact in at least this first season, more than anything because of the dramatic changes likely involved. Last season, there was basically no coach and no real system - this season we get a coach and a system. The PP should be different, the attitude in the lockerroom will be vastly different which should help some of our underachievers from last year bounce back.

    Anyway, just a different perspective from another number geek. :) It doesn't mean I necessarily disagree with your point predictions (although one can hope for better) but as I said, I feel you've underplayed the importance of a coach in the success/failure of his team.

    By Anonymous Sara, at 4:24 PM  

  • I hear what you're saying regarding Scotty Bowman. Let me throw out two points for your consideration:

    1) Bowman had a lot of Hall of Fame level players. Go back and look at those rosters. The only "not great" roster he coached was St. Louis Blues IIRC.

    2) Just as there are phenomenal players, I assume there are phenomenal coaches--but just like elite players there simply are not a ton of Scotty Bowman's in the NHL.

    If you think about coaching talent it probably looks a lot like player talent. A few REALLY good coaches, a few REALLY bad coaches and a bunch in the middle who all possess some mixture of strengths and weaknesses.

    I think the REASONABLE assumption is that Anderson is probably part of that middle group--until such time as he proves otherwise. You can't count on him being the next Scotty Bowman when you're forecasting. Hopefully he is the next great coach--if he shatters my forecast I will be open to considering that possibility.

    By Blogger The Falconer, at 4:33 PM  

  • Falconer - absolutely one cannot assume Anderson will be the next coming of Bowman (although wouldn't it be nice). However, although Bowman had some considerable talent to utilize, that talent had struggled to fully maximize its potential until he worked his magic (at least with the Wings). Likewise, some coaches have managed to pull off a lot with seemingly a little (or at least average ie Carolina - they have never really been stacked, but they have managed to achieve high levels of success mostly due to the blue-collar-like system they've incorporated).

    As I think we can both agree, there is no way to predict either objectively or substantively what impact Anderson will have on the team next season (and those to follow). But I'm looking forward to finding out! :)

    By Anonymous Sara, at 4:55 PM  

  • The Thrashers trailed after the first period 36 times last year, only Toronto had a worse record. Many times it looked like players had set their watches on Central Time, alarm bells didn't go off until the 2nd period. Does a coach have responsibility for the player's mental preparations? Or is the first period just the crucible where the chinks in a weak teams' armor are exposed?
    You don't mention line changes, length of shifts, matchups, all coaching responsibilities. I've seen NHL coaches eat opposing coaches for lunch in this area (I'm thinking Carlyle vs Murray in the 07 Cup Finals). You have to have the personnel to do this but even with a weak team, smart coaching might "steal" a few games.

    By Blogger Big Picture Guy, at 5:06 PM  

  • Big Picture:
    Interesting stuff about the 1st periods. Was it a coaching or a lack of talent? My instinct is to lean towards the latter, but I could be persuaded otherwise.
    (Maybe the Thrashers need more guys from Newfoundland on their roster with their body clocks set ahead)

    I would agree that coaching differences become more apparent/important in the playoffs. There you have a "repeated iteration" of playing the same opponent multiple times so customized strategies can pay off in wins.

    In the playoffs the talent gap between two teams is often smaller than it is during a random regular season matchup. All the playoff teams are decent or they wouldn't have qualified. So when you get into a situation with more evenly matched teams the little things like line changes, faceoffs, line matching become more important.

    I still tend to think that the "coach effect" is small in regular season.

    One fairly important way a coach can make an impact is by simply giving ice time to the correct guys. There are certain NHL players who are young and under-utilized.

    Jason Pominville is a guy who is amazingly productive and should get more ice time. Alexander Radulov is another guy who should see more ice time at ES and PP. The coaches are missing out on points by not giving them more ice time in offensive situations.

    By Blogger The Falconer, at 7:39 PM  

  • Excellent point about "repeated iteration". In theory that should also work when you play a team eight times (that's changing this year, isn't it?) but I guess the focus isn't there over a six month schedule. The list of "trailing after one period" is mostly populated by the worst teams in the league, so it probably is a talent deficit: the only reason I hesitated about that is that I coached several team sports in high schools (not hockey) and I definately can say the coach has a big role in getting players at that level mentally prepared.Professionals, I can't speak to that.
    Looks like Radulov is going to get his ice time in Yekateringburg Time (YEKT): Ufa is there. I have heard that Russian coaches sometimes barely use their 3rd and 4th lines (radulov, of course, will get a lot of ice time):it's one of the complaints of younger players defecting to the NHL.

    By Blogger Big Picture Guy, at 8:21 PM  

  • Falconer,
    I saw this on on the THN roundup segment:

    The Flyers are around $2 million above the $56.7 million cap ceiling and given their depth at right wing, it’s thought GM Paul Holmgren could move one of them.

    Recent reports suggest the best candidates are Mike Knuble ($2.8 million) and Joffrey Lupul ($2.3 million) as moving either would get the Flyers below the cap.

    You were right. What other teams are over the cap and what other players are some teams looking to unload as a result?

    I also read on that Detroit will probably workout a deal with Filippula by Aug. 1, but McCarty and Downey are testing the market.

    By Anonymous Steve, at 11:44 PM  

  • Steve:

    These teams are all in a tight spot

    By Blogger The Falconer, at 12:41 AM  

  • Hey Falconer - do you have an e-mail addy I can reach you at?

    By Anonymous FrankD, at 10:19 PM  

  • frankd:

    yes, if you look at my profile I have a gmail account for the blog. It is Falconer18 at sign

    By Blogger The Falconer, at 5:21 AM  

  • Apparently the Flyers aren't going to move anyone. They will put Hatcher ($3.5M) on the long term injury list to get under the cap. Forget about Knuble or Lupul, for now.

    By Blogger fromthepoint, at 12:59 PM  

  • If Anaheim and Chicago were to give up players, who do you think that they would give up?

    By Anonymous Steve, at 2:09 PM  

  • From the Point:

    My understanding is the Flyers will still have to be under the cap counting Hatcher's salary on day 1 of the season. IIRC to qualify for the LTI list you have to actually miss some regular season games first--it isn't like the DL in baseball where a team can pick and choose--only the NHL office grants LTI exceptions and they have some very specific conditions that must be met first.

    By Blogger The Falconer, at 4:17 PM  

  • Steve: the situation is so fluid it is really hard to say. Schneider and Khabibulin are the most frequently mentioned trade candidates. We will just have to wait and see.

    By Blogger The Falconer, at 4:18 PM  

  • Yeah, you are right. I have heard those two names the most. Khabibulin is mentioned in a trade to Ottawa. Not sure who Schneider would go with? Do you think he would come here? We certainly could use him.

    By Anonymous Steve, at 5:13 PM  

Post a Comment

Links to this post:

Create a Link

<< Home

Who links to my website?
View My Stats