NHL Scouting Report Analysis: Playmakers Vs. Snipers
This is the fourth post in a series where I dive into text analysis and how it relates to biases in the NHL draft. I added hundreds of Future Consideration's scouting reports to my dataset to dig deeper into NHL draft biases. Today we will use these scouting reports to compare the NHL outcomes of "Playmakers" and "Snipers."
What is a Draft Bias?
First, let's talk about draft biases and how to identify them. Again, I will use a methodology similar to the previous series mentioned in the introduction to find draft biases. You may want to skip this section for those following along with the series. For those new or interested in a refresher, this section is for you.
This has been done in a previous article which outlines the process behind this draft pick model (you can find that article here).
Note: This article will use Evolving Hockey’s Goals Above Replacement (GAR) metric. GAR's a catch-all metric measures how much a player contributes to their team’s performance. Higher GAR values represent more productive players.
The pick-value chart above gives us an expected value for each player based on their draft slot. Now, all we do is take the player’s cumulative GAR in their first seven seasons and subtract off expectations, and we have their GAR relative to expected.
For example, let’s look at Nail Yakupov. Yakupov was picked No. 1 overall in the 2012 NHL Draft by the Edmonton Oilers. As a result, his expected GAR from the chart is about 65. Unfortunately, he only produced about eight goals above replacement in the seven seasons after 2012.
As a result, Yakupov’s value above expected is -57 (8-65). In other words, He dramatically underperformed his draft position. Yakupov is a generational draft bust, so a large negative number for him checks out.
Now, all we have to do is apply this formula to every player drafted from 2007 to 2014 and start looking to see if any statistics can help us predict value above/below expectations. If certain players have been more likely to produce negative numbers, it suggests that NHL teams have overvalued those players because they generally fail to live up to their draft position.
Finding Playmakers and Snipers
The next step is to find playmakers and snipers. I could do this with point totals, but where is the fun in that? Additionally, scouts often mention there is more to prospect output than point totals, and they are right. So, by using scouting reports, hopefully, we can get some information beyond just NHLe.
To find these cohorts, we will continue looking for keywords within scouting reports. First, to find our playmakers, we will use the words and phrases; "playmaker," "set-up," "puck-moving," "great vision," "tape-to-tape," and "seeing-eye-pass." Next, to find our snipers, we will use the following words and phrases; "sniper," "goal scorer," "good shot," "great shot," "accurate shot," and "quick release."
Once defined, the results from this article will serve a second purpose. We know from previous articles that high-scoring prospects have been undervalued. So if playmakers and snipers show as undervalued, it will help reinforce the idea we measure what we think we are with this text analysis.
Now, with our playmakers and snipers defined, let's look at how much value each group produced relative to their expectations.
So are playmakers and snipers undervalued during the NHL draft? Early signs suggest these highly productive offensive players have outperformed their draft position. To start, the worst cohort of prospects is those who don't make the sniper or playmaker cohort. The next cohort has not performed particularly well, and it's our pure snipers. Surprisingly, the snipers have produced slightly less value than expected based on their draft position. On the contrary, the playmakers have produced far more value than expected. And finally, prospects grouped into both the playmakers and have performed even better than any of the other three cohorts.
Before moving on to success rates, one final comparison for those following along with the entire text analysis series. Here is how our playmakers and sniping cohorts compare to the other groups we have looked at in previous articles.
Success Rates
Before making sweeping conclusions, let's look at success rates too. Why are success rates important? A handful of incredibly successful or unsuccessful players could drive their entire bin up or down. To account for these outliers, we can also look at success rates.
What is a success rate? Success rates will simply be a 0 or 1 for each player. If a player produced more GAR than expected based on draft position (for example, fourth-round pick Johnny Gaudreau), they get a 1. If a player was worse than expected (think Yakupov), they get a 0. From there, the average number for all our cohorts is the average success rate of the group.
What are the success rates of our groups? Let's find out.
The prospects who were neither playmakers nor snipers only outperformed their draft position about 18% of the time. The next best cohort is still the snipers, who only outperformed their draft position about 28% of the time. This is a massive jump but not as large as the next one. After the sniping cohort is playmakers who outperformed their draft position 41% of the time. And finally, the most successful group of all is again the prospects who were both, who outperformed their draft position 42% of the time.
So the findings when using success rates instead of average value above expected are directionally similar. Both playmakers and snipers outperformed other prospects, but the two groups don't look equal. After accounting for draft position, the playmakers have been dramatically more valuable than snipers. Next, the prospects who were both playmakers and snipers were technically the most successful cohort. However, surprisingly, both groups barely outperformed the playmakers.
And finally, for those following the entire series, here are the success rates of all our cohorts.
Conclusion
We know from previous articles that high-scoring prospects have been undervalued. This analysis leverages scouting reports to investigate if passers and shooters have been equally undervalued. Like every article in this series, we must preach caution because of the smaller samples here.
With the small sample in mind, the playmaker's cohort has dramatically outperformed their draft position. These passers have been the most undervalued cohort by a massive margin using success rates and value above expected. While the snipers haven't performed poorly, they have had far worse results than the playmaking group. This suggests that while high production prospects have been undervalued, it is probably the strongest playmakers driving most of this relationship.