Standardized test-score shaming is a staple of every NFL draft season.
Prospects take the timed, 50-question career aptitude test known as the Wonderlic each February during the NFL Scouting Combine. NFL commissioner Roger Goodell told teams in 2012 that results are supposed to be confidential, but like clockwork, they leak every year prompting personnel, media and fans to apply Goldilocks-level thinking to each reported score.
Alabama quarterback Tua Tagovailoa reportedly scored the lowest among draft-eligible quarterbacks, but Sports Illustrated's Albert Breer reported Saturday that Tagovailoa's leaked score was incorrect. The initial report of Tagovailoa's results prompted ESPN’s Dan Orlovsky to note Friday in a since-deleted tweet that the rumored score score “bothers me more than [Tagovailoa’s] injuries.”
Richard Sherman didn’t directly respond to Orlovsky, but the outspoken 49ers cornerback slammed the Wonderlic’s usefulness in a tweet Friday night.
Lol if ppl only knew how guys really treat the wonderlic. Lol no one cares! https://t.co/0WU52F2yQm— Richard Sherman (@RSherman_25) April 17, 2020
Orlovsky, to his credit, deleted the tweet questioning Tagovailoa’s reported score and said he was wrong to tweet it in the first place. But the analyst, who played seven seasons as an NFL quarterback, admitted he was still “sorting out” how Tagovailoa’s football IQ and decision-making were so good while his Wonderlic score was so low.
There shouldn’t be anything to sort out for Orlovsky, or anyone else. Sherman's right: Wonderlic scores have little to do with on-field success.
Study after study has shown the Wonderlic has next to no predictive ability for NFL careers. Joseph Stromberg noted in a 2014 Vox piece that multiple studies indicate Wonderlic scores have no correlation with NFL success. A pair of studies even found a slight negative correlation, meaning that lower scores were a better predictor of players’ success than high scores.
Michael Callans, a former Wonderlic executive, argued in 2012 that the studies are inaccurate because they rely upon leaked scores that Callans contended are often incorrect. Given the initial report of Tagovailoa and other quarterbacks' scores this year were disputed within a day, that's an important point to consider when examining the Wonderlic's usefulness.
So is the fact that the Wonderlic has also faced criticism for having a built-in racial bias, as have many standardized tests. As writer Stephanie McCarroll observed in a Twitter thread Friday, a 2012 study showed that teams drafted white prospects nearly 15 spots sooner if they had a higher Wonderlic score as opposed to just six spots for black players.
McCarroll also pointed to the Supreme Court’s unanimous 1971 ruling that using the Wonderlic and other standardized tests that aren't “demonstrably a reasonable measure of job performance” as a “controlling force” in hiring violated the 1964 Civil Rights Act, which barred discrimination on the basis of race, color, religion, sex and national origin. The Wonderlic isn’t the only piece of the NFL draft evaluation puzzle, but that context begs the question of why it’s a piece at all.
The NFL clearly recognized the Wonderlic’s inherent flaws in 2013, when it instituted the Player Assessment Tool (PAT) to be used alongside it. The NFL said in a 2013 memo that the PAT “measures a wide range of competencies.” Cyrus Mehri, an attorney and the co-founder of the Fritz Pollard Alliance, told USA Today Sports’ Jarrett Bell that year that the PAT “kind of levels the playing field from a socio-economic point of view.”
Yet, PAT scores don’t seem to leak with the same frequency as Wonderlic results.
Sherman, and other players who joined in, are right to rip the Wonderlic test’s utility. The test itself is a bygone tradition, which has bred another bygone tradition of pundits dedicating column inches and air time to interpreting its results without much consideration of its inherent flaws.
Maybe the 2021 draft will be the year analysts stop falling into the same trap.