How closely does recruiting success forecast winning in Conference USA?

“Stars don’t matter”

It’s a refrain we hear time and time again from college football fans, often those that support a program that isn’t pulling in top talent.

Despite overwhelming evidence that recruiting does in fact precede success on the gridiron, many fans will use anecdotal counter-examples to argue against this stone-cold fact until they’re blue in the face. And hey, I get it! It’s hard to come to grips with the realization that customized cookie cakes and photoshopped graphics from 3-4 years ago have a bigger impact on the outcome of any given college football game than a coach’s decision to go for it on fourth and one.

To be fair to recruiting truthers, every data analysis on the influence of recruiting on college football success I’ve ever come across online has primarily focused on national championship contenders and Power 5 conferences. How can we look down on those people when no one is giving them the empirical data they need to make an informed decision?

I’ll admit I was a recruiting truther once. I fully recognized that at the top levels of the sport, yes there was a clear difference between a three-star and a four-star and a five-star, and that difference would absolutely chart out in the Power Five conferences’ standings. I was, however, a bit more skeptical that recruiting rankings for the Group of Five conferences were as reliable.

Foremost, many athletes who sign Letters of Intent to G5 programs never get fully evaluated by the national scouts who assign grades for Rivals and 247. I’ve seen enough Aaron Jones and Marcus Davenport film to know as much. The tragically smaller and less fervent fanbases of G5 programs don’t drive as many clicks and subscriptions as foaming-at-the-mouth SEC crazies, so it doesn’t make business sense for those services to devote their resources to evaluating some unknown cornerback out of the backwoods of America who signed with a non-power program.

Secondly, recruiting classes at the G5 level are much more volatile than in power conferences. G5 schools are much more likely to take risks on partial and non-qualifying athletes who face an uphill battle to even make it on campus. Same deal for athletes who have known behavioral issues. My review of each C-USA program’s top recruit of all time shows a bit of how this plays out over time.

Lastly, what’s really the difference between a 2 star and a 3-star athlete? Isn’t a 3 star just a guy who happened to be noticed by a national scout and got a basic evaluation? How many recruits actually get an honest review and yet still end up as a 2 star? How are we to know that the unranked recruit at the bottom of the class isn’t a hidden gem ready to take the world by storm?

Are these concerns valid, or has the growth of the recruiting industry and the proliferation of social media improved the accuracy of recruiting rankings to the point where they actually do predict on-field success at the G5 level?

Let’s let the data guide our decision.

---

To get to the bottom of whether or not a team’s recruiting rankings influenced their football outcomes I gathered two key pieces of data.

The first is a figure which most readers of this site will be familiar with. SP+ is an opponent and tempo adjusted efficiency rating which has a stellar success rate in predicting wins and losses on the field. At a high level, SP+ gives us the ability to say Team A is better than Team B without the two meeting on the field. In a vacuum, if Team A takes Alabama to double overtime in Week 1, while Team B has to kick a game-winning field goal to beat Saginaw State at home, SP+ will likely rank Team A quite a bit higher than Team B.

Our second data point is a new one for me — the 247 Sports Team Talent rating. This statistic is meant to address one of the problems that made this type of analysis so difficult years ago. Due to player attrition, having a top-ranked class three years ago could have very little impact on your current day roster if all of your top recruits were kicked off the team or never made it to campus. This statistic from 247 allows us to see an aggregation of the recruiting rating of each player actually on a team’s roster for the team’s first game of each season.

The methodology of the statistical analysis is fairly straightforward. I grabbed each Conference USA program’s Team Talent rating for the past five years and matched that up with the team’s SP+ rating at the end of that year. In an attempt to restrict the data a bit to just compare across the conference and not the entire FBS, I also ranked each C-USA team by their per-year Team Talent rating and SP+ final rating.

Next, I performed a simple Pearson product-moment correlation calculation. This sounds fancy, but it’s just a formula that tells us how one set of data influences another. The coefficient of this output (sorry for the math) can range from -1 to 1, where at -1 there is an absolute negative relationship (the worse you recruit the better your team is), at 0 there is no influence (“Stars don’t matter!”), and at 1 there is an absolute positive relationship (the better you recruit the better your team is, no matter what).

After crunching the numbers, it became pretty apparent that having a top recruiting rating in C-USA doesn’t guarantee a spot on top of the C-USA standings, but it definitely puts you in the driver seat when comparing strictly against other C-USA teams, and maybe allows you to ride shotgun when expanding out to compare against all FBS teams.

Overcoming a poor talent rating to finish as one of the top teams in C-USA isn’t unheard of, but it’s a very steep uphill battle to say the least.

I’ve crudely plotted a few of the correlation coefficients on this chart to show how strongly they indicate an association between Team Talent and SP+ rating.

Point A - Overall correlation for last five years (.255)

Point B - Overall correlation for last five years by ranking in C-USA (.442)

We can see here that Point A shows a decent indication of Team Talent aligning with the SP+ rating, but when we restrict the comparison to just C-USA ranking in Team Talent vs C-USA ranking in SP+, the relationship strengthens to a fairly significant one.

With that baseline established, let’s take a closer look. The following chart maps out both the overall correlation as well as the C-USA ranking correlation by year. You’ll notice an outlier. Anything crazy happen in 2020?

The pandemic was clearly a jolt to the system and the data for Conference USA bears this out. While there was a moderate to very strong association between Team Talent and SP+ in 2016 through 2019, the 2020 season presented no statistically significant relationship between a team’s recruiting ratings and its final outcomes for the season. At least for one year of our lives, stars truly did not matter.

I can think of many potential reasons for this, so many in fact that I think it would be hard to definitively say why this happened other than “Covid”. Cancelled seasons, cancelled games, player opt-outs, contact tracing, and much more played a part in making the 2020 season so unpredictable.

When we look specifically at what happened in Conference USA last season to buck the trends, I spy three teams that really influenced this shift in outcome predictability.

First off, UAB was given a Team Talent rating of 383.95, slotting them at 11th place in the conference. UAB’s recruits have historically been underrated by the 247 Composite, but this Team Talent rating seems extremely off. Since UAB finished #40 in the nation in SP+, this differential between Team Talent and the final result of the season put 2020 on a weird footing right off the bat.

Additionally, both FIU and Louisiana Tech massively under-performed their Team Talent rating expectations. FIU had the second most talented roster in the conference but went 0-5. The Bulldogs had the third most talented roster and yet they also grossly underperformed, finishing in the bottom third of the conference in SP+. Looking back over the past five seasons, there’s typically one major under performance relative to Team Talent rating, however we observed two of these in a single season between FIU and Louisiana Tech. Both programs were both hit hard by Covid-19, while Ruston was also struck by a hurricane.

And then there’s ODU. The Monarchs were Conference USA’s only team to opt out of the 2020 football season. Old Dominion was filtered out of the formula for 2020, but there could have been intangible impacts to the rest of the conference, such as conference members having to adjust their schedule in a less than optimal fashion, or, to be frank, losing out on a likely win over the Monarchs who would have held the second least talented roster in the conference had they proceeded to play their 2020 schedule.

So now that we know 2020 bucked the trends for a myriad of reasons, how does the Team Talent to SP+ relationship look with that outlier removed? Quite a bit stronger. We see a 17% increase in correlation for both overall rating comparison, as well as the C-USA ranking comparison. This moves us from the “moderate” section of the graph to the “strong” section in correlation measurement for the overall talent to SP+ comparison, while the C-USA ranking comparison moves even deeper into the “strong” octet.

Point 3 - Overall correlation for last five years minus 2020 (.309)

Point 4 - Overall correlation for last five years minus 2020 by ranking in C-USA (.528)

Assuming 2021 is back to “normal” (ha.. ha... ha.....) then we should see the conference’s top recruiters jump further up the standings than they did in 2020.

---

To wrap things up, you may be a Middle Tennessee or UTEP fan feeling rejected after reading this analysis. While, yes, the Blue Raiders’ and Miners’ odds of winning the conference this year are rightfully low, the odds are never a guarantee, and breakthrough seasons do happen.

We mentioned UAB’s 2020 season above, but two other C-USA teams have defied the odds in recent year and enjoyed hugely successful seasons despite a low Team Talent rating at the start of the season.

In 2018 the North Texas Mean Green held a 368.58 Team Talent rating, 10th in the conference for that year. A two-star athlete from Locust Grove, Oklahoma, by the name of Mason Fine led the Mean Green to the top finish in Conference USA in SP+ that year at 34th in the nation.

Similarly, Old Dominion enjoyed a breakout year in 2016 as they wrapped up their season ranked third in C-USA in SP+ (73rd in the nation) despite a Team Talent rating of 347.2 (11th in C-USA).

With those two notable exceptions aside, all of the C-USA teams to finish with a top three SP+ rating during the years of 2019 to 2016 recruited better than at least half of the conference. Unfortunately, most of the outliers were in the negative direction, as teams like FIU and Marshall squandered their recruiting success and finished near the bottom of the conference.

At the end of the day, the message is clear. If you plan on winning Conference USA you need to either recruit with the best of them, or employ Bill Clark.