FPI, SP+ and the Pac-12: post-week-2

Well, it's been an interesting couple weeks in the Pac-12 conference – including when looking at the various computer rankings out in the world. For everyone who missed this column of couple weeks ago, I'm tracking the performance of teams in the Pac-12 per a couple main computer algorithms: Bill Connelly’s SP+ (with Week 2 rankings available here on ESPN, plus regular analysis on Connelly’s twitter account) and ESPN's FPI (available here). Each system had posted preseason rankings for every team in the country – and a couple weeks ago, I went through the rankings and season predictions. If you want to check that out, just head over here.

So first off, let's look at the scores and records for the past couple weeks:

After 2 weeks, 6 undefeated teams remain – and only 2 teams are left are winless. While Oregon lost to an out-of-conference ranked foe in Week 1 (in heartbreaking fashion), Cal and U$C both beat ranked in-conference opponents in Week 2. UCLA – which was ranked ridiculously high in the FPI preseason rankings (and pretty darn low in SP+) lost to a couple Group-of-5 teams, including a Cincinnati team that got destroyed by Ohio State this week. As I'll show later, that made them plummet down both computer polls.

Now, how does this compare to the preseason projections and the ones that come out the week before the game? Below is a refresher on the preseason projected win percentages for both FPI and SP+, as well as SP+’s projected margin.

... Each computer also puts out new projections each week (SP+ only shows the upcoming week's games, while FPI recalculates the whole season). What did those look like so far? (Note: the FPI includes upcoming games on Week 3, but SP+ does not, since those were not released at the time of this article's writing).

Now that the games are over, we can look at the success of each model from the beginning of the season, and week-by-week.

FPI comes first. The following tables show the accuracy compared to the predictions from the beginning of the year, and the predictions for each revised week. It's important to note that the "count" at the bottom of the table counts each team's prediction as one game – so a single conference game counts as 2 (so this week, Cal’s rectangle at Washington and Washington's rectangle vs Cal both count). I'll do my best to change that in upcoming weeks.

Then, SP+...

Of course, a couple things stand out… In week one (preseason predictions for all 4 tables), both formulas whiffed on the Arizona @ Hawaii game, while only FPI missed on UCLA @ Cincinnati (SP+ predicted a loss). In week 2, both got the UCLA v SDSU game wrong (preseason and week-of), and incorrectly predicted a Washington win against Cal (also both preseason and week-of). Both incorrectly predicted a Colorado loss vs Nebraska in the preseason, but accurately revised the predictions before kickoff; only FPI incorrectly predicted a Furd win @ USC in the preseason, but both formulas correctly had a Trojans win before kickoff.

The final tally? SP+ was more accurate both in the preseason (19-5 vs 16-8 for FPI) and on a week-by-week basis (20-4 vs 19-5). Looks like Bill Connelly has done pretty well formula-wise.

Changes in Rankings – How's the Progress?

Now, what have the changes been for each week for both formulas – and for offense and defense?

Let's take a look at overall rankings, first. Here is the current, post-week-2 FPI scores and ranks (national and conference).

And now, national ranks and how much things have changed since the preseason…

A few things: Colorado has jumped a full 14 spots, from 63 up to 49, since the beginning of the season – and the Bears are in 2nd place with a 10-spot change from 55 to 45. Meanwhile, ASU has dropped a full 17 places nationally from #36 to #53 – but not nearly as much as the tumbling Bruins, as UCLA fell 27 spots from #20 to #47. All the other changes – both positive and negative – are in the single digits, with OSU the most stable in the mid-80s. Although half of the teams gain spots and half dropped in the rankings, the overall conference fell to the equivalent of 1.83 spots per team – largely thanks to ASU’s and UCLA's dramatic falls. However, the North actually moved up at the equivalent of 1.17 spots per team, while the South saw a division-wide fall averaging 4.83 – largely thanks to ASU's and UCLA's dramatic falls.

FPI’s Conference rankings have seen quite a bit of change, as well, with the Bears jumping the most since the preseason (3 spots) and from last week to this week (for spots). UCLA, on the other hand, saw a cumulative 5-spot loss, while the team we beat crashed a full 4 spots in one week.

What's great about this little graph is that, in terms of conference rankings, the North jumped up to the tune of 0.5 spots per team, while the South fell down the same level. The Bears have the unfortunate dis-privilege of playing in the tougher division, but also the benefits that brings (in recruiting, conference recognition and any national media actually focusing on the Pac-12).

What's SP+ have to say? Here’s where things stand:

How about our weekly changes? First, the national:

On the national level, Colorado again saw the largest jump at 15 places, from #68 to #53, followed by WSU's 14-spot jump from #36 to #22 – OSU also jumped a respectable 9 spots from #105 to #96. The Bears moved up ever-so-slightly (5 spots from #60 to #55), but SP+ is still more cautious about the Bears than FPI is. Despite SP+’s preseason caution about UCLA compared to FPI, the Bruins fell even further – a full 18 spots from #63 to #81. And, beautifully, Stanfurd saw the largest tumble – a full 29 spots from #32 to #61. The conference's average rank from preseason to now is much more balanced compared to FPI, but both divisions fell – the average drop was the equivalent of 0.42 per team, while the North had a smaller drop at 0.17 per team and the South fell an average of 0.67 spots per team.

Within the conference, Colorado saw a conference-leading 4-spot jump from the preseason (which also made room for UCLA to fall from #10 to #11). The Bears skipped upward from #9 to #8, which was made possible by Stanfurd’s 3-spot tumble in Week 2, from #6 to #9. In fact, the Cardinal fell a conference-leading 4-spot drop from the preseason. Life is good!

Finally, here are the averages for both national and in-conference rankings. I won't do any major analysis, but go ahead and take a look at the data if you want…

How are the offenses doing, week-to-week? Let's take a look – and note that SP+ provided preseason offense & defense rankings, while FPI only started after week 1. Because of that, the tables for FPI and averages note the difference from Week 1 to Week 2, while the ones for SP+ show the difference from preseason to Week 2.

Long story short, things have been pretty wacky compared to Week 1 and the Preseason. Just from Week 1 to Week 2, FPI saw changes everywhere from a drop of 37 (Washington) to a jump of 46 (USC). Since the preseason SP+ predictions, the biggest loser has been ASU (66) while the biggest winner has been Colorado (69 – from #73 up to #4!). Cal has jumped up notably in both formulas: 27 spots in one week for FPI and 29 spots in 2 weeks, for SP+. Go Bears!

Take a look at the Pac-12 rankings and take away what you will…

Next up is defense. So, what did the national numbers say? (The "change" figure is the same as with Offense – so FPI and averages are from week 1 to week 2, while SP+ is from the preseason).

And now, the conference numbers for defense…

The changes in defenses have gone all over the place. SP+’s preseason predictions were inaccurate by double-digits for all but 2 teams in the conference, and varied by as much as 65 spots in the rankings (for Arizona, which dropped from #55 in the preseason to #120 after 2 weeks). UCLA actually saw a jump from the preseason of 29 spots, despite its losing/embarrassing record, and ASU jumped 53 spots – all the way from #60 up to #7. Sadly, the Bears dropped 20 spots, from a preseason #5 down to #25, but I wouldn't be surprised for them to move back up as the season goes on. The differences from Week one to Week 2 are more modest: 9 teams were in the single digits and 2 teams held steady (OSU at #128 and UCLA at #43). The biggest single-week winner was ASU with a 20-spot jump, from #27 up to #7 – while the biggest loser was the evil Cardinal, falling 28 spots from #2 all the way to #30 (and from #1 to #6 in-conference).

FPI was much more volatile in its one-week change, as only 4 teams were in the single-digits (compared to SP+’s 9). Cal was the biggest winner, jumping 27 spots from #40 to #13 nationally – and from #5 in-conference to #1. Washington, our nemesis of the week, dropped a full 34 spots from #37 to #71 – while the conference's biggest one-week loser was Stanfurd, falling 49 spots from #29 down to #78. Most teams went downward, with 7 falling, one staying even (USC at #53), and only 4 gaining ground.

The changes are sure to be less wild as the season goes on and more data comes together, but for now it looks like lots of teams in the conference are still finding their identity… or the computers still need to figure out the actual quality of opponents (and their phases of the game) along the way.

Balance – Offense to Defense

In the preseason, I put forward the idea of the balance between offense and defense for the various teams in the conference. Basically, calculating the difference in national rankings for offense and defense – and the ratio between them. A perfectly balanced team would have a difference of 0 and a ratio of 1.0. Meanwhile, a team with #5 offense and #25 defense would have a difference of 20 and a ratio of 5.0 – and a team with the #5 defense and #25 offense (so the mirror image of the first team) would also have a difference of 20 and a ratio of 5.0.

So for the final set of tables, let's see how balanced the teams are for both FPI and SP+. The numbers are just for this week, but we'll see how tables go in the future…

Overall, FPI does put forward some bigger gaps than SP+ has – for a conference average of 50.5 spots (FPI) compared to 64.33 spots (SP+). Both agree that Arizona has the biggest gap , with offense favored by 113 spots – although SP+ has them tied with Colorado (also offense by 113), while FPI puts Colorado 4th at #83 spots in favor of offense. OSU also has a big gap toward offense (109 for SP+ and 66 for FPI), as does WSU (93 for FPI and 48 for SP+). The biggest defensive differences are ASU (93 spots for FPI and 102 spots for SP+), followed by our Sturdy Golden Bears with differences of 56 for FPI and 71 for SP+. More boring teams include Utah (zero spots for FPI and a modest 13 toward defense for SP+), Oregon (7 spots on defense for FPI and 22 spots on defense for SP+), and Washington (27 spots toward offense for FPI and 13 spots toward defense for SP+).

The ratios tell a slightly different story, partly because of the computers' disagreement on who has top-10 offenses and defenses. WSU's offense is a full 47.5 times better than its defense, per FPI, given that it's offense is ranked #2; SP+ puts the Cougars' offense at #12 nationally, shrinking the ratio to a mellower 5.0. The computers agree that Utah is a super-balanced team, with a dead-even 1.0 ratio per FPI and a best-in-conference 1.5 per SP+. Cal sits somewhere in the middle of the conference per SP+ (ratio of 3.84 in favor of defense) and on the middle-high side per FPI (5.31 in favor of defense). Who knows how Cal’s performance will go moving forward – but it's certainly better than massive gap we had at the beginning of the season, where SP+ gave Cal a defense rank of #5 nationally and offense of #125. Cal’s gap was a nation-leading 120 spots and the ratio was a 3rd-in-the-country 25.0. Both were also most dramatic in the conference. Hopefully we are finding some balance, although with the defense moving back up to a top-10 unit. Time will tell…

So, that's all for this week. The column will either be back after Week 3's games, or after Week 4. Either way, keep a lookout…