Cal ranked #13: Stupid computer polls

Distrust toward computer polls stems in part because humans analyze college football very differently than computers do. Human brains do not do particularly well when they are asked to work as parallel processors, at least not on the conscious level. On the other hand, college football seasons are like the intersecting concentric circles created when you threw two stones in a pond. They meet in ways that no human can fully take in or observe at once. The fact that computers, or humans using spreadsheets, can perform this task is an excellent supplement because it compensates for our chief weakness.

This is not to say that computers have no flaws either. As you noted a lot can change based on the weight that you assign particular metrics. It would be an interesting – and monumental – effort to test enough different data points that you could determine the dp with the highest correlation to victory. Is it offensive yards? Is it turnover margin? Is it red zone conversions? Or is it a hidden stat, such as the one that Ivan Maisel suggested, aggregate number of starts by the starters on the offensive line? This is a research effort that I am very happy to delegate.

Nor is assigned weights the computer polls only problem. Computer polls fail to take into account program prestige, which I can tell you was one of the most important techniques I used to develop my year-over-year #1 recruiting classes in NCAA Football 2006.

Finally, there is another more sinister source of our distrust, one that we are all loathe to acknowledge but secretly know is true. We distrust the computers because we know that in 2029 they will rule the world, thanks to Skynet, a creation of Cyberdyne Systems. Remember gentlemen, first they came for the AP poll.

P.S. First!

Although the computer methodology is a nice component, it should never be the sole component in determining how you rank your top . Unlike baseball, where the Moneyball strategy is quite useful because the variables are simple to analyze (hitching to pitching is two dimensional at best), football requires measuring a number of variables, such as offensive blocking, receiver management, how the quarterback goes through his progressions, how quickly linebackers can get to the edge, etc. There’s still a wide gap of uncertainty in measuring a team solely by statistics. The human viewpoint is necessary in the college ranks, where a championship must be determined by polls.

Football Outsiders has made some nice inroads, but it’s still a long way from making the proper measurements of the value of a team.

I endorse this post.

Rec’d!

Well if you have any doubts about such a system, consider this: Google’s famous and incredibly successful PageRank algorithm uses the same principle.

Ever notice that Google’s first few entries for a search are really good and that they get progressively bad as you flip through the pages? Ever wonder why? Basically it’s just noise after the first few entries just like with this ranking.

That said, awesome post.

thank you

Since I prefer the “resume” polling philosophy over the who-would-beat-whom-on-a-neutral-field-gut-feel approach, I very much am a fan of the computer polls—especially at the end of the season when they have a lot more data points to work with.

Year in and year out, the Sagarin rankings seem the most on-target to me, although as he explains, his numbers still include this week a subjective component based on his pre-season start values. Thus, it’s pretty hard to justify the gap he has between Cal (#13) and Maryland (#41), and if both teams keep winning, I suspect the gap will close considerably as his formula becomes non-biased.

It’s interesting to note that Cal is actually ranked #8 in Sagarin’s preferred “Predictor” model, which takes into account margin of victory and predicts a 21-point win this Saturday for the Bears over the Devils. That remains to be seen, but there is a (homerish) side of me that thinks Cal isn’t that overrated by Sagarin right now. The struggles on offense, and in particular the passing game, are obvious to us. But we’re forgetting what a fantastic start the defense is off to. By my figuring, the defense has played only a single bad half of football—the first against Maryland—and that wasn’t so bad as to knock the Bears out of the game. Indeed, it only would have taken a correct Safety call here, and a key third-down stop (or two) there, and Cal is probably still looking at an undefeated season. Beyond that game, the week-one shutdown of Javon Ringer looks more and more impressive every week, and holding WSU and CSU to six and seven points respectively—despite having an offense that provided very little in terms of Time of Possession—is likewise worthy of note, no matter how bad those teams might be.

Is Cal a Top-Ten team in my subjective estimation right now? No. But Top 20 seems quite reasonable, particularly from a “resume” point of view, so I’m inclined to believe the computers more than the pollsters, who are downgrading Cal simply because of the failed promise of post-season 2004, pre-season 2006, and post-Oregon 2007. Thrice burned, Cal would have to win the Rose Bowl before some voters would be willing to put the Bears back into their Top 10. The good thing about the computer polls, they’re not programmed to have such carry-over memory from the past.

It’s also interesting to check out the Sagarin ratings by conference:
http://www.usatoday.com/sports/sagarin/fbc08.htm

Yes, he has the SEC in first, and No, he does NOT have the Mountain West (#7) ahead of the Pac-10 (#5). But he does have the Mountain West scoring in the same range as the 6 BCS conferences, with a huge drop-off to C-USA in 8th. There’s obviously room for argument here, given the Mountain West’s head-to-head record against the Pac-10. But keep in mind that his rankings emphasize a true average—the teams at the middle of each conference—rather than the marquee teams at the top.

Personally, I am glad that the BCS has the sense to use computers at all. They can obviously see statistical variation no human has time or energy to sift through. But then again I am also glad they know to put in the human element. If only they included the Blogpoll, though.

I have a question, though:

Teams start out with a base ranking at the start of the season (it’s obvious USC is better than App State, etc).

Who/what determines this base ranking? 2007? Sagarin? The AP poll? Monkeys? The Maharg? Because as I see it that ranking is pretty darn crucial, and the computer bases it off zero statistics gathered for the 2008 season…

(surprise, surprise, another EECS grad here…)

There’s a reason we link to Sagarin’s rankings just above our own Top 25 on the ‘Polls and Rankings’ widget; I generally really his rankings, and in the past have often referenced them in my posts. However, it’s important to understand the limitations of computer rankings, as several posters have mentioned: they can’t watch the games, only the box scores. I think it’s important to look at both expert human opinions and computer-generated rankings. To disregard either would be to ignore valuable information.

I think an interesting method of coming up with a Top 25 would be to start with a computer ranking, and then make adjustments from there. Instead of starting with preseason preconceived notions of strength, we’d start with an unbiased ranking and make “corrections” for data that the computer is unable to see. Perhaps I’ll try that one of these weeks.

I tend to think the human polls are bogus.

Based on that statement, I would assume you think the BCS was justified in leaving USC out of the BCS title game after the 2003 season over the objections of the pollsters. Correct?

I think you EECS majors spent way too much time with your heads stuffed inside a textbook. Put the star trek toys down, turn down the Rachmaninoff blaring the background (ok, business majors listen to classical too from time to time), stop gulping your Mountain Dew, and recognize that controversy makes money, and that if we left everything to computers and the computer scientists that build algorithms, then there’d be no controversy and less money pumped into the machine that is sports today. Think about it….what gets us angry and excited, and hopeful and debating….its the subjectivity of bad referee calls on the field, the coaches poll, the AP poll, all that subjectivity makes for good conversation. If we left everything to computers and EECS majors…….

Twist, you’re too sweet already, no sugar for you

I haven’t tested it and I don’t know if it’ll even terminate. It’s just a speculation.

Let’s have a directed graph with nodes for each D-IA team that start out at 0 rating. An edge will lead out of node x to node y iff the x defeats y. At the end (after graph is drawn out) we add up the the ratings of all teams defeated by a node i (outgoing edges) plus one and subtract from that ratings of all teams that defeated node i (incoming edges). Applying this algorithm each weeks allows for the ratings to converge to the correct or optimal rating for each team. And of course the top 25 teams are the ones with the top 25 ratings.

This could be modified with a bias term and possibly weightings depending on the final score, injuries, covering a spread, etc.

The inspiration for this idea came when I stumbled upon my CS188 (AI) project from last semester on neural networks/reinforcement learning.

Again emphasis: this could all be bullshit