The scarcity of head-to-head non-conference games between top FBS teams magnifies the real culprit in creating a fair way to determine the national champion: bias built into the existing ranking system. Now, the rank of a team is primarily determined by votes of persons who are FBS football experts (65 sportswriters/broadcasters in the AP poll, 57 FBS coaches in the USA Today poll, and 115 school representatives and members of the media for the Harris Interactive poll). Theoretically, the voting system is designed to spread out personal bias so that its effect is negligible. For instance, the voters are chosen from all parts of the country, different age groups, and a wide variety of college backgrounds, which should dilute out all that emotional stuff, like regional bias, historical favorites, and conference or alma mater loyalty.
However, the selection systems are not designed to root out “expert” and “consensus-building”biases. What do I mean by this? With their first vote, members independently rank order the top 25 teams by whatever system they wish to devise, and send their choices to a central group which compiles all the votes and publishes a top 25 list that everyone (including the experts) can see. Early in the season, the “expert” bias is in play because the voters have little head-to-head information available to them, so they choose teams based on their potential, which has proven to be deceiving. The “expert” bias can be clearly seen nearly every year, when early favorites lose a game but do not drop out of the top 25, and sometimes they don’t even drop below a team that beat them on the field. The “expert” bias can also be recognized at the end of the season by checking the final records of the top 25 teams selected in the first poll of the season. While some pre-season favorites complete the season with two or fewer losses, there are always several “surprises” that finish mediocre at best, with five or more losses.
After the first poll or two, the “consensus-building” bias is in play – once members know what the consensus results are, they start to make adjustments to their votes to fall more in line with the other voters. The “consensus-building” bias can be seen by comparing the AP and USA Today polls after a few weeks in the season, relative to their results in the pre-season poll. At the fourth week or so of the season there are many undefeated teams, yet the two polls always look remarkably alike with regards to who is ranked where. Another visible aspect of the “consensus-building”bias is that a team tends to keep its position in the poll when it wins, moving up only if a team a few notches above it loses, and usually keeping the same position relative to the other winners (for example, if the #5 and #7 teams lose, then #6, #8 and #9 will move to positions 5 through 7 respectively, regardless of their opponents or the margins of victory). It is also evident when looking at teams with the same number of losses – the one with the earlier loss in the season is more likely to be ranked higher, because it has more time to recover.
One might argue that the “expert” bias doesn’t really matter because the voters have time to correct their early mistakes and choose the best teams by the end of the season. That argument would be reasonable, except for the “consensus-building” bias which tends to keep teams at the top if they continue to win, aggravated by the fact that only 2 teams (or 4 starting in 2014) are selected for the national championship game(s). In a 2-team or 4-team playoff, inevitably there are going to be several teams with nearly identical records vying for that last playoff spot, and the “consensus-building” bias is going to have a lot of influence. At the end of the season, a team that was considered to be one of the best at the start of the season is more likely to be ranked higher in the polls, compared to another team with an identical win-loss record which was ranked lower at the start of the season. Ultimately, end of season rankings are biased by the perception of who is best at the beginning of the season, which is clearly an unreliable predictor of talent.
If an expert evaluation system is to be employed to rank football teams, it can only be unbiased if the evaluators are “blind”; that is, the voters may be provided any desired statistics or information about the FBS teams, except for their identity, the conference they belong to, and the previous week’s rank. That may be an unusual proposal, but it’s what’s necessary to make the system truly fair.
No comments:
Post a Comment