Resume Ranking
We define only out of despair, we must have a formula.. to give a façade to the void. Emil Cioran
In ranking teams, our intention should be to observe and record objective information as games are played and to use this information to update our existing perceptions of the ordinal standing of college football teams. In my opinion:
1. Teams should be ranked on all available information.
2. Objective statistical information should be emphasized.
3. Preseason and early season rankings should be based on team characteristics such as recent performance, returning starters, depth, recruiting, and coaching staff.
4. As the season progresses, ranking should be based less on perceptions of team characteristics and more on actual team performance.
5. Teams that are likely to be very good, but that have not been tested in tough games and/or in adverse circumstances should not be ranked above teams with similar characteristics but a superior resume of on-field performance.
6. The worthiness of a ranking methodology should be judged by one simple criterion: the number of games correctly predicted. The best ranking methodology is the one which produces a ranking which most often predicts winners. Higher ranked teams should beat lower ranked and unranked opponents.

As I understand it, SMQ defines resume ranking as a methodology in which rank is 1) "based strictly on the measurable" and 2) "takes into account only games played to date this season".
The first criterion seems laudable at first glance. However, a strict adherence to ranking based only on "measurable" attributes would lead to mathematical ranking schemas which omit subjective criteria. I find the clarity of computational ranking methods attractive, but do not believe any computational method is sufficiently robust to incorporate every relevant factor.
More importantly, many attributes that are objectively and easily measured do not lend themselves to objective comparisons between teams. We can easily determine whether a win occurred at home or away. But how do we objectively compare the degree of difficulty of winning an away game at Ann Arbor versus the difficulty of winning an away game at Palo Alto? We could measure decibels, distance from home, variations in climate, etc... I suppose, but this is ultimately a matter of subjective opinion. Is Nebraska's Memorial Stadium more or less hospitable than Clemson's Memorial Stadium? And does it depend upon who is visiting?
I would suggest that resume ranking emphasize objective and quantitative criteria, while allowing subjective criteria so long as the potential bias is considered and noted, and so long as the subjective criteria are subordinate to the objective criteria. If we are to strictly limit ourselves to objective measurable criteria, we should use a computational method which eliminates all bias and error.
The second criterion (consider only games played to date) is simply ludicrous if one wishes to form a rational ordinal ranking. Prior to the first week's games, the current resume ranking methodology would compel us to ignore a wide variety of relevant information that is easily measured and objectively comparable (past performance, returning starters, depth, recruiting), and instead rank every team in a 119 way tie for first, last, and only place.
The situation is little better after the first games are played. Suppose for example that a panel of experts has assigned a preseason rank for each team on the basis of the factors mentioned above (past performance, returning starters, depth, etc...). Let letters appearing earlier in the alphabet represent the higher ranked teams. Now suppose that A beats Z, and T beats S. All else equal, by virtue of defeating a tougher opponent, T has a more impressive resume than does A. Resume ranking thus requires that we rank T ahead of A.
This seems unreasonable. We've learned that A is better than Z. And we've learned that T is better than S. But no new information has been generated from the first games that allow T to be objectively compared to A. The resume ranking methodology is therefore requiring us to ignore the only available objective information and form an ordinal ranking based on the subjective value judgment that a team beating a "tougher" opponent should have the higher rank.
In most sports it is the case that either every league team plays every other league team at least once and/or a championship is determined via a playoff. In this situation we have a plethora of pair wise comparisons to facilitate an ordinal ranking. But in college football, most teams do not play each other and there is no playoff. This paucity of information makes ranking by any methodology troublesome, and it is a particularly difficult problem for those bold enough to publish an ordinal ranking early in the season. How do we compare A and Z when they do not play, and when A plays tough opponents while Z plays easy opponents? As more games are played this problem will become less pressing, though we will have to contend with the rise of contradictory information (A beats B, B beats C, and C beats A).
If you insist on resume ranking based only on statistics from this season, this computational method might be of interest. It produces a ranking based on something very much like the "resume" and does so without human bias or error.
0 recs |
0 comments








