Dawg Sports: An SB Nation Community

Navigation: Jump to content areas:



Sports blogs for fans, by fans.
Around SBN: Steve McNair Dead - Former NFL QB Shot and Killed


Resume Ranking

We define only out of despair, we must have a formula.. to give a façade to the void.  Emil Cioran

In ranking teams, our intention should be to observe and record objective information as games are played and to use this information to update our existing perceptions of the ordinal standing of college football teams.  In my opinion:

1.    Teams should be ranked on all available information.
2.    Objective statistical information should be emphasized.  
3.    Preseason and early season rankings should be based on team characteristics such as recent performance, returning starters, depth, recruiting, and coaching staff.  
4.    As the season progresses, ranking should be based less on perceptions of team characteristics and more on actual team performance.  
5.    Teams that are likely to be very good, but that have not been tested in tough games and/or in adverse circumstances should not be ranked above teams with similar characteristics but a superior resume of on-field performance.  
6.    The worthiness of a ranking methodology should be judged by one simple criterion: the number of games correctly predicted.  The best ranking methodology is the one which produces a ranking which most often predicts winners.  Higher ranked teams should beat lower ranked and unranked opponents.  

Star-divide

As I understand it, SMQ defines resume ranking as a methodology in which rank is 1) "based strictly on the measurable" and 2) "takes into account only games played to date this season".

The first criterion seems laudable at first glance. However, a strict adherence to ranking based only on "measurable" attributes would lead to mathematical ranking schemas which omit subjective criteria.  I find the clarity of computational ranking methods attractive, but do not believe any computational method is sufficiently robust to incorporate every relevant factor.

More importantly, many attributes that are objectively and easily measured do not lend themselves to objective comparisons between teams.  We can easily determine whether a win occurred at home or away.  But how do we objectively compare the degree of difficulty of winning an away game at Ann Arbor versus the difficulty of winning an away game at Palo Alto?  We could measure decibels, distance from home, variations in climate, etc... I suppose, but this is ultimately a matter of subjective opinion.  Is Nebraska's Memorial Stadium more or less hospitable than Clemson's Memorial Stadium?  And does it depend upon who is visiting?

I would suggest that resume ranking emphasize objective and quantitative criteria, while allowing subjective criteria so long as the potential bias is considered and noted, and so long as the subjective criteria are subordinate to the objective criteria. If we are to strictly limit ourselves to objective measurable criteria, we should use a computational method which eliminates all bias and error.  

The second criterion (consider only games played to date) is simply ludicrous if one wishes to form a rational ordinal ranking.  Prior to the first week's games, the current resume ranking methodology would compel us to ignore a wide variety of relevant information that is easily measured and objectively comparable (past performance, returning starters, depth, recruiting), and instead rank every team in a 119 way tie for first, last, and only place.

The situation is little better after the first games are played.  Suppose for example that a panel of experts has assigned a preseason rank for each team on the basis of the factors mentioned above (past performance, returning starters, depth, etc...).  Let letters appearing earlier in the alphabet represent the higher ranked teams.  Now suppose that A beats Z, and T beats S.  All else equal, by virtue of defeating a tougher opponent, T has a more impressive resume than does A.  Resume ranking thus requires that we rank T ahead of A.

This seems unreasonable.  We've learned that A is better than Z. And we've learned that T is better than S.  But no new information has been generated from the first games that allow T to be objectively compared to A.  The resume ranking methodology is therefore requiring us to ignore the only available objective information and form an ordinal ranking based on the subjective value judgment that a team beating a "tougher" opponent should have the higher rank.  

In most sports it is the case that either every league team plays every other league team at least once and/or a championship is determined via a playoff.  In this situation we have a plethora of pair wise comparisons to facilitate an ordinal ranking.  But in college football, most teams do not play each other and there is no playoff.  This paucity of information makes ranking by any methodology troublesome, and it is a particularly difficult problem for those bold enough to publish an ordinal ranking early in the season.  How do we compare A and Z when they do not play, and when A plays tough opponents while Z plays easy opponents? As more games are played this problem will become less pressing, though we will have to contend with the rise of contradictory information (A beats B, B beats C, and C beats A).  

If you insist on resume ranking based only on statistics from this season, this computational method might be of interest.  It produces a ranking based on something very much like the "resume" and does so without human bias or error.

0 recs | Comment 0 comments

Story-email Email Printer Print

Comments

Display:

Comments For This Post Are Closed


User Tools

Welcome to the SB Nation community devoted to the Georgia Bulldogs.
Start posting about the Bulldogs »

Join SB Nation and dive into communities focused on all your favorite teams.

FanPosts

Community blog posts and discussion.

Recent FanPosts

Flowerchucker_small
Quick Question re: Branden Smith
Small
College Football Preview Luncheon
Photo_69_small
Individual/Composite calendars for iCal/Outlook?
Small
College Football Preview Luncheon
Small
muschamp recruits in georgia
Associated_20press_clayliston_1965_l_small
How to Taunt a Georgia Fan
Mtsu_sm_small
Going to UGA
Small
oklahoma state blog
Small
2009 Competition Improves
Small
100 Cocktails

Post_icon New FanPost All FanPosts Carrot-mini


Managers

Beard_47_series_wins_and_42_points_in_2007_small T Kyle King

Official Partner of CBS Sports