Dr. Saturday - NCAAF

Dr. Saturday

Mon Nov 24, 2008 2:50 pm EST

Either James Madison is a lot better than you think, or there's one more thing rotten in the BCS

By Matt Hinton

Before we get started, a quick primer on James Madison, from the unlikely, dramatic finishes of its wins over Villanova and Richmond:

Now: On Sunday, I noted some of the indefensible results spit out by the BCS computers this week, most notably that they collectively rank Texas Tech and Utah ahead of Florida and USC, and two of them -- those run by Jeff Sagarin and Peter Wolfe -- still ranked Texas Tech ahead of Oklahoma a day after the Sooners' 44-point demolition left the Raiders' season lying in itty bitty pieces strewn across the heartland. I also explained why I don't think this is the computers' fault, as they're forced to throw out margin of victory by BCS rules, eliminating the difference the humans see in winning 65-21 and winning, say, 3-2. When there are so few games to compare, no win is really just a win, period, but them's the rules.

No, if you want to excoriate the machines, there's another source for you scorn: the inexplicable rise of JMU. The esteemed Jeff Sagarin's submission to the BCS this week ranks the Dukes No. 24, making the James Madison -- 31-7 loser to Duke in their only game against I-A competition -- the first ever I-AA/FCS team to earn BCS poll points of any kind.

This has been brewing for a while. Every Monday morning for the last few weeks, a JMU alum has messaged me on the Dukes' latest conquest in the Sagarin poll. Last week, they were ranked ahead of West Virginia. This week, they're ahead of not only the Mountaineers, but according to the ELO_CHESS rating (not Sagarin's "official" ranking, but the one actually submitted to the BCS, with margin of victory removed), they also trump LSU, Iowa, Michigan State, North Carolina, Cincinnati, Oregon, South Carolina, Nebraska, Pittsburgh, Wake Forest, Northwestern, UConn, Tulsa, South Florida, Western Michigan, Tulsa, Central Michigan and Rice, all of which are either in or received at least one vote for this week's Associated Press or Coaches polls.

Sagarin isn't alone, though there is a high degree of disagreement here: Of the two other BCS computer polls that include specific I-AA teams (the Colley Matrix shuffles I-AA into groups in some inscrutable way), Kenneth Massey ranks JMU No. 37, ahead of Northwestern, Iowa, UConn, Kansas, Vanderbilt, South Florida, Rutgers, Kentucky and Wisconsin, all BCS conference teams with winning records (and, yes, ahead of Duke, too). Peter Wolfe, on the other hand, ranks the Dukes 105th, alongside the likes of unabashed bottom dwellers New Mexico State, Ohio U. and UAB.

In its defense, James Madison is a solid team in its own right, with 10 straight wins, the No. 2 scoring offense in I-AA and the No. 1 seed in the upcoming playoffs. But the question here isn't, "How high can a I-AA team go?" It's "What is wrong with these computer formulas?" James Madison's best wins are over fellow playoff participants Appalachian State, Maine, Richmond and Villanova, each of which has had its head handed to it by a I-A/FBS team currently unranked in any of the mainstream polls. The Dukes' only foray into the top division was a beatdown at the hands of Duke, which is Duke, and which has subsequently lost six ACC games in a row.

In other words, there's no evidence JMU can compete in the I-A ranks at all, much less in the top 40. So if two different BCS algorithms are already making that kind of leap, what else might they be capable of?

digg delicious
more

26 Comments

Post a Comment
  1. todd.murray@...
    1. Posted by todd.murray@... Thu Sep 03, 2009 10:14 pm EDT

    Report Abuse

    Sagarin's rankings seem to have a horrible slant toward ACC strength of schedule this year. Virginia, Duke Wake Forest and NC State are somehow 1-4 in SOS rating. How is that even possible?? Playing William & Mary, James Madison, Navy (2 times), and East Carolina (2 times) with a Baylor and USC thrown in somehow equals the toughest schedules in the nation? The downtrodden ACC?? By getting waxed by the worst team in the ACC, James Madison is somehow rewarded for that?? How can Sagarin not see how skewed this is and not adjust for it? I used to respect his rankings, but they've gone off the reservation on this one. His bastardized ELO_CHESS rankings that the BCS uses are even worse.
  2. Steve
    2. Posted by Steve Thu Sep 03, 2009 9:25 pm EDT

    Report Abuse

    i know its fashionable to beat up on the computer pollsters, the "nerds" of college football who don't know what they are talking about 99% of the time. but really, its not completely their fault. sagarin has a perfectly fine predictor which takes mov into account and which we all agree is a better poll, but it goes unused. instead, the bcs tells them they can't use mov in their polls, and with this restriction their polls suffer. garbage in, garbage out.
    lets remember here that the human pollsters make up 2/3 of the bcs standings, and their judgement can often be described as questionable. for example: why is texas tech still in the top 10? top 10 teams should not be destroyed 65 ways from sunday. maybe they lose. maybe they lose bad. but to be utterly obliterated like that exposes them as not worthy of the top 10.
  3. Landon C
    3. Posted by Landon C Thu Sep 03, 2009 4:56 pm EDT

    Report Abuse

    It should also be noted that Sagarin's preferred ranking system has James Madison ranked... wait for it... #33. Doc, you can rail on the computers not taking into account margin of victory all you want, but the problem still remains (though admittedly not as bad) when margin of victory IS taken into account. How any computer system can rank a team that got smacked by Duke and needed two miracle finishes to be undefeated in 1-AA is almost beyond comprehension.
    All college football polls are flawed. Human polls are biased, generally based on limited watching of actual games, and the information considered often uses false assumptions. Computers, while eliminating the bias (in most cases, I believe one computer uses the last years rankings as a starting off point) is too limited in sample size to be accurate.
  4. Phil K
    4. Posted by Phil K Thu Sep 03, 2009 8:10 pm EDT

    Report Abuse

    I agree with your point that the computers do just what they're told.
    I will be non-objective and suggest that perhaps Florida is overrated. To our eyes, we see Florida blow out all their opposition in about 60 seconds. The computers just see a victory (as you stated, no margin of victory). Florida has only beat two good teams - Miami and Georgia. The computers are looking at what Florida has done and it is no more impressive than what the rest of the 1 loss teams have done so far - and I feel considerably less. I feel that you're off-base with this criticism of the BCS.
    Of course, Florida will rightfully move up if they beat Alabama.
    Perhaps things would be better if the computer rankings were not published until after the season...
  5. Matt H
    5. Posted by Matt H Thu Sep 03, 2009 9:48 pm EDT

    Report Abuse

    James Madison's ranking is not presented as a result of margin of victory being removed. It's just presented as flawed, period. I only used the absence of MOV to excuse the polls that still have Texas Tech ranked ahead of Oklahoma, and for the computers' collective preference of Tech and Utah to Florida and USC. But MOV (or lack thereof) had nothing to do with the criticism re: James Madison.
  6. gtne91
    6. Posted by gtne91 Thu Sep 03, 2009 8:43 pm EDT

    Report Abuse

    I think the JMU problem is called the "Mount Union" effect amongst those who program this kind of stuff regularly. I have run into it. All computer ranking systems suffer from 1 of 2 problems (actually, there is a 3rd but its more technical) - Either teams with great records get ranked too high (which is this problem) or teams with poor records but strong strength of schedule get ranked too high. A few years back, the RPI for basketball had a Clemson team with something like a 13-14 record in the top 25. It was just silly.
    Balancing these too and keeping mathematical sensibility is almost impossible. For me, the obvious way to avoid it is to avoid mixing the rankings from different divisions. This problem can even occur with MOV in play.
  7. gtne91
    7. Posted by gtne91 Thu Sep 03, 2009 8:43 pm EDT

    Report Abuse

    Just threw together a pure power ranking program, hopefully no errors, but I did it in the 20 minutes since my last post - I had all the parts available, just had to fix some stuff up, I didnt do it from scratch in the last 20 minutes. Using only MOV and who you played, here is the top 5 in nearly all of college football, NCAA and NAIA included, I think. 1. Florida (10- 1) 55.379 2. Oklahoma (10- 1) 51.612 3. Texas (10- 1) 49.511 4. Southern Cal ( 9- 1) 49.189 5. Mount Union (11- 0) 47.459
    Top 4 seems reasonable for a power rating, doesnt it? And what do you know, the Mt Union effect rears its head. According to this, USC would beat Mt Union on a neutral field by less than 2 pts. BTW, JMU came in at 26th.
    The problem is lack of connectivity between levels, I think Sagariin's mistake is ranking all of division 1 together instead of splitting it in 2 with Colley-style adjustments. Clearly having Div 3 teams in with Div 1 doesnt work.
  8. gtne91
    8. Posted by gtne91 Thu Sep 03, 2009 8:43 pm EDT

    Report Abuse

    Just as a sanity check, the power ranking in #6 has Florida as 48.7 pts better than The Citadel, adjust for home field for Florida and what do you know, Florida won by 51.
  9. Patrick A
    9. Posted by Patrick A Thu Sep 03, 2009 8:01 pm EDT

    Report Abuse

    Colley's adjustments aren't great - he just lumps division IAA teams together until they get enough connection to division IA to be somewhat reasonable. it's fairly arbitrary.
    The problem *absolutely* is lack of connectivity between levels. If you completely disconnect two groups, and then rank them individually, the two groups will tend to have the same high team and the same low team. Obviously as you add connection, the groups will start to align, but with weak connectivity, you always tend to "stretch" the major groups out. You have a similar, but weaker, problem with conferences themselves.
    There is absolutely no reason for games against division IAA teams to be included in BCS statistical ranking systems. wins or losses. there's just way too much uncertainty as to where to place the groups relative to each other. Toss them all out - if a major team loses to a division IAA team, the human polls will tar and feather them appropriately.
  10. gtne91
    10. Posted by gtne91 Thu Sep 03, 2009 8:43 pm EDT

    Report Abuse

    #8 - as you probably know, Colley use to do as you suggest, he changed his after Appy St beat Michigan last year. If he hadnt, he would have been crucified for overrating Michigan. I prefer to treat all FCS teams as a single team, which is unfair if you play JMU or Appy St instead of a weak FCS team, but oh well, their isnt any single good and fair adjustment.
  11. Jammmick
    11. Posted by Jammmick Thu Sep 03, 2009 4:28 pm EDT

    Report Abuse

    Sagarin's ratings are always silly. Check out his No. 1 football conference. It's the ACC. Let me say that again: It's the ACC. Silly, no? (Unless he means futbol.)
  12. Trey P
    12. Posted by Trey P Thu Sep 03, 2009 3:56 pm EDT

    Report Abuse

    Sagarin's conference ratings are really only dependent on how the teams did in their non-conference games.
    After their awful start in the first couple of weeks, the acc has been very strong in conference games. Forget your perceptions and look at what they've actually done on the field -- that's the whole point of objective ratings like the sagarins.
    The acc doesn't have any elite teams but it has no really awful teams either which makes it much stronger than any other conference at the bottom. The ACC has a lot of pretty good teams that have beat up on each other, and they've gotten much better through the season unlike some conferences (hello lsu/auburn/tennessee/vandy/uga/arkansas).
    Bama didn't physically dominate Clemson any worse than some of their SEC opponents. Miami played UF tougher than most of their SEC opponents did. Ole Miss is the second best team in the SEC-West and they are probably going to the Cotton Bowl, but they lost to wake forest. GT beat MSU worse than any SEC team did, and worse than GT beat any of their ACC opponents. Vandy went 4-4 in SEC play, but lost at home to the ACC's worst team, Duke.
    Duke is the best example of why the ACC is rated highest -- not because of how good the best teams are, but because of how good the worst teams are. Duke has been mashed for the most part in ACC play but they were hell on wheels out of conference. They beat an SEC bowl team on the road, soundly beat a decent mid-major in Navy, just barely lost to 9-3 Northwestern, and crushed JMU who is the #1 seed in the FCS playoffs.
    That is a stellar non-conference record for the worst team in a league, and much better than the bottom couple of teams in any other conference could do against that non-conference schedule. There have been no breaks at all in the acc this year, no wazzus or texas a&m's or arkansas's or michigans or syracuses, and that's part of why noone has gone better than 5-3.
    It's kinda funny -- it used to be that FSU dominating the league was a sign of its weakness. Now noone dominating the league is a sign of its weakness. Funny how people tend to see what they want to see.... :-)
  13. Rawk
    13. Posted by Rawk Thu Sep 03, 2009 4:52 pm EDT

    Report Abuse

    "the first ever I-AA/FCS team to earn BCS poll points of any kind. "
    Considering the BCS throws out the high and low computer rankings, I believe the BCS is awarding JMU no points.
  14. Matt H
    14. Posted by Matt H Thu Sep 03, 2009 9:48 pm EDT

    Report Abuse

    They're still points, Darkmane ... of the thrown-out variety, but still points.
  15. mangere_bridge
    15. Posted by mangere_bridge Thu Sep 03, 2009 3:12 pm EDT

    Report Abuse

    Sagarin claims James Madison's schedule is of about the same degree of difficulty as Ball State's, which, given that JMU's played one of the toughest I-AA schedules, is only a little unreasonable. But then I think Ball State is too high.
    I don't think the ACC is the premier conference, but it's a reasonable claim if you value top-to-bottom consistency. They certainly have the best worst team, unless you think NC State have been extremely lucky these last three weeks.
  16. hr209
    16. Posted by hr209 Thu Sep 03, 2009 4:21 pm EDT

    Report Abuse

    I run an amateur college football ranking system that seems to work on a general basis, and it too spits out that ACC is the best conference. The main problem is that the ACC is the only conference that isn't top heavy. When people think of "the best conference", they always think of top teams attacking one another (Texas and Oklahoma could beat Penn State and Ohio State, USC could beat West Virginia, Alabama and Florida would beat BC and FSU, etc.). But if you want to know who the best total conference is, you have to factor in the bottom feeding teams as well, and Washington, Washington State, Iowa State, Syracuse, Indiana, and Arkansas are terrible teams. The ACC's worst four teams are Duke, Clemson, NC State, and Virginia, all of which have been semi-competitive.
    You can't just think of the big teams with flashy offenses when determining the best conference. At least until bowl season, that is.
  17. derrich.com
    17. Posted by derrich.com Thu Sep 03, 2009 3:41 pm EDT

    Report Abuse

    Didn't App St. crawl into the BCS last year? Or was it only the human polls? I forget.
  18. pantsmcpants1
    18. Posted by pantsmcpants1 Thu Sep 03, 2009 7:59 pm EDT

    Report Abuse

    #13 hr209 nails it with the overall problem that causes SOS arguments all the time- when you are trying to determine the best teams in the country, you need to measure them by the BEST teams that they play.
    Is it harder to go undefeated against a schedule of 10 "5-5 teams", or 5 "9-1 teams" and 5 "0-10" teams? The former has a much harder SOS per the books, but an elite team like UF, USC, etc, should have little trouble beating a bunch of 5-5 teams, little more so than the 0-10 teams. But beating one, two or more 9-1 teams is a much bigger task, yet the computers do not see it that way.
  19. PC
    19. Posted by PC Thu Sep 03, 2009 8:04 pm EDT

    Report Abuse

    I don't think anyone here believes JMU should be ranked 24th across the board. I could be wrong but doesn't Sagarin's system factor how each team did against each other in the calculation? Meaning there is some transitive effect to scores? What probably helps JMU is that Duke beat JMU 31-7 and beat UVA 31-3 but UVA beat Maryland 31-0, Maryland Beat Delaware 14-7 and JMU beat Delaware 41-7. Sagarin's formula apparently factors all that info in.
  20. Ken
    20. Posted by Ken Thu Sep 03, 2009 8:27 pm EDT

    Report Abuse

    Sagarin's "preferred ranking" is his predictor version, taking MOV and schedule into account. That ranking has JMU 41st, not 33rd. And 41st is defensible. This article is not an indictment of Sagarin; it's an indictment of the BCS for using a version of his rankings with MOV removed, and they are doing so over his objections and advice.
  21. gtne91
    21. Posted by gtne91 Thu Sep 03, 2009 8:43 pm EDT

    Report Abuse

    Ken,
    Sagarin's preferred version is the synthesis of the ELO and the predictor, not just the predictor.
  22. Laughing Stock
    22. Posted by Laughing Stock Thu Sep 03, 2009 7:08 pm EDT

    Report Abuse

    The key problem with college football ranking systems are interconnections between teams and the lack of including Margin of Victory.
  23. Patrick A
    23. Posted by Patrick A Thu Sep 03, 2009 8:01 pm EDT

    Report Abuse

    Whoops, New Hampshire also beat 3-8 Army 28-10. So Division IAA's sole victories are over a 3-8 team and a 2-10 team. Yes, clearly their best team should be ranked in the top half of Division IA. That makes sense.
  24. Michael
    24. Posted by Michael Thu Sep 03, 2009 7:17 pm EDT

    Report Abuse

    How about the fact that teams get better throughout the season. JMU is a different team than 12 weeks ago when it played Duke to open up their season. Put the teams on a neutral field and play right now I think it would be a different story. I agree with the fact that JMU shouldn't be ranked that high but I follow them and they play very very similar to WVA in offensive setup. The ACC is a joke this year.

Dr. Saturday

Add to My Yahoo! RSS

Matt Hinton

Dr. Saturday is a college football blog edited by Matt Hinton. Email him tips and feedback.

Contributors:
Holly Anderson, Doug Gillett, Dan Shanoff,

Related Photo Gallery

More NCAAF Photos »

Y! Sports Blogs

Dr. Saturday Recent Readers

(provided by MyBlogLog)