On Academic Rankings

There is nothing more important in human society, so it would seem, than relative status. It doesn’t matter how great your wealth is, if someone else is wealthier. It doesn’t matter how great your university is, if someone else’s is ranked higher. But how do we actually rank universities? How can we compare such dissimilar items?

I believe university rankings follow from sports rankings. Sure, West Dakota A&M may be ranked number one in Ice Lacrosse, but we’re smarter. How do you prove that.

If it were a business, we might look at sales or profitability or market share or customer satisfaction ratings. For universities, we need other metrics. And there are a lot.

Ranking Universities

Having been in universities a long time, I have seen rankings come and go. Wikipedia has an amazing list. Comparing the lists, on most, year after year, the same set of schools, more or less, are at the top. But their positions vary widely.  West Dakotah A&M rarely makes it. There are a variety of ranking systems profferred.

In the US, the most famous is the US News and World Report College Rankings. USNWR is a defunct weekly newsmagazine, the RC Cola to Time’s Coke and Newsweek’s Pepsi. But their College Rankings live on. They use an arbitrary weighting of a variety of factors, a weighting which changes regularly, to develop a ranking.

Now if this were just about status, it wouldn’t matter except for alumni egos. It does however have a self-fulfilling element. If you are an international student and are offered admission to 3 US universities, with otherwise equal offers, you will tend to choose the one with the highest ranking. If you are a national employer, you will recruit from the highest ranked university for what are perceived as the best self-selected students.

However it is important to note the US college system is not nearly as an effective filter as those in Asian countries. While in the US, the best student at West Dakotah is almost assuredly better than the bottom two-thirds of Minnesota, in some countries which take entrance exams very seriously (Iran, China, India, e.g.), the worst student at the best school might very well be better, or at least smarter, than the best student at the next ranked school. (Well probably I exaggerate, but you get the point).

Now perhaps you aren’t looking at USNWR. Maybe it’s Shanghai Jiaotong (which incidentally means transport in Chinese) University’s ARWU rankings. Or maybe it’s the Times Higher Education Survey, or one of two-dozen  others. It doesn’t matter much. It’s good there is so much competition, many schools can claim to be number 1 or top 10 or some such.

My University of Minnesota once wanted to the be among the top 3 public universities in the US. It didn’t say which list though. I assume whichever put them at number 3. (I always assumed this would involve treating the different University of California campuses as one university). This goal was abandoned quietly it appears, though it still is mentioned in this 2011 document, which says top 3 public research university “in the world in a decade”.

These numbers are so meaningless, I wish people would stop talking about them. But no.

Ranking Programs

Rankings also give status to individual programs. This is even harder. Whose Civil Engineering program is the best? The one with the most money per faculty member? The one with the most graduates placed? The one with the most publications?  Caltech, a fine school which doesn’t even have an undergraduate civil engineering program, was highly ranked by one of the surveys one year. (Sorry can’t find the link).  Reputation is so strong it can rank non-existant programs.

The granularity of this is getting deeper. Joseph Chow has ranked  Transport Engineering programs. (Based on H-index). The University of Minnesota did well. Obviously H-Index is a function of size, so larger programs will tend to score better. Sydney (ITLS) also scored really high. My alma mater, UC Berkeley topped out though. While I don’t agree with every detail, and it is biased to larger programs, it seems truthy. But really, what’s the difference between an h-index 29 and 30? Three more citations. (The 29th ranked paper needs one more and the 30th ranked paper needs two more).

Rankings of Transportation Engineering Programs by Joseph Chow (2007-2016)
Rankings of Transportation Engineering Programs by Joseph Chow (2007-2016)


Ranking People

Individual academics are presumably the limit of this. Tom Sanchez has ranked US planning academics. Despite being an engineer, I show up on this list (I do have a Humphrey School affiliation, and I do some planning stuff). Still, a lot of my work is more engineering-ish than planning-ish, so I am not sure about what the right methodology would be. Google Scholar tends to be noisy, so I am reluctant to rely on that as the be-all, end-all, but ok, I’ll brag about it.