A better way of rating universities

One of the slightly embarrassing things about working for a university, at least for those of us brought up to believe that big-noting yourself is bad form, is the academic obsession with status. The Jiao Tong rankings and the Times Higher Education Supplement rankings are the most anxiously awaited, though any vaguely credible assessment will grab the attention of academics and university administrators (here is a list of various rankings of Australian universities). At the micro level, all the indicators that help make up these rankings – research grants, publications in journals of varying prestige, awards received by staff etc – are the subject of much personal academic angst.

Universities regularly tell us how well they have done. ‘ANU ranks first among Australian universities’, its marketing and communications department tells us. ‘Melbourne moves up key world ranking’ says Melbourne University’s UniNews. La Trobe University’s trumpeting of its performance was pure spin. According to its press release:

La Trobe University has been ranked among Victoria

11 thoughts on “A better way of rating universities

  1. Even if you are interested in a research career, it is not clear that the ranking of a university in general is what you are looking at. You would be better off looking at the ranking of the departments that teach the subject or subjects you are interested in. Why should the fact that Physical Sciences Uni is ranked number one be of any relevance to someone interested in studying Anthropology, if its Anthropolgy department is ranked 5, 985, 285 in the world? When it comes top picking grad schools, you might want to actually look even closer at the research strenghs within particular departments to see which one best suits your own areas of interest within the subject. Of course, there are a number of other factors you would be interested in as well.

    Like

  2. Interesting post, Andrew. I suspect though that there is quite a good correlation between the research strength of the university and its essential qualities for students, such as employment prospects (for the same fields). At least this was my impression when I compared what I read in Good Universties Guide (thanks to the reference in your book) to research rankings. It is up to specialists (eg yourself) to tell why.

    Like

  3. Also worth considering:

    1. Murdoch owns the Times and attended Oxford (top ranked UK uni according to his newspaper). His family has invested time and money in unimelb (and made a loss thanks to erroneous predictions). He has published these fabricated statements without investigating the University of Riga or the the 700 year old university of Pecs in Hungary. In fact, there is a whole layer of non anglophone universities which have made serious differences to the way we live and think which do not even rate a mention. Given that some of them dont charge fees, they would rank very highly on gauranteed value for money.

    2. If ANU is so excellent why cant it attract enough undergrad students to fill its quota? Is the move out of the family home to Canberra just too expensive to consider when you can stay at home until you marry and then become prime minister? Why are do so many of the academics there discourage original research in the fields they claim to know about?

    3. Even at Beijing U and TsingHua U there are whole subject areas which the Chinese are just ignorant about and vice versa with the west. How are they in any position to rank universities when they dont have a grasp of the subject matter?

    Ranking should be based on philosophical, technological, legal and medical etc contribution to humanity. Per capita this would however favour Scotland’s central belt universities Glasgow, Strathclyde and Edinburgh which have provided telephones, television (thereby the screen you are reading this from), Adam Smith, penecillin, all of Australia’s universities through the establishement of USydney, medical degrees that involve hands on training, internal combustion engines, the US constitution, Hume, the concepts of an “Ice Age” and global climate change, ending African slavery (Livingstone), the list goes on and on..

    The only reaon Murdoch doesn’t rate Scotland is because his family were kicked off the land centuries ago as peasants and they cant even run a successful record label with half a degree from Harvard and unlimited finance.

    Dont believe or worry about 80% of what you cut and paste into your blog and little of what you predict because BigHeed is lookin out fur ye..

    Like

  4. “suspect though that there is quite a good correlation between the research strength of the university and its essential qualities for students, such as employment prospects (for the same fields).”

    Boris – Path dependency in both cases. The first universities in each state always win research funding because that is based on past performance and still have the prestige advantage, in that the less talented students tended to go to the second and later universities in each state, establishing the pecking order that persists to this day. I have not examined the student satisfaction data in recent times, but when I wrote my book there was generally a negative correlation beween research strength and student perceptions of teaching.

    Like

  5. What Damien said, the idea of ranking the whole uni is a holistic error, with all the silliness of comparing incomparables. Stiglitz wrote a good essay on the economics of recruiting top people, it breaks the budget to buy more than a dozen or score of the best people in the world so unis need to specialist and concentrate on selected faculties.

    Like

  6. Rafe and Damien — you are both incorrect. If you are interested in a real research career, then the university you go to makes a lot of differences for picking up things like post-docs, academic jobs etc. thanks to snobbery (whether you like it or not)

    My suggestion is that people try and go to a research university in the US to do both their PhDs and postdocs, since PhDs from Australian univerisities don’t have half the status of US ones and you will have to be a huge amount better than other candidates when competing for the same thing if your PhD says e.g., La Trobe vs Harvard.

    Like

  7. Andrew — the obvious reason univerisities rank themselves against the SJT index (and it isn’t just Australian universities), is that it is
    a) the most widely publicized;
    b) done completely on metrics so ignorant people think it doesn’t have much bias;
    c) the most widely known in Asia (i.e., the OS students know about it);
    and d) apart from the US, almost no-one cares about teaching quality, including the students applying to do the courses (I don’t think I’ve ever been asked about the teaching quality of a university but I’ve been asked a zillion other questions on sell-university days) and a fair chunk of the staff teaching them (its not like you get promoted for getting 4.3 on a student survery vs. 3.6).

    Like

  8. There were attempts to do this sort of customized rankings with data from the US National Research Council’s 1995 Assessment of Research Doctorates. The NRC is now coordinating the next of those assessments and they are looking at providing data for similar customizable rankings.

    Interestingly, American universities dominate the international rankings but Americans – including university leaders – could not care less about the THES and SJTU rankings. They have enough difficulty sorting out the problems associated with how US News calculates its (undergrad) rankings: these international rankings give new meaning to the word shoddy. How SJTU attributes Nobel laureates

    Like

  9. Conrad said
    (its not like you get promoted for getting 4.3 on a student survery vs. 3.6)

    um but it helps….

    And now that funding is being given to universities that get good reports on teaching & learning in the CEQ, the pressure’s on to make sure students learn well (and know they are learning well.)

    Like

Leave a comment