Breaking down university rankings

Francesca Mirolo


With her experience of business backing mining and petroleum exploration over other studies, geology student Francesca Mirolo says university rankings should be taken with a pinch of salt.

Every year, shortly after the Leaving Cert results and the merry scramble for places in college, we get the release of the university rankings list. Frankly, I have a rather jaded view of the rankings; it feels like the rankings given to secondary schools escalated to a global level.

It seems only natural that each university will aim as high up the list as possible, for prestige and to attract funding and students. While great care is taken in the calculations for the rankings, I’m not convinced the rankings are a full reflection of each university. In the words of my secondary school English teacher: you must have context. I’ll use Trinity College, Dublin, my home university, as an example.

This year Trinity ranked 110th in the Times Higher Education (THE) rankings, up seven from last year, with an overall point score of 56.2. The individual scores that make up this average indicate Trinity is weakest in research and industry investment.

But these scores say nothing about the university’s economic environment: just three years ago, in 2009, before the beginning of the global economic downturn, Trinity was sitting in a rather comfortable tie with Osaka University at 43rd place. Since then, Ireland’s small, open economy has drastically worsened, naturally resulting in funding cuts to all areas of society, particularly universities.

The THE industry score is calculated by how much a university’s research work promotes economic development, as measured by the amount industry invests back in the university. Let’s be honest – industry favours work that will turn a profit. That means maths, engineering, the sciences and business attract the most attention from industry. This automatically disqualifies large swathes of work done in the arts and humanities, and favours science-focused institutions.

Even within a single science, more money will go to certain disciplines than to others. To take another example that I know well, industry investment in geology will focus primarily on mining and petroleum exploration, with some interest in tectonics and palaeontology since they can help pinpoint new areas to exploit.

The research score, valued at 30% of the total, is the other weak point in Trinity’s breakdown. This score is calculated using the university’s research income, a reputation score based on a survey of academics, and the paper output of its academic staff. Not only is research income sensitive to economic conditions, but the sample size of the reputation survey is only 16,000, and thus unlikely to provide a truly global perspective.

As for paper count per academic, while this is a helpful indicative measure, it is well known not to be a good indicator of real research output; it also fails to take into account the problem of balancing teaching and research work.

Furthermore, research is not just the province of academics, and this measure explicitly does not count postgraduate students or postdocs. But, when there is a lack of funding for the latter, research output will suffer. I suspect such a lack of research funding to be the problem for Trinity: if the research produced here is so poor, then why is the citations score – based on the impact of a university’s work and the number of times its papers are cited – Trinity’s best score at 88.1? It appears what is being produced is quality over quantity.

The lack of funding makes itself known in other ways as well. In Ireland, to be accepted to a master’s program with any hope of funding, typically you need a 2.1. Yet in geology, there are no such positions available at all, save for industry-funded research masters.

To pursue postgraduate work in geology in Ireland, you therefore need to pursue a PhD, and to have any chance of funding for that you need to have a first. And that is without taking into account competition with the international students vying for the position. By contrast, at Oxford, it’s perfectly possible to be admitted to a fully-funded doctoral programme with a 2.1.

The thing I’m afraid of is that prospective students will use these rankings to choose between universities without understanding the caveats attached to them. Realistically, university rankings might give some indication of the relative quality of their courses, but they certainly don’t give a complete picture. They need to include a lot more contextual information, and only once they do so will they be truly useful tools for picking your course. Until then, take them with a pinch of salt.