Five Important Things to Keep in Mind when a Student Asks About University Rankings
19/09/18
A strong believer in the transformative nature of higher education, in his current role Sebastion Marx leads UQ's School Liaison team - providing advice to prospective university students on which degree pathways will best help them achieve their career aspirations.
This article is an adaptation of his well-received presentation at the 2018 CDAA annual conference in Hobart. Sebastion takes a close and critical look at what university rankings measure, and more importantly, what they don’t, dispelling some prevailing myths and assumptions.
Photo by Steven Lelham on Unsplash
The media frenzy from the release of the annual university ranking tables is an Australian winter tradition. It shows itself in many forms on a slow-news-day, from the media beat up over falling standards of Australian higher education to the self-congratulatory smugfest when the latest cut of the data falls in a certain university’s favour.
Because of this prevalence in the media-cycle, it’s only natural that parents and students attempt to adopt rankings as a definitive comparison tool between institutions.
It can be hard to dissuade students, or their parents, of the apparent importance of rankings. Here are five thoughts to keep in the back of your mind when you are asked about university rankings.
1. The major rankings don’t focus on teaching quality
Many of the international rankings say that they consider teaching quality in their methodology, but these claims rarely stack up.
For example, let’s consider the Times Higher Education (THE) university rankings.
Within the Time’s methodology, 30% of an institution’s score is attributed to ‘Teaching’. But let’s break that category down and investigate what is actually being measured:
Teaching (total 30%)
- Reputation survey - 15%
- Staff-to-student ratio – 4.5%
- Doctorate to bachelor ratio – 2.25%
- Doctorates awarded to academic staff – 6%
- Institutional income – 2.25%
Not exactly the metrics you expect from a category titled ‘teaching’, is it?
2. Some rankings rely on opinion polls to determine their outcomes
Even when student experience and teaching is considered by ranking organisations, the scores from these outcomes are often heavily diluted through the subjective opinions of the academic community.
In fairness, an institutions reputation can play a role in the validity of its publications and experienced researchers are more likely to have reviewed another universities research.
When it comes to teaching however, I don’t see any way that the opinions of the academic community could be an informed one, except within their own institutions. An academic from one university is not likely to have met many students from other institutions. Now I might not have a PhD, but I can think of a few more effective ways to measure teaching quality than based on hearsay.
Bias is also a major issue with these opinion polls. Virtually all survey participants would be active users of the data previously collected and are therefore undoubtedly influenced by it. This makes it harder for rankings to change from year to year. Essentially, they changed this year’s result by measuring (and publishing) it last year.
3. Research outcomes dominate ranking outcomes
The methodology of most major ranking organisations is dominated by two things. Research outcomes and academic opinions of research outcomes. In every major ranking, at least 50% of the result is measured by these two factors.
For most course-work bound students, it’s unlikely this will bear much impact on the best institution for them.
4. Employability information is available through government sources
Over the past five years the Federal Government has introduced a raft of measures designed to make it easier for students to compare Australian institutions – as demonstrated by the recent admissions transparency changes that impacted the terminology of university admissions.
The most important of these innovations is the Quality Indicators for Learning and Teaching (QILT) website at www.qilt.edu.au.
QILT pulls results from several different surveys and data analysis to provide student comparisons on a huge range of different metrics. Students can compare discipline areas, universities or discipline areas at different universities.
Most importantly, career counsellors can access field of study graduate outcome data and compare based on institution. Graduate employment, median starting salaries and proportion of students in further study.
This data trumps any ranking I’ve yet to find.
5. Context is king when comparing data
QILT is an amazing tool for gaining an insight into different institutions and their program offerings. It is also potentially misleading to those who might approach the data it contains without context.
Many of the surveys are subjective, asking participants their opinion on resources, support, and the quality of their education. The problem with comparing these results is that every university has a different student body with a unique demographic and psychographic profile.
What a student considers to be a high-quality education will change based on background and interests. An ATAR 99 school-leaver on a Law or Medicine track will naturally have different expectations than say a mature-age student at a small regional uni.
Subjective surveys are a great way to get a feel for an institution, but generally are only truly comparable within the demographic context of that institution.
For institution comparison on QILT, you might be better off sticking to hard and fast metrics such as graduate outcomes.
Contact Sebastion Marx:
https://www.linkedin.com/in/sebmarx
[email protected]