UK universities might misattribute changes in their rankings to their institutional strategies when random fluctuations in the ranking system are a more likely explanation. Preprint: osf.io/preprints/ps... BSky author: @tvpollet.bsky.social #UKHE #AcademicSky - ThreadSky

markrubin.bsky.social • 41 days ago

UK universities might misattribute changes in their rankings to their institutional strategies when random fluctuations in the ranking system are a more likely explanation.

Preprint: https://osf.io/preprints/psyarxiv/fdkj5

BSky author: @tvpollet.bsky.social

#UKHE #AcademicSky

Comments

rkapitany.bsky.social•41 days ago

I love that universities have dozens of departments with experts on every field imaginable, but don't consult any of them on huge, long-term, costly decisions.

This is so obviously the case! I'm glad it's been written down now. Maybe in 8 years something will be done about it.

amcgregor.bsky.social•41 days ago

We should just reply ‘regression to the mean’ whenever universities boast about their rankings

rkapitany.bsky.social•41 days ago

I haven't read the article, but it might even be dumber than that. These scores are dependent on all other scores. Uni x could shift away from their true mean due to error variance in other unis scores (~local to them on the ladder)!

rkapitany.bsky.social•41 days ago

That is, imagine jittering all scores except one (the one with a true known real mean). I suspect more than half the time it'll shift away from it's own mean.

rkapitany.bsky.social•41 days ago

(is this possible? it's 3am where I'm at. I'll model this tomorrow perhaps)

spyhistorian.bsky.social•39 days ago

Ah! It's absolutely classic.

marcinko.bsky.social•41 days ago

I reached out a couple times years ago to offer research-backed best practice on some diversity initiative or other.

I received a firm "no thanks." 😂

markrubin.bsky.social•41 days ago

Seems like an obvious way to save on external consultancy fees in these tight financial times. 🤷‍♂️

emmadw.bsky.social•41 days ago

So, improve - praise your strategy, go down blame the algorithm ...
Ranking is always going to be an issue, as everyone wants to go up it, rather than aiming to improve and do better *compared to yourself, last year*

emmadw.bsky.social•41 days ago

Ironic really, class results lists have long gone, we no longer list who came first in the exam, rather encourage students to do better than before. Rankings are the anthesis of that.

hitchikerspie.bsky.social•40 days ago

Well discussed by some Nottingham Uni lecturers and professors here:

https://youtu.be/3zSkxkXkJHE?si=bZFqzgclPzFMyO9B

gnoli.eu•41 days ago

So we should do our job and not care too much about statistics !

drboothroyd.bsky.social•41 days ago

Ooo nice one Thomas!
@tvpollet.bsky.social

Comments

Posting Rules

Reply