bioinformatics, genomes, biology etc. "I don't mean to sound angry and cynical, but I am, so that's how it comes across"

Why you probably shouldn’t be happy at Stern’s recommendations for REF

If you are a British academic then you will be aware that Lord Stern published his recommendations for REF (the research excellence framework) this week.  REF is a thoroughly awful but necessary process.  Currently your academic career is distilled down to 4 publications and assessed every 4-5 years.  Papers are classified via some unknown system into 2*, 3* or 4* outputs and your value as an academic recorded appropriately.   Given that higher REF scores result in more money for Universities, your individual REF score has a very real impact on your value to your employer.  This has pros and cons as I will set out below.

Here are some of the recommendations and my thoughts:

  • Recommendation 1: All research active staff should be returned in the REF
  • Recommendation 2: Outputs should be submitted at Unit of Assessment level with a set average number per FTE but with flexibility for some faculty members to submit more and others less than the average.
  • What currently happens?  At the moment your employer decides whether your outputs make you “REF-able”.  In other words, if you don’t have 4 good outputs/publications, you won’t be submitted and you are REF-invisible
  • Stern recommendation: Stern recommends that all research-active staff be submitted and that the average number of outputs is 2.  However, there is a twist – the number of submissions per person can be between 0 and 6.  Therefore you may be submitted with zero outputs, which is perhaps even worse than being REF-invisible.  Given the formula for the number of expected outputs is 2*N (where N is the number of research-active staff), if a University has less than 2*N good impacts, there must surely be a pressure to transfer those with few outputs onto a teaching contract rather than a research contract.  And given the range of 0 to 6, I can see established Profs taking up all 6, with early career researchers being dumped or submitted with zero outputs.  So I’m not impressed by this one.


  • Recommendation 3: Outputs should not be portable.
  • What currently happens?  At the moment, an output stays with the individual.  So if I publish a Nature paper during a REF cycle and then move to another University, then my new employer gets the benefit, rather than my old employer.  This has resulted in REF-based recruitment, whereby individuals are recruited by Universities (often with high salaries and incentives) specifically because they have good REF outputs.
  • Stern recommendation: that outputs are not portable.  Specifically that publications remain with the employer present when they are accepted for publication.   It’s worth reading what the Stern report says here: “There is a problem in the current REF system associated with the demonstrable increase in the number of individuals being recruited from other institutions shortly before the census date. This has costs for the UK HEI system in terms of recruitment and retention”.   Read and re-read this sentence in context – high impact publications directly influence how much money a University gets from the government; yet here Stern argues that this shouldn’t be used for “recruitment and retention” of staff who produce those publications.  In other words current REF rules are pitched not as some sort of incentive to reward good performance, but as some kind of unnecessary cost that should be banished from the system.   Yes – read it again – potential staff rewards for good performance (“retention”) are quite clearly stated as a “cost” and as a “problem” to HEIs.
  • What the old REF rules did, in a very real way, is give power to the individual.  Publish some high impact papers and not only will other HEI’s offer you a job, but your existing employer might try and keep you, offering incentives such as pay rises and promotions.  What Stern is recommending is that power is taken from the individual and handed to the institution.  Once you publish, that’s it, they own the output.  No need to reward the individual anymore.
  • This also has the perverse outcome that an institution’s REF score shows how good they were not how good they are.  Take an extreme toy example – University A might have 100 amazing researchers between 2010 and 2014 and achieve an incredible REF score in 2015; yet they all may have left to go to University B.  How good is University A at research?  Well, not very good because all of their research-active staff left – yet they still have a really good REF score.


I don’t really have any major objections to the other recommendations; I think Stern has done a pretty good job on those.  However, I’m not at all happy with 1-3 above.   There are actually very few incentives for pay rises amongst UK academics, and REF was one of those incentives.  Stern wants to remove it.  You can see how healthy your University’s accounts are here (from here);  you will see that the vast majority (about 110 out of 120) UK universities generated an annual surplus last year, and the whole sector generated a surplus of £1.8Bn.   Yet somehow, incentives to promote, recruit and retain staff who are performing well is  a “cost” and a “problem”.  I also don’t think that the recommendations help ECRs as they could remain invisible to the entire process.

In conclusion, I don’t think the recommendations of Stern – or to give him his full title, Professor Lord Stern, Baron of Brentford, FRS, FBA – do anything positive for the individual researcher, they don’t provide much help for ECRs, and they hand power back to Universities.




  1. I don’t see the recommendations themselves as negatively as you do…. I could see if they were implemented poorly it could result in some of the negative scenarios you paint.

    I agree that all researchers should be included as the different attempts to play the system isn’t a good use of everyone’s time and skews results and interpretation of the results in too many ways. Between the REF and TEF, particularly if viewed together all academic staff should be accounted for. What would be great would be a way of value the impact of the support/technical staff …. Perhaps something in the environment section.

    The REF transfer window was not great and the costs to academia, financially and in terms of staff motivation wasn’t great… Just look at the difference in pay between academic staff appointed during the transfer window (http://www.ed.ac.uk/files/atoms/files/grade_10_professorial_and_equivalent_equal_pay_audit_2015.pdf) which means there’s a tiny proportion of staff that moved that benefited versus those that stayed and are effectively maintaining the research capacity in the reporting organisation that you’re lamenting potential loss of in your blog. People remain put for a number of reasons… Mainly personal I’m guessing and therefore the ref transfer game was potentially biased. I’m pleased to see that game ending as it benefited only a small proportion of staff!

    If the outputs aren’t portable then the researcher FTE during the reporting period shouldn’t be so the FTE count should reflect proportion during the reporting period so people submitted actually match the outputs being assessed… I know it’s retrospective but that’s the only thing that can be done. Perhaps the way to highlight the difference between the review of ref and the current research status is to get the institutions reporting the proportion of staff retained from those submitted… But that might hinder mobility!

    My reading of the recommendations is that the papers aren’t necessarily going to be matched to one individual researcher but that it’s a body of workthat equates to 2*FTE submission. Last time I was submitted with 4 outputs but I was involved on another 6/7 across the piece. If it’s a body of researchers and their associated body of papers where it equals fte*2 does it matter that the output goes against multiple people? After all is the outputs that are scored not the individual researchers…. We’re simply multipliers!

    Other comments/blogs http://blog.history.ac.uk/2016/07/stern-review-an-initial-bibliography/

    • biomickwatson

      3rd August 2016 at 2:36 pm

      Its not just about moving but also retaining staff – publishing 4* papers works both ways as employer tries to keep good staff as well as recruit new ones. Its still taking power away from individuals whichever way you look at it…

      • I don’t see how the REF transfer of staff helped in retaining people – I am not happy about people being hired at inflated hire salaries in the months leading up to the cut off census date compared to those performing to REF standards but just in situ – that doesn’t help me feel more valued bur rather the opposite!

        It is a fact that it is harder to seek the extra bucks to your wage packet if you’re already employed there and if you don’t want to move and participate in the game you’re at a disadvantage.

        Message is as always – write papers that are good and you will be retained because there will be some Fat Cat Professor wanting to ride on your coat tails! Once you get to count the outputs too!

        • biomickwatson

          4th August 2016 at 2:46 pm

          Eileen I can’t see in the document you referenced any mention of REF or how it influences pay? You cited an analysis of gender equality….

  2. I entirely agree with this assessment . People who generate ideas and get them funded will not get rewarded. Instead Universities will easily be able to get shot of them and attribute the research to other researchers/teachers who might not have had anything to do with the research work. It will strengthen the hand of teaching staff who are not research active – which is a University priority. Research is all about the individual and not necessarily the Institution. Often the institutions show little interest in research until it looks like being a “cash cow” leaving the researcher to shoulder the intellectual input and burn the midnight oil. Lord Stern has not thought through these points in a cogent way and certainly does not recognise what Universities have become e.g. businesses. This is a very sad day for research excellence and will deliver little in terms of new theories and understanding. A compromise would be a good way forward e.g transferring some outputs to a new position. After all in other spheres of work individuals are hired for their experience, ability and track record – lets make sure this principal stays within research.

Leave a Reply

Your email address will not be published.




© 2017 Opiniomics

Theme by Anders NorenUp ↑