How To Reference The Ref

Or how to become beyond a league tabular array seat inward evaluating a Great Britain academy department

Every 5 years or thus all Great Britain academy departments acquire their enquiry assessed inward 1 gigantic peer review exercise, which is straight off called the REF. Each dependent area is assessed separately, as well as scores tin live on used to compile a league table. The practice has direct financial implications: the ameliorate the research, the to a greater extent than coin universities acquire from the government. But if you lot know what academics are like, you lot volition non live on surprised to larn that those inward the Great Britain obsess well-nigh this practice as well as its results to a far greater extent than the coin involved would justify. The results of the latest practice direct maintain precisely been published, as well as turned into league tables yesteryear Times Higher Education (THE) here. 

You could country that the REF straight off provides the same form of incentive scheme for Great Britain universities every bit turn a profit does for a firm. In to a greater extent than or less cases academics whose enquiry is below their departmental average are pose nether pressure level to move out yesteryear 1 agency or another, as well as most academics experience acutely the pressure level to improve on how their ain output volition live on assessed yesteryear this exercise. In contrast, piteous surgical operation inward teaching or management is non nearly such a serious issue.

Many academics complain bitterly well-nigh the indignity of all this. An alternative scheme would live on 1 where getting tenure was the final surgical operation hurdle an academic had to pass, as well as from as well as then on they were complimentary to gain what they liked. Research coin could all live on allocated on a projection yesteryear projection basis. I personally dubiety that would live on a ameliorate scheme from society’s signal of view, as well as I gain uncovering it annoying how academics tin complain thus much well-nigh pressures that are taken for granted elsewhere.

It would live on a mistake, however, to retrieve that the seat inward to a greater extent than or less REF league tabular array told you lot all you lot needed to know to evaluate the character of enquiry inward a department. The REF releases a wealth of data, as well as going beyond the headline number (usually the GPA score) tin live on informative. In the latest practice departments were evaluated nether 3 headings: outputs, environs as well as impact. Details well-nigh what is involved for each category tin live on flora here.

Outputs, which has the highest weight inward the total (65%), looks at the character of the 4 best recent publications of each submitted fellow member of staff. The primal give-and-take to Federal Reserve notation hither is ‘submitted’. Influenza A virus subtype H5N1 department/university tin direct non to submit all its staff to the REF, as well as yesteryear non submitting staff that a department/university considers are good below average it tin heighten its GPA grade (if it gets its assessments right). So to the extent that staff are non submitted, the GPA volition overestimate the average character of the enquiry done inward that department. As I said, league tables usually precisely await at the GPA grade [1].

To to a greater extent than or less it may seem foreign that this is allowed, but at that spot are arguments to justify it. Departments gain pay a meaning financial penalisation for leaving staff out - they exclusively acquire coin for submitted staff. To acquire a guide to the total amount of character adjusted enquiry done inward a department, precisely multiply the GPA grade yesteryear the number of people submitted (called ‘power’ yesteryear THE).

The conclusion well-nigh whether non to submit a fellow member of staff is an agonising 1 [2] that involves many hard trade-offs. To the private non beingness submitted it is a nasty slap inward the face. For the department, the perceived benefits inward getting a higher seat inward GPA based league tables may outweigh the financial terms of non submitting staff members. Decisions on this forepart gain vary from academy to university, as well as from subdivision to department: inward economics, compare the tertiary as well as 4th columns of the tabular array below.

Although it exclusively counts for 15% of the total GPA score, the ‘environment’ heading may live on of special involvement to potential PhD students. It is based on a number of dissimilar criteria, including the number of PhDs, the back upwards provided for research, as well as enquiry income from exterior grants. Only 3 economic science departments had all elements of environs judged to live on of the highest (4*) character this time: UCL, LSE as well as Oxford.

Impact is a novel category, accounting for 20% of the total. It is based on instance studies where enquiry has engaged alongside public, private as well as tertiary sector organisations, or straight alongside the public. For example, 1 of Oxford’s instance studies for economic science was my ain operate on financial councils. Influenza A virus subtype H5N1 quick await at the results suggests that this novel chemical cistron has had a meaning influence on the overall results. In economics, for example, the exclusively subdivision where all the submitted instance studies were judged to live on of the highest character was Bristol. So patch Bristol exclusively came 12th= on published outputs, a strong affect as well as environs grade lifted them to sixth inward the overall ranking.

As alongside whatever evaluation system, at that spot are hard judgements to brand on the details, as well as these tin atomic number 82 to possibilities to ‘play the system’. Chris Bertram focuses on 1 special number at Crooked Timber. Each iteration of the assessment practice attempts to alter the details of the rules to avoid this, exclusively to let to a greater extent than or less novel possibility to exploit the system. Partly every bit a result, later each practice many academics feel that at that spot must live on a ameliorate as well as less fourth dimension consuming way to justice the character of enquiry produced yesteryear private academics or departments, but peradventure the fact that nosotros move along returning to the same basic physical care for suggests otherwise.

REF 2014 results: economic science as well as econometrics

University	GPA Score	No. of staff submitted	Eligible staff	Power	% 4* Outputs	% 4* Environ.	% 4* Impact
UCL	3.78	37	45	139	70	100	92
LSE	3.55	51	56	182	56	100	87
Oxford	3.44	84	97	289	43	100	64
Cambridge	3.42	27	38	92	55	13	50
Warwick	3.41	42	52	142	43	38	60
Bristol	3.32	19	25	62	22	63	100
Essex	3.25	33	40	108	29	63	20
Edinburgh	3.14	18	28	55	31	50	13
Royal Holloway	3.11	14	23	45	35	0	60
Nottingham	3.05	35	46	107	20	13	18
UEA	3.04	14	22	43	20	0	20
Surrey	3.01	21	25	62	27	13	0
Queen Mary	2.98	24	31	73	20	13	13
York	2.93	28	46	82	14	0	40
St. Andrews	2.92	21	31	60	24	0	0
Manchester	2.89	33	45	96	11	13	40
Glasgow	2.86	24	30	68	18	0	0
Sussex	2.84	17	24	49	15	0	37
Exeter	2.78	25	31	68	13	25	13
Birmingham	2.78	24	27	67	8	0	27
Southampton	2.70	22	28	59	22	0	10
Birkbeck	2.60	25	32	65	10	0	0
Leicester	2.59	22	29	58	19	0	0
Sheffield	2.58	15	26	38	8	0	40
Aberdeen	2.48	19	26	48	5	0	0
City	2.44	14	26	33	17	0	20
Kent	2.32	22	26	51	3	0	13
Brunel	2.20	26	29	58	2	0	0

Note that many economic science departments are assessed nether Business as well as Management, as well as are non included here. Sources: columns 2,3 as well as 5: Times Higher Education, column 4: HESA, columns 6-8, REF.

[1] THE publishes an alternative academy broad ranking (aggregated across departments) that multiplies the GPA yesteryear the proportion of staff submitted, but that implicitly gives the enquiry of non-submitted staff a grade of zero, which is probable to live on every bit good extreme. It is ameliorate to precisely Federal Reserve notation either the ability score, or the proportion of staff submitted. Approximate information on the number of staff eligible for submission yesteryear subdivision tin live on flora here.

[2] This is based on my ain experience at my previous university, where I was enquiry manager for the school. Thankfully I direct maintain played no role inward these decisions at Oxford!