The phrase "we will need to undertake further work..." crops up several times in HEFCE's consultation document on the new Research Excellence Framework.
The biggest area that is still up for debate is how to deal with the arts, humanities, social sciences, mathematics and statistics. For these, HEFCE will "develop a quality assessment regime involving a light touch form of peer review informed by quantitative indicators". But, so far, "we have not undertaken significant development work on this," says HEFCE. The consultation is seeking preliminary input on how this system should work. The time line for this is not as tight though, it only needs to be in place by 2013, to inform funding from 2014.
Other areas that are still under development include how researchers should be assigned to institutions and subject groups, the algorithm to produce the quality indicator based on citations, and indicators of research income and numbers of research students will be used. HEFCE is also looking for suggestions for quantitative indicators that can be used to capture user value and the quality of applied research.
HEFCE has commissioned further technical advice, to report in early 2008, on some of the data and methodological issues, and will run a "substantial" pilot exercise next year.
All this shows that the REF is still very much a work in progress, and although the general framework has been decided, there is still room for academics to tweak it here and there to try and deal with the problems that are bound to crop up.
But the timetable is tight, as HEFCE acknowledges:
November 2007 to February 2008 - Consultation on key elements of the framework and on bibliometric indicators. In parallel, further work on developing bibliometric techniques.
March to August 2008 - Substantial pilot of proposed approach
Autumn 2008 - Further consultation and decisions on the framework and indicators to be used for the science-based disciplines
Early 2009 - Launch of full bibliometrics exercise for science-based disciplines
November 2009 - Output of bibliometrics exercise available for use in funding; decisions on new funding approach to be phased in from 2010
From late 2009 - Consult on light touch peer review to run in 2013