News

Society News

EPS response to REF consultation on UoA4 costing exercise

As a Learned Society, the EPS was invited to consider the current REF consultation exercise, and submit a response to a variety of questions. The deadline was 15 October 2018. The Officers do their best to express a view in response to consultations that fits with their understanding of Members’ interests and attitudes, and with the overall field of experimental psychology, bearing in mind the constraints in how such exercises are timed. The broader strategy of the Society can be, and often is, actively debated at AGM.

At a number of points in the REF cycle, the EPS has attempted to offer constructive suggestions and views to inform decision making at the discipline-level, whilst refraining from commenting on many issues that fall within the remit of institutional decisions or practices. The EPS committee agreed to comment specifically on the proposal that REF introduce a costing exercise for UoA4. For transparency, we report on the EPS submission below.

12a. How feasible do you consider to be the approach set out at paragraphs 267 to 271 for capturing information on the balance of research activity of different costs within submitting units in UOA 4? (300 word limit)

The Experimental Psychology Society opposes the costing of research outputs from UoA 4. Our concerns about feasibility relate to measurement, coherence, and risk of perverse incentives.

We recognise that research costs vary across projects. This is the case in many UoAs, and therefore it is not clear why costing of research outputs should be applied idiosyncratically in UoA 4.

The consultation document proposes costing of methods used in research, rather than the actual total cost of a piece of research, in the sense of its Full Economic Costing (FEC).  It is easy to think of experiments with low-cost methods but high FEC, or vice versa.  The UK science funding framework is based on FEC, and it is hard to defend using different costing frameworks for research funding (FEC) and research assessment.  Two key factors that affect the FEC of research are not mentioned: one is researcher time investment, and the other is size of a dataset.  Both have a positive effect on research quality, and particularly on research reproducibility.  The current proposal risks rewarding small, irreproducible studies with high infrastructure costs, and penalising careful reproducible studies with lower infrastructure costs. This would be a major scientific mistake, and runs contrary to the current, consensual focus on improving the reliability of research.

REF focuses on the evaluation of research outputs; it is not tuned for the assessment of cost, an input measure, or for the assessment of outputs relative to inputs. We endorse this focus on outputs, and see a serious risk that conflation of input and output measurement will encourage institutions to believe they can gain by driving up the cost base of research. It is likely to encourage the use of expensive research methodologies, which is not the same as the REF’s stated aim of encouraging excellent research.

12b. Are the examples of high cost and other research activity sufficiently clear to guide classification? (300 word limit)

Based on the examples, we do not have confidence that classification will provide valid and reliable indicators of cost. One problem relates to change in costs over time. If an institution invests in expensive equipment, the costs of the first output (based on the initial capital) will be different from the next (in which the initial capital is no longer directly relevant). Another problem relates to distribution of effort. Use of expensive methods often involves collaborative teams, often working internationally. If we understand the consultation document correctly, a UK researcher who submits a paper with cutting-edge fMRI data collected at another university will bring money into the university that employs them, not the university that bore the cost of the neuroimaging facility.  (The Experimental Psychology Society advised against such ‘portability’ in a prior REF consultation.) We see no reliable way of directing the rewards towards the institutions that actually incurred the costs, which brings the risk of a perception of unfairness.

Published papers will not generally provide enough information to make accurate estimates of the cost of the methods used.  Misconceptions abound: brain stimulation, for example, is mentioned as a high-cost method, yet sample sizes are small, analyses are often simple, and the equipment can be cheap.  In animal studies, stains, reagents and vectors vary dramatically in cost, but the cost will probably be known only to those who buy them. Assessors may not have information to make accurate estimates of cost, and should not therefore be asked to do so.

12c. Please provide feedback on any specific points in the guidance text as well as the overall clarity of the guidance. (300 word limit)

Para 270 proposes a classification into three bands based on the percentage of research activity that is classed as high cost. This may encourage departments to select outputs based not on the quality of the research but on the cost of the infrastructure in order to ensure classification in a preferred band. This has the potential to be divisive, and runs counter to the central aim of the REF: to reward excellence.

The Experimental Psychology Society agrees that advanced research methods are important, and REF should encourage rather than discourage investment in key research infrastructures, such as animal labs and neuroimaging.  In our view, this should be done through the *environment* assessment, rather than through output assessment.  That way, funding for high-cost infrastructures is guaranteed to go to the HEI that bears the cost, which is not the case for output assessment.  We recommend that the REF team consider how evaluation of methods-based facilities in the environment assessment can take account of the research productivity of methods-based facilities, as well as their existence.  HEIs should be rewarded for facilities that produce useful research, not for facilities that are poorly used.

Forthcoming Workshop

Research Workshop: The psychology of upper-limb prosthetic use. To be held at Manchester Metropolitan University, 19 March 2019 – organisers Gavin Buckingham, Same Vine and Greg Wood

The human hand is remarkable, and the loss of an upper limb represents a huge challenge for one’s sensorimotor system. To address this challenge, we will bring together psychologists, prosthetists, bioengineers, and prosthetic-users to present and discuss work which furthers our understanding of the psychology of upper limb prosthetic use.

This day-long workshop will take place on the 19th of March 2019 at Manchester Metropolitan University, immediately prior to Trent International Prosthetics Symposium which is being held in the same city.

Further details, including registration and abstract submission, can be found here:  https://prostheticsworkshop.wixsite.com/event

Society News

Small grants and study visits: open for open science!

The EPS committee have agreed to make explicit in the guidance for small grant and study visit  applicants that we welcome proposals that specify relevant open science practices. These awards are used for a wide variety of purposes and without being restrictive, we are simply encouraging applicants to frame their proposals in ways that help convince an audience of their potential value to applicants and the community.

Forthcoming Workshop

Research Workshop: Memory malleability over time. To be held at University of Kent. 10-11 January 2019

Traditional research into the neurocognitive mechanisms that underlie long-term memory has typically focused on how we encode information from a single exposure into memories that are then retrieved once in a test phase. Through these type of paradigms, we have learned a lot about the brain mechanisms that underlie memory for unique experiences. However, in real life, very similar experiences are often encoded multiple times, and memories are often retrieved/reactivated repeatedly. Recent evidence suggests that repeated encoding and retrieval can change memories in different ways. The aim of this workshop is to bring together researchers who investigate long-term memory malleability over time from different theoretical angles and using different cognitive neuroscience methodologies.

 Further details can be found here: https://kentmemoryworkshop2019.weebly.com/

Organisers: Zara Bergstrom and Robin Hellerstedt

Journal News

New video material of Andy Young’s Bartlett paper

Andy YoungIn what we hope will be the first in a series of commissioned video resources, Andy Young talks about the research and the ideas that are central to the 45th Bartlett Lecture on “Faces, people and the Brain”. This is now hosted here

We hope this resource will be useful to everyone from students to colleagues, and provide a helpful supplement to the paper itself. We should also note that, as part of the Journal partnership with SAGE, the Bartlett paper is free to view.

Andy’s Society Lecture is also available to members here

Member News

Obituaries

It is with great sadness that we record the following deaths:

Anne Treisman, February 2018. Anne was a recipient of the EPS Frederic Bartlett Prize award in 1987. Her Bartlett lecture can be found here:

http://journals.sagepub.com/doi/pdf/10.1080/02724988843000104

 

Peter Venables, April 2017. Peter was elected and served as Honorary Secretary of the EPS in 1965-1967 and President of the EPS in 1969-1970.

Peter Venables

Annette Karmiloff-Smith, December 2016. Annette was a recipient of the EPS Frederic Bartlett Prize award in 2012.

 

Anyone interested in writing an obituary in respect of any of the above, for the website/newsletter is welcome to contact the Hon Secretary for more details.

Journal News

Update to journal format

QJEP has made changes to the existing format of two types of journal article – comments and book reviews.

First, comments will be allowed to alleviate the problem that readers have few options to raise their concerns (or support) about an article published in QJEP. Comments are short (1000 words at most), deal with articles published in QJEP or with general issues faced by psychological researchers, and will be published at the end of an issue. Normally they will not go to reviewers but be decided upon at the Editorial level. Given our experiences at the Meetings of the Experimental Psychology Society, it is our conviction that such commentaries can become a vital and very informative part of the journal.

We also discovered that many readers miss the Book Reviews section, which had to be dropped a few years ago because the publication lag was becoming too long. Now that the journal has many more pages (and could further extend if needed), there is an opportunity to revitalise that part. Philip Quinlan kindly accepted to be the new Book Review Editor of the journal and readers are invited to send him suggestion of must-be-reviewed books. More importantly, readers who want to help making this section a success, are invited to send in their names as possible reviewers (please also include your subjects of expertise/interest).