Blogs

Submission to ERA and EI assessment review [10/2020]

By Jenny Pham posted 24-04-2024 15:29

  



RACI commentary on the call for public consultation on the review of the  ERA and EI assessment

Executive Summary 

It is the view of the RACI that the ERA and EI schemes are used by the Government solely as a means to demonstrate the effect of their commitment to supporting University research and how this is leading to “well above world standard” performance; without the Government actually providing a sufficient level of genuine support that is needed to deliver such achievements in real terms.

Who we are

The Royal Australian Chemical Institute (RACI), has been serving the profession of chemistry and supporting research in chemistry for more than 100 years. Most of the leading University researchers working at the forefront of efforts to expand Australia’s scientific capability in the discipline of chemistry are members of the RACI.

The RACI's feedback

The RACI is of the opinion that a cost benefit analysis of the ERA assessments of University standards, especially in the discipline of chemistry, would reveal that these exercises have a significant cost and deliver minimal benefit over and above what can be discerned from other internationally recognised metrics.

The RACI is of the view that the institutional time costs in undertaking ERA exercises might be avoided if the ERA relied more on information in recognised electronic databases (ORCID, Scopus, Web of Science, etc) such that less time and effort was spent optimising the data submission and writing unquantifiable impact statements.

A clear issue with focussing on metrics, from whatever source, means that the ERA does not capture important research occurring at discipline boundaries. The highest quality research and the research that is most likely to lead to significant impact is interdisciplinary research. This is especially true for research in chemistry which can often translate into other fields such as physics, environmental science, engineering, biology and medicine. Interdisciplinary research, by its very nature, does not suit the level of institutional data curation that is required to achieve good outcomes in the current ERA assessment process.

Of particular concern to the RACI is the comment in the ERA Review Consultation Paper (3.2.1) that ‘The results of ERA have shown that over time university research has improved in quality (see the ERA outcomes on the ARC Data Portal)’. The RACI views this comment to be either naïve or perhaps deliberately misleading, but certainly incorrect. Whilst undoubtedly the metrics represented in the Data Portal show improvement, it is the view of the RACI that this comes from improved institutional experience at how best to present their research data.  

Research quality has not been improved by the ERA exercise and indeed the effort spent on undertaking the ERA assessment comes with sector-wide and institutional-level costs that negatively impact on achieving quality research.

With regards to the comment in the same section of the Consultation Paper that ‘ERA results were used widely by universities for strategic planning’ it was noted that some University recruitment decisions have become skewed towards maximising scores in the ERA, rather than making better strategic appointments for solid academic reasons. This has significant impact on teaching capability and can lead to poorer outcomes for students at both undergraduate and postgraduate levels. 

The RACI believes that for the ERA review process to have any merit is must accurately assess and justify the cost the ERA imposes on the sector. This cost analysis must not only determine the impact on the institutions involved, but also the costs to the ARC which manages the assessment process. Increasing resources are dedicated to each assessment cycle for no return on that investment and indeed only a negative impact on University finances and staff time. These costs and time commitments are not trivial. It is not unusual for research intensive institutions to have >50,000 outputs to process for ERA. The review must also assess the value of repeating the ERA assessments so frequently (4 times in one decade). It is not clear how the proposal to move to annual data collection would be an improvement.

Conclusion

In conclusion, the RACI believes that the current model of the ERA is flawed. The review of the ERA must justify the costs of the ERA exercise in terms of the impact on the sector and the ARC itself, and in terms of the number of potential grants or fellowships that could have been awarded with these resources and how not supporting quality research leads to enhanced research outcomes.

Mr Roger Stapleford (CEO)

On behalf of the Board of the Royal Australian Chemical Institute:

Dr. Vicki Gardiner (President)

Prof. Steven Bottle (President -elect)

Dr. Renate Griffith (Hon Secretary)

Dr. Matthew Sykes (Treasurer)

Dr. Cynthia Cliff (Board member)

Mr Greg Ewing (Board member)

Mr David Springer (Board member)

Ms Maree Stuart (Board member) 

0 comments
3 views

Permalink