HSRAANZ Response to the ARC Engagement and Impact Assessment Paper

The HSRAANZ’s response to the Australian Research Councils Engagement and Impact Assessment Consultation Paper argues that a key issue with the impact measurement of Medical and Health Sciences research through the analysis of citations in peer reviewed journals is that Public Health and Health Services research is generally mostly relevant to the jurisdiction in which it is conducted and therefore research undertaken in Australia is less likely to be cited in other jurisdictions than laboratory-based or clinical Medical and Health Sciences research.

Analyses of citations in peer reviewed journals do not reflect impact on healthcare policy and practice and on patient outcomes and population health within Australia. Policymakers do not tend to report in peer reviewed journals, but are more likely to cite research that influences policy and practice in non-peer reviewed government reports and other publications found in the grey literature.

Consideration should therefore be given the expansion of citation analyses to include the grey literature and the inclusion of case studies or exemplars of research that has influenced policy and practice in the assessment of research impact and engagement in the Medical and Health Sciences division.

For a more general research assessment exercise, it might be more appropriate to count significant examples of research translation to active researchers. Analogous to the analysis of average citation rates across research papers within an ANZSRC group, the average frequency and grade of policy or practice impact and engagement across eligible researchers assigned to each ANZSRC group could be estimated.

Assessments of research impact based on citation analyses and case studies or exemplars could be reported separately.

A single, pooled measure of research impact would be more consensus-based and subjective than the numbers driven assessment of citations (noting that quantitative citation measures may provide false quality assurance by not explicitly accounting for differences in research applicability across jurisdictions).