Hexbyte Glen Cove
‘Where researchers are in their career, be they exciting newcomers or distinguished professors, can also have a strong effect on current metrics.
‘The Epsilon Index is a new way to reduce systemic biases in assessing researcher quality via citations by providing career-stage, gender, and opportunity corrections to citation-based performance metrics,’ he says.
The tool is freely available as a ready-made app—simply punch in a few data for a sample of researchers from open-source databases like Google Scholar, and it does the heavy lifting to produce the result, enabling comparison of researchers at any stage of their career and from any discipline on the same scale.
It’s a boon for anyone who wishes to use an objective metric to rank researchers, be it for grant applications, job interviews, promotions and awards, or even as a staff performance indicator.
The approach to develop and test the ε-index was itself an exercise in multi-disciplinarity and specifically engineered to ensure gender balance.
The tool was tested and refined through collaboration on sample data with an archaeologist—Assistant Professor Dr. Stefani Crabtree (Utah State University/Santa Fe Institute), a geologist and vertebrate palaeontologist—Professor Kate Trinajstic (Curtin University), a chemist—Professor Justin Chalker, a microbiologist—Professor Bart Eijkelkamp, a palaeontologist—Professor John Long, an ophthalmologist—Professor Justine Smith, and an evolutionary biologist—Professor Vera Weisbecker (all Flinders University).
The sample comprised 480 researchers with Google Scholar profiles, stratified evenly into eight disciplines (archaeology, chemistry, ecology, evolution and development, geology, microbiology, ophthalmology, palaeontology), three career stages (early-, mid-, late-career), and two genders.
Professor Justine Smith, who’s been a ‘superstar of STEM’ actively promoting women’s engagement in science, technology, engineering and mathematics, hopes the ε-index will make a difference to how women in science are perceived, and in turn encourage more women to enter the sciences.
‘More accurately assessing the contribution of women to the sciences and celebrating their successes is pivotal to encouraging future generations of girls into the sciences; as the saying goes, “you can’t be what you can’t see”. The ε-index gives fairer and greater visibility to women’s achievements, and that it does so across a range of disciplines makes it especially beneficial,’ Professor Smith says.
How it works
The new index is a ranking algorithm that can be standardised across disciplines, can be corrected for career breaks, and provides a sample-specific threshold that can determine whether individual performance is greater or less than expected relative to the other researchers in a sample.
- the number of citations acquired for the researcher’s top-cited paper
- the i10-index (number of articles with at least 10 citations)
- the h-index, and
- the year in which the researcher’s first peer-reviewed paper was published.
The tool also provides a simple method to scale the index across disciplines with variable citation trends (ε′-index) to enable fairer comparison of researchers in different areas.
The ε-index easily allows benchmarking of subsets of researchers into women-only or men-only to adjust the threshold such that the ranks are more comparable between these two genders. Alternatively, dividing the genders and benchmarking them separately followed by a combined re-ranking effectively removes the gender bias in the ε–index, which is difficult or impossible to do with other ranking metrics.
‘No ranking metric is perfect, but the ε-index is a big leap forward in overcoming bias and giving a clearer, fairer picture of research performance across the board,’ Professor Bradshaw says.
Corey J. A. Bradshaw et al, A fairer way to compare researchers at any career stage and in any discipline using open-access citation data, PLOS ONE (2021). DOI: 10.1371/journal.pone.0257141
Banishing bias across disciplines, genders and experience—new tool for fairer research metrics (2021, September 10)
retrieved 11 September 2021
Read More Hexbyte Glen Cove Educational Blog Repost With Backlinks —