Do Scholars Trust Their Altmetrics?

The next speaker in this AoIR 2019 session is my colleague Kim Osman, presenting outcomes from our research project in collaboration with The Conversation and the Cooperative Research Centres Association in Australia. We are interested in assessments of the public value and impact of scholarly work, which are also increasingly demanded by the governments that fund scholarly research. Slides here:

Increasingly, platforms like The Conversation as well as social media are also critical to the engagement with and impact of scholarly research, and there has been a rise in the development of scholarly ‘altmetrics’ that complement conventional bibliometrics for traditional academic publications. The role of amplifier platforms like The Conversation is especially interesting here: these take content by academics, ensure that it is accessible for lay audiences, and publish such content under licences that enable republication and circulation through mainstream news and media outlets.

The Conversation’s own metrics show that this model works: its content is republished by major media outlets from ABC News to CNN, as well as by specialist science news outlets like IFLScience. Altmetrics are critical to assessing such circulation across multiple sites and outlets, and also enable scholars to understand their own impact in the world as well as plan their further dissemination efforts.

So how do scholars understand their impact? Our project interviewed scholars in Australia and Canada at various career stages, some of whom had and some of whom had not published in The Conversation. In Australia, where it started in 2011, the site is now seen as a default platform for publishing research insights to the general public, and scholars as well as their universities are also using its impact metrics for career planning purposes; the same is now true for a number of the other countries where The Conversation has been established. In Canada, where it is only just getting started and not yet embedded into scholarly processes, this is not yet the case, and scholars publish there mainly out of an intrinsic motivation to bring their work to the attention of a larger public, whatever the metrics that result from doing so.

Scholars do recognise the distinctions between bibliometrics and altmetrics in this. They are generally resigned to the use of bibliometrics as a means of impact assessment, but views towards altmetrics are more diverse at this stage; such alternative impact metrics are not yet consistently used in impact assessment across academia. Where altmetrics are used more thoroughly, an understanding of their value and a trust in their reliability is also growing, however.

Scholars see such altmetrics as useful especially because they enable them to trace more clearly where their work is going – how many people read it, and where it is being republished. The value for effort ratio is also positive here – a few hours of work writing a short Conversation article can expose this work to a much larger number of readers than several months of work on a scholarly article.