Do Journalists Trust Journalistic Metrics?

It’s Thursday morning, and after the fabulous opening keynote by Bronwyn Carlson last night the AoIR 2019 conference at QUT in Brisbane is now getting started properly. This morning I’m in a panel on metrics in journalism, academia, and music that begins with a paper I’ve been involved in, and which my colleague Aljosha Karim Schapals will present. The slides are here:

Our key question here is whether journalism metrics are trustworthy enough to be used in editorial decision-making. This is part of a larger project on the future of journalism in a post-journalism era, which explores news work and news engagement beyond the traditional confines of the journalism industry. There are now many more modes of fact-based discourse beyond conventional journalism itself.

This also steps beyond the culturally pessimistic narrative of journalism in crisis, and instead highlights the transition and transformation of journalistic and news work in the current era. Innovative fact-based news outlets, in particular, are emerging and evolving into new forms of news work, and they are different in their forms and styles, in their professional practices, and in their engagement with users. This also influences how they affect the democratic societies they operate in.

Our study draws in part on a large number of interviews with journalists across Australia, Germany, and the UK – and in part this focusses on their engagement with journalism metrics and the role of such metrics in editorial decision-making. The emergence of such metrics can be understood as a quantitative turn in journalism, which measure elements such as page views, reading time, arrival and departure pathways, and so on, and sometimes combine these raw metrics into single engagement scores for individual stories.

Stories that are seen to be successful on the basis of such metrics may then be further highlighted and pursued in journalistic work, while others might be backgrounded and will not receive further investigation in future. All of this can happen very quickly, with story promotion and positioning happening in real time within minutes of publication.

But journalists and other news workers are also concerned about how such metrics might be exploited or gamed, and about how such metrics might oversimplify editorial decision-making processes. These concerns also play out differently in different news media environments, across different countries, depending on their exposure to journalism metrics – for instance, a number of German journalists expressed significant concern about the potential power of metrics in influencing editorial directions.

Others take a more balanced view, which sees metrics as a useful tool but not as directly influencing the story line-up in a particular outlet. Some suggest that there’s a fight going on in trying to achieve a balance between maximising audience exposure in order to keep advertisers happy, and maximising the quality of news content even when such content is not always popular with audiences.

Some journalists, in fact, suggest that the metrics data are most useful once they are unshackled from simple performance indicators – when they can be used in more sophisticated ways to understand audiences rather than just as a way to chase audience eyeballs.

What emerges from these interviews already is a new comparative perspective on audience metrics in journalism, across different cultural and industry settings. There is overall a prevailing sense of ambiguity in engaging with these metrics; journalists perceive a mix of threats and opportunities in their use of metrics, but how they achieve a balance between these diverging trends is a continuing conundrum between journalistic mission and economic imperative that has yet to be solved. Journalists trust metrics only to a point.