Designing Better Fact-Checking Reports

The final speaker in this ICA 2024 conference session is the great Damiano Spina, who begins by highlighting the current challenges to the global information environment. In the IPIE survey of disinformation experts, politicians, social media platforms, and governments were seen as the most problematic sources of mis- and disinformation.

Computational methods are critical to addressing these problems. They can assist in fact-checking by engaging in claim detection and assessing check-worthiness, supporting evidence retrieval, and enabling veracity and truthfulness classification. But they can also assist in the reporting and dissemination of fact-checks. This does not remove the role of the human in the loop, however, but it enables the greater involvement of non-experts in such processes.

Fact-checkers currently take a broad range of approaches to presenting the results of their work, but these are mostly text- and screen-based. How could this be done instead in an auditory environment, for instance (e.g. using smart speakers)? Stripped back, what are the most important design elements in a fact-checking report? Damiano’s project explored this through a participatory workshop, to examine key presentation elements and strategies and opportunities to customise fact-checking reports for diverse audiences.

Themes emerging from this related to the structure of the report, trust towards the verdict, and personalisation of the presentation. These were then used to generate mock-ups own fact-checking presentations that were then tested in crowdsourcing experiments through Mechanical Turk. Clear patterns emerged from this that should help with the development of future presentation modes, but these were also related to the personal attributes of different users: these included age, previous experience with misinformation, trust in science, and political leaning.