Using Screen Captures in Digital Media User Research

The next speaker in this session at the P³: Power, Propaganda, Polarisation ICA 2024 postconference is Andrew Fitzgerald, whose interest is in the use of longitudinal mobile screenshot data in research. This is another response to the emerging challenges in doing research on the power of platforms – platform infrastructures continue to change in their interface design and affordances, algorithmic curation affects what actual content users encounter, access approaches to platform data keep evolving, and new platforms emerge all the time. This means that we need independent data collection methods, beyond what the platforms themselves do or do not provide, that can cope with all of these issues.

This might also better capture how users are receiving and truly engaging with the content they encounter on digital and social media platforms. Andrew is exploring this through an approach described as ‘screenomics’, and part of the interest here is especially in how users encounter and respond to unexpected events – for instance major crises and terror attacks.

The approach here is to take regular screenshots of a participant’s devices – including the screens of both computers and mobiles –, which reveal how events are co-constructed by audiences and the media; this draws substantially from cultural studies influences and explores the micro-social environments that constrain and mold the construction of media meanings as text and technology. Such situated meaning-making re-articulates contexts, discourses, territorialities, identities, and structural power relations as well as our perceptions of them.

Focussing on surprise events, this examines in situ how participants and their ecosystems respond to and make sense of terror attacks, and how their habitual media use unfolds as they become aware of these events. It addresses the processes of media fragmentation across channels as well as media consolidation in fewer key devices, and traces the many small acts of engagement that take place in a user’s information milieu as part of their mediatised rituals. Platforms in this sense serve as preparatory media connecting diverse groups, incite action, and legitimise movements that respond to and engage with developments in the world.

But how do we make sense of the vast amounts of data that such data gathering approaches produce? One approach is through optical character recognition that enables keyword searches of the data; this is effective for instance for major terror events where key terms such as the location can be easily identified.

Several key genres emerged from this: most centrally, liberal micro-rituals, which focus on compensatory connection (e.g. including sympathy flags) and self-care, engage with the micro-political scale of self-referential taking care of each other, and foreshorten temporality to a focus on the present or immediate future; and far-right rituals, which highlight the futility and superficiality of liberal micro-rituals and instead call for more forceful responses against perceived enemies, gleefully anticipate future conflicts, and in doing so centre on a future-oriented temporality that makes possible the conflicts it anticipates.

Habit, interface design, and recurrence also play a key role: ritually checking engagement metrics updates on social media will surface certain events, for instance. Far-right mis- and disinformation also emerged as a key source of content on terror attacks for a subset of participants.

This is of course a very invasive method as it captures so much about participants’ online activities, so privacy concerns need to be taken extremely seriously; they are recompensed for their participation, but this too might affect the group of people who participate. Participants may also engage in self-censorship if they remain aware that their content is being captured. And such surveillant approaches to data gathering may play into especially the far right’s suspicions about academic research. Further work is using the open-source ScreenLife Capture tool which is now available.