Exploring Automated Visual Analysis Tools

And the final speaker in this ICA 2024 conference session is Ahmed Al-Rawi, who is interested in assessing the automated visual analysis of news and social media images. His study draws on the GDELT dataset of news content metadata from around the world, which (using the Google Vision API) also OCRs, labels, and detects logos in broadcast TV content. He extracted some 813,000 news items from the GDELT CloudVision dataset, and from this drew some 10,000 items addressing mis- and disinformation. He extracted similar data from the Google AI TV Explorer dataset.

Separately, he took 12,000 images addressing racism from Canadian users on Twitter over the platform’s existence, and analysed these using the Amazon Rekognition tool. This produces moderation labels: flagging hate symbols, rude gestures, or visually disturbing content, for instance.

GDELT data indicated that CNN was ahead of MSNBC and Fox News in terms of its misinformation-related TV footage; in online news, the Daily Mail was well ahead in its visual coverage of misinformation. Trump, coronavirus, pandemic, COVID, and Biden were key terms related to misinformation here; surprise, joy, and sorrow were the most common types of sentiments associated.

Both tools appeared to perform adequately in their labelling of such visual content; however, their features differ. The full quality of the performance of these tools still needs to be examined in much more detail to determine their usefulness in research, however.