Manual and Automatic Video Coding Approaches

Bremen.
The next speakers in this ‘Doing Global Media Studies’ ECREA 2010 pre-conference session are Tobias Kohler and Jan Müller, whose interest is in the computer-based analysis of television footage from multiple countries. This is part of a larger study into automated TV content processing, covering German, US, Brazilian, and Chinese television content. The material examined here, in particular, are the annual year-end review broadcasts. There are substantial format differences here, of course (in length as well as original placement in the broadcast schedule – German clips are longer stand-alone review shows, while US content was broadcast during news bulletins).

This content was coded by a team of students, annotating videos according to a variety of criteria (presence of key actors, setting, topics, etc.). This was done segment by segment, since a universal coding sheet which was applicable across some very different segment types proved very difficult to develop. This allows for a precise analysis – for example, comparing the amount of broadcast time devoted to media actors from specific countries.

The international nature of the content required the use of a team of coders with different language abilities; as a result, detailed intercoder reliability testing was also necessary. Even then, some notable problems remained – not least when coding for features occurring relatively rarely. Working with and responding to coder feedback was essential for the research team in this process – for example, coders were eventually instructed to process the video in a number of run-throughs, focussing on different aspects of the video. Other problems included coder drop-out, and the subsequent addition of new coders, and similar projects need to be prepared for such problems (and plan for feedback rounds).

This work by the coders was compared to the results of an automatic shot detection tool, and the project team also tested automatic face recognition and scene detection tools. The accuracy of such tools is generally better than random – but not necessarily by much. To address this, the research team developed the Kivi video coding application, which combines manual and automatic aspects.