Extracting User-Generated Multimedia Metadata

Leuven.
The post-lunch session here at EuroITV 2009 is the one that my paper is in as well, so I've refrained from sampling the fine Belgian beers available during lunch. We start with Marcelo Manzato, whose interest is in the peer annotation of multimedia content. Digital television makes it easier for user to interact with multimedia content, and this is happening for example through YouTube and similar services, of course, as well as through the proliferation of mobile devices (and the necessary adaptation and personalisation of content for such contexts).

Personalisation requires a selection of content based on personal information; the extraction of such semantic information from multimedia content is very difficult, however. Multimedia metadata can be authored on a hierarchical basis, on the content provider side during or after content production; peer-level annotation, on the other hand, is done by users themselves and is therefore less systematic.

Hierarchical metadata creation is time-consuming and expensive, and what metadata are being created constitutes a subjective choice by a limited number of staff. Peer-level metadata are richer and more useful for providing personalised services to the users who provided annotation. Such annotation does not follow a defined, restrictive set of metadata terms, however.

What is required, then, is an architecture which extracts peer-level metadata and processes such information to provide a reliable metadata source. For multimedia, this may involve the selection of individual frames or scenes that are chosen by the user; these can then be further analysed on an automatic basis in order to group them with similar content. Users may also provide further, more detailed information (such as regions of interest in the frame). Metadata content may also be able to be shared with fellow users, users may provide links to similar or related material, or directly interact within the metadata creation interface.

In Marcelo's project, such annotation would not necessarily take place, by the way, not on a computer screen, but uses mobile or hand-held devices which are used while watching television. The project has developed media object models that combine MPEG4 media content directly with MPEG7 descriptors. Additionally, users have personal profiles that track their personal interests and preferences.

Technorati : EuroITV 2009, annotation, euroitv09, metadata, multimedia, semantic, television
Del.icio.us : EuroITV 2009, annotation, euroitv09, metadata, multimedia, semantic, television