Developing More Advanced Television Engagement Metrics for Twitter

The final AoIR 2015 session is our panel on social television, and starts with a co-authored paper presented by Darryl Woodford (slides to follow soon below).

Darryl begins by noting that raw social media engagement numbers for television are useful only to an extent: they are usually not normalised to account for specific factors, and simply offer raw quantities.

Nielsen SocialGuide's Twitter engagement statistics for social media follow that pattern, for example, and obviously shows on major TV channels do better than those on niche cable channels. Beamly's social media rankings are skewed by the Twitter terms they track: any tweet containing the letters 'yr' is counted as engagement with The Young and the Restless, for example, which is obviously wrong.

A new approach to deal with this needs to draw on more advanced statistical modelling and analytics techniques. Darryl's work draws on the sabermetrics techniques developer for sports analytics, and he thus calls it telemetrics: such techniques seek to take into account underlying factors such as the dimensions of the ballpark and the quality of the opposing team in assessing a player's average performance.

For television, there are similar underlying factors that need to be taken into account. Darryl's Weighted Tweet Index adjusts for the network, month of year, time of day, and other factors, for example; the Excitement Index measures the volatility of social media conversation; the Hype Score covers engagement between episodes of shows and uses this as a predictor of engagement during the broadcast.

The WEI builds on seasonal models for broadcast seasons. Tweet volumes tend to be greater at the start and end of seasons, as well as around mid-season finales and premieres. The EI builds again on sporting metrics, which are able to determine the closeness of a match as an indicator of how exciting it is; reality TV shows often have a greater excitement index than drama.

There is some industry resistance to such data – TV is still programmed to some extent by gut feeling rather than by using actual data on viewer engagement, and the data are often mistrusted especially if they tell programmers and producers something other than their own views. However, it has been shown that data on social media hype before new movie releases are a very good indicator of box office takings.

Hype between shows is also a useful metric for viewers themselves, if they want to engage through social media with other viewers. Darryl's Hypometer provides such predictive stats back to viewers, showing them the likely social media engagement with upcoming shows to enable them to make a more informed choice about what to watch. Similarly, such predictive metrics are also successful in predicting Big Brother evictions.