TALK: Content-Aware Adaptive Playback for Low-Latency Live Sports

This talk can be attended in-person and virtually. To obtain a virtual joining link, please contact Dr Laura Toni.

Abstract
There are two main factors that determine the viewer experience during the live streaming of sports: latency and stalls.

Latency should be low and stalls should not occur. Yet, these two factors work against each other and it is not trivial to strike the best trade-off between them. One of the best tools we have today to manage this trade-off is the adaptive playback speed control, which allows the streaming client to slow down the playback when there is a risk of stalling and accelerate the playback when there is no risk of stalling to maintain the target latency.

While adaptive playback generally works well, the artifacts due to the changes in the playback speed should preferably be unnoticeable to the viewers. However, this mostly depends on the part of the audio/video content subject to the playback speed change.

In this talk, we present the latest enhancements on the content-aware playback speed control (CAPSC) algorithm we developed for dash.js along with the encoder support for FFmpeg. The enhanced version of the CAPSC algorithm keeps the playback speed close to the nominal speed (1x) during the important parts of the content and gracefully adapts the playback speed changes to provide a more pleasant viewing experience, all in an automated fashion.