Video Quality Models from ITU-T — An Overview
Originally recorded for a hybrid conference, we’d like to share this topic. Our CEO Werner Robitza has recorded a short summary about the current standardization efforts at ITU-T for monitoring HTTP adaptive streaming quality. These include the P.1203 and P.1204 recommendation series. Those models have been (co-)developed by members of Ilmenau University of Technology and AVEQ in coorperation with Deutsche Telekom — and so we’re happy to give you a small introduction.
P.1203
The idea of the P.1203 model was “to find one model that rules them all” — providing one model which uses technical data as input and delivers, as a response, the user’s opinion of a video session for up to 5 minutes length. That score includes the combination of intial loading delay, stalling effects, and quality fluctuations throughout the video.
Members at ITU-T individually performed subjective experiments where video degradations were shown to viewers, to better understand the opinion of real users who viewed the provided streams and rated the quality on a scale of 1 (bad) to 5 (excellent). The studies ran independently from each other. Each member company developed their own models to predict the expected user ratings based on the technical impairments. In a competition, the best model contributions were selected and united into a single set of models. That was the moment P.1203 was born.
P.1203 consists of individual modules that are responsible for the prediction of short-term audio and video, and longer-term overall quality. You can find an open-source reference implementation of P.1203 on GitHub — which we co-maintain — and read a joint report if you want to look at the model in more detail.
P.1203 — Extended
As standardization is sometimes a long-term process, new technologies have been released in the meantime, e.g., the VP9 or AV1 codecs which have been primarily used by YouTube and thus account for a considerable portion of video traffic. As the original P.1203 models are based on the technologies that existed at the time of the conduction of the experiments, those new codecs were not supported. That’s why Ilmenau University of Technology and AVEQ have implemented an extension for the P.1203.1 model to add coefficients for those other codecs. Based on our extended software, the standard can be used directly for OTT services without risking calculating the wrong scores.
Our extended version not only adds support for new codecs, but also includes additional Key Performance Indicators that are useful for benchmarking the performance of a video streaming setup.
P.1204
P.1204 is a set of standards to extend the capabilities of short-term video quality models such as the ones used by P.1203. Different model types were developed by ITU-T members. They are described in an open-access paper.
One of the P.1204 models is a bitstream-based model (P.1204.3), which can deliver highly accurate predictions of quality — without needing access to the source video. Based on initial tests by TU Ilmenau, it can reach performance equal to VMAF, despite not knowing the original source video’s quality. Our tech report covers that topic as well and shows you detailed performance graphs.
