Video Quality Measurements with the MOS

How video quality models can help you compare and improve your service

Download the factsheet

Why QoE and User Experience Matter

From Key Performance Indicators to Real User Experience

Download the factsheet

Surfmeter – Robust solutions to monitor QoS & QoE

Quality measurements for ISPs and OTTs.

Download the factsheet

Are you still watching? Streaming Video Quality and Engagement Assessment in the Crowd

In this paper, we present first results from a large-scale crowdsourcing study conducted with AVEQ’s measurement technology. We compare YouTube, Amazon Prime Video and Netflix across five major national ISPs in Germany. We not only look at streaming performance in terms of loading times and stalling, but also customer behavior (e.g., user engagement) and Quality of Experience.

Download the publication

Scientific Publications

Here are selected, peer-reviewed publications that we have (co-)authored:

  • Raake, A., Borer, S., Satti, S. M., Gustafsson, J., Rao, R. R. R., Medagli, S., List, P., Göring, S., Lindero, D., Robitza, W., Heikkilä, G., Broom, S., Schmidmer, C., Feiten, B., Wüstenhagen, U., Wittmann, T., Obermann, M., & Bitto, R. (2020) Multi-Model Standard for Bitstream-, Pixel-Based and Hybrid Video Quality Assessment of UHD/4K: ITU-T P.1204. IEEE Access, 8, 193020–193049.
  • Herglotz, C., Robitza, W., Raake, A., Hossfeld, T., & Kaup, A. (2023). Power Reduction Opportunities on End-User Devices in Quality-Steady Video Streaming. 15th International Conference on Quality of Multimedia Experience (QoMEX).
  • Braun, F., Rao, R. R. R., Robitza, W., & Raake, A. (2023). Automatic Audiovisual Asynchrony Measurement for Quality Assessment of Videoconferencing. 15th International Conference on Quality of Multimedia Experience (QoMEX).
  • Herglotz, C., Robitza, W., Kränzler, M., Kaup, A., & Raake, A. (2022). Modeling of Energy Consumption and Streaming Video QoE using a Crowdsourcing Dataset. 14th International Conference on Quality of Multimedia Experience (QoMEX).
  • Robitza, W., Ramachandra-Rao, R. R., Göring, S., Dethof, A., & Raake, A. (2022). Deploying the ITU-T P.1203 QoE model in the wild and retraining for new codecs. Proceedings of the 1st Conference on Mile-High Video, 121–122.
  • Robitza, W., Rao Ramachandra Rao, R., Göring, S., & Raake, A. (2021). Impact of spatial and temporal information on video quality and compressibility. 2021 13th International Conference on Quality of Multimedia Experience, QoMEX 2021, 65–68.
  • Schwarzmann, S., Hainke, N., Zinner, T., Sieber, C., Robitza, W., & Raake, A. (2020). Comparing fixed and variable segment durations for adaptive video streaming. Proceedings of the 11th ACM Multimedia Systems Conference, 38–53.
  • Robitza, W., Dethof, A. M., Göring, S., Raake, A., Polzehl, T., & Beyer, A. (2020). Are You Still Watching? Streaming Video Quality and Engagement Assessment in the Crowd. 12th International Conference on Quality of Multimedia Experience (QoMEX 2020). Athlone, Ireland.
  • Rao, R. R. R., Göring, S., Robitza, W., Raake, A., Feiten, B., List, P., & Wüstenhagen, U. (2020). Bitstream-based Model Standard for 4K/UHD: ITU-T P.1204.3 – Model Details, Evaluation, Analysis and Open Source Implementation. 12th International Conference on Quality of Multimedia Experience (QoMEX 2020). Athlone, Ireland.
  • Kara, P. A., Robitza, W., Pinter, N., Martini, M. G., Raake, A., & Simon, A. (2019). Comparison of HD and UHD video quality with and without the influence of the labeling effect. Quality and User Experience, 4(1), 4.
  • Robitza, W., Ahmad, A., Kara, P. A., Atzori, L., Martini, M. G., Raake, A., & Sun, L. (2017). Challenges of future multimedia QoE monitoring for internet service providers, (June). Multimedia Tools and Applications.
  • Rao, R. R. R., Göring, S., Vogel, P., Pachatz, N., Juan, J. V. V., Robitza, W., List, P., Feiten, B., Raake, A. (2019). Adaptive video streaming with current codecs and formats: Extensions to parametric video quality model ITU-T P.1203. In Human Vision and Electronic Imaging (HVEI). Burlingame, CA.
  • Asan, A., Robitza, W., Mkwawa, I.-H., Sun, L., Begen, A. C. (2018). Optimum Encoding Approaches on Video Resolution Changes: A Comparative Study. In IEEE International Conference on Image Processing (ICIP 2018). Athens.
  • Robitza, W., Göring, S., Raake, A., Lindegren, D., Heikkilä, G., Gustafsson, J., List, P., Feiten, B., Wüstenhagen, U., Garcia, M.-N., Yamagishi, K., Broom, S. (2018). HTTP Adaptive Streaming QoE Estimation with ITU-T Rec. P.1203 – Open Databases and Software. In 9th ACM Multimedia Systems Conference. Amsterdam.
  • Robitza, W., Kittur, D. G., Dethof, A. M., Göring, S., Feiten, B., & Raake, A. (2018). Measuring YouTube QoE with ITU-T P.1203 under Constrained Bandwidth Conditions. In 10th International Conference on Quality of Multimedia Experience (QoMEX 2018). Santa Margherita di Pula.
  • Singla, A., Robitza, W., & Raake, A. (2018). Comparison of Subjective Quality Evaluation Methods for Omnidirectional Videos with DSIS and Modified ACR. In Human Vision and Electronic Imaging (HVEI). Burlingame.
  • Singla, A., Fremerey, S., Robitza, W., Lebreton, P., & Raake, A. (2017). Comparison of Subjective Quality Evaluation for HEVC Encoded Omnidirectional Videos at Different Bit-rates for UHD and FHD Resolution. In ACM Multimedia (ACMMM 2017), Thematic Workshops. Mountain View.
  • Asan, A., Robitza, W., Mkwawa, I., Sun, L., Ifeachor, E., & Raake, A. (2017). Impact of Video Resolution Changes on QoE for Adaptive Video Streaming. In IEEE International Conference on Multimedia and Expo. Hong Kong.
  • Robitza, W. (2017). Towards Behavior-oriented Quality of Experience Assessment for Online Video Services. In ACM International Conference on Interactive Experiences for TV and Online Video (TVX 2017), Doctoral Consortium. Hilversum.
  • Raake, A., Garcia, M.-N., Robitza, W., List, P., Göring, S., & Feiten, B. (2017). A Bitstream-based, Scalable Video-Quality Model for HTTP Adaptive Streaming: ITU-T P.1203.1. In 9th International Conference on Quality of Multimedia Experience (QoMEX 2017). Erfurt.
  • Robitza, W., Garcia, M.-N., & Raake, A. (2017). A Modular HTTP Adaptive Streaming QoE Model – Candidate for ITU-T P.1203 (“P.NATS”). In 9th International Conference on Quality of Multimedia Experience (QoMEX 2017). Erfurt.
  • Singla, A., Fremerey, S., Robitza, W., & Raake, A. (2017). Measuring and Comparing QoE and Simulator Sickness of Omnidirectional Videos in Different Head Mounted Displays. In 9th International Conference on Quality of Multimedia Experience (QoMEX 2017). Erfurt.
  • Kara, P. A., Robitza, W., Raake, A., & Martini, M. G. (2017). The Label Knows Better: The Impact of Labeling Effects on Perceived Quality of HD and UHD Video Streaming. In 9th International Conference on Quality of Multimedia Experience (QoMEX 2017). Erfurt.
  • Robitza, W., Kara, P. A., Martini, M. G., and Raake, A. (2016). On the Experimental Biases in User Behavior and QoE Assessment in the Lab. In Fifth IEEE International Workshop on Quality of Experience for Multimedia Communications (QoEMC). Washington.
  • Kara, P. A., Robitza, W., Martini, M. G., Hewage, C. T., and Felisberti, F. M. (2016). Getting Used to or Growing Annoyed: How Perception Thresholds and Acceptance of Frame Freezing Vary Over Time in 3D Video Streaming. In IEEE International Conference on Multimedia and Expo (ICME). Seattle.
  • Dethof, A. M., Robitza, W., and Garcia, M.-N. (2016). StreamSim: A Video Streaming Simulation Toolchain for Unreliable Transport Mechanisms. In Eighth International Workshop on Quality of Multimedia Experience (QoMEX). Lisbon.
  • Robitza, W., and Raake, A. (2016). (Re-)Actions Speak Louder Than Words? A Novel Test Method for Tracking User Behavior in Web Video Services. In Eighth International Workshop on Quality of Multimedia Experience (QoMEX). Lisbon.
  • Robitza, W., Schönfellner, S., and Raake, A. (2016). A Theoretical Approach to the Formation of Quality of Experience and User Behavior in Multimedia Services. In 5th ISCA/DEGA Workshop on Perceptual Quality of Systems (PQS). Berlin.
  • Robitza, W., Garcia, M.-N., and Raake, A. (2015). At Home in the Lab: Assessing Audiovisual Quality of HTTP-based Adaptive Streaming with an Immersive Test Paradigm. In Seventh International Workshop on Quality of Multimedia Experience (QoMEX). Costa Navarino.
  • Garcia, M.-N., Robitza, W., and Raake, A. (2015). On The Accuracy of Short-Term Quality Models for Long-Term Quality Prediction. In Seventh International Workshop on Quality of Multimedia Experience (QoMEX). Costa Navarino.
  • Robitza, W., and Hlavacs, H. (2014). Assessing the validity of subjective QoE data through rating times and self-reported confidence. In 2014 Sixth International Workshop on Quality of Multimedia Experience (QoMEX) (pp. 297–302). Singapore: IEEE.
  • Robitza, W., Pitrey, Y., and Hlavacs, H. (2013). The NappingPlayer — Projective Mapping Experiments on Android Tablets. In Fourth International Workshop on Perceptual Quality of Systems (PQS). Vienna.

Newsletter

Sign up and find out what’s happening in the world of video streaming.