Mobile Video Quality Measurements with Surfmeter

While video streaming traffic is growing globally, the amount of time spent watching video on mobile devices and smart TVs is increasing year on year, according to various studies. Desktop usage remains stagnant. To understand the quality of video streaming on mobile devices, we need a dedicated solution that gives us detailed insight into the performance and quality of experience of video delivery. Read on to learn more about our new Surfmeter Video Quality SDKs for Android!

Our Surfmeter set of tools is a comprehensive suite of applications that allow you to automatically measure video quality from various devices. By using Docker, you can run Surfmeter almost everywhere — in the cloud, on virtual machines, or on automated measurement devices in the user’s home.

But what if you are an OTT provider packaging your own video streaming application for mobile devices? Or what if you offer a performance testing application but lack support for video streaming measurements? We now have you covered.

Surfmeter Video Quality SDK

The latest product in our portfolio is a mobile video quality software development kit (SDK) compatible with Android SDK v23 and above. It enables mobile network operators, broadcasters, system integrators and application developers to measure video streaming performance on Android mobile devices, both in the lab and in the field.

Our SDK actually consists of two similar but different plugins:

• A client analytics adapter to track video streaming performance from existing players
• A standalone video quality test solution

For example, you can use our mobile client analytics SDK to analyze every view made by your customers — just like other analytics frameworks.

Our quality test SDK could instead be used to provide a “video speed test” capability for your application, in case you would like to have your customers troubleshoot their connection. Or you could wrap our quality test SDK around a dedicated testing application. Maybe you are a provider of speedtest tools but you don’t have video capability yet? We can help you!

How do the Measurements work?

For our client analytics SDK, we attach to your player as it’s running. We collect everything that we need to determine the quality of streaming — you only need to call the SDK once. The data is sent in the background and will be stored on our servers for you to analyze later.

The quality testing SDK instead provides its own video player that can be configured freely to play and DASH, HLS, or Smooth Streaming manifest. DRM is of course also supported. The player is based on the popular ExoPlayer. The SDK starts the video and collects technical measurements such as initial loading times, video codecs and resolutions, bitrates, and any buffering events.

At the end of the measurement — no matter if done through client analytics or a dedicated test — we calculate a unique video streaming quality score: the Mean Opinion Score (MOS). On a scale from 1–5, it shows you how well the streaming works. Or why it doesn’t. Because we track all underlying Key Performance Indicators (KPIs), we know what causes bad video quality. Be it a too low video resolution, or frequent stalling events.

Various Use Cases

If you are an OTT provider and use ExoPlayer already, you can simply attach our SDK to the existing player and obtain analytics data from it. This works just like many other analytics frameworks. With Surfmeter, you’re very flexible. Our SDK is lightweight and allows you to pass various metadata to the measurements, so you can later easily map the results to other data you might have. For instance, you’ll get information on the used ISP or geographical location. Or you supply us with the video asset IDs, user IDs, or other markers that you can later link to your internal data sources.

If are a network provider or integrator and you want to use our SDK to perform a video speed test, you can configure the integrated player according to various assumptions about the video service to test. Here, you can use the video quality data results to pinpoint any problems to a certain geographic point. Correlate video streaming performance with specific locations — which can be very helpful for drive testing, for exapmle. The SDK is compatible with 5G networks, enabling customers to accurately monitor video QoE on both 5G networks and legacy networks.

Demo

Here’s a demo application integrating our own video quality testing SDK. We’ve configured it to use a test manifest from a streaming provider. The test is set to run for a few seconds and report the results afterwards.

When you launch the test, a view of the video player can be shown, although that is not required for the test itself. It can also be run in the background in case you want to present something else to the user, or if you want to run everything in a scheduled fashion.

When the test is done, the results are calculated and shown to you in the form of a JSON object that can be easily parsed and stored/forwarded to your own backend.

Above we can see the the detailed breakdown of the video streaming session itself in the “segments” object, including used codecs, bitrates, and resolutions. On the left you can see a medium performance streaming session that only reached 720p despite a 1080p HD video being available. On the right, the session was able to load the 1080p video, and it was able to do so faster (see the “stalling” attribute).

Now, these data are quite technical and available on the device or through an API, but what if you need everything visualized centrally?

Analytics

Our client analytics SDK also includes a server-side analytics backend — providing a comprehensive dashboard of all collected network and video quality metrics as well as relevant KPIs such as video start-up times and average bitrates. You can create custom dashboards and visualizations for all important metrics, and get alerted when certain thresholds are reached.

For example, here is a breakdown of initial streaming delay for various clients plotted over time. These are devices that stream regularly. You can easily spot the outliers with low performance, where startup time reaches 8 seconds or more.

Any visualization is possible, and thanks to the collected metadata, you can break down the charts by whatever explanatory factor you need — be it the CDN, the client operating system version, or the used ISP.

Test the Video Quality SDK now!

Our Surfmeter video quality SDKs are the perfect tools to measure video streaming performance on mobile devices, both in the laboratory and in the field. Use it to analyze your mobile video applications, to offer a “video speed test” function to customers, or to get detailed analytics information on network performance.

At AVEQ, we proud ourselves to be flexible to work with. We provide our SDKs in various versions and offer differnt integration options to fit any kind of use case. Just get in touch and we’ll discuss your needs. Try out our a demo now and see how our SDKs can help you improve your networks or streaming video quality!