Measurement/QoE

Overview

Traditional linear broadcast has established a set of consumer expectations around watching video that have not translated well to online streaming. Consumers want a “broadcast quality” experience regardless if they watch on their television or their mobile phone. But what metrics do video distributors need to gather and measure? What data will identify problems with online streaming and offer opportunities to fix underlying issues? And how do distributors implement the mechanism to capture that information? In this working group, participating members focus on answering these questions by providing a framework for the collection, analysis, and reporting of measurement/QoE metrics.

Objectives

The Streaming Video Technology Alliance Measurement/Quality of Experience (QoE) Working Group has three objectives. First, to identify streaming video metrics to gather; second, to establish guidelines on how to calculate those metrics; and third, to create best practices on implementing a system to capture QoE and other measurement data.

Chairs

Current Projects

Proofs of Concept

This group has not initiated or completed any proofs of concept.

Presentations

Below are presentations delivered during working group sessions. Click on the presentation name to open it (many files are for members only and will require authentication) and a presenter name to view their profile. Note: if a presenter’s profile is not public, it will redirect to their LinkedIn page.

Best Practices for End-To-End Workflow Monitoring

  

There is no presenter.

Best Practices in End-to-End Monitoring and Touchstream's End-to-End Vizualisation

  

Presented by Brenton Ough

CTA Common Media Client Data Presentation

  

Presented by Thomas Edwards

Lisbon QoE Meeting Agenda/Presentation

  

Presented by Thomas Edwards

Measurement/QoE May 10 2018 Meeting Presentation

  

Presented by Thomas Edwards

QoE WG Session

  

Presented by Antonio Gentile

Presented by Christopher Kulbakas

Presented by James Hall

Presented by Josh Evans

Presented by Thomas Edwards

QoE Working Group Session

  

Presented by Antonio Gentile

Presented by Christopher Kulbakas

Presented by Josh Evans

Presented by Thomas Edwards

QoE WG Project Updates

  

Presented by Christopher Kulbakas

Presented by James Hall

Presented by Josh Evans

Presented by Thomas Edwards

QoE WG Video Quality Tags Update

  

Presented by Christopher Kulbakas

SVA Q1 2019 Meeting - QoE WG - End-to-end Workflow Monitoring

  

Presented by Christopher Kulbakas

QoE WG Session

  

Presented by Antonio Gentile

Presented by Chris Lemmons

Presented by Christopher Kulbakas

Presented by Gordon Lai

Presented by James Hall

Presented by Josh Evans

Presented by Rajeev RK

Presented by Thomas Edwards

Documents

Published

Best Practices for End-to-End Workflow Monitoring

As the video streaming industry continues to mature, QoE expectations continue to rise in importance both as a value proposition to the consumer as well as a differentiator between competing video streaming services. Yet, consistent, systematic, standardized, objectively measured, and business-rules-driven QoE assurance continues to be an elusive goal. This document outlines factors that impact consumer QoE, sheds light on some of the underlying challenges to diagnosing QoE impact and provides a set of best practices for end-to-end QoE monitoring in video streaming workflows.

Key Network Delivery Metrics

This document describes key network delivery metrics for streaming Internet video. The Streaming Video Alliance Quality of Experience (QoE) working group (WG) developed these metrics as a common language to describe effectiveness of network delivery. Although many more metrics could have been documented, these particular metrics represent the most common and important metrics according to the QoE WG group membership. It is expected that these metrics can be used across the streaming video ecosystem to provide guidance that can lead to enhancement of QoE.

Draft

Note that to download draft documents, you must be an employee of an active member company with an ARO account. If you are part of a member company and don’t yet have an ARO account, you can request one here.

This group does not currently have any draft documents.

Blog Posts

SVTA Distributed Tracing: Working Towards Improved Observability and QoS/QoE

As the streaming tech stack has grown in complexity, ensuring a high-quality viewing experience has become increasingly challenging. Part of that is the nature of a distributed architecture frequently assembled from third party services and technologies as well as just more moving parts. But part of it is also a lack of well-established standards which meet the specific needs of streaming content providers. The combination of these two issues results in operational “blind spots” when tracing issues through the workflow. Because of the critical importance of data within streaming operations, any blind spots can significantly increase the Mean Time to Diagnose (MTTD) and, more importantly, the Mean Time to Resolve (MTTR). When these operational streaming metrics go up, QoE and viewer satisfaction goes down often resulting in loss of revenue and/or increased churn. There is a need then, for a standardized approach to trace data across the different vendors in the streaming video technology stack with an obvious starting point: CDNs.

Participating Companies