How Consumers Perceive Video Quality and How to Measure | MediaMelon
At the heart of the debate on how best to improve the quality of video streams for...
MediaMelon’s SmartSight QBR (aka SmartPlay) is a streaming optimization solution that reduces bandwidth use by 35% or more and improves QoE while working with your existing ABR content without re-encoding. As you’ll read in this post, SmartSight QBR works by analyzing the quality of every segment in an ABR package using a objectively researched technique that produces an iMOS hintfile, which is used by the SmartSight enabled video player to improve the logic that selects how video segments are retrieved.
Rather than simply retrieving the highest bitrate stream the viewer’s bandwidth supports, operators can set quality preferences on a per-content or per-subscriber basis. This saves bandwidth and allows the player to buffer ahead for hard-to-encode scenes that require a high-bitrate segment to achieve target quality.
The principle of QBR works for live and VOD delivery, supports static and dynamic packaging and dynamic ad insertion, and doesn’t increase latency. This makes SmartPlay a practical, affordable and easy to implement solution for any operator looking to control costs or quickly reduce outbound bandwidth for COVID-19 related limitations.
This is the first of three posts on SmartSight QBR, and it will cover some of the unique iMOS and QBR technology - and how they combine to optimize the business approach to streaming. The second post in the series will discuss how to implement SmartSight QBR and its ROI. The final post will be a case study from Telecine, Brazil’s leading movie streaming service.
How Most ABR Technologies Work
Most ABR technologies implicitly try and optimize video playback by seeking to retrieve the highest quality stream in the encoding ladder that the viewers device bandwidth can support. Particularly for operators using a fixed bitrate ladder, with a lot of less detailed content, the quality of the top rungs of the bitrate ladder may be perceptually indistinguishable from lower rungs, so the operator is paying for the extra bandwidth of each and every delivery without improving the quality of experience experienced by the viewer. This could be termed economic madness!
Figure 1. Typical ABR operation (on top) is bandwidth driven. SmartSight QBR operation is quality driven.
SmartSight QBR (Streaming Optimization Solution) Overview
SmartSight QBR has two fundamental,, but very simple, elements that must be added within your production workflow and playback architecture. The first is the iMOS Content Analyzer, which uses a MediaMelon technology called iMOS (for Mean Opinion Score) to analyze every ABR segment of any encoded video asset and score it on a simple quality scale from 1-5.
After analyzing the ABR package, the iMOS Content Analyzer produces at least two “hintfiles;” for the video asset one for small screens and one for larger ones. The iMOS process is trainable via machine learning, and during installation, MediaMelon will tune the algorithm for the operator’s content and encoder, a process that continues to be enhanced periodically as the system collects more asset data.
The second SmartSight QBR integration is via the SmartSight SDK installed within each device player, which is accomplished using formal APIs and callbacks offered by all commercial and open-source players. The SmartSight SDK here is the same SDK used to instrument all stream delivery and generate rich analytics via SmartSight QoE and SmartSight Ads.
QBR Enhanced Video Player Operation
During playback, the QBR enhanced video player targets the quality level specified for that subscriber, which can be driven by business rules related to subscriber level, content type, and other variables. Operationally, the QBR player uses the hintfile to retrieve segments that meet the specified quality level, which is often less costly in bandwidth used than those offered by the higher bitrate ladder rungs, particularly for operators using a fixed bitrate encoding ladder.
Figure 2. QBR downloads lower bitrate streams in simple scenes to
reduce bandwidth and buffers ahead to download complex scenes to improve quality.
In operation, SmartSight QBR reduces bandwidth costs by downloading the stream that meets the quality level specified for that subscriber, as opposed to the highest bitrate stream that the viewer’s bandwidth can support. QBR enhancement improves quality by looking ahead for complex scenes that require a higher bitrate to maintain target quality. If there is sufficient buffered content on the player, the player will download these scenes in advance, improving the overall quality of the session.
The QBR process has three operating modes, Quality, Bitrate, and Cost Save, that allow the operator to tune the system to meet their specific QoE and cost saving goals. Both Quality and Bitrate modes improve quality and reduce bandwidth usage, with Quality mode delivering higher quality and lower bandwidth savings and Bitrate mode delivering greater bandwidth reductions and less quality improvements. In Cost Save mode, QBR prioritizes bandwidth savings and never buffers ahead to improve quality. In all modes, retrieving the lowest possible bandwidth stream that meets the target quality level saves bandwidth and reduces the possibility of re-buffering and other bandwidth-related issues.
Beyond these three operating modes, operators can set video quality delta targets that minimize the perceptible quality shifts in the video, which can degrade viewer QoE. You see the results in the Statistics screen in Figure 3, in the upper right of the video player window (click the figure to see at full resolution). The Segment uplift of 13.79% indicates the number of retrieved segments that improved video quality scores. This uplift positively impacts two other statistics shown on the screen. First is the 40.17% increase in quality consistency, which means many fewer perceptible quality shifts during playback. Second is the 1.1 increase in the Least iMOS score from 3.4 to 4.5, meaning that the quality of the lowest quality segments viewed by the subscriber jumped from 3.4 to 4.5, which should eliminate most, if not all, short-term quality glitches. Both statistics impact viewer QoE which QBR significantly improved in this case.
Figure 3. SmartSight QBR improves Quality Consistency and Least MOS score as well.
This short (4:19) YouTube video further demonstrates how SmartSight QBR works to improves overall quality and QoE:
SmartSight QBR and Content Adaptive Encoding
Content Adaptive Encoding (CAE) techniques, also called per-title encoding, adjust the encoding settings for the ABR package to reflect the encoding complexity of the video content. There are many CAE technologies and they all operate differently, so it’s impossible to create a blanket statement about how CAE impacts SmartSight QBR performance and benefits. That said, as you’ll see in the next section, all of the real-world data shown from Telecine’s SmartSight QBR installation was generated with video encoded using AWS Elementals QVBR, a form of per title encoding, and SmartPlay still produced a bitrate savings of 42.45% while slightly improving quality.
In addition, SmartSight QBR has several key advantages over CAE, most notably that you don’t have to re-encode your entire library to use it. Also, where CAE is fixed, the QBR process is dynamic and network sensitive, so if you need to adjust bandwidth and quality on-the-fly, as many producers have had to with COVID-19, you can. With CAE, you can’t.
Now that we’ve covered how SmartSight QBR works let’s have a look at the benefits it delivers.
Analytics Detail the Bandwidth Savings and Quality Improvements
SmartSight QBR is built on the same platform as SmartSight QoE which produces the data you see in the figure below. The metrics shown on this and the following pages are from 24 hours of Telecine operation in Bitrate mode while encoding with AWS Elemental’s QVBR, which is a form of per-title encoding.
Figure 4. SmartSight QBR shares a common platform with SmartSight QoE, here showing
the performance from 24 hours of real-world operation.
As you see, high-level metrics compare QBR-enhanced service delivery to default ABR operation and tracks data bandwidth saved, quality density increase, the bitrate reduction produced by SmartPlay, and iMOS change. The dashboard also graphically displays the difference between QBR and ABR performance over the tracked period and presents quality buckets for the content actually delivered.
Data saved and bitrate reduction are obvious and need no definition. As the name suggests, Quality Density is a measure of the video quality achieved over the bandwidth that you deliver.
Consider an example. Assume you encoded a static newsreader with a fixed bitrate ladder. For most of the video, the 1080p rung would deliver little perceivable quality over the 720p rung but the data rate might be 2 Mbps higher. In this case, the 1080p rung has a much lower quality density than the 720p rung. If SmartPlay can meet the quality target by delivering the 720p stream, it has increased the Quality Density, and the quality bang for your bandwidth buck.
The iMOS change is the difference between the quality of the QBR delivered segments and what the ABR player would have delivered without enhancement. In this case, despite a bitrate reduction of over 40%, iMOS quality actually improved slightly.
Finally, iMOS bucketing groups the iMOS quality scores into the five MOS buckets and shows the number of segments delivered in each bucket. During the covered period, the vast majority of segments delivered were in the top two quality buckets. This, the improvement in quality density, and the positive iMOS change of 0.23 shows the SmartPlay increased viewer QoE while reducing bandwidth by 42.45%.
File-by-File Results
Operators can also view the results on an asset-by-asset basis as you see below, again real-world stats for 24 hours of operation. As you can see, columns show ABR bitrate, QBR bitrate, and bitrate reduction.
Figure 5. File-by-file performance.
The next two columns show the Quality Density of the ABR and QBR streams with QBR significantly higher, indicating a much more efficient use of bandwidth. For these files, which were sorted for highest average QBR iMOS, ABR iMOS scores were slightly higher, though as you saw in the previous figure, SmartPlay improved iMOS overall by 0.23. The last column shows the Quality Density increase, measured as the percentage increase from ABR quality density (2.82) to QBR quality density (7.59).
Telecine’s Real-World Savings
Telecine is a joint venture of Globo Group with Paramount Pictures, Universal Pictures, Metro-Goldwyn-Mayer and The Walt Disney Company, and also streams content from Sony Pictures and Warner Bros.
I present the Telecine case study in the third SmartPlay blog post and the data below as a teaser. For perspective, Telecine implemented SmartPlay in Bitrate mode and streams video encoded using AWS Elemental’s QVBR per-title encoding technology.
Figure 6. One week of real-world Telecine results.
The chart above shows one week of Telecine results. As you can see, SmartPlay saved 79.26 TB of bandwidth over those 7 days while slightly improving quality as shown by the 0.24 iMOS change.
This short (3:23) video shows SmartPlay analytics in action using 4 days of actual results from the Telecine installation.
Summary
Let’s summarize the SmartSight QBR key features, benefits, and advantages:
- Reduces bandwidth costs, particularly for delivery to mobile devices
- Improves subscriber Quality of Experience by:
– Improving overall MOS scores
– Reducing buffering events
– Improving quality consistency
– Improving the Least MOS Score - Key implementation features
– Works for both live and VOD
– Works with existing VOD content – no re-encoding required
– Operators can choose between three operating modes to best match operating goals
– Operators can customize quality level on a per-subscriber or per-content basis
– Operational efficiency improves over time via machine learning
