Understand the 5 playback metrics Mux uses to measure quality of experience.


Overall Viewer Experience Score is a metric that describes the overall Quality of Experience (QoE) of video streaming in a single number. A score of 100 means every viewer had a satisfying experience, and a score of 0 means that every viewer had a frustrating experience.

Overall Viewer Experience Score is based on four other Viewer Experience Scores, which each describe one of the four elements of video streaming performance: Playback Success, Startup Time, Playback Smoothness, and Video Quality.

Viewer Experience Scores are useful as a way to describe streaming performance from the perspective of an end viewer and not just from the perspective of system-level metrics. The Viewer Experience Scores are QoE (Quality of Experience) metrics, which describe the actual end-user experience of watching video. This is in contrast to QoS (Quality of Service) metrics, which describe a specific system’s performance without reference to user experience.

For example, "Downscaling Percentage" is a useful metric to track when it comes to QoS, since a high Downscaling Percentage means a service is delivering more video than necessary to fill the player or display. But users don't see downscaling, so Downscaling Percentage doesn't come into QoE when describing video quality. However Upscaling Percentage is a QoE metric because upscaling creates visual artifacts in the video, affecting the viewer’s experience.

Both QoE and QoS metrics are important for different purposes. QoS metrics are useful when troubleshooting specific system problems, while QoE metrics are useful when evaluating technologies or prioritizing problems to improve.

### Formula
Each individual video view is given an experience score of 0-100, and the Overall Viewer Experience Score is calculated by averaging the experience of all video views. Each view’s experience score is measured by first calculating a score between 0 and 100 for each of the elements of streaming performance (Playback Success, Smoothness, Startup Time, and Quality). Those scores are then averaged using higher weights for the more impactful elements of the viewer experience.

The weights are created by measuring the relative importance of each element of the experience. Mux conducted user surveys and research across millions of video views on the relative tradeoffs between increasing one metric at the expense of another. For example, you can increase Quality at the expense of Startup Time and vice versa. However, doing so would be a bad idea because Startup Time is more valuable than Quality. Generally, we found that Playback Success is the most important, followed by Smoothness then Startup Time, and finally Quality.

We want to make sure the Overall Score captures these complex relationships between metrics since developers may decide to make certain tradeoffs in order to improve their QoE scores. So, instead of just averaging the scores, Mux creates a set of tradeoff scores first and combines that into the overall score. The exception is Playback Success, which is a multiplier applied to the Overall Score.


The Overall Viewer Experience Score is defined as:
$$
Availability\ Score * \frac{T_{Sm, Q} + T_{Sm, Su} + T_{Su, Q}}{3}
$$
where
$$
T_{Sm, Q} = \text{Tradeoff(Smoothness, Quality)}
$$
$$
T_{Sm, Su} = \text{Tradeoff(Smoothness, Startup)}
$$
$$
T_{Su, Q} = \text{Tradeoff(Startup, Quality)}
$$

This way of combining the metrics is a more accurate representation of the viewer’s quality of experience, and increases the usefulness of Mux Data’s scoring system.

### Use this metric to:
* See trends in your overall viewer experience over time
* Begin prioritizing your efforts in areas of your platform (devices, regions, etc.) that have the lowest score and can use the most improvement


Overall Viewer Experience Score

Overall Viewer Experience


Playback Success Score focuses on whether a video played back successfully.

Successful playback includes two components:

* Did the video play without an error?
* Did the user actually get to playback, or did they exit before playback started?

### Formula
**Playback Success Score** is fairly simple. A failure that ends playback is a `0`, while a video that plays through without failure is `100`. A view that is terminated by the viewer before playback starts (an “Exit Before Video Start,” or EBVS) is given a score of `50`.  EBVS views that occur in less than 1 second is given no score.

```
    100: successful playback
    50: exit before video start
    0: playback failure
    N/A: exit before video start <1 second
```

Why are EBVS views given a score of 50? The reason is that while exits can often point to streaming problems (e.g. the video took too long to load), some percentage of exits before video start are normal. A user might click the wrong video or see a link to a different video they want to watch more. If a view is abandoned in less than one second, we assume the video start was unintentional or programmatic and exclude those play attempts from the score.

### Use this metric to:
* Understand how playback failures impact the overall viewer experience
* Compare playback success performance to other areas of viewer experience
* Find areas where playback success can be optimized and improved


Playback Success Score


The Playback Failure Percentage metric gives the percentage of video views that failed to play due to an error. Playback failures can happen at any point during video playback, causing the playback to end prematurely.

### Use this metric to:
* Understand where playback failures are happening most frequently
* Watch for spikes in playback failures due to new errors

Visit the Errors section to see which specific errors are happening the most frequently.


Playback Failure Percentage


Viewers will sometimes abandon a video (e.g. close the page/app or click the back button) because it is taking too long to load. The Exits Before Video Start Percentage metric captures how frequently this happens.

For this metric we count the number of video views where the viewer clicked play (or the player began to autoplay) but the video never began to play back (Video Startup Time was never recorded), excluding playback failures.  We then divide that number by the total number of video views.

### Use this metric to:
* Watch how changes in Video Startup Time directly impact viewers abandoning the video
* Compare players and understand if factors other than startup time may be causing viewers to leave, for example visual cues like loading indicators and poster frames.

Note: Viewers may leave for reasons other than long startup times, for example deciding that they clicked the wrong video or clicking on a related video. Before becoming concerned with your platform’s specific percentage you should attempt to improve your Video Startup Time and see how that impacts your Exits Before Video Start.


Exits Before Video Start

Playback Success


Startup Time Score describes how happy or unhappy viewers are with startup time. Longer startup times mean lower scores, while shorter startup times mean higher scores. Once startup time reaches a certain point (around 8 seconds), we begin to decrease the rate of score decay since additional seconds of startup becomes less impactful for long startup times.

### Formula
$$
\frac{8}{8 + startup\_time\_in\_seconds} * 100
$$

This score decreases at a greater rate after 500ms of starting up.

Note that EBVS views do not receive a Startup Time Score.

Example values:
* 400 ms: 95
* 2 seconds: 80
* 8 seconds: 50
* 20 seconds: 29

### Use this metric to:
* Understand how problems with startup time impact the overall viewer experience
* Compare startup time performance to other areas of viewer experience
* Find areas where startup time can be optimized and improved


Startup Time Score


Video Startup Time measures the time that the viewer waits for the video to play after the page is loaded and the player is ready. It specifically measures from when the player has been instructed to play the video, either directly by the user or via autoplay, to when the first frame of video is showing and the playhead is progressing. In the case that the player is configured with a pre-roll ad, Video Startup Time is the time until the first frame of the pre-roll ad is displayed.

Mux provides two percentiles of this metric:
* Median (50th Percentile) – Helps understand a typical experience (half are better than this number, half are worse)
* 95th Percentile – Helps understand what a poorer experience is like on your platform, while excluding outliers and happening frequently enough (1 in 20 views) to always be worth your attention.

Our data shows that viewers can be very impatient when waiting for a video to start, leaving in as little as two seconds for certain content types.

Network performance and initial rendition selection have the greatest impact on this number.

Preloading the video data before the viewer clicks play can also have a positive impact on this metric, however this should only be done when the video is the primary piece of content.


Video Startup Time


Player Startup Time measures the time from when the player is first initialized in the page to when it is ready to receive further instructions.

While Player Startup Time is usually low, it can point to subtle difference in the operations of players. When combined with Page Load Time and Video Startup Time we can see Aggregate Startup Time, and understand the full amount of time a viewer waits on a video watch page.

To get Player Startup Time data, you must pass [`player_init_time`](https://docs.mux.com/v1/docs/metadata) in your client integration.


Player Startup Time


Page load time measures the time from the initial user request for a page to the time when the video player is first initialized. Use this metric to understand the impact of new page resources (JavaScript files, CSS, etc.) on the viewer wait time. This can also be used to compare video players, and the size and loading speed of their files impacts the wait time.

Page Load Time is only recorded for the first video view on a page, so you may see a smaller number of total views for this metric.

To get Page Load Time data, you must pass [`player_init_time`](https://docs.mux.com/v1/docs/metadata) in your client integration.


Page Load Time


Aggregate Startup Time combines Page Load Time, Player Startup Time, and Video Startup Time to show the total time a viewer waits for a video to play after requesting to watch the video on the previous screen or page.

On the web we often have web pages that are dedicated to individual videos. These pages are referred to as watch pages (e.g. `mydomain.com/watch?video=1234`). Viewers get to these pages by clicking on search results and lists of video thumbnails. In the case of watch pages we need to not only understand the Video Startup Time, but the full time that the viewer waited from when they clicked/tapped to watch the video.

Use this metric to optimize your watch pages for shorter wait times. For example, waiting to load secondary content (comments, related videos) until the video is playing, or choosing a video player that has the lowest impact on aggregate startup time.

This metric is only recorded for the first video view on a page, so you may see lower total view counts represented in this metric.


Aggregate Startup Time


The Seek Latency metric measures the average amount of time that viewers wait for the video to start playing again after seeking to a new time. Seeking is any time the player is asked to jump backward or forward to a new time in the video, outside of normal playback. Aside from the viewer clicking on the progress bar, seeking can also happen programmatically, for example when jumping ahead to resume where the viewer previously stopped watching.

### Use this metric to:
* Look for cases of extreme seek startup times
* Compare video players and their ability to respond quickly to a viewer’s seek request


Seek Latency


The Video Startup Preroll Request Time measures the total amount of Video Startup Time that is spent making ad requests, waiting for the ad responses, and parsing the VAST/VMAP response. Specifically,
his measures the amount of time the viewer is waiting for the video to start in which the player does not yet know which ad to play, if any.

It is important to call out that this time _only_ includes time during video startup (i.e. after the user initiates playback, or auto-play does). Any time spent requesting ads before playback is initiated is not included in this metric.

### Use this metric to:
* Understand the performance of your ad server as it affects video startup time
* Attribute slow startup times to ad network performance, versus ad asset performance


Video Startup Preroll Request Time


The Video Startup Preroll Load Time measures the total amount of Video Startup Time that is spent loading the first preroll ad asset. Specifically, this measures the amount of time the viewer is waiting for the first preroll ad to start playing after all ad responses have been received and parsed.

It is important to call out that this time _only_ includes time during video startup (i.e. after the user initiates playback, or auto-play does). Any time spent preloading the ad asset before playback is initiated is not included in this metric.

### Use this metric to:
* Understand the performance of your ad asset delivery as it affect video startup time
* Attribute slow startup times to ad asset performance, versus ad network performance


Video Startup Preroll Load Time


This metric measures the number of ad requests that are made up to the point of preroll ad playback beginning. Depending on your ad architecture, it is possible that your player may make sequential ad requests in the case that the previous request returned no playable ad in order to fill all possible impressions. In this case, multiple ad requests can lead to slow startup times, which can potentially be improved by reducing the possible waterfall/fallback calls, ensuring playable ads being returned in the first request, or other means within your ad server.

### Use this metric to:
* Understand performance correlation with the number of ad requests made to retrieve the first playable ad


Requests for First Preroll

Startup Time


Smoothness Score measures the amount of rebuffering a viewer sees when watching video. A higher Smoothness Score means the viewer experiences less rebuffering, while a lower score means a viewer sees more rebuffering.

### Formula
Average of:
$$
\frac{1}{\sqrt{1 + \Big(\frac{\text{rebuffer\_count}}{2}\Big)^2}}*100
$$
and
$$
e^{-10 * rebuffer\_percentage} * 100
$$

Rebuffering can be measured as a combination of the number of rebuffering events and the rebuffering percentage. We measure rebuffering in both ways in order to account for views where rebuffering events are short but occur often. We consider multiple interruptions to be worse than a single interruption, even if total time spent rebuffering is the same. Averaging both measurements helps account for this, and provides a truer representation of the viewer experience.

Rebuffering time is measured as a percentage because a 5-second rebuffering duration is much more meaningful when viewing a 10-second clip versus a 2-hour movie. Our research shows that watch time rapidly decreases from just a single percentage point of rebuffering. We use an exponential curve to model this rapid initial decrease from rebuffering, which then slows down after a rebuffering percent of ~10%. This is because rebuffering is now high enough that incremental amounts of rebuffering becomes less impactful.

Rebuffering counts are slightly less negative under certain circumstances. For example, a single half-second rebuffering event during a long video might be barely noticeable. However, multiple rebuffering events do become noticeable and will rapidly decrease your score. This curve differs from the percentage curve in that a single rebuffering event is not harshly penalized. The score decreases rapidly in the 2 to 4 range, and then slows down past 5 rebuffering events due to the diminishing impact of additional events.

Both rebuffering percentage and rebuffering counts degrade the user experience, so we average the score. This helps balance for cases when either count or percentage is high while the other metric is low. A single rebuffering event of 1% isn’t great, but it’s better than 10 rebuffering events of 0.1%.

Note that EBVS views do not receive a Smoothness Score.

Examples:
* No rebuffering: 100
* 5 minute video with a single 5s rebuffer: average of 80 and 90 = 85
* 20 minute video with four 15s rebuffers: average of 44 and 60 = 54

### Use this metric to:
* Understand how problems with rebuffering impact the overall viewer experience
* Compare rebuffering performance to other areas of viewer experience
* Find areas where rebuffering can be optimized and improved


Smoothness Score


Rebuffer Percentage measures the volume of rebuffering that is occurring across the platform. Total Rebuffer Duration is the sum of the time viewers spend rebuffering on the platform or the given set of video views. Total Watch Time is the sum of all time viewers spend watching video, inclusive of rebuffering time. The Total Rebuffer Percentage then measures the total rebuffer duration as a percentage of total watch time.

### Use this metric to:
* Understand how much time viewers spend waiting for videos to rebuffer
* Optimize an adaptive algorithm or rebuffering strategy
* Compare players and CDNs


Rebuffer Percentage


Rebuffer Frequency measures how often rebuffering events happen. It’s important to track this number because it can reveal issues of video stuttering, where the player is being too aggressive when restarting playback and has to frequently stop to rebuffer. This issue can be lost when measuring the rebuffering time, but can be just as frustrating as longer rebuffering events.

### Use this metric to:
* Understand how frequently viewers are interrupted by rebuffering
* Optimize an adaptive algorithm or rebuffering strategy
* Compare players and CDNs


Rebuffer Frequency


Rebuffer Duration is the amount of time in seconds that viewers wait for rebuffering per video view. Videos with longer durations have more opportunities for rebuffing events to occur and can make comparisons with shorter videos difficult, making Total Rebuffer Percentage the safer metric to optimize with. However Rebuffer Duration can be a useful metric for understanding the true viewer experience because it’s measured in seconds as opposed to a percentage.

### Use this metric to:
* Understand how long viewers wait for videos to rebuffer per video view
* Optimize an adaptive algorithm or rebuffering strategy


Rebuffer Duration


Rebuffer Count shows the number of rebuffering events that happen during video views. Compared to Total Rebuffer Frequency, Rebuffer Count can help you easily understand how many views are seeing more than zero rebuffering events.

### Use this metric to:
* Understand how often viewers are interrupted by rebuffering per video view
* Optimize an adaptive algorithm or rebuffering strategy


Rebuffer Count

Smoothness


Video Quality Score measures the visual quality a user sees by comparing the resolution of a video stream to the resolution of the player in which it is played. If a video stream is significantly upscaled, quality generally suffers, and viewers have an unacceptable experience.

Note that video quality is notoriously difficult to quantify, especially in a reference-free way (without comparing a video to a pristine master). Bitrate doesn't work, since the same bitrate may look excellent on one video and terrible on another.

Several factors contribute to actual video quality: bitrate, codec, content type, and the quality of the original source. However, if content is encoded well and at the right bitrates, upscaling correlates reasonably well to video quality. We use a combination of average and max upscaling in order to account for extreme drops in quality, even when it only occurs for brief moments.

### Formula
$$
e^{-0.33 * (0.15 * U_{m} + 0.85 * U_{a})} * 100
$$
where
$$
U_{m} = \text{Max Upscale Percentage}
$$
$$
U_{a} = \text{Average Upscale Percentage}
$$

Video Quality Score is inversely related to the upscaling percentage for each view. 85% of the score is based on average upscaling, and 15% is based on max (peak) upscaling.

Note that EBVS views do not receive a Video Quality Score.

Examples:
* No upscaling: 100
* 50% upscaling throughout: 85
* 200% upscaling for the first 30 seconds, and no upscaling for the next 20 minutes: 92


### Use this metric to:
* Understand how video quality impacts the overall viewer experience
* Compare video quality to other areas of viewer experience
* Find areas where video quality can be optimized and improved


Video Quality Score


Upscaling is when the video player has to increase the size of the video to fill the player’s display. For example, if the video source is 320x240 and the player size 640x480, the player will stretch the video to fill the player dimensions. In that process the quality of the video degrades.

Upscale Percentage is measured as the change in one dimension, specifically the dimension that fits the player first when upscaling. In the 320x240 to 640x480 example, the Upscale Percentage would be 100%, calculated as (640-320) / 320.

However while the video plays the upscaling percentage may change if a new video rendition is selected or if the player goes to fullscreen. For this reason in the Total Upscale Percentage metric we multiply the upscale percentage by the amount of time the video was upscaled. If the video was upscaled 100% for half of the video, and 0% for half of the video, the Total Upscale Percentage would be 50%.

### Use this metric to:
* Optimize the dimensions of videos to reduce poor quality due to stretching


Upscale Percentage


Downscaling is the inverse of upscaling, measuring when the video source is too big for the player and has to be reduced in size to fit the display. In the process of shrinking the video pixels are
thrown out and essentially wasted. While this does not mean a reduction in video quality it does mean wasted bandwidth for you and the viewer, and significant occurrences of downscaling should be addressed.

### Use this metric to:
* Optimize the dimensions of videos for bandwidth and cost savings


Downscale Percentage


While Upscale Percentage helps understand the volume of upscaling that’s occurring on your platform, the Max Upscale Percentage can help reveal points of significant upscaling, even if they don’t last the full video. It can also be more clear which video rendition may be the culprit as the percentage will exactly match the difference between a rendition and the player dimensions.

### Use this metric to:
* Optimize the dimensions of videos to reduce poor quality due to stretching


Max Upscale Percentage


While Downscale Percentage helps understand the volume of downscaling that’s occurring on your platform, the Max Downscale Percentage can help reveal points of significant downscaling, even if they don’t last the full video. It can also be more clear which video rendition may be the culprit as the percentage will exactly match the difference between a rendition and the player dimensions.

### Use this metric to:
* Optimize the dimensions of videos for bandwidth and cost savings


Max Downscale Percentage


Weighted Average Bitrate is the time weighted average of the indicated bitrates that a viewer experiences during a video stream. The  weighted average is calculated from the amount of time spent at each bitrate while a video is played. The bitrate value is the indicated bitrate from the video manifest for the rendition that is used for each segment of playback.

For example, if during a view lasting 3 minutes a video plays for 1 minute at 1Mbps and 2 minutes at 2Mpbs, the Weighted Average Bitrate would be: [(1Mbps * 1min) + (2Mbps * 2min)] / 3min = 1.67Mbps

This metric only includes the bitrates of video segments that are actually played. It does not include the segments that are downloaded but unplayed due to, for instance, an ABR algorithm that switches to a higher bitrate and discards previously downloaded lower bitrate segments in the cache.

### Use this metric to:
* Measure and optimize the visual quality of the videos that viewers experience


Weighted Average Bitrate


Request Throughput measures the average throughput, in Mbps, for all media requests that were completed. Throughput is measured as the number of bits received per second from the time a request is initiated until it is completed.

Note that request metrics are only available for certain SDKs and playbacks. See our docs for more information.

### Formula
Each request has a total time loaded and a total bits downloaded. All of the bits loaded are added up and divided by the total request time (the sum of all time spent downloading those bits). If two requests are sent in parallel, they still have independent measurements which are added to the totals respectively.

```
    sum(bits downloaded) / sum(request time)
```

For example, if there were three requests that each downloaded 40 Megabits (5MB) in one 4 second request and two 2 second requests then the request throughput is 15 Mbps, `(40 + 40 + 40) / (4 + 2 + 2)`.

### Use this metric to:
* Compare the performance of multiple CDNs
* Troubleshoot throughput problems by CDN, ASN (ISP), or geography
* Understand the bandwidth of your users across different geographies, devices, etc.


Request Throughput


Request Latency measures the average time to first byte for media requests, that is the time from when the request is initiated to the time when the first byte of data is received from the server.

Note that request metrics are only available for certain SDKs and playbacks. See our docs for more information.

### Formula
```
    sum(time to first byte) / (number of requests)
```

For example, if the time for first byte for five requests was 100ms, 200ms, 100ms, 75ms, and 150ms, the Request Latency would be:
```
    (100 + 200 + 100 + 75 + 150) / 5 = 125
```

### Use this metric to:
* Compare the performance of multiple CDNs
* Troubleshoot latency problems by CDN, ASN (ISP), or geography


Request Latency


Max Request Latency measures the maximum time to first byte for a media request, that is the maximum time an individual request took from the time it was initiated to the time when the first byte of data was received from the server.

Note that request metrics are only available for certain SDKs and playbacks. See our docs for more information.

### Formula
```
    max(time to first byte)
```

For example, if the time for first byte for five requests was 100ms, 200ms, 100ms, 75ms, and 150ms, the Max Request Latency would be 200ms.

### Use this metric to:
* Compare the performance of multiple CDNs
* Troubleshoot latency problems by CDN, ASN (ISP), or geography


Max Request Latency

Video Quality

Views counts the total number of views that started during the selected time interval. This is calculated differently from the views in Quality of Experience metrics, which counts the total number of views that ended during the selected time interval.

Views

Unique Viewers counts distinct viewers based on the start time of the associated view, using Viewer ID to determine uniqueness. If a Viewer ID isn't provided, the default Viewer ID generated by the Mux SDK is used. We recommend setting a meaningful, anonymized Viewer ID to get an accurate Unique Viewer count. The Viewer ID should not use any value that contains personally identifiable information (such as email address, username, etc).

Unique Viewers

Playing Time is the total time (in hours) that viewers watched playing video, and excludes rebuffering, seeking, and paused time.

Docs

Docs

Understand metric definitions