This blog post is based upon the “Multi-CDN Webinar” which was recorded on October 22, 2024.
What is Multi-CDN?
The idea of using a number of different CDNs to delivery content more effectively has been around for quite some time. Streaming operators understand that no one CDN can provide both the capacity and geographic reach that might be needed for large-scale events. What’s. more, some CDNs perform better than others in certain geographies. The long-and-short of it is that a multi-CDN approach provides operators the best chance of ensuring a higher QoE. In addition, using multiple CDNs can provide redundancy and resiliency in the event that a single CDN (or the transit provider they are using) experiences an outage.
But there are a number of different ways to implement multi-CDN (midstream switching):
- Centralized. This is using a single entity, like an OVP, to handle everything involved. While this is simplest way to enable multi-CDN delivery, it’s also a blackbox as the single entity controls everything. In addition, service providers like DLVR, can do this as well (in DLVR’s case, through manifest manipulation), which can allow the streaming operator to separate out CDN decisioning from their content platform.
- Centralized/De-centralized. This is using client-side programming which acts against a pre-populated list of CDNs where the CDN is selected based on third-party analytics data (provided by a company like MUX, Conviva, or NPAW for example)
- Decentralized. This is putting everything entirely into the client, using client-side programming and the CDN list but with data generated from the client itself to make the decisions (rather than data provided by a third-party).
Of course, there are challenges to using a multi-CDN approach which complicates the way streaming operators can implement. For example, not all CDNs have the same features or implement features in the same way. Take access tokens, as an example. The result of a lack of feature parity can force operators to create advanced business logic to account for it.
Multi-CDN Versus Content Steering
While utilizing multiple CDNs is nothing new (and there are multiple ways to accomplish it), content steering is a relatively new idea that has been embraced by HLS and DASH. It is a way of directing requests for content segments to different locations (i.e., a specific CDN) using a content steering server. So one could say that this is a decentralized approach to multi-CDN but it is specific to the two formats and, more importantly, to JavaScript-based players that can playback the two formats.
According to the dash.js content steering specification:
Content steering describes a deterministic capability for a content distributor to switch the content source that a player uses either at start-up or midstream, by means of a remote steering service. The DASH implementation of Content Steering also supports the notion of a proxy steering server which can switch a mobile client between broadcast and unicast sources.
The big difference then, between other multi-CDN architectures and content steering, is the steering server. By encapsulating a lot of the business logic within the server, rather than the client, the operator can simplify the implementation of CDN switching. When the decisioning logic is in the player, it requires the operator to continually update and manage player code across all devices. This may require touching, and changing, multiple player code bases. When the logic is in the server, there is a centralized place where the operator can make adjustments to how switching happens.
Switching CDNs Isn’t As Simple As It Seems
Imagine you are traveling down the highway and there is traffic ahead. To circumvent that traffic, your map app (Google or Waze or Apple) tells you to get off on the next exit. But, it’s also telling everyone else that too. So, suddenly, the way ahead is congested but the bypass gets congested as well. This is a problem that operators face when switching CDNs midstream. If CDN A is having a problem and the operator just makes a switch to send everyone to CDN B, then CDN B might become overwhelmed as well. It’s important for operators to utilize algorithms and business logic in their multi-CDN systems to try and load balance requests so that no one CDN is overwhelmed when a CDN has issues that requires traffic to be diverted. Just because a CDN is out performing all the others at a specific point-in-time during a live event doesn’t mean the operator should switch all the traffic to it
But it’s not just the business logic that’s important. It’s also the parameters against which that business logic operators. For example, a streaming operator might want to avoid a CDN that peers with a particular ISP in a specific country. Perhaps the CDN performs poorly on ingest for live events in that scenario. So the mix of traffic balanced across the CDNs will be different in that country than it might be in other countries. Identifying and updating parameters like those, which impact how the business logic operators, further complicates the process of CDN switching.
Finally, there is the trade-off between having a multi-CDN system that is dynamic and reactive with one that is simple and easy-to-operate and maintain. In other words, operators must answer the question, “how much complexity can I handle?” Streaming operators can feed the data needed to intelligently switch CDNs to a third-party that handles the multi-CDN architecture for them. But, in that sense, they lose a lot of flexibility and control. Or, they can bring in all the data themselves and do the decisioning and business logic, which is significantly more complex, but allows the operator to react in real-time.
Balancing The Data With The Cost
One of the unseen challenges of employing multi-CDN is the cost of the data analysis. The goal of any multi-CDN architecture is 2-fold. First, it’s about ensuring the best possible delivery performance so that viewer’s get the highest QoE. The second, though, is about economics. The operator wants to ensure that the CDNs they use are the most economical to ensure the best possible margin. But these two facets can sometimes be mutually exclusive requiring a lot of data analysis to find the best path forward for optimizing the business logic to enact the CDN shifts. So sometimes, the worst performing CDN may be the best economical choice. The issue gets more complicated when the cost of that data analysis impacts the economics of the switch. So what happens when it costs more to analyze the data to find the best CDN than it does for saving money on switching to a less expensive CDN?
Operators must track a wide variety of data impacting the financials of multi-CDN, from the cost of the infrastructure to the cost of the delivery to the cost of the decision to switch, so that the very system they are using to ensure the best possible financials of the delivery doesn’t cost more than it’s worth.
The Role of the CDN in a Multi-CDN Architecture
You might think that the CDNs are just an idle bystander in this scenario, where the content operator just moves content between CDNs. But that is not the case. The CDN can play a very active role by providing data to the content operator in real-time. When a CDN is switched away from, this kind of data can be instrumental in demonstrating that, for example, performance issues are fixed. A live event war room, for example, may involve operational resources from all of the CDN providers (in a multi-CDN delivery architecture), third-party analytics providers, and the streaming operator. Real-time CDN data, then, is paramount. CDN providers can supply deep analysis of errors in their logs which can be triangulated against client data. The result of all this cooperation can be more efficient and effective use of CDN resources.
Multi-CDN is Not a Panacea For Delivery Problems
One of the issues streaming operators face in employing multi-CDN is not relying on it to solve delivery problems. Switching from one CDN to another doesn’t necessarily solve the underlying issues. There may be fundamental configuration problems within the stream itself which is causing one particular CDN (or maybe even multiple) to have issues. Operators need to see CDN switching as an overall part of the strategy to ensure the best possible viewer experience. CDN switching can’t be seen as the primary means of solving delivery issues. In some cases, delivery issues may be caused by something deep in the ISP. If that ISP happens to be the primary peering partner for the CDNs in a specific region, switching won’t solve the problem. Streaming operators must work closely with all network operators within the delivery chain, whether that’s an ISP or a CDN, to understand root cause of a delivery issue before resorting to switching the CDN.
What’s Next for Multi-CDN?
As CDN switching continues to get smarter and more efficient, one of the underlying challenges, the issue of CDNs operating differently, needs to get addressed. This might happen through such systems as Open Caching, which can provide common APIs through which CDNs can be communicated with by the operator. This would reduce the complexity of having to deal with multiple CDNs, perhaps even allowing operators to increase the pool of delivery networks without adding more complexity.
Another aspect of research, being carried out within the SVTA, is around developing a common set of CDN KPIs that all operators can use to influence or direct CDN switching. Again, most operators who manage their own multi-CDN and CDN switching implementations, have their own data and KPIs they use. But defining and optimizing those metrics just adds to the complexity of the entire system. By having an industry-accepted set of KPIs can help align everyone, streaming operator, CDN, and ISP, on a single set of metrics making it easier to make decisions when switching CDNs.
Finally, addressing security within a multi-CDN architecture is paramount. But right now, token authentication, which is a primary means by which to secure delivery streams at the CDN edge, is done very differently across all the CDNs. This can complicate switching from one CDN to another. Thankfully, the SVTA and CTA-WAVE have worked together on a standardization project, the Common Access Token, which should be ratified and published shortly, solves that problem enabling all CDNs to support a single method for token authentication.
Conclusion
Multi-CDN, content steering, and CDN switching are all integral parts of a streaming operator’s approach to delivering high-quality video at scale. But it’s not simple and there’s not one correct way of doing it. Built on data and working relationships with everyone in the delivery chain, it can be either a complicated system which reacts dynamically in real-time or a simple offload to a third-party provider of multi-CDN. At the end of the day, though, a multi-CDN strategy is just one part of the overall approach to video delivery at scale. And, when it comes down to it, operators can’t rely blindly on it. They must still have people looking at screens, evaluating the effectiveness of the switching, and, if needed, make different decisions than automated processes might be suggesting.
While multi-CDN has been around for quite some time, it is only recently that we, as an industry, have begun to get smarter about this. But, we are only at the cusp of that intelligence. As more standardized methods for switching are implemented, the better the systems will become, not only through dynamic and reactive methods, and the more integrated the parties involved in switching can become.
Jason Thibeault
Jason is the CEO of the Streaming Video Technology Alliance, the international technical association for streaming video which brings companies from across the streaming ecosystem together to collaborate on technical solutions to delivering high-quality video at scale. In this role, he runs day-to-day operations, finances, member recruitment, strategy, and evangelizes the organization at events around the world. He is also the co-founder of a big data startup, datazoom.io. Jason is a contributing editor at Streaming Media Magazine and has written several books.