One of the impacts of the COVID-19 situation is an increased consumption of on-demand and streaming services. Although that may be a bright spot in this challenging time for some businesses, it also means a strain on their system, infrastructure, and resources.
Your prospects and clients come to you for informational, educational, or entertainment content. The time from initial visit to conversion is critical. If the user experience is negative, you will lose revenue, and even worse, you may be called out on social networks that can impact your brand and business going forward.
Valtira’s 10+ years of experience driving streaming services for organizations like AMC, SeekaTV and, Tasteocracy can help you prepare you to scale and optimize your streaming services.
The Key: Staying Ahead Of The Game
To help make sure you are on top of the scaling needs, a dashboard and alerting system is paramount. This will allow you to monitor the components to see when additional infrastructure or resources are needed. An alerting system will help you know when and how much to increase your resources to meet the scaling demand.
Above is an example of a system that isn’t healthy – you can see clearly that it is constrained and cannot scale to meet its needs.
The auto-scaling rules governing the pace of scaling to meet the load needs to be adjusted to allow the service to allocate more resources. The “MaxInstanceCount” looks like it should be increased 25%.
The following graph shows an example of what a healthy service looks like.
This graph shows that this service may scale (quickly!) to meet demand, peak under the limit of instances and then slowly reduce the number of instances to “scale-in”. This auto-scaling rule optimizes both service availability and cost.
Keep in mind that usage patterns change over time, so it is important to set up a Dashboard to monitor activity and make adjustments as necessary.
What to Scale
To make sure your service meets the needs of your clients, you need to ensure you have the scaling in place for the following components.
Web Solutions
To make sure your clients have a smooth account management and store experience, your web solutions should be able to scale with demand.
SSL imparts a heavy CPU cost for each request to the web server, so offloading SSL to a Content Delivery Service (CDN) will drastically reduce CPU utilization on your web servers.
API layer
It is critical to scale the performance of the API layer to ensure entitlements and other components are processed in a timely basis.
Typically, the API is running complex SQL queries to build a response to a request from a Web site or App. By implementing read replicas to parallelize the work, APIs are free to scale to meet the needs of the users.
Persistent services
A critical component of the performance of your system you’re your databases and no sql services.
As we highlighted above, many API endpoints run expensive SQL queries. Adding auto-scaling to the database infrastructure is the next step to free the APIs to scale.
Technical infrastructure
Whether you are using AWS, Azure, or another platform, the goal is to optimize the cloud or hardware infrastructure to meet your client demand.
If you are deploying advanced cloud infrastructure like Kinesis Streams or DynamoDB for your NoSQL persistence, setting up a Dashboard to monitor throughput for reading and writing activity will help you decide if and when to scale. Scaling these services is non-trivial as they have major cost implications.
Optimizing Costs
Just as important as scaling your services to meet demand, you need to optimize the costs and resources to maintain or grow your margin on your service. And not only now in this heightened stay home period, but going forward as well.
Of note, it is difficult to optimize for cost. It takes a higher level of knowledge to architect, and an understanding of the current state of useful tools, to balance the complex problems of optimizing an environment within the framework of performance vs. cost over time. Your in-house team or your partner will need to focus on this, or you risk missing out on what should be an escalating ROI on your services.
How Valtira Can Help
We work with clients to analyze traffic patterns, resource utilization, application logs, and infrastructure. This guides and implements changes to free up systems to scale automatically to increased demand and match the peaks and troughs of that demand to cost. We will guide you through deciding what issues are low-hanging fruit and likely to make a big performance difference, then work on the thornier problems which require a heavier amount of analysis and planning.
Examples of low-hanging fruit:
- Configuring Scaling Dashboards – huge amount of value here
- Offloading content and SSL termination to Content Delivery Networks
- Auto-scaling Azure App Services, AWS ECS Clusters, and AWS Aurora
We have the experience and knowledge to get you started out of the gate and make a huge impact right away. For the longer view, we know how to do the complex work of optimizing the environment for performance and cost over time.
We wish you the best during this unique and challenging time.
If you would like to discuss your streaming or on-demand environment please contact us.