Welcome to this blog series about media infrastructure automation. Over the course of the past year, many of you asked about how to design your media infrastructure operation and management—how to provide your teams with access to the right resources, with the right level of monitoring, and with required flexibility without the need to become an AWS services expert. To help with this, I invite you to follow the steps that describe how to automatically deploy a channel on AWS Elemental MediaLive (MediaLive), a broadcast-grade live video processing service, using a video file as a looping source.
This is the first post in a series that explores ways in which you can automate your media infrastructure and streamline your operation processes. Doing so will help increase the repeatability of your workflows and reduce potential failures, as well as provide you with agility in the management of your media infrastructure.
In this post, we make use of the AWS Cloud Development Kit (AWS CDK), an open-source software development framework, to create a media infrastructure that generates a channel using MediaLive to loop through an MP4 file. The AWS CDK generates the associated AWS CloudFormation, which lets users model, provision, and manage AWS and third-party resources by treating infrastructure as code, in order to provision it.
Once we have a running infrastructure, we see how to set up a continuous integration and continuous delivery (CI/CD) workflow using AWS CodePipeline, a fully managed continuous delivery service, to inform you that appropriate processes are in place to maintain infrastructure for future code updates and deployments.
Finally, after setting up version control using AWS CodePipeline, we set up deployment control using AWS Service Catalog, which lets organizations create and manage catalogs of IT services that are approved for use on AWS. This way, only authorized operators can deploy and access your media infrastructure. It also helps structure the deployment of your media infrastructure with custom parameters that you define for personalization.
AWS CDK structure
AWS CDK is an open-source software development framework that lets you define your cloud application resources using familiar programming languages. For hands-on experience, please visit the AWS CDK introduction workshop.
For this project, we make use of the Python version of the AWS CDK. We will create a Python app using the AWS CDK, which abstracts the AWS CloudFormation stack and resource creation. The application code is structured as follows:
- Core stack
- Permission stack
- Automation stack
- AWS Elemental services stack
The core stack contains shared resources. The other stacks are nested to the core stack and have their own domain of specialty.
- The permission stack holds the roles and policies required to run the application.
- The automation stack hosts the function from AWS Lambda (a serverless, event-driven compute service) and the activations necessary for Amazon EventBridge (a serverless event bus that makes it easier to build event-driven applications at scale).
- The AWS Elemental services stack handles the deployment of all the components to the media infrastructure.
The AWS CDK sample code references the following resources:
- 2x for AWS Identity and Access Management (AWS IAM) roles, which provides fine-grained access control across all of AWS
- The first role provides permission to MediaLive to access resources and process content.
- The second role allows the AWS Lambda function to access AWS Elemental services to run any necessary action.
- 1x AWS Lambda function
- This AWS Lambda function automatically starts the channel once the stack deployment is complete.
- 1x Amazon EventBridge rule
- The EventBridge rule activates the AWS Lambda function.
- 1x AWS Elemental MediaLive input
- This MediaLive input holds the necessary information for the service to access the MP4 file.
- 1x AWS Elemental MediaLive channel
- The MediaLive channel holds the necessary configuration to process the MP4 file referenced in the input.
In order to run the application from the GitHub sample, follow these instructions:
- Deploy: the following command copies the sample code to your environment and deploys the application in your AWS account.
- Destroy: the following command removes all components of the media infrastructure from your account—make sure to stop any running channel before using this command.
I highly recommend using AWS Cloud9, a cloud-based integrated development environment (IDE), to deploy the script because this will simplify the installation of the AWS CDK, AWS CLIv2, and Python >=3.6.
Once you clone the sample code to your AWS Cloud9 environment, you need to customize the application by updating the following parameters in the file app.py to fit the purpose of your project:
- my_region: region in which to deploy your application
- my_stack_name: custom name for your AWS CloudFormation stack
- my_customer_name: customer name in case of multi-tenancy and/or project
- channel_name: name of your MediaLive channel
- source: HTTP path to your MP4 source file
- destination: IP address to publish your content
- port: destination port to publish your content
- auto_start: allow automated start of the channel upon completion of stack deployment
More information is available on the AWS CDK best practices documentation page.
We begin with the following infrastructure that may look simple—a broader set of samples are available in GitHub under aws-cdk-mediaservices-refarch—nevertheless, this infrastructure provides you with all the necessary components to get started on media infrastructure automation.
The live encoding loop sample provides you with a ready-to-use application to build a live channel.
The generated channel can be used for testing and debugging and to help familiarize yourself with building and writing media infrastructure as code.
This sample code builds a media infrastructure that generates a MediaLive channel that in turn processes an MP4 file stored in a bucket on Amazon Simple Storage Service (Amazon S3)—an object storage service offering industry-leading scalability, data availability, security, and performance—in a continuous loop simulating a live channel.
This kind of setup is particularly useful as a dummy input to feed a more complex infrastructure and to build your proof of concepts until you are ready to send real-time content to your account.
While deploying the sample code available in GitHub, you will make use of the following services that could incur charges to your account:
- Amazon Simple Storage Service (Amazon S3)
- AWS Elemental MediaLive
- AWS CloudFormation
- AWS Lambda
The MediaLive input will point the channel to an MP4 file stored on an Amazon S3 bucket.
Then the channel input configuration will be set to loop.
Next, the application will set up a single UDP output group for the channel that points to the provided destination IP referenced in the app.py file.
To play this content, you need to publish it to a valid IP or modify the application to publish the content to an origin server such as an Amazon S3 bucket, AWS Elemental MediaStore (a media optimized cloud storage), AWS Elemental MediaPackage (which prepares and protects your video for delivery over the internet), or any compliant third-party origin or RTMP(s) server.
Content security is important for today’s streaming business models. Applying in-depth security principles make it harder for unauthorized users to access your content. So, make sure to use encryption-at-rest for your assets with the bucket encryption capabilities and secure the transport of your content with HTTPS, s3ssl, or any other supported secured protocol.
For demos and debugging purposes, this solution runs a single pipeline to process your content.
However, in a production environment, make sure to remove any single point of failure by using the STANDARD mode, which allows for dual pipeline creation to process your content in the cloud.
Activating logs on the channel gives you more insight about what is happening in your infrastructure should you need to investigate any issue.
You can enhance your AWS CDK application with API call-to-automate operational tasks based on activators (tasks such as “start channel,” “stop channel,” etc.) that can be activated using events recorded by the API call in AWS CloudTrail, which monitors and records account activity across your AWS infrastructure, to automate those actions.
MediaLive is a per-minute encoding service. This means the more profile and publishing points you have, the more costly it will become. Hence, review your encoding settings to optimize your ABR ladder and use only the appropriate profile and publishing point for your application.
Consider reservation for 24/7 workflows and a single pipeline for testing and debugging purposes.
Make use of bandwidth optimized control rates, such as QVBR, to save bandwidth on CDN usage when possible.
To find more information about media infrastructure best practices, review our AWS Well-Architected Streaming Media Lens white paper.
This post describes some best practices about how to design your media infrastructure code. You gained access to the aws-cdk-mediaservices-refarch repository, which contains a collection of code samples for media infrastructure deployment. In the next post of this series, we will explore how to set up version control and validation for your AWS CDK sample code using AWS CodePipeline.