© 2015, Amazon Web Services, Inc. or its Affiliates. All rights reserved. May 10, 2016 AWS Media & Entertainment Symposium-New York
Jan 15, 2017
© 2015, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
May 10, 2016
AWS Media & Entertainment Symposium-New York
Welcome CONTENT PRODUCTION 9:15-9:45 am Digital Media Ingest and Storage Options on AWS 9:45-10:15 am AWS Cloud controls for securely storing your digital content and running media workloads 10:15-10:30 am Break 10:30-11:15 am How Wazee Digital and Amazon S3 Enable Bloomberg’s Customers with Global Access to
Business and Financial Content 11:15-12:00 pm Cost Effective Rendering in the Cloud with Spot Instances 12:00-1:00 pm Lunch CONTENT DISTRIBUTION AND MEDIA SUPPLY CHAIN 1:00-2:00 pm “All In”: Cloud Transformation of the Media Industry 2:00-2:45 pm Fox Networks’ End-to-end Video Supply Chain in the Cloud 2:45-3:30 pm L Leveraging Cloud to Reshape the Broadcast Supply Chain: Sony Ci Media Cloud Platform and Public Media Management 3:30-3:45 pm Break CONTENT PUBLISHING AND MONETIZATION 3:45-4:30 pm Cloud-enabled Innovation at Dow Jones 4:30-5:15 pm Content Monetization in the AWS Cloud 5:15-5:30 pm Wrap Up and Networking Reception
Thank You to Our Sponsor
Industry Business and Technology Challenges
Cost Pressures Data center investment,
management; cost alignment
Content Growth More content, mediums (VR),
higher quality (4K, HDR…) leads to more storage & compute needs
Peak Demand Servicing unpredictable demand =
overprovisioning and idle capacity
Competitive Pressures Changing consumption patterns
(cord cutting, unbundling); nimble competitors
Global Market Competition for customers across channels and GEOs
Core Competency Resources deployed on datacenters vs. core biz
Cloud Benefits and Outcomes
Benefit from massive Economies of scale
AWS helps you adapt your media storage and compute needs
Stop guessing capacity Handle unpredictable & bursty
media needs
Trade capital expense for variable expense
Pay for media you store and process, as you go
Go global in minutes Global availability instantly, with
no commit
Stop spending money on running & maintaining data centers
Focus your resources on your media needs
Increase speed and agility Shorten time-to-market, test out
new approaches
Innovating on Behalf of Customers
Building and managing cloud since 2006
70+ services to support any cloud workload
History of rapid, customer-driven releases
12 regions, 32 availability zones, 55 edge locations
51 proactive price reductions to date
Thousands of partners; 2,100+ Marketplace products
Experience
Service Breadth & Depth
Pace of Innovation
Global Footprint
Pricing Philosophy
Ecosystem
Global Footprint
AWS is available today in the U.S., Brazil, Europe, Japan, Singapore, Australia, and China. Additional regions in India, Korea, the UK, and Ohio are expected to come online over the next 12 – 18 months. Over 1 million active customers across 190 countries 2,000 government agencies 5,000 educational institutions 12 regions 32 availability zones 54 edge locations
Region
Edge Location
ENTERPRISE APPS
DEVELOPMENT & OPERATIONS MOBILE SERVICES APP SERVICES ANALYTICS
Data Warehousing
Hadoop/Spark
Streaming Data Collection
Machine Learning
Elastic Search
Virtual Desktops
Sharing & Collaboration
Corporate Email
Backup
Queuing & Notifications
Workflow
Search
Transcoding
One-click App Deployment
Identity
Sync
Single Integrated Console
Push Notifications
DevOps Resource Management
Application Lifecycle Management
Containers
Triggers
Resource Templates
TECHNICAL & BUSINESS SUPPORT
Account Management
Support
Professional Services
Training & Certification
Security & Pricing Reports
Partner Ecosystem
Solutions Architects
MARKETPLACE
Business Apps
Business Intelligence Databases DevOps
Tools Networking Security Storage
Regions Availability Zones
Points of Presence
INFRASTRUCTURE
CORE SERVICES
Compute VMs, Auto-scaling, & Load Balancing
Storage Object, Blocks, Archival, Import/Export
Databases Relational, NoSQL, Caching, Migration
Networking VPC, DX, DNS CDN
Access Control
Identity Management
Key Management & Storage
Monitoring & Logs
Assessment and reporting
Resource & Usage Auditing
SECURITY & COMPLIANCE
Configuration Compliance
Web application firewall
HYBRID ARCHITECTURE
Data Backups
Integrated App Deployments
Direct Connect
Identity Federation
Integrated Resource Management
Integrated Networking
API Gateway
IoT
Rules Engine
Device Shadows
Device SDKs
Registry
Device Gateway
Streaming Data Analysis
Business Intelligence
Mobile Analytics
AWS Cloud Services Highlights Amazon
STORAGE
Amazon DELIVERY
Amazon COMPUTE
Amazon PROCESSING
Amazon INGEST
CloudFront – Global Content Delivery Network with Analytics and customization at the edge
S3andGlacier–durable,scalableandsecuresolu.onsforon-lineandarchivalcontentstorage
Elastic Transcoder; Elemental Technologies – Scalable and cost effective video processing and transcoding
EC2 – Resizable general purpose compute capacity featuring instance types optimized for processing video, analytics
Direct Connect; Snowball; S3 Transfer Accelerator – Upload options for content and files of all sizes
Elemental Product Portfolio Elemental SERVER
Elemental LIVE
Elemental CONDUCTOR
Elemental DELTA
Elemental CLOUD
Real-time video & audio encoding
Faster than real-time video processing for file-based workflows
Video delivery origin for monetization, packaging, DRM, and distribution.
Unified management & orchestration
Platform for managing & deploying workflows in AWS
AWS Digital Media Customers
NASA/JPL
WHO WE OBSESS OVER: SOME OF ELEMENTAL’S 850+ CUSTOMERS
Pay TV Operators Content Owners Broadcast / Sports Corp/Edu/Gov’t
12
AWS Partner Ecosystem INGEST STORE MANAGE SECURE PROCESS
CREATE MONETIZE
INTEGRATE DELIVER
SaaS BYOL PaaS
Media Software on AWS Marketplace
• Launch Software on AWS with 1-Click • Pay-by-the-hour, monthly, or annual • Single invoice for AWS usage and ISV
software • Free Trials
Key Media Workloads Migrating to Cloud
v
Acquisition
DAM & Archive
Media Supply Chain
Publishing
Playout & Distribution
Analytics
OTT
VFX & NLE
Dive Deeper at: aws.amazon.com/digital-media
Content Production
v
Acquisition
DAM & Archive
Media Supply Chain
Publishing
Playout & Distribution
Analytics
OTT
VFX & NLE
Digital Media Ingest and Storage Options on AWS
Erik Durand Amazon Web Services
Content has Gravity and is getting heavier …
…it’s easier to move processing to the content
Partner 4k/8k Content
Where is the problem?
MoreBandwidth$$$$$
MorePowerfulCompute$$$$$
WaymoreStorage$$$$$
SomeProgress(ABR,HEVC,VP10)
Where is the sliding scale on my Infrastructure?
Amazon EFS
File
Amazon EBS Amazon EC2 Instance Store
Block
Amazon S3 Amazon Glacier
Object
Data Transfer
AWS Direct Connect
AWS Snowball
ISV Connectors Amazon Kinesis Firehose
S3 Transfer Acceleration
Storage Gateway
AWS Storage is a platform
A Concept - the Content Lake Inspired from Data Lake (Coined by James Dixon in 2010)
A single store of all of digital content that you create and acquire in any form or factor Don’t assume any resolutions/formats (for now or future) It is up to the consumer (application consuming the content) to use the appropriate infrastructure for processing
Amazon S3 : the Content Lake
Durable, cost-effective and fast Highly scalable front-end
• Multi-part uploads (parallel writes) • Range-gets (parallel reads)
No need for capacity planning or provisioning Use Amazon S3 with on-premises storage in a hybrid model Secure
S3 scalability: buckets and objects
1 PB raw storage
800 TB usable storage
600 TB allocated storage
400 TB application data
S3 capacity pricing—pay only for what you use
AWS Cloud Storage
Object Storage Options
S3 Standard
S3 Standard - Infrequent Access
Amazon Glacier
Active data Archive data Infrequently accessed data
- Transition Standard to Standard-IA - Transition Standard-IA to Amazon Glacier
- Expiration lifecycle policy - Versioning support
- Prefix support
Data Lifecycle Management
T T+3 days T+5 days T+ 15 days T + 25 days T + 30 days T + 60 days T + 90 days T + 150 days T + 250 days T + 365 days
Data access frequency over time
Securing your data on AWS
AWS alignment with the latest MPAA cloud based application guidelines for content security – August 2015 • VPC private endpoint for Amazon S3 – enables a true private workflow capability • Encryption & key management capabilities • Amazon Glacier Vault for high-value media/originals
Save money on storage
58% saving over S3 Standard
44% saving over S3 Standard-IA
* Assumes the highest public pricing tier
Hydrating the Content Lake
AmazonS3 AmazonS3(mulJ-partUpload)
Direct Connect
Nx1G|10G
MassivelyScalableFront-end
S3 Transfer AcceleraJon AWS Snowball
Storage Gateway
Avere - Demonstrated M&E Success
Who uses Avere? Movie studios for the top-20 blockbusters of 2015 for special effects
Customer Challenges • Scale rendering and transcoding performance • Cost, space & power • Managing storage silos • High latency of data access over WAN • Add compute resources at peak times • Need for 2-3 months, no long-term commitment • Do NOT want to rewrite applications
Avere Benefits • Hot data stored on RAM & SSD within FXT cache • Bulk of data can remain on-prem or on inexpensive
S3 • Caching of remote data eliminates WAN latency • Clustering provides scalable NAS performance • Hybrid model - FXT filer on-prem and/or vFXT on EC2 • Pay only for what is used
Optimizing Data Transfer Method #1: The Highway
What is AWS Snowball? Petabyte-scale data transport
E-ink shipping label
Ruggedized case “8.5G impact”
All data encrypted end-to-end
Rain- and dust-resistant
Tamper-resistant case and
electronics
80 TB 10 GE network
How it works
Use cases: AWS Import/Export Snowball
Cloud Migration
Disaster Recovery
Data Center Decommission
Content Distribution
How fast is Snowball?
• Less than 1 day to transfer 250 TB via 5x10 G connections with 5 Snowballs’ less than 1 week including shipping
• Number of days to transfer 250 TB via the Internet at typical utilizations
InternetConnectionSpeedUtilization 1Gbps 500Mbps 300Mbps 150Mbps
25% 95 190 316 63250% 47 95 158 31675% 32 63 105 211
Optimizing Data Transfer Method #2: The Internet
Complicated Setup and Management
Hard to Optimize Performance
Expensive
Optimizing Internet Performance Is Hard
Prioprietary
Introducing Amazon S3 transfer acceleration
S3 Bucket AWS Edge Location
Uploader
Optimized Throughput!
Typically 50%–400% faster
Change your endpoint, not your code
54 global edge locations
No firewall exceptions
No client software required
Rio De Janeiro
Warsaw New York Atlanta Madrid Virginia Melbourne Paris Los Angeles Seattle Tokyo Singapore
Tim
e [h
rs]
500 GB upload from these edge locations to a bucket in Singapore
Public Internet
How fast is S3 transfer acceleration?
S3 Transfer Acceleration
We have customers uploading large files from all over the world. We’ve seen performance improvements in excess of 500% in some cases. - Emery Wells, Cofounder/CTO ”
“ Use case: media uploads
Regional Lakes…
AWS is available today in the U.S., Brazil, Europe, Japan, Singapore, Australia, and China. Additional regions in India, Korea, the UK, and Ohio are expected to come online over the next 12 – 18 months. Over 1 million active customers across 190 countries 2,000 government agencies 5,000 educational institutions 12 regions 32 availability zones 54 edge locations
Region
Edge Location
Source (Virginia)
Destination (Oregon)
• Only replicates new PUTs. Once S3 is configured, all new uploads into a source bucket will be replicated
• Entire bucket or prefix based
• 1:1 replication between any 2 regions
Use cases Compliance - store data hundreds of miles apart Lower latency - distribute data to remote customers/partners)
S3 cross-region replication Automated, fast, and reliable asynchronous replication of data across AWS regions
AmazonS3AmazonS3(range-gets)
Direct Connect
Nx1G|10G
MassivelyScalableS3Front-end
EBS
Instance Store
c MassivelyScalableComputeonAWSCloud
On-PremApps
Consuming the Content Lake
Media Workloads (redefined)
EBSInstance
Store
AmazonEBS/EFS/EC2InstanceStore
Process
Partner/Affiliate/ServiceProvider
UserDelivery/Consump.onVFX/Produc.on
On-PremApps
Archive
AmazonGlacier(LifeCyclePolicies)
c
c
Direct Connect
Content Access Transfer
Disposable Infrastructure
Auto-scaling Workload specific
AmazonS3
EFS
Q&A
Learn more at: http://aws.amazon.com/s3/ http://aws.amazon.com/glacier/ http://aws.amazon.com/importexport/
Securely storing your digital content and running media workloads
Konstantin Wilms – Specialist Solutions Architect Amazon Web Services
AWS Cloud Controls
Who is attacking and why?
Cyber Criminal
Hacktivist Advanced Persistent Threat (APT)
Deface & Destroy
Manipulate
Highly Targeted
Associated Press – Hacked Twitter Account
• Internal password phishing • 1% drop in S&P 500 • $136 Bn market drop • US Treasury bond yield drop • $ weakens against ¥
TV5Monde Outage
• State sponsored phishing attack • 11 TV channels off air for 3 hours • Website & Facebook page defaced • Email server taken offline
Attack types against media vs other industries
Higher than Average • DDOS • Brute Force • Application Attacks
Lower than Average • Part of a botnet • Scanning • Recon
Content Production
Content Distribution
Processing & Management
Content Storage
§ Modelling § Rendering § Video editing § Post production § Broadcast signal
acquisition § Digital dailies/
approvals
§ B2C streaming of live and VOD content
§ B2B distribution § Video advertising
insertion
§ High speed ingest § Library storage and
archiving § Tier management § Content/asset
management
§ En/Transcode § Packaging § Encryption,
watermarking § Digital Rights
Management § Workflow, job
scheduling, automation
Content Consumption
§ Analytics, reporting, log analysis
§ Real-time monitoring
§ Content discovery § Content
recommendation engine
Studio
Post House + Other Service Providers
Affiliates + Broadcasters + Distributors
Digital Media Workloads
Content Production
Content Distribution
Processing & Management
Content Storage
Content Consumption
Shared IT Services Network Security Operations Infrastructure
Partner Solutions
Storage | S3, Glacier, EBS, Instance Store, EFS
Processing | EC2, Database (RDS/DynamoDB), EMR, ECS, Lambda, SNS, SQS, SWF
Network | VPC, VPN, Direct Connect
Access | IAM, AWS Config, CloudTrail, CloudWatch
…from the Lens of a Security Architect
A Layered Security Approach
Security of the Cloud
Security on the Cloud
Cloud Security
Organization &
Management Operations Data Security
Application Security
Development Lifecycle
Authentication & Access
Secure Coding & Vulnerability Management
Digital Security
Content Management Content Transfer
Facili.esPhysicalsecurity
PhysicalinfrastructureNetworkinfrastructure
Virtualiza.oninfrastructure
§ Certifications
§ MPAA best practices alignment
https://aws.amazon.com/compliance/mpaa/
Security of the Cloud
What’s in scope for MPAA (BP) Alignment? …the entire AWS Services stack
MPAA Guidelines
MPAA Alignment
MPAA Best Practice Alignment
SOC ISO 27001
PCI DSS Level1 FEDRAMP }
Media Workflow Security
Content Production
Processing & Management
Content Storage
§ Modelling § Rendering § Video editing § Post production § Broadcast signal
acquisition § Digital dailies/
approvals
§ High speed ingest § Library storage and
archiving § Tier management § Content/asset
management
§ En/Transcode § Packaging § Encryption,
watermarking § Digital Rights
Management § Workflow, job
scheduling, automation
Security of Media Workflows in the Cloud
• Highly Valued Pre-Released Assets • Secure Transfer (physical in many cases) • Encryption & Key Management • Access Control • Deletion Protection • Isolated from public access (internet) • Logging and Monitoring • Content location • Patriot Act/PRISM
12 Regions
30 Availability Zones
54 Edge locations
Where is my Content?
Media Workflow Migration to AWS
corporate data center
AWS cloud
users
Content
Servers
disk
tape storage Amazon S3 Amazon Glacier
Content Encrypted at Rest
Encrypted in Transit Using my Keys
Over Private Connection Access Policies
Protection
Processing Layer Amazon EBS
MFA, Users,
Groups, Roles
Linked Accounts,
Alerts
Resource Separation
Log & Audit
CloudFormation
Infrastructure
Key & Secret Storage
Amazon S3 AWS KMS Request
Managed Policy
Keys managed centrally in Amazon KMS with permissions and auditing of usage
Additional Options such as AWS CloudHSM, Hashicorp Vault & others
Application Level Security
Development Lifecycle
Authentication & Access
Secure Coding & Vulnerability Management
AWS Config AWS IAM AWS CloudTrail AWS Inspector
Application Security
Encryption & Security Options
corporate data center
users
Content
Servers
disk
tape storage
Processing Layer
Amazon S3
Amazon EBS
Amazon Glacier
KMS/ HSM Client side
encryption
role IAM
role
AWS Import/ExportSnowball
AWS cloud
EncryptedContent
Private Hybrid Model - Non Internet Facing
corporate data center
users
Content
Servers
disk
tape storage
Processing Layer
Amazon S3
Amazon EBS
Amazon Glacier
KMS/ HSM Client side
encryption
role IAM
EncryptedContent
role Direct Connect
S3
VP
C E
ndpo
int
AWS cloud
Key Management Service
Provide CPK for S3 encryption at rest
EC2, ETS can request the data-key on behalf
of customer Store and deliver object specific keys in Dynamo
S3 Ingest For Source, Renditions, Metadata Sidecar Files
Ingest
AWS Elastic Beanstalk
Content Consumption
CloudFront Distribution
Amazon DynamoDB
Individual Key Storage
Other Media processing on EC2 Elastic
Transcoder
Processing
Authentication/ Authorization
Content owner provides the master key
Sample End to End Media Security Workflow
Storage Security Controls
Amazon Glacier Vault lock
Permissions Access Logs AWS CloudTrail Versioning Durability
Amazon S3
Launch a CloudFormation stack with all the infrastructure
resources for a specific project
Autoscale the stack as appropriate
AMI
CloudFormation Launch Template
CloudFormation Terminate Template
Infrastructure Recycling
VPC Flow Logs
Amazon SNS
CloudWatch Logs
Private subnet
Value-add Service for High Valued assets
AWS Lambda
If SSH REJECT > 10, then…
Elastic Network Interface
Metric filter
Filter on all SSH REJECT Flow Log group
CloudWatch alarm
Source IP
You are making API calls and accessing your content ...
On a growing set of services around the world accessing your content
Amazon CloudTrail is continuously recording API calls…
And delivering log files to you…
Elas.cLoadBalancing
Amazon S3 Amazon Glacier
Amazon CloudFront
Amazon S3/Amazon CloudFront/App Logs
Access Logs
Feed Logs in Amazon Cloudwatch or monitor patterns on Logs
Act Fast or automate based on realtime notifications and alerts
Amazon Redshift
Amazon EC2
AWS IAM
Amazon RDS
Amazon Elastic
Transcoder
Log, Monitor, Act - Proactively
Content Distribution
§ B2B distribution
Distribution (B2B) Workflow Security
Content Delivery
Security of Distribution Workflows
• Secure Transfer (physical in many cases) • Encryption & Key Management • Access Control (bucket policies, …) • Logging and Monitoring (source, destination) • Multiple Accounts (destination consumable media) • Consumption Models (Requester Pays) • Centralized Logging (security account)
Security of the Distribution (content transfer) Workflow (B2B)
AWS cloud
Proxy Layer (Optional) Amazon S3
KMS/ HSM IAM
role
S3 VPC Endpoint
Vendors / Partners
Internal Users
Affiliates/Distributors
Fine grained temporary access
Temporary Access
Access Logs
Distribution account
INGEST STORE MANAGE SECURE PROCESS
CREATE MONETIZE
INTEGRATE DELIVER SECURE
Media Security Software on AWS
Comprehensive Cloud Controls for Media
§ AWS CloudTrail, Config & S3 Logs § Log-based Alerting (Splunk, AWS Elasticsearch, …) § HIDS Solutions (MP, Partners, …) § NIMS Solutions (MP, Partners, …) § OS Controls (SELinux, …) § AWS Inspector (CVE, …) § Pre-Authorized Pentesting (Qualys, …) § AWS Well Architected Program § AWS Security Playbooks § Disposable & Burnable Infrastructure § Pre-Baked AMIs (Packer) § AWS ECS (Docker / Containers) § Patch Management (AWS, MP, BYO)
Thank You!
How Wazee Digital and Amazon S3 Enable Bloomberg’s Customers with Global Access to Business and Financial Content
Company Overview April 2016
About Us
With over 10 million video files managed, Wazee Digital is the global leader in content capture, cloud-based video management and monetization services for sports, media and entertainment companies around the world. !!
5/18/16 Copyright © 2016, Wazee Digital, Inc. All rights reserved.
81
More than 300,000 Global Customers
5/18/16 Copyright © 2016, Wazee Digital, Inc. All rights reserved.
82
A Robust and Extensible Platform
5/18/16 Copyright © 2016, Wazee Digital, Inc. All rights reserved.
83
Core, Live Event Services and Commerce
5/18/16 Copyright © 2016, Wazee Digital, Inc. All rights reserved.
84
Wazee Digital's open supply chain means never compromising on technology choices !!
Core
5/18/16 Copyright © 2016, Wazee Digital, Inc. All rights reserved.
85
Wazee Digital’s services platform, combined with our experienced team, enables managed services that provide solutions to current industry challenges around global content supply chains and workflows…
REVIEW! MONETIZE! MANAGE!
DISTRIBUTION! ARCHIVE!
Core: Workflow Orchestration, Bloomberg
5/18/16 Copyright © 2016, Wazee Digital, Inc. All rights reserved.
86
Overview Customer since 2014 The Challenge Provide high velocity ingest, discovery and delivery service of images and videos to media syndication customers worldwide outside of Bloomberg’s proprietary terminal interfaces The Core Solution Acquire via Amazon S3 discovery policies with strict SLA expectations Manage transcode services via cloud-based services for elasticity Deliver using a robust, non-proprietary web interface
Key Points
• Enables licensing, direct purchase, subscription, and delivery of photos, interviews, short form and TV episodic content
• Close to a million assets under management
• Supports fast ingest services via Amazon S3 for immediate syndication of news content
Live Event Services
The sports industry generates the world’s most compelling moments every day to global audiences. These moments are witnessed by millions of viewers on a grand scale across broadcast and digital mediums alike. These moments live forever, offering a unique value to sports properties and their rights-holders. From ingesting and tagging content; to locating and distributing highlights with ease, our clients look to us to amplify and monetize every moment of their content.
5/18/16 Copyright © 2016, Wazee Digital, Inc. All rights reserved.
87
Sports News Studio Brands
Commerce Customers
Wazee Digital represents the rights of more than 400 media partners (aka Contributors) Sports, News, Studio, Major Brands and Independent categories…
5/18/16 Copyright © 2016, Wazee Digital, Inc. All rights reserved.
88
5/18/16 Copyright © 2016, Wazee Digital, Inc. All rights reserved.
89
May 18, 2016
The Bloomberg Content Service is a leading provider of global business and finance news, photos, video and data for more than 1,600 media outlets in 126 countries. 2,400
news professionals in 152 bureaus in 72 countries
Who we are
Clients include:
All Platforms. Everywhere.
Most Innovative Publisher of the Year 2014
TELEVISION 330M+ households worldwide 70M+ US households
DIGITAL 20.5M unique visitors
BLOOMBERG PROFESSIONAL SERVICE 325K+ subscribers in 174 countries
MOBILE 18.6M users
RADIO 27MM+ Listeners Stations in NY, SF, Boston and Sirius/XM nationwide
Print 1.3M+ circulation Bloomberg Businessweek, Bloomberg Markets & Bloomberg Pursuits
Flexible Models
Bloomberg Content Service licenses News,
Photos, Video, Linear TV, Data, and Bloomberg
Terminal to partners through:
• Subscriptions
• Partnerships
• Royalty arrangements
Rights Clearance & Ops Workflows • 100% Owned Content Ownership
• Standard Tagging
• Extensible Video Formats
• Multiproduct Integration
Media Source: Bloomberg’s Visual Media Portal
Why Bloomberg Chose Wazee Digital
Bloomberg Needs • Opportunity in footage licensing • Risk in existing photo portal/archive • One partner to deal with the storage,
transformation and delivery of assets • White-label portal solution
Wazee Capabilities • Core technology in place for its own
licensing business
• Video-native company with scale
• Experience with white-label portal solutions for premium brands
• Focused on custom needs
Wazee Core Benefits • Critical points of integration with AWS allowing for
seamless workflow • Additional points of monitoring to limit interruptions in
content flow • Normalizing of metadata to work across multi-asset
search on portal • Managing subscription rules and users access
AWS Integration • Ingest leverages AWS storage, monitors, queues,
transcodes and permissions • Limits transfers of large, mezzanine files and keeps
metadata current with origin file • Elasticity of AWS enables scale for both short-term
(news) and long-term (archive) needs • Avoiding capital investment facilitates workflows beyond
legacy systems
Content Ingest Leveraging Wazee Digital and AWS
Media Source: One Search For All Assets
Challenges and Learnings
• Demands for a “real-time” news service
• Archive versus news
• Integration of latest tech with reliable, and fully developed legacy systems
• Unifying metadata and search terms
• Understanding customer behavior and their needs
• Managing permission rules
• Transfer of large files to distant and low-bandwith markets
What’s Next
• Content Expansion with other prominent media brands
• Video products in other languages
• Packages of content across assets
• Improved API access
• Better integration with our own asset storage systems
• Combining Media Source front-end with Bloomberg’s news portal è
Thank you! Questions?
Cost Effective Rendering at Scale with EC2 Spot Usman Shakeel | Principal Solutions Architect M&E Amazon Web Services
Agenda
Cost Effective Rendering at Scale with EC2 Spot
VFX/Animation Rendering Computationally intensive Batch Process Non-deterministic Compute usage patterns Customer Sizes/Types Hybrid/All-in Cloud Workflows Architectures
AWS’s Spare Capacity at Scale Spot Features that make it super easy Terminations – What is it worth? Real world examples
Under 2 pennies per core hour What is the definition of “large” in scale Is it really cost effective? 1 3
2
VFX/Animation Rendering - workflow components
Composting Modeling Rendering
Asset Management
Collaboration and Task Management
Challenges in the VFX/Animation Industry
Increasingly Shrinking Budgets Cap-ex / Op-ex conundrum and flexible hardware needs Increasingly Global Workflows Increased Demand for Computation
• High Resolutions (4K, 8K and beyond) • 3D Stitching • VR, AR Stitching
Project based Infrastructure requirements • Budget, Quality, Render Time
A broad and complex Software toolset per project Security of Crown Jewels
The challenge of making a film
The challenge of making a film
On-premise capacity
The challenge of making a film
On-premise capacity
Rendering in the Cloud
The challenge of making a film
On-premise capacity
Rendering in the Cloud Cloud provides you the capability to scale fast and get the outputs faster
Initial project on-boarding artwork
Rendering Workflow Components
Storage
Render Farm
Pipeline and License Manager
Graphics Artist Workstations
Rendering Workflow Components (move to the cloud)
Storage
Render Farm
Pipeline and License Manager
Graphics Artist Workstations
• Content has gravity • Network Bandwidth • Hybrid/All-in Cloud • IO Performance
• Ability to burst at a very short notice
• Cost? • Performance • Security
• License mobility/Elasticity • Dependency Management
(hybrid scenario)
• Interactivity • High Performant
Storage • Hardware Support
Rendering in the Cloud - Hydrating the Cloud Renderfarm
S3 as the content repository for your content/data • On AWS Marketplace/SaaS
(Aspera, Signiant, File Catalyst, Expedat) • S3 Multi-part Upload • AWS import/export Snowball • S3 Transfer Acceleration NEW ! Direct to Shared File Systems • EFS throughput scales linearly to the storage • Lustre can hydrate from an S3 bucket • Avere can be fronted to S3 or an on-premise NAS • AWS Snowball NEW ! AWS Direct Connect
EFS S3 Multipart
AWS Snowball
Rendering in the Cloud - Shared FS Everywhere (some ideas)
Shared Storage On-prem Storage AWS Direct Connect
Storage Cache
Amazon S3
Luster on EC2
Avere on EC2
EFS
AWS Direct Connect
Hydrate workers
EC2 Spot
Shared Storage
FXT on-prem
Rendering in the Cloud - Shared FS (Content/Data Share) Everywhere
Elastic File System (Amazon EFS) • Designed to support Peta-Byte scale file systems • Throughput scales linearly to storage • Same latency spec across each AZ • Thousands of concurrent NFS connections • Works great for Large I/O sizes • Pay for only what you use not what you provision • Managed with multi-copy durability Amazon EFS
• BYOL • SaaS • AWS Marketplace • Elastic Licensing models
Thinkbox Deadline 8 Usage Based Licensing • Render nodes pull metered licenses from Cloud-based license server • Usage is tracked per minute • Bulk minutes will be available via Thinkbox’s online store • Hosts 3rd party licensing (Nuke, VRay, etc)
Rendering in the Cloud - Licensing at Cloud Scale
AWS M&E Partner Eco-System
INGEST STORE MANAGE SECURE PROCESS
CREATE MONETIZE
INTEGRATE DELIVER
SaaS BYOL PaaS
Rendering in the Cloud - Move the Graphic Artist to the Cloud …
Rendering is going Global • NVIDIA GPU based EC2 instances • Nice DCV • Teradici PCoIP • Windows and Linux (VNC+VirtualGL)
3D Modeler
Modeling Dumb Client
Remote Application running on a G2 instance
G2
Rendering in the Cloud - Managing your “disposable” infrastructure
Launch a CloudFormation stack with all the infrastructure
resources for a specific project
Autoscale the stack as appropriate
AMI
CloudFormation Template
CloudFormation Terminate Template
Rendering in the Cloud – Securing the Crown Jewels
• AWS alignment with the latest MPAA cloud based application guidelines for content security – August 2015
• VPC private endpoint for S3 – enables a true private workflow capability
• Encryption & key management capabilities • Glacier Vault for high-value media/originals
Rendering in the Cloud - A Sample Architecture (All in Cloud Pipeline)
Shared Storage
Renderfarm
On-Prem Storage
Pipeline and License Manager
3D Modeler
Remote App Visualization
AWS Direct Connect
Modeling Dumb Client
Storage Cache
Amazon S3
Avere on EC2
Scalable Renderfarm on EC2
Appstream or Teradici running on a G2 instance
Pipeline Manager running on EC2
G2
EC2 SPOT
EFS
Hydrate workers
EC2 Spot
Render Farm
Rendering in the Cloud - A Sample Architecture (A Hybrid Pipeline)
Shared Storage
Renderfarm
On-Prem Storage AWS Direct Connect
Storage Cache
Amazon S3
Avere on EC2
Scalable Renderfarm on EC2
EFS
Hydrate workers
EC2 Spot
On-premise Renderfarm
EC2 SPOT
Cloud renderfarm as an extension of on-prem renderfarm
FXT on-prem
Pipeline and License Manager (also manage cloud renderfarm)
Lets dig deeper into EC2 Spot
$1 ¢
On-Demand
Pay for compute capacity by the hour with no long-term commitments For spiky workloads, or to define needs
AWS EC2 Consumption Models
Reserved
Make a low, one-time payment and receive a significant discount on the hourly charge For committed utilization
Spot
Bid for unused capacity, charged at a Spot Price which fluctuates based on supply and demand For time-insensitive or transient workloads
Spare capacity at scale
AWS has more than a million active customers in 190 countries. Amazon EC2 instance usage has increased 93% YoY, comparing Q4 2014 and Q4 2013, not including Amazon use.
With Spot the rules are simple
Markets where the price of compute changes based on
supply and demand
You’ll never pay more than your bid. When the market exceeds your bid you get 2 minutes to
wrap up your work
Capacity pools
AZ1
AZ2
SYD Total Capacity
T2 C4 M4 I2 R3 D2
Shared
Dedicated
Shared
Dedicated
$0.27 $0.29 $0.50
1b 1c 1a
8XL
$0.30 $0.16 $0.21 4XL
$0.07 $0.08 $0.08 2XL
$0.05 $0.04 $0.04 XL
$0.01 $0.04 $0.01 L
C3
$1.76
On Demand
$0.88
$0.44
$.22
$0.11
Show me the markets!
Each instance family
Each instance size
Each Availability Zone
In every region
Is a separate Spot Market
50% Bid
75% Bid
You pay the market price
Bid Price Vs Market Price
25% Bid
¢
- ClusterK acquisition!- Spot Bid Advisor!- Spot fleet!- Spot blocks!
What Happened In 2015?
Amazon EC2 Spot – in the wild
1) We make this easy using the Spot bid advisor
2) With deliberate pool selection and bidding, you will keep your Spot instance as long as you need to.
3) And with new features like Spot fleet diversified we do the heavy lifting for you...
¢
Spot Bid Advisor – aws-spot-labs ¢
Spot fleet helps you
Launch Thousands of Spot Instances with one RequestSpotFleet call.
Get Best Price Find the lowest priced horsepower that works for you. or Get Diversified Resources Diversify your fleet. Grow your availability. And Apply Custom Weighting Create your own capacity unit based on your application needs
¢
It is easy!
aws ec2 request-spot-fleet --spot-fleet-request-config file://config.json { "IamFleetRole": "arn:aws:iam::781603563322:role/fleet-role", "TargetCapacity": "100", "SpotPrice": "0.03", "ValidFrom": "2015-09-15T00:56:19Z", "ValidUntil": "2016-09-14T07:00:00Z", "TerminateInstancesWithExpiration": true, "LaunchSpecifications": [ { "ImageId": "ami-0d4cfd66", "InstanceType": "c3.large", "WeightedCapacity": 2, "SubnetId": "subnet-d0dc51fb" }, { "ImageId": "ami-0d4cfd66", "InstanceType": "c3.large", "WeightedCapacity": 2, "SubnetId": "subnet-64531413" }, { "ImageId": "ami-0d4cfd66", "InstanceType": "c3.large", "WeightedCapacity": 2, "SubnetId": "subnet-0b1b8052" }, { "ImageId": "ami-0d4cfd66", "InstanceType": "c3.xlarge", "WeightedCapacity": 4, "SubnetId": "subnet-d0dc51fb" }, { "ImageId": "ami-0d4cfd66", "InstanceType": "c3.xlarge", "WeightedCapacity": 4, "SubnetId": "subnet-64531413" }, { "ImageId": "ami-0d4cfd66", "InstanceType": "c3.xlarge", "WeightedCapacity": 4, "SubnetId": "subnet-0b1b8052" }, { "ImageId": "ami-0d4cfd66", "InstanceType": "c3.4xlarge", "WeightedCapacity": 16, "SubnetId": "subnet-d0dc51fb" }, { "ImageId": "ami-0d4cfd66", "InstanceType": "c3.4xlarge", "WeightedCapacity": 16, "SubnetId": "subnet-64531413" }, { "ImageId": "ami-0d4cfd66", "InstanceType": "c3.4xlarge", "WeightedCapacity": 16, "SubnetId": "subnet-0b1b8052" }, { "ImageId": "ami-0d4cfd66", "InstanceType": "c3.8xlarge", "WeightedCapacity": 32, "SubnetId": "subnet-d0dc51fb" }, { "ImageId": "ami-0d4cfd66", "InstanceType": "c3.8xlarge", "WeightedCapacity": 32, "SubnetId": "subnet-64531413" }, { "ImageId": "ami-0d4cfd66", "InstanceType": "c3.8xlarge", "WeightedCapacity": 32, "SubnetId": "subnet-0b1b8052" }, { "ImageId": "ami-0d4cfd66", "InstanceType": "c3.2xlarge", "WeightedCapacity": 8, "SubnetId": "subnet-d0dc51fb" }, { "ImageId": "ami-0d4cfd66", "InstanceType": "c3.2xlarge", "WeightedCapacity": 8, "SubnetId": "subnet-64531413" }, { "ImageId": "ami-0d4cfd66", "InstanceType": "c3.2xlarge", "WeightedCapacity": 8, "SubnetId": "subnet-0b1b8052" } ] }
¢
An easy to use interface that lets you launch spare EC2 instances in seconds
Helps you select and bid on the EC2 instances that meet your applications requirements
Simple to use dashboard lets you modify and manage your application’s compute capacity
EC2 Spot Console
Spot Fleet – Focus on application needs
Using a single additional Parameter
Run continuously for up to 6 hours
Save up to 50% off On-Demand pricing
EC2 Spot block
$1 ¢
Capitalizing on two minute warning
When the Spot price exceeds your bid price, the instance will receive a two-minute warning Check for the 2 minute spot instance termination notification every 5 seconds leveraging a script invoked at instance launch
Sample script – two minutes left!
1) Check for 2 minute warning 2) If YES, run shutdown scripts 3) OTHERWISE, do nothing 4) Then sleep for 5 seconds
#!/bin/bash while true do if curl -s http://169.254.169.254/latest/meta-data/spot/termination-time | grep -q .*T.*Z; then /env/bin/runterminationscripts.sh; else # Spot instance not yet marked for termination. sleep 5 fi done
Real-Life Examples… Cost, Scale & Performance
¢
A Customer Example – Large Scale, Cheap, High Performant
A large scale example for animation rendering on AWS: • Hybrid Environment using Avere • All in Cloud Rendering using EFS • Automated environment leveraging Spot Fleet • Launched 40K cores in 20 min at < $0.02/core/hr for the particular rendering workload Findings: EFS performance for rendering Hybrid Rendering Scenarios
http://www.slideshare.net/ AmazonWebServices/ cmp404-cloud-rendering-at-walt-disney-animation-studios
Customer Example - Spot Fleet Deployment
Core Count
./aws_spot_fleet_request-preinvent--cpu8--ram64-m4.7-c1500
EFS Performance in a real rendering scenario - Average Open Latency
EFS Performance in a real rendering scenario - Average Read Latency
0
100
200
300
400
500
600
700
100 500 800 1200 2400 4000
Tim
e (µ
s)
Render Processes
Mid-TierA Mid-TierB Mid-TierC Archive EFS
Customer Example Rendering in the Cloud vs. On-Premise
!"!!!!
!5,000!!
!10,000!!
!15,000!!
!20,000!!
!25,000!!
!30,000!!
1! 10! 20! 30! 40! 50! 60! 70! 80! 90!
Ren
der T
ime
(s)
Frame #
EC2/EFS!
On!Prem!
Lower is better
The $9 Billion Experiment 50,000 physical cores to meet the 1500 scientific researchers demand
Over 5 days, less than 1% of instances were terminated, leaving them with a significant margin of safety. Instead of building a 50,000 core data center they were able to successfully use AWS Spot for 5 days and pay just $45,000
Another customer example - Large Scale, Cheap, High Performant
Parting thoughts
VFX/Animation rendering workloads can be streamlined on the cloud • Avoid Data/Content movement • Distribute Single job across multiple nodes • Manage state often • Segregate subworkflows (winthin a single pipeline) between incloud and on-premises based on dependancies
Rendering in the Cloud is possible and can be more performant over traditional hardware setup
• All-in Cloud vs. Hybrid • Technical Feature set has come a long way from even a year ago
AWS EC2 has a VERY Large Capacity @ CHEAP • EC2 Spot (Fleet, Block) and Reserved Instance models
Questions / Comments / Feedback
v
Acquisition
DAM & Archive
Media Supply Chain
Publishing
Playout & Distribution
Analytics
OTT
VFX & NLE
Content Distribution & Media Supply Chain
All In: Cloud Transformation of the Media Industry Alex Dunlap – GM, CloudFront, AWS Keith Wymbs – CMO, Elemental Technologies May 16, 2016
“Why cloud computing?”
2006
Today, the Benefits Are Very Well Known
?Move from operational
to variable cost Lower variable cost than most
companies can achieve No need to
guess capacity
Agility, speed & innovation
Remove undifferentiated heavy lifting
Go global in minutes
Customers are experiencing these benefits across all
Industries
Workloads
Company Sizes
Bold Move in 2009
Time Inc. is going all-in on AWS, migrating five of its global data centers to AWS. The company has already reduced costs by 75% across 80 web properties that deliver more than 120 million impressions each month.
The Destination for Many Enterprises: All-in
But isn’t media different?
Why are Media Customers migrating now?
-0.8%
-0.6%
-0.4%
-0.2%
0.0%
0.2%
0.4%
1-10 11-20 21-75 75-150 150+
Y/Y
Cha
nge
in %
of T
otal
Vie
win
g H
ours
Cap
ture
d
Network Groupings: Ranked in Order of C3 Hours Viewed
Y/Y change in % of total viewing hours captured (TTM thru Feb. 2016)
Top networks and tail networks have gained share at the expense of the middle.
Y/Y Change in Distribution of Viewing
Sources: Rentrak, Pacific Crest Securities Sources: Rentrak, Pacific Crest Securities
Source: Pacific Crest, Rentrak 3/16
Trade capital expense for variable expense
Pay for media you store and process, as
you go
Benefit from massive economies of scale
AWS can adapt to your media storage and compute needs
Stop guessing capacity
Handle unpredictable and bursty media needs
Increase speed and agility
Decrease time-to-market, test out new approaches
Go global in minutes
Global availability instantly, with no commit
Stop maintaining data centers
Focus your resources on your media needs
Digital Media & the Cloud
Perfecting the Media Experience
Connecting viewers with content at massive scale
Three use cases: Building a Better On-Ramp to the Cloud
coverage
coverage
Solution Challenge
Benefit
Rapid delivery of content with the flexibility to easily adopt new formats and standards ensuring service availability on all devices
Cloud-based resources provide elastic video processing capacity with dynamic scaling to absorb spikes in demand
Serve catch-up programming to millions of users per day across 1,000 multiscreen device types
BBC USE CASE: DAILY CATCH-UP
Minimum: 5 nodes
Peak usage: 80 nodes
• 19k hours / month
• 4,200 steady state
Example daily Elemental Cloud usage
Excess capacity
• 13k hours capacity
• 6,600 hours unused
On-premises comparison
BBC Video Factory Workflow
• Video Factory captures live broadcasts and sends to Amazon cloud storage
• Processing jobs get picked up by idle transcoder or create new instance
• Elemental Cloud processes multiple H.264 ABR video streams
• Elemental Cloud ramps dynamically with processing demand on AWS EC2
Live TV
SOURCE CDN
AWS DataTransfer
STORAGE
AWS S3
ELASTIC VIDEO PROCESSING DATA TRANSFER
AWS Direct Connect
iPLAYER DISPLAY DEVICES
coverage
coverage
Solution
Quickly and efficiently prepare 3,900+ hours of live and catch-up World Cup coverage for delivery to viewers on any OTT device
Challenge
Benefit
The platform provides an end-to-end second-screen solution for live streaming feeds, multi-angle content and VOD assets to sports fans
Elemental Cloud on AWS EC2 provided encoding elasticity with the ability to reliably deliver high quality, high resolution live content across multiple streaming protocols across the globe
Quickly and efficiently prepare 3,900+ hours of live and catch-up World Cup coverage for delivery to viewers on any OTT device
Case Study: Live Events for 2014 World Cup
• Peak Usage: 140 nodes
• 15k total usage hours
Elemental Cloud usage
• 111k hours capacity
• 96k hours unused
excess capacity
On-premises comparison
2014 World Cup Live Streaming Workflow
• 10Mbps live HLS input streams delivered from Rio to Dublin
• Elemental Cloud services ingested up to 48 concurrent inputs
• Each input converted to 9 ABR outputs, 2 thumbnails, and 1 RTMP feed
• Content delivered to regional broadcasters via CDN
ELASTIC VIDEO PROCESSING
Live TV
STADIUMS FIBER
Accelerated
GLOBAL
Broadcasters
DISPLAY DEVICES
HLS Inputs to AWS S3
S3 STORAGE DELIVERY
AWS Data Transfer
Case Study: Amazon Video Live Streaming
Satellite and Fiber Network
SOURCE
LOAD BALANCER
DISPLAY DEVICES CDN 2
CDN X
Amazon CloudFront
ELEMENTAL CLOUD
LOAD BALANCER
AWS REGION 2
Direct Connect
INGEST LOGIC
AWS REGION 1
Direct Connect
INGEST LOGIC
ELEMENTAL CLOUD
Amazon Redshift
Amazon SQS
Amazon EC2
Amazon SNS
OTHER AWS SERVICES USED Amazon Route S3
Amazon Video Live Streaming Workflow
Improving CloudFront’s Media Delivery Capabilities
AV’s Long Tail Demand Profile Presents a Challenge to CDNs
More Popular Titles Less Popular Titles
% o
f Ses
sion
s w
ithou
t err
ors
and
rebu
ffers
Other CDNs
Why doesn’t the traditional CDN work for long tail media?
Limited storage at the edge
Least recently used eviction
policy
Media files are big and getting bigger
Objects requested frequently
High cache hit ratios
Good playback experience
Objects requested infrequently High cache miss ratios Poor playback experience (latency, rebuffers)
Solution: Build Infrastructure Optimized for Throughput and Storage
Interconnect Fabric
Tier 1: Transit Layer
Tier 2: Caching Layer
Border
Transit / Peers
Interconnect Fabric
Border
Transit / Peers
Large-Object Store
Large-Object Store
Large-Object Store
Backbone
Benefits
Benefits
Direct peering with major ISPs in multiple internet
exchange facilities
Improved Throughput Better Caching Lower Costs
Petabytes of content storage at the edge
Decouples storage from network
capacity; scale each separately as
demand warrants
Results: CloudFront winning traffic where we have deployed new sites and raising the bar on quality
CloudFront Traffic Growth CloudFront QoS Improvements
wee
k 1
wee
k 3
wee
k 5
wee
k 7
wee
k 9
wee
k 11
w
eek
13
wee
k 15
w
eek
17
wee
k 19
w
eek
21
wee
k 23
w
eek
25
wee
k 27
w
eek
29
wee
k 31
w
eek
33
wee
k 35
w
eek
37
Peak
Thr
ough
put (
Gbp
s)
Peak Throughput of Amazon Video served via CloudFront
wee
k 1
wee
k 3
wee
k 5
wee
k 7
wee
k 9
wee
k 11
w
eek
13
wee
k 15
w
eek
17
wee
k 19
w
eek
21
wee
k 23
w
eek
25
wee
k 27
w
eek
29
wee
k 31
w
eek
33
wee
k 35
w
eek
37
% o
f Ses
sion
s w
ithou
t err
ors
and
rebu
ffers
Quality of Service of Amazon Video served via CloudFront
Results: Dramatically improve performance on long tail portion of content
More Popular Titles Less Popular Titles
% o
f Ses
sion
s w
ithou
t err
ors
and
rebu
ffers
CloudFront Other CDNs
Our relationship with AV also influenced other media streaming specific improvements to CloudFront
Intelligent Pre-fetching
5 Mbps
2.5 Mbps
1 Mbps
512 Kbps
256 Kbps
Dynamic Manifest Support
CloudFront Edge
Pre-fetch video fragments into cache at the requested and adjacent bitrates to reduce cache misses.
AV URL Vending Service
Dynamic Manifest Service
Media Fragments Media
Fragments
Built support for requesting a dynamically generated manifest to optimize bitrate availability based on device to improve quality of playback.
.m3u8
AWS Digital Media Customers
NASA/JPL
In Summary
The cloud enables media and entertainment companies to connect premium content with viewers at massive scale Elastic cloud platforms help:
• On-demand applications cope with peak loads • Optimize live event workflows • 24/7 live linear content with maximum resiliency
Media-optimized CDN offerings can improve performance and quality
Thank You
Cloud Technology in the Media Supply Chain, Advertising and Analytics Chris Blandy, EVP Technology Solutions, Fox Networks Engineering & Operations Simon Eldridge, Chief Product Officer, SDVI Corporation Joshua Rangsikitpho, Chief Technology Officer, True[X]
18th May 2016
Today’s Discussion
Why Cloud? Industry Trends SDVI True[X] Q & A
Why Cloud?
Agility Time to Market Pay-as-you-go Cap Ex vs Op Ex Enables “Software-Defined” platform future
Trends
Cloud innovation is outpacing traditional on-premise solutions Broadcast technology moving to IP Rapid growth in non-linear/VOD consumption Data & Analytics Evolving advertising model
Fox NE&O at a Glance • NE&O underpins our broadcast and cable networks business • Supports 35 networks and 40k hours of live content annually • 3,984 programs produced and ~330k hours of content played out per year across 3
NE&O managed production facilities
CreaJve
FuncJons Clients
DistribuJon
Engineering&FaciliJes
• FoxSports• FBC• FX/FXX/FXM• FXP
• FoxSports• FBC• FX/FXX/FXM• FXP
• FBC/FOXNow• MyNetworkTV• MundoFOX• 20thDom.Synd• 20thInt’lSynd• FoxTVSta.ons• FXSuite/FXNow• NGSuite/NGTV• FSRacing
• FS1/FS2• RSNs(14)• FCS(3)• FoxSoccer+• FSNNetBase• BigTen• FoxSportsGo• FOXSports.com• FoxDeportes
ThirdPartyClients• FOXNews/Business• OtherFBCAffiliates• MVPDs• FNG/21CF
KeyMetric LosAngeles Woodlands Charlo[e Total
Produc.onControlRoomsSupported:
10 - 3 13
GraphicSystemsSupported: 105 - 30 135EditSystems: 221 - 24 245AudioEdi.ng: 15 - 1 16MasterControlRooms 24 39 - 63
SatelliteDistribu.onPaths 46 36 - 82
DesktopEdi.ng/MediaHandling:
284 - 80 364
NE&OByLoca,on
Fox NE&O is a factory with raw content inputs and varying distribution outputs
CreaJv
e
‒ DeliverySpecs‒ Schedules‒ ProgramPlanning
Distrib
uJon
‒ Schedules‒ Commercialdublists‒ Logs
Strategy&Planning1
ProducJonContent/Prep
2LiveProducJon
3DistribuJon
4
InformaJonGathering/Prep
1ContentIntake/
Staging
2Assembly/Packaging
3DistribuJon
4MVPDs
Affiliates
3rdPartyPladorms
‒ Content‒ LiveFeeds‒ FilesIn
puts
Inpu
ts
‒ Commercials‒ 3rdPartyPrograms
NE&O’s Long Term 2020 Strategy A B C
CreaJv
eDistrib
uJon
NE&OCurrentlyPursuing
ShibFromHardware-BasedtoSobware-BasedSystemsforMulJ-PlacormDistribuJon
DevelopFullCloudCapabilityForEnd-To-EndNE&O
Processes
CompleteMigraJontoCloud&
DynamicContentAssembly
ScaleforDirecttoConsumer?
Phase1 Phase2 Phase3 2020
FY15-FY16 FY16-FY18 FY17-FY19 FY20&Beyond
NE&O Efficiency Benchmarks
1.8 1.6
1.11.0
0.80.7
0.00.20.40.60.81.01.21.41.61.82.0
2001 2006 FY11 FY12 FY13 FY14
PerN
etworkHeadcou
nt
$1.0
$0.7 $0.6$0.5
$0.2$0.2
$0.0
$0.2
$0.4
$0.6
$0.8
$1.0
$1.2
2001 2006 FY11 FY12 FY13 FY14
PerN
etworkCo
st($in
millions)
CableNetworksOperaJonsLaborCosts
#ofNetworks: 5 8 12 12 14 16
CableNetworkManagementOverheadEfficiency
#ofNetworks: 5 8 12 12 14 16
$1.0
$0.7 $0.6$0.5
$0.2$0.2
$0.0
$0.2
$0.4
$0.6
$0.8
$1.0
$1.2
2001 2006 FY11 FY12 FY13 FY14
PerN
etworkCo
st($in
millions)
Introducing SDVI
SDVI provides a suite of SaaS-based infrastructure management applications and services that enable dynamic management of the 3rd Party Applications and Resources required to publish premium content to consumers, via television or digital distribution channels.
What does SDVI solve?
• Spin up or spin down media infrastructure in minutes • Shared versus dedicated infrastructure • Infrastructure resources on-demand • Best-in-class 3rd party applications, with no lock-in • Facilitates the move to opex for media supply chains • Accurate cost tracking and reports • Infrastructure analytics and modeling
Linear Broadcast
Television & Network Feeds
Video Content and Metadata
(from suppliers)
Fox Broadcast Center Los Angeles
Manual Verification
Content Normalization
Content Captioning
Metadata Systems
Content Quality Check
Playout Systems
Existing Content Factory
The SDVI Platform SDVI Applications
• Cloud-resident, workflow specific SDVI Applications that leverage the 3rd Party Applications, SDVI Platform Services and virtualized infrastructure to address common operational problems
SDVI Platform Services • Back-end SDVI services that provide functionality to the platform such as
resource management, analytics & optimization
SDVI Adapters • Connectors to 3rd Party Applications, and on or off premise processing, storage &
networking resources
3rd Party Applications • 3rd Party Applications such as transcoding, file-based QC and network/ router
control
Infrastructure • The infrastructure includes processing, storage and networking resources which
may be located in private or public cloud, or on premise
SDVI Applications
SDVI Platform Services
SDVI Adapters
3rd Party Applications
Infrastructure
Infrastructure Management for the Media Supply Chain
Fox Broadcast Centre Los Angeles
Content Quality Check
Content Normalization
Content Captioning
Distribution-ready content and BXF metadata
Future Distribution Platforms
Single sign-on & 2-factor authentication via Bypass Path
Manual Upload
Video Content (from suppliers)
Customer Portal (content suppliers)
Content Metadata
Expected results of the migration
• Reduced labor costs associated with supply chain • Increased operational resilience • Vanguard of ‘cloud-first’ strategy • Transition to pay as you go model, transparent pricing on
a unit cost basis • Elastically absorb peaks in volume • Leading the ecosystem of broadcast engineering tool
providers to software-defined, cloud ready solutions
true[X] is the gold standard for engagement advertising. We deliver the most effective format to gain real consumer attention and form meaningful brand connections in the digital space.
Introducing True[X]
Joshua Rangsikitpho Chief Technology Officer, True[X]
Responsible for: • Ad technology • Data engineering
Engagements
Your ad is full screen ad without any competition or distractions. It’s just the consumer and your
message.
100% SOV
Your ad runs on the most viewable ad
network, verified by MOAT. You run an ad
and people really see it. It is that simple.
100% Viewable
This is guaranteed fraud-free engagement. Only real humans with real eyes and real ears interact with your brand.
100% Bot Free
Consumers initiate the brand experience, which means they only see your ad if they choose to. The
most meaningful interactions happen when
both parties agree to engage.
100% Consumer
Opt-in
Engagement Serving Platform
Load balancer distributed traffic across multiple availability zones (ELB, EC2)
AutoScaling + Scheduled scaleups for live events. Stress tested to 20k requests/second. (ELB, EC2, AutoScaling Groups)
All system performance is tracked and anomalies trigger alerts with pre-defined escalation paths (Cloudwatch, Sensu, Pagerduty)
Elastic Scale
Monitored
Highly Available
Audience
Data collected directly on the Fox (ie. CRM, subscription data)
Party Data
Data offered by external data collection companies (ie. behaviorally modeled data)
Data collected by the advertiser (ie. purchase history)
1st 1st
Party Data 2nd
Party Data 3rd
Audience Platform
Multiple AZs via synchronous data replication (DynamoDb) Highly Available Data Backend
Real Time Targeting
Data pipeline loads external data sets into internal DMP (EMR) 1st, 2nd and 3rd Party Data Ingest
Data backend supports dynamic scaling (DynamoDb)
Analytics
Campaign Performance
Brand lift metrics determined through
surveys
Brand Lift
All user interactions within the engagement (ie. mouse
click/movement, demographic distribution,
frequency)
Viewership and monetization health
across the Fox ecosystem
Fox digital performance
Analytics Platform
Flexible, fault tolerant connection between ad serving and data layer (SQS, Aurora)
Simple, SQL enabled data warehouse for adhoc analysis and visualization layers (Redshift, Tableau)
Highly scaleable and cost effective analytics engine supporting complex data extractions and transformations (Spark, EMR, Spot Instances)
Big Data Analysis
Big Data Processing
Resilient Data Pipe
Q&A
WGBH Chief Technology Officer - Stacey Decker
§ Spend the time to understand your problems ü What is costing so much money?
ü Where are your roadblocks?
ü Are we structured properly?
ü What are the organizations strategies?
§ How can technology investments be realized?
What's not working? Why…..
207
WGBHCONFIDENTIAL
Set a Strategy….
Goals
Objectives
§ Create efficiencies in the PBS system § Retain local control of content and distribution
§ Technology that prepares for the future § Content, anytime, on any device § Redundant and high availability (Accessible) § Reduce local expenses § Leverage cloud computing and network services
WGBH and Sony create PMM
• Public Private Partnership
• Created for Public Media
• Service Orientated • Strong Vendor
Partners
Current PTV state
With PMM
Major functions • Processes RT & NRT files (Transcode/rewrap, QC and metadata entry) • Transfers files to Sony Cloud • Manages overall health of system (NOC, node and connectivity to Ci)
Network Operations Center
Crispin/Harmonic workflow roles • Manage real-time recordings via
Harmonic Amberfin/Digimetrics workflow roles
• Rewrap content to AS03 specs • Provide auto and manual QC
Aspera file acceleration workflow roles • Manage content uploads to Ci
Network Operations Center
NOC operators workflow roles • Perform manual QC and logging as required • Oversee health of end-to-end PMM system
Network Operations Center
Your PMM node
Major functions • Receives information on available assets
from cloud • Automatically downloads the required
content – based on your unique schedule • Provides approx. 30 days of local node
storage • Supports local ingest from file, tape and
satellite • Utilizes local PBS station services
platforms
§ Crispin/Harmonic workflow roles – Automate real-time play-out &
recordings via Harmonic, as well as live switching when needed
§ Ci workflow roles – Provide online content storage and
services including MAM functions
Your PMM node
§ Benefits to a station – No need for costly dedicated fiber – Automatically downloads the required
national content — based on your schedule
– Inclusive branding software providing ability to air local snipes, lower thirds, bugs, etc.
– Disaster preparation — multiple days of content stored in node
Your PMM node
PMM timeline to date
§ Proof of concept to live-on air in less than a year
218 CPB recommends JMC solutions
PMM Cloud overview
Major functions today
§ Creates multiple proxies § National content stored reducing
duplication § Living archive that can be accessed
and managed § Content sharing and collaboration
PMM Ci Cloud
Where will the take us ???
§ Cloud based services – Closed Captioning – Cloud editing – Social tools – Data driven distribution – Emotionally driven content
PMM Ci Cloud
Lessons learned
§ Determine your problems…..
§ Set your goals (what are you trying to accomplish)
§ Good project management is key
§ Work before the work (storage strategies)
§ Leverages existing relationships with “the good ones”
§ Don’t be afraid to pivot
Project highlights
Sony | Ci Media Cloud Platform
Connects Teams, Facilities and Systems
Broadcast Facility, Post House, Vendors.
Traffic, Automation, OVP, MAM, etc.
Team NYC Workspace
Real-time contribution, collaboration, secure delivery and content processing for media workflows
Shoot + Acquire Pack + Ship Rough Cut + Review Integrate + Orchestrate
Automate and Simplify Redundant Tasks
Validate
Extract Metadata
Transcode Encrypt
Index
Notify
CDN
Store & Archive
Caption Transform
Ci Workspace gives you total access every step of the way.
Common Workflows
Collaboration Upload
Workspace + VideoReview &
RoughCut
Send MediaBoxes
Editorial Upload
Create edit-ready
proxies
Auto-deliver to facility
DR copy archived
Content Prep Upload
Transcode
QC
Caption Transform
Pull to air
100’s Terabytes processed every month Multiple Petabytes being managed In April… 6000 hours of video ingested (Peak was 50,000 hours) 200,000 images ingested 50,000 MediaBoxes sent
Built on AWS natively for scale
Scalability with Controls
Intelligent and tunable scaling limits per customer Spot instances when appropriate Optional features enabled per Workspace
Who uses Ci?
Who uses Ci?
Some Customer Platform Examples
Customer Examples
Who: Largest independent station group of major affiliates in top 25 markets with 46 stations What: All syndicated programming flows thru Ci where it is prepped for QC and pulled to air by each station.
Customer Examples
Who: Motion Picture, Television, Digital Studio What: Built their next generation DAM on top of Ci. “Runner” used by all divisions of the company. Archived > 2PB of content with Ci.
Customer Examples
Who: Fan engagement for modern sports organizations What: Built their FanCam experience on top of Ci. User Generated content is uploaded to Ci, curated by operations and pushed to in stadium displays
Customer Examples
Who: JV with Library of Congress/WGBH Archives What: 40,000 hours of broadcast content ingested into Ci and integrated with their customer portal
In Summary
Ci leverages AWS to eliminate constraints and provide customers with a simple on-ramp to cloud Our goal is to enable media workflow at scale with simple integration capabilities More at: http://sonymcs.com
Thanks!
Thank you
Content Publishing
v
Acquisition
DAM & Archive
Media Supply Chain
Publishing
Playout & Distribution
Analytics
OTT
VFX & NLE
5/18/2016
Shaown Nandi VP, Head of Infrastructure and Cloud
CLOUD TRANSFORMATION JOURNEY
Save $100 Million annually Reduce from 50 data centers globally to 6 75% of computational power in the Cloud
AUDACIOUS GOALS SET IN FY14 BY NEWS CORP
Increase in AWS instances
400%
Dow Jones c loud compute
53%
Replicate True Lift and Shift
ReHost
Update Infrastructure (OS / DB / MW)
ReFactor Redesign to Cloud Native, leverage PaaS
Remediate Minor Application Changes, CI/CD Pipeline
Cloud Migration starts with an Operating Model
Multi-modal Operations
Be Proactive in Building Relationships!
“Automated Efficiency” allows non-DevOps apps the ability to leverage the cloud to be more scalable & resilient while reducing cost
Legacy Model
Digital Apps Transforming
Building a new ordering process on AWS
THANK YOU! QUESTIONS?
Digital Advertising and Content Monetization Dmitri Tchikatilov, Business Development Amazon Web Services
Content Monetization Trends Web
• Content discovery platforms
• Improved Mobile Web
• Traditional Display spend is declining
Video • Ad Spend
Desktop $5B Mobile $5B
• Subscription Spend $5B
• US Mobile Video is up 41% (2016)
Mobile
• Mobile ad spend up 50%
• Search • Social • Video
All numbers US – courtesy of eMarketer
Digital Content Strategy and Monetization
Content Strategy
Content Creation
& Lifecycle
Content Delivery
Payments Analytics
Ad Revenue
Where Does Cloud Make a Difference?
Paid subscribers
registered
identifiable
anonymous Integration with 3rd parties Prospecting, retargeting
Content recommendations Optimized user experience
Premium content Content control and distribution
User cross sell, lifetime value, churn
Dat
a In
sigh
ts
Advertiser Solutions
Ad Networks and Exchanges
Publisher Solutions
Content Publishers
Brands and
Marketers Data Providers
Digital Advertising Industry Scope for AWS
Industry Scope
Publishers
”
“ • Decided to migrate its brands and data centers to AWS cloud
• Worked with AWS tools and partners to ensure security of customer data
• Found successful solutions for wide range of security challenges
• AWS tools and solutions expected to save millions in security cost savings
Time Inc. All-in on AWS
Time Inc. is a leading global media and entertainment company. It is based in New York City.
AWS gives us the environment to be 100% ready as we complete
our move to the cloud. Keith O’Sullivan
Vice President, Global Information Security Time Inc.
”
“
”
“ • 3 months project • reduced costs by 40% and increased
operational performance by 30-40% • over 500 servers • one petabyte of storage • various mission critical applications
(such as HR, Legal, and Sales) • over 100 database servers
Conde Nast: All-in on AWS
AWS enables the business to create content better and
faster... We can also adapt as we need.
Joe Simon EVP-CTO, Conde Nast
”
“ Hearst: 30 TB of Clickstream Data Daily on AWS
Hearst is one of the world’s largest media and information companies, with more than 360
businesses.
I don’t know how we could have made our clickstream data pipeline work without
Amazon Kinesis.
• Needed to develop a platform to analyze real-time clickstream events and trending content.
• Uses Amazon Kinesis Streams and Amazon Kinesis Firehose to transmit 30 TB of daily clickstream data.
• Processes data from more than 300 websites.
• Delivers clickstream data to editors in minutes.
• Increases recirculation of trending content by more than 25 percent.
Peter Jaffe Data Scientist,
Hearst Corporation
”
“
AWS Data Pipeline
Buzzing API
API Ready Data
Amazon Kinesis
S3 Storage
Node.JS App- Proxy Users to
Hearst Properties
Clickstream
Data Science Application
Amazon Redshift
ETL on EMR
100 seconds 1G/day
30 seconds 5GB/day
5 seconds 1G/day
Milliseconds 100GB/day
LATENCY THROUGHPUT Models
Agg Data
Mobile Advertising
”
“ Mobile Application Monetization
Inneractive is a mobile ad exchange that provides technologies for the buying and selling
of mobile advertising space.
The company has saved tens of thousands of dollars. That’s between 20 and 30 percent of
our total monthly AWS bill.
• Mobile Ad Exchange
• 450 million unique users per month
• 15–20 terabytes of raw data each day
• Uses EC2 Spot Market to reduce the cost – partner solution by Spotiinst
• Uses Amazon Redshift for data analytics
Gal Aviv Research & Development Group Manager
”
“
”
“ Mobile Monetization - Local Advertising
Localytics is a provider of mobile application monetization solutions
AWS helps us get new services to our customers faster. For a startup, faster
time to market is key.
ü Mobile engagement and analytics platform
ü 37,000 apps
ü 3B devices worldwide
ü AWS Kinesis, Lambda Mohit Dilawari Director of Engineering
”
“
Latest Developments - Big Data on AWS
Analysis of streaming data Apache Spark streaming, Kinesis Shorter time to insight
Serverless and event-driven architectures Shorter time to market
Data processing and machine learning in the same framework (Spark, Flink) Agility & flexibility
On the Buy Side
New Ad Technologies: In Image
New advertising formats IN-IMAGE ü 400M daily visitors across 2,000
premium publishers ü 1 billion events (approximately 6 TB
of data) every day. “With AWS, we can smoothly process billions of events daily with 24/7 reliability, and can scale quickly to meet spikes in demand...” Ken Weiner Chief Technology Officer
GumGum Architecture
“We run the RTB platform on more than 2,500 machines, approximately eight hours a day globally, at a cost of less than $0.05 per day per machine...” “Because we’re running on AWS, we’re able to focus 95 percent of our staff on new product development. Using AWS allows us to focus on innovating our platform and solving customer problems.” Valentino Volonghi, CTO AdRoll
Efficiency & Scale: Enabling Real Time Bidding
AWS Regions as Centers of Gravity
AdverJser
AdExchange
AdNetwork Publisher
AdNetwork
AdNetwork
AdNetwork
AdverJser
Publisher
Benefits: ü Lower latency ü Lower traffic costs ü Large scale secure
B2B data sharing
High Growth in ü Mobile ü Video
What's dangerous is not to evolve... Jeff Bezos
Thank You
Learn more at aws.amazon.com/digital-media For follow up, feedback and questions contact: Ian McPherson – [email protected]