AWS Service Drill Downs - AWS Symposium 2014 - Washington D.C.

AWS Government, Education, and Nonprofits Symposium Washington, DC | June 24, 2014 - June 26, 2014


AWS Service Drill DownsLarry [email protected]


AWS Global Infrastructure

Application Services

Networking

Deployment & Administration

DatabaseStorageCompute



10 AWS Regions51 AWS Edge Locations



Availability Zone A

Availability Zone B

Availability Zone C

EU (Ireland)

Availability Zone A

Availability Zone B

South America (Sao Paulo)

Availability Zone A

Availability Zone B

Asia Pacific (Sydney)

Availability Zone A

Availability Zone B

GovCloud (OR)

Availability Zone A

Availability Zone B

Availability Zone C

Availability Zone D

US East (VA)

Availability Zone A

Availability Zone B

US West (CA)

Availability Zone A

Availability Zone B

Asia Pacific (Singapore)

Availability Zone A

Availability Zone B

Availability Zone C

Asia Pacific (Tokyo)

Availability Zone A

Availability Zone B

Availability Zone C

US West (OR)

Customer Decides Where Applications and Data Reside

US REGIONS GLOBAL REGIONS

Note: Conceptual drawing only. The number of Availability Zones may vary.




Networking



Networking


AWS Government, Education, and Nonprofits Symposium Washington, DC | June 24, 2014 - June 26, 20142013 AWS Worldwide Public Sector Summit

Networking• Complete networking isolation and private network addressing inside

the AWS cloud

• Connect existing infrastructure to a set of isolated AWS compute resources via a Amazon Virtual Private Network (VPN) connection

• Bring your own address space and naturally extend existing networking and management capabilities

• Rich routing features, Network ACLs, Elastic Network Interfaces (virtual network cards) for Amazon EC2 instances, etc.

• Network Security Groups (hypervisor-enforced firewall rules) provide comprehensive, fleet-wide, API-driven control over all network flows

Amazon VPCIsolated Cloud Resources

Amazon Virtual Private Cloud


NetworkingAmazon VPCIsolated Cloud Resources

EC2

10.0.2.12

AWS Region – Amazon VPC network isolation

AZ A AZ B

VPC 10.0.0.0/16

SN 10.0.1.0/24 SN 10.0.2.0/24

(23.20.103.11)

Internet

EC2

10.0.1.11

Internet GW


Networking

• Route end users to Internet applications and endpoints

• Answers DNS queries with low latency by using a global network of highly available DNS servers

• Latency based routing to closest AWS endpoint (e.g. Amazon EC2 instances, Elastic IPs, or ELBs)

• Deep integration with other AWS services (ELB, Amazon EC2 Elastic IPs, Amazon S3, Amazon CloudFront, etc.)

• DNS service health-checks and automatic failover


Amazon Route 53

Amazon Route 53Scalable DNS


Networking

• Establish a dedicated Layer 2 network connection from your premises to AWS

• Segment traffic on the customer side using industry standard 802.1q VLANs

• Multiple virtual interfaces may be configured to access AWS services such as Amazon EC2 and Amazon S3 using public IP space, or resources in a VPC using private IP space.

• Choose 1 Gbps and 10 Gbps port speeds, one or more links


AWS Direct Connect

Amazon Route 53Scalable DNS

AWS Direct ConnectDedicated network connection to AWS




Networking


DatabaseStorageComputeCompute

Networking


Compute• Resizable compute capacity in 18 instance types

• Reduces the time required to obtain and boot new server instances to minutes or seconds

• Scale capacity as your computing requirements change

• Pay only for capacity that you actually use

• Choose Linux or Windows

• Deploy across Regions and Availability Zones for reliability

• Support for virtual network interfaces that can be attached to Amazon EC2 instances in your VPC

Amazon EC2Virtual servers in the cloud

Elastic Compute Cloud


Compute• Building blocks of Amazon EC2 instances; an AMI is like a generic

template of a computer's root volume

• One-click creation from a running VM of your choice

• Can be private, public, or shared with selected accounts

• Create hardened or “gold images” of your Amazon EC2 infrastructure; use AWS Identity and Access Management (IAM) permissions to limit access to non-blessed images


Amazon Machine Image


Compute• Block storage devices from1GB – 1TB for use with Amazon EC2

instances – create, attach, snapshot (backup), restore and delete

• Storage volumes are attached to an Amazon EC2 instance and exposed as a block device for raw or formatted (file system) access

• Volume lifecycle can be completely independent from instance lifecycle

• Optionally create RAID configurations for any server

• Ideal use cases:

– OS Boot device / root file system; secondary volumes/file systems

– Typical basis for database storage

– Raw block devices for RAID, some databases

• Available in both standard and provisioned IOPS (up to 4K IOPS)

• Integration with Amazon S3 storage service (snapshots) for regional access


Amazon Elastic Block Storage (EBS)


Compute• Automatically scale instances based on a rich set of policy options

• Scale your Amazon EC2 capacity automatically once you define the conditions (from 1 to 1000’s of servers)

• Can scale up just a little…doesn’t need to be massive number of servers (may be simply 2 servers, or 1 server with Auto Scaling for high availability)

• Well suited for applications that experience variability in usage

• Set minimum and maximum scaling sizes, use any Amazon CloudWatch metric for rules, also time-of-day, day-of-week, etc. policies


Auto Scaling


Compute• Supports the routing and load balancing of HTTP, HTTPS and generic

TCP traffic to Amazon EC2 instances

• Supports SSL termination and Proxy protocol

• Supports health checks to detect and remove failing instances

• Dynamically grows and shrinks required resources based on traffic

• Seamlessly integrates with Auto Scaling to add and remove instances based on scaling activities

• Single CNAME provides stable entry point for DNS configuration

• Supports internal load balancing within an Amazon VPC


Elastic Load Balancing


Compute

• Managed Hadoop 0.20.205, 1.0.3, 2.0.2, and 2.0.4 infrastructure

• Amazon EMR supports the MapR M7, M5, and M3 Hadoop Distributions.

• Reduces complexity of Hadoop management

– Handles node provisioning, customization, and shutdown

• Provides tight integration with AWS services

– Optimized for Amazon S3

– Amazon EC2 integration with automatic re-provisioning on node failure

– Cluster monitoring/alarming through Amazon CloudWatch

• Leverages significant operational experience

– Monitor thousands of clusters per day

– Use cases span from university students to Fortune 50

Amazon EMRManaged Hadoop Framework


Amazon Elastic Map Reduce (EMR)




Networking


DatabaseStorageCompute StorageCompute


Storage• A “Bucket” is equivalent to a “folder”

• Able to store unlimited number of Objects in a Bucket

• Objects from 1B-5 TB; no bucket size limit

• Highly available storage for the Internet (object store)

• HTTP/S endpoint to store and retrieve any amount of data, at any time, from anywhere on the web

• Highly scalable, reliable, fast, and inexpensive

• Annual durability of 99.999999999%

• Designed for 99.99% availability

• Trillions of objects stored

• Peak requests 1,500,000+ per second

Amazon S3Scalable Storage in the Cloud

Amazon Simple Storage Service (S3)


Storage• A low-cost storage service for data archiving and backup

• $0.01 per GB / Month

• Optimized for data that is infrequently accessed

• Retrieval times measured in hours not days or weeks (typical retrieval job is 3-5 hours)

• Annual durability of 99.999999999% for an archive

• AES 256 data at rest encryption

• Data stored as archives within a vault. Vaults are located within a specific AWS region

• Move data from Amazon S3 to Amazon Glacier using data lifecycle policies


Amazon GlacierArchive Storage in the Cloud

Amazon Glacier


Storage• Storage Gateway connects an on-premises software appliance with

cloud-based storage

• On-premises software appliance solution to store data on Amazon S3’s storage infrastructure

• Exposes standard iSCSI interface to on-premises applications, while maintaining low-latency data access

• Data in Amazon S3 stored as Amazon EBS snapshots for local & Amazon EC2-based recovery

• Cached volumes

• Use Cases

– Backup/Restore on-premises data

– Set up a test/dev environment with production data

– Migrating applications to the cloud

– On-premises DR/COOP to AWS

AWS Storage Gateway



AWS Storage GatewayIntegrate On-Premises IT Environments with Cloud Storage


Storage• Accelerates moving large amounts of data into and out of Amazon S3

or Amazon EBS

• Transfers your data directly onto and off of USB or SATA storage devices shipped to AWS with manifest file

• Final copy uses high-speed datacenter network AWS Storage GatewayIntegrate On-Premises IT Environments with Cloud Storage

AWS Import/Export



AWS Import/ExportBulk Data Transfer


Storage & Content Delivery• Web service for content delivery

• Distribute content to end users with low latency, high data transfer speeds, and no commitments

• Delivers your content using a global network of 51 edge locations

• Supports download, streaming, live streaming, and dynamic content

– Key features: RTMP Streaming, HTTPS Delivery, Private Content for HTTP & Streaming, Programmatic Invalidation, Detailed Logs for HTTP & Streaming, Default Root Object

• Use Cases: Video and Rich Media, Online Gaming, Interactive Agencies, Software Downloads, Static Websites

– Static web content that must be delivered to global user base at Highest bandwidth / Lowest latency / Lowest cost

AWS Import/ExportBulk Data Transfer

AWS Storage GatewayIntegrate On-Premises IT Environments with Cloud Storage



Amazon CloudFrontGlobal Content Delivery Network

Amazon CloudFront




Networking


DatabaseStorageCompute DatabaseStorage


Database• Fully managed NoSQL database.

• Eliminates the administrative burden of data modeling, index maintenance, and performance tuning.

• Durability and high-availability - stores data on Solid State Drives (SSDs) and replicates it synchronously across multiple AWS Availability Zones in an AWS Region.

• Scalability - With AWS Console, you can grow your Amazon DynamoDB table from 10 to 100,000+ writes per sec.

Amazon DynamoDBScalable NoSQL Data Store

Amazon DynamoDB


Database

• Fully-managed, tuned MySQL, Oracle 11g, PostgreSQL or Microsoft SQL Server

• Cost-efficient and resizable capacity

• Manages time-consuming database admin tasks

• Code, applications, and tools you already use today work seamlessly

• Automatically patches the database software and backs up your database

• Flexible Licensing: BYOL or License Include

• Multi-AZ deployment option

Amazon RDSManaged Relational Database Service


Amazon Relational Database Service (RDS)


Database• Fully managed scalable data warehousing service

• Scale from a single 2TB XL node to a hundred 16TB 8XL clustered nodes for a total 1.6PB of compressed user data

• Standard PostgreSQL JDBC or ODBC drivers

• Massively parallel processing (MPP) architecture

• Validated integrations with Jaspersoft, MicroStrategy and many partners

• Priced as low as $1,000 per terabyte per year

• Continuously backed up to Amazon S3


Amazon Redshift


Amazon RedshiftManaged Petabyte-Scale Data Warehouse Service


Database• Fully-managed, distributed, in-memory cache

• Memcached compliant cache cluster on-demand

• Manages patching, cache node failure detection and recovery

• Simple APIs calls to grow and shrink the cache cluster

• Seamlessly caches in front of Amazon RDS instances

• Integrated with Amazon CloudWatch and Amazon SNS for monitoring and alerts


Amazon ElastiCache


Amazon RedshiftManaged Petabyte-Scale Data Warehouse Service

Amazon ElastiCacheIn-Memory Cache




Networking




Database



• Hosted queue for storing messages as they travel between computers

• Move data between distributed components of their applications

• SQS messages can contain up to 256 KB of text data, including XML, JSON and unformatted text.

Amazon SQSMessage Queue Service

Amazon Simple Queue Service



• Set up, operate, and send notifications

• Publish messages from an application and immediately deliver them to subscribers or other applications

• Publishers, Topics, and Subscribers

– Subscribers can be SQS, HTTP/S, Email, and SMS endpoints

Amazon SNSPush Notification Service


Amazon Simple Notification Service



• Easily manage workflows, including state, decisions, executions, tasks and logging

• Coordinate processing steps across distributed systems

• Ensure tasks are executed reliably, in order, and without duplication

• Simple API calls that can be executed from code written in any language and run on your Amazon EC2 instances, or any of your machines located anywhere in the world that can access the Internet

• Amazon Simple Workflow Service


Amazon SWFWorkflow Service




• Bulk and transactional email-sending service

• Eliminates the hassle of email server management, network configuration, and meeting rigorous Internet Service Provider (ISP) standards

• Provides a built-in feedback loop, which includes notifications of bounce backs, failed and successful delivery attempts, and spam complaints


Simple Email Service



Amazon SESEmail Sending Service



• Highly scalable video transcoding service

• Specify Amazon S3 input and output buckets

• Outputs SD and HD H.264/MP4/ACC and WebM

• Input formats include: 3GP, AAC, AVI, FLV, MP4 and MPEG-2





Amazon Elastic TranscoderScalable Media Transcoding

Amazon Elastic Transcoder



• Fully-managed search service

• Integrate fast and highly scalable search functionality into applications

• Scales automatically: with increases in searchable data or as query rate changes

• AWS manages hardware provisioning, data partitioning, and software patches

Amazon Elastic TranscoderScalable Media Transcoding



Amazon CloudSearchManaged Search Service



Amazon CloudSearch




Networking






Deployment & Admin Services

• IAM enables customers to create and manage users in AWS’s identity system

– Identity Federation with local directory is an option for enterprises

• Very familiar security model

– Users, groups, permissions

• Allows customers to

– Create users

– Assign individual passwords, access keys, multi-factor authentication devices

– Grant fine-grained permissions

– Optionally grant them access to the AWS Console

– Organize users in groups

IAMSecure AWS Access Control

Identity and Access Management


Deployment & Administration Services

• Visibility into resource utilization, operational performance, and overall demand patterns

• Metrics such as CPU utilization, disk reads and writes, and network traffic

• Accessible via the AWS Management Console, web service APIs or Command Line Tools

• Add custom metrics of your own

• Alarms (which tie into auto-scaling, Amazon SNS, SQS, etc.)

• Billing Alerts to help manage charges on AWS bill

Amazon CloudWatchResource Monitoring


Amazon CloudWatch



• Create templates of stack of resources

• Deploy stack from template with runtime parameters

• Templates are simple JSON formatted text files

• Amazon CloudFormer supports generating templates from running environments

• Amazon CloudFormation


Amazon CloudFormationTemplated AWS Resource Creation


"Resources" : { "Ec2Instance" : {

"Type" : "AWS::EC2::Instance", "Properties" : {

"SecurityGroups" : [ { "Ref" : "InstanceSecurityGroup" } ], "ImageId" : { "Fn::FindInMap" : [ "RegionMap", { "Ref" : "AWS::Region" },

"AMI" ]}, "Tags" : [{

"Key" : "MyTag", "Value" : "TagValue"

}] } },



• Simply upload your application (Java, NET, PHP, Node.js, Ruby and Python)

• Automatically handles the deployment details of capacity provisioning, load balancing, auto-scaling, and application health monitoring

• Retain full control over the AWS resources powering your application


Amazon Elastic Beanstalk



Amazon Elastic BeanstalkAWS Application Container



• DevOps service for applications in the AWS cloud

• Helps manage complete application lifecycle:

– Resource provisioning

– Configuration management

– Application deployment

– Software updates

– Monitoring

– Access control

• Visualized through application layers

• Uses Chef recipes used to deploy and configure software components on Amazon EC2 instances

Amazon Elastic Beanstalk

AWS Application Container




Amazon OpsWorksDevOps Application Management

Amazon OpsWorks



• Automates the movement and processing of data using data-driven workflows and built-in dependency checking

Amazon OpsWorksDevOps Application Management

Amazon Elastic BeanstalkAWS Application Container


Amazon Data PipelineOrchestration for Data-Driven Workflows



Amazon Data Pipeline


SDKs

Java Python PHP .NET Ruby nodeJS

iOS Android AWS Toolkit for Visual

Studio

AWS Toolkit for

Eclipse

Tools for Windows

PowerShell

CLI


Amazon CloudHSMProtect and store your cryptographic keys with industry standard, tamper-resistant HSM appliances (SafeNet Luna).

No one but you has access to your keys (including Amazon administrators who manage and maintain the appliance).


AWS Services are a few clicks away…

https://console.aws.amazon.com



What is AWS Marketplace?AWS app store for customers•Broad selection, over 1500 listings - IT and business titles for Enterprise production, simple pricing and reviews•Free Trials of selected production software – low risk pilot

– AWS infrastructure charges still apply•Instant fulfillment using 1-Click and Cloud Formation•No overprovisioning, use only what you need, pay with integrated AWS procurement/billing•Seamless license management and ‘compliance by default’•Deploy in minutes, not weeks•Use vetted, tested, secure productsAWS sales channel for Partners•Customers of all sizes – F500 and SMB•Access to hundreds of thousands of AWS customers, from Enterprise to SMB•Hourly pricing and free trials remove risk for customers - enable Partners to run fast, production environment PoCs with customers•Reduce sales cycles from months to weeks (or days)•‘Paperless’ contracting for customers•Provide software bits and pricing – we will do the rest


What are we announcing today?Education & Research: A new category of software for Public Sector! •Broad selection of products focused on the specific needs of EDU customers•Education and Research products already listed in the category include:

• Canvas, Moodle, MATLAB, Adobe Connect, and others

•Industry-leading products in Education and Research will actively be added to the category moving forward



Questions?Larry Pizette

[email protected]

AWS Service Drill Downs - AWS Symposium 2014 - Washington D.C.

Technology

aws government

aws cloud

aws regions

aws wo

availability zone b

scalable dns aws

aws services elb

closest aws endpoint