Systems and Internet Infrastructure Security Laboratory (SIIS) Page
Module: Cloud Computing Security
Professor Trent JaegerPenn State University
1
Systems and Internet Infrastructure Security (SIIS) Laboratory Page
Cloud Computing Is Here
2
Why not use it?
Systems and Internet Infrastructure Security (SIIS) Laboratory Page
What’s Happening in There?
3
Systems and Internet Infrastructure Security Laboratory (SIIS) Page
Overview
• Cloud computing replaces physical infrastructure
• Is it safe to trust these services?
4
Systems and Internet Infrastructure Security (SIIS) Laboratory Page
From Data Center to Cloud
5
Systems and Internet Infrastructure Security Laboratory (SIIS) Page
Reasons to Doubt
• History has shown they are vulnerable to attack
‣ SLAs, audits, and armed guards offer few guarantees
‣ Insiders can subvert even hardened systems
6
‘06 ‘07 ‘08 ‘09 ‘10 ‘11
903
678695
986
770641
Data Loss Incidents
External54%
Unknown7%
Insider16%
Accidental23%
Incident Attack Vector
Credit: The Open Security Foundation datalossdb.org
Systems and Internet Infrastructure Security (SIIS) Laboratory Page
Cloudy Future
• New problem or new solution?
‣ New challenges brought on by the cloud (plus old ones)
‣ Utility could provide a foundation for solving such challenges
7
Systems and Internet Infrastructure Security (SIIS) Laboratory Page
Cloudy Future
• Improve on data centers? On home computing?
‣ Seems like a low bar
8
Systems and Internet Infrastructure Security Laboratory (SIIS) Page
What is Cloud Computing?• Cloud vendor provides managed computing
resources for rent by customers
• What do you want to rent?
‣ (Virtualized) Hosts (Infrastructure as a Service)
• Rent cycles: Amazon EC2, Rackspace Cloud Servers, OpenStack
‣ Environment (Platform as a Service)
• Rent instances: Microsoft Azure, Google App Engine
‣ Programs (Software as a Service)
• Rent services: Salesforce, Google Docs
• Other variations can be rented9
Systems and Internet Infrastructure Security Laboratory (SIIS) Page
What is Cloud Computing?
10
Systems and Internet Infrastructure Security (SIIS) Laboratory Page
IaaS Platform: OpenStack
11
Client
SchedulerNetworkController
CloudDatabase
Message Queue
VolumeStore
ImageStore
Cloud API
CloudCustomer
CloudNode
Instances
CloudVendor
Systems and Internet Infrastructure Security Laboratory (SIIS) Page
PaaS Platform: Google App• Platform for deploying language-specific apps
‣ Java, Python, PHP, etc.
• Vendor provides OS and middleware
‣ E.g., Web server, interpreters
• Customers deploy their customized apps
‣ You focus on custom code
• Clients use these apps
‣ Analogously to IaaS
12
Systems and Internet Infrastructure Security Laboratory (SIIS) Page
How to Build an IaaS Cloud?• Vendors obtain hardware resources for
‣ Various cloud services: API, Messages, Storage, Network, ...
‣ Compute nodes for running customer workloads
• Install your hardware
‣ Need to choose software configurations specific for services and compute nodes
• Start your hosts
‣ Join the cloud - services and available compute nodes
• Now your cloud is running
‣ Have fun! Customers are ready to use your services and nodes
13
Systems and Internet Infrastructure Security Laboratory (SIIS) Page
How to Use an IaaS Cloud?• Customers choose an OS distribution
‣ These are published by the cloud vendor and others
‣ Obtain cloud storage necessary to store these and your data
• Configure your instance (VM)
‣ Prior to starting - enable you to login and others to access the instance’s services
• Start your instance
‣ Boots the chosen OS distribution with the configurations
• Now your instance is running
‣ Have fun! Login via SSH or ready for your clients
14
Systems and Internet Infrastructure Security (SIIS) Laboratory Page
Multiple Stakeholders
15
Cloud Node
Cloud Instance (VM)
Client Data
Clients
Service Providers
Cloud Administrators
Is my platform secure?
Are my services running correctly?
Are my data protected?
Systems and Internet Infrastructure Security Laboratory (SIIS) Page
VM
Cloud Complexity• Cloud environment challenges
‣ Opaque, Complex, Dynamic
‣ Insiders, Instances, Co-hosting
16
Client Service
CloudNode
CloudNode
CloudNode
CloudNode
VM
VMVM
CloudPlatform
Systems and Internet Infrastructure Security Laboratory (SIIS) Page
What Could Go Wrong?• What do customers depend on from the cloud?
‣ Trust Model
‣ Are those parties worthy of our trust?
• Who are potential adversaries in the cloud?
‣ Threat Model
‣ Are customers protected from their threats?
• What would be ideal from a security standpoint?
‣ Ideal Security Model
‣ How many trusted parties and how many threats?
17
Systems and Internet Infrastructure Security (SIIS) Laboratory Page
Published Instances
18
our case, operates the IaaS cloud infrastructure, authenti-cates users and bills them for the resources they consumed.
The Publisher creates and publicly o↵ers cloud apps, calledAmazon Machine Images (AMIs). For this, he selects an ex-isting AMI (AMI-1 in Fig. 1), instantiates it (Instance-1AMI-1),logs into the running instance to configure it, and finallypublishes a snapshot as a new AMI (AMI-2).
The Consumer selects this AMI from a list of availableAMIs, instantiates it (Instance-2AMI-2), and uses it for herpurposes. Optionally, a Publisher can declare an AMI aspaid AMI to earn money from Consumers invoking it.
!"#$%&'((&)*#+,&
-.&/#012$+,& 3.&405*6076*,&
8.&$5,&
9.&($:"45;&
<.&405*6076*,&
!"#$%&'()*
+,-&".()*
!),/%0()*
=05*60/,>3'?=>3&
=05*60/,>-'?=>-&
'?=>3&
'?=>-&
Figure 1: Basic System Model of Cloud App Store
The Cloud App Store poses security challenges for both,Consumers and Publishers (see also [48, 17]).
Security of Consumer. The Consumer must trust thePublisher not to include any malware into the AMI. Sucha malicious AMI could contain a Trojan horse that spieson or modifies the Consumer’s data, or a backdoor for mali-cious remote login. Even though full protection against suchmalicious AMIs is almost impossible, filters, virus scanners,and rootkit detectors could provide at least some level ofprotection [48].1
Security of Publisher. The Publisher on the other handmight accidentally publish AMIs that contain highly sensi-tive information. Examples include keys, credentials, pass-words, command history/log files, or source code.
Although Amazon’s user guide recommends to ensure thatall confidential information is removed before publishing anAMI [12, Sharing AMIs Safely], many users seem to be un-aware of the crucial consequences of ignoring these recom-mendations, do not have the appropriate tools at hand, orsimply forgot private data in their AMIs.
The Gap between Theory and Practice. The Pro-vider could filter AMIs for Trojans, backdoors, or confiden-tial information to reduce the chance of malicious or sen-sitive data within AMIs. This was proposed in [48], butalthough the automated filtering system presented in thatpaper seems to be used already within the IBM Smart-Cloud [32], the explicit filtering rules are not available tothe public.
In contrast, Amazon currently does not provide automatedscanning of public AMIs as they are not responsible/liablefor what users do with their own data. Though Amazonquickly reacts on incidents reported to their security hotline1In principle, this is similar to mobile app stores wheredownloaded apps must be trusted as well. Recently,Google’s mobile app store withdrew 25 Android apps thatwere infected with malware [13]. As such attacks alsoharm the reputation of the mobile app store provider, someproviders already review new apps submitted to the store toensure that they perform as expected [9, 31].
and informs a↵ected customers, e.g., those running an AMIin which a backdoor was found [15].2
In this paper we show that these previously reported inci-dents are only the tip of the iceberg and many of the publiclyavailable AMIs have severe security vulnerabilities leakinghighly sensitive data.
Our Contribution and Outline.After summarizing related work in §2 and giving back-
ground information on the Amazon Web Services (AWS)in §3 we present the following contributions.
Extraction of Sensitive Information from PublicAMIs (cf. §4). Through an extensive analysis we were ableto extract highly sensitive information from several publiclyavailable EC2 AMIs. To make the analysis cost and timee↵ective we developed an automated tool that uses di↵erentsearch strategies and exploits technology specific aspects ofthe Amazon cloud. The costs for running our attack wereless than $20 while the information we extracted from theAMIs would allow an attacker to cause financial damage ofseveral $10, 000 per day and could severely harm the reputa-tion of several companies that operate services in the cloud.After testing overall 1225 AMIs we got hold of the sourcecode repositories, administrator passwords and other typesof credentials of various web service providers.
SSH Vulnerabilities in AMIs (cf. §5). We discoveredseveral vulnerabilities in AMIs that are introduced by incor-rect usage and configuration of SSH. About one third of thetested 1100 public AMIs in Europe and the US-East regioncontain an SSH backdoor, i.e., a (forgotten) public key thatallows remote login for the Publisher. We identified multi-ple instances that use the same SSH host key which allowsan external attacker to correlate these instances running thesame or a similar AMI, identify candidates for correspondingpublic AMIs, and mount several attacks, e.g., host imper-sonation.
Countermeasures (cf. §6). We provide several mech-anisms to protect against our attacks on public AMIs. Be-sides organizational measures we propose to use our tools toenhance the security of the interfaces for publishing AMIsand also extensions to the interface of the Cloud App Store.
2. RELATED WORKIn this section we briefly revisit previous work on the se-
curity challenges of publicly sharing Virtual Machine (VM)images (AMIs in our terminology) on which we build ourpractical attacks. Afterwards we review the main relatedwork on general cloud security, security aspects specific tothe Amazon cloud, and methods for searching private data.
VM Image Analysis.As summarized in §1, security and privacy risks for the
Consumer and Publisher when sharing VM images havebeen identified in [48]. Shared VM images may contain ei-ther malware that was intentionally or unintentionally in-cluded by the Publisher. To protect against these threats,the authors propose filtering of VM images by the Providerwhich has been implemented in the Mirage image manage-
2“For security reasons, we (Amazon) recommend that anyinstance based on a publicly available AMI that is dis-tributed with an included SSH public key should be con-sidered compromised and immediately terminated.” [14]
390
Consumers use published instances
Who do you trust? What are threats?
Systems and Internet Infrastructure Security Laboratory (SIIS) Page
SSH Study [AmazonIA]
19
• Publisher left an SSH user authentication key in their AMI
• Fortunately, Amazon agreed that this is a violation
‣ Unfortunately, it was not an isolated problem
• 30% of 1100 AMIs checked contained such a key
‣ Also, pre-configured AMIs had SSH host keys
• Thus, all instances use the same host key pair
• Implications?
Systems and Internet Infrastructure Security (SIIS) Laboratory Page
Security Configuration ‣ Zillions of security-relevant configurations for instances
• Do you have the right code and data installed?
• Are you running the expected code?
• Discretionary access control
• Firewalls
• Mandatory access control
‣ SELinux, AppArmor, TrustedBSD, Trusted Solaris, MIC
• Application policies (e.g., Database, Apache)
• Pluggable Authentication Modules (PAM)
• Application configuration files
‣ Plus new configuration tasks for the cloud - e.g., storage 20
Systems and Internet Infrastructure Security (SIIS) Laboratory Page
Cloud Service Vulnerabilities• Vulnerabilities have been found in cloud services
‣ E.g., OpenStack identity service, web interface, and API service
• Adversaries who compromise such services may launch a variety of attacks
‣ E.g., Key Injection Attack
21
Response Modification. Compromised cloud servicescan tamper with responses returned by compute the ser-vice, causing the client to see attacker crafted responsesinstead of the legitimate compute service reponse (i.e.Console Output Attack).
In next few sections, we exemplify and analyze eachcategory.
A. Snapshot Attack
In this attack, attacker can obtain sensitive datasaved on instance’s filesystem through taking a snapshotof it.
Background. The cloud supports a quick and easy wayto create new instance image by taking a snapshot of arunning instance. The new instance image will containall files on the running instances filesystem at the timeof snapshot creation. This is useful as for backup andimage building purposes. The detailed procedure to takea snapshot is shown as follows: Client makes a requestto cloud API service to take a snapshot. Cloud APIservice invokes the snapshot_instance method ofthe compute service that hosts the instance. Computeservice will interface with local virtualization driver(e.g. Libvirt Driver) to take a snapshot of the instance.
Attack. A malicious cloud API service under controlof attacker can obtain sensitive data from a client’sinstance by directly making snapshot request to computeservice. This request is not on behalf of the client.Nevertheless, compute service will still serve it as itperforms no authentication and authorization (the cloudidentity service only authenticates and authorizes theclient’s initial request)
Launching this attack requires the attacker to invokethe The difficulty for the attacker to launch Snap-shot Attack relies on attackers ability to invoke thesnapshot_instance method of the compute ser-vice. Unfortunately, this is fairly straightforward. SinceOpenStack cloud services are interconnected throughthe message queue, any service can publish messages onthe queue to the compute service. The compute serviceneither verifies nor authorizes the message as it believesthe request is already authenticated and authorized.Therefore, attacker only needs to compromise one singlecloud service that is connected to the message queue.
There are many similar security sensitive meth-ods supported by compute service. For example, theset_admin_password method of compute serviceallows resetting the admin password of an instanceand inject_file allows injecting arbitrary files intoa running instance. These security sensitive methodsare open to attackers once they compromise any cloudservice connected to the message queue.
Attack Analysis. The root cause of the Snapshot Attackis the lack of authorization of incoming requests to
APIService
ComputeService
Database
APIService
nova keypair-add
mykey
nova boot --key-name
mykey
mykey : ssh-rsa ABC
mykey : ssh-rsa ABC
ssh-rsa ABCssh-rsa DEF
Step 1
Step 2
Fig. 3: Key Injection Attack
the compute service. The compute service serves anyrequest from any service without verifying that therequest is on behalf of the client.
B. Key Injection Attack
Even if the request is verified to be on behalf ofclients, services can still modify the request in maliciousways. An example is the key injection attack. In thisattack, attackers can gain privileged access to client’sinstance by modifying the SSH connection key injectedinto instance.
Background. When client starts up an instance, it isoften based on a clean image retrieved from the cloudimage service that contains no user-specific credentials.To enable client login, the cloud provides a mechanismto dynamically inject SSH connection key (a publickey belonging to client) into the instance. The clientcan later can use his private key to authenticate andlog in to the instance. The detailed procedure is shownin Figure 3. First, the client makes a request to cloudAPI service to create a key pair. The keypair is named“mykey” in this case. While client keeps the private key,the public key will be uploaded to cloud and saved incloud database. In the second step, client starts up anew instance with “mykey”. Cloud API services querythe database to retrieve the actual public key that mapsto “mykey” and sends it to the compute service. Thecompute service injects the public key into the instanceimage and boots up the instance. The client can thenlogin to her instance by establishing an SSH connectionusing her private key.
Attack. We now consider a malicious cloud API servicecontrolled by an attacker. A malicious API service canmodify the public key during either step 1) saving it todatabase or step 2) retrieving it from database. With amodified public key (a public key belongs to attacker)injected into client’s instance, attacker can thus log in toclients instance as privileged user. Attacker can increasethe subtlety of the attack through appending a publickey instead of modifying it. Therefore both client and
Systems and Internet Infrastructure Security (SIIS) Laboratory Page
Insiders‣ Although the vendor may have a good reputation, not every
employee may
22
Embracing the cloud Trust me with your
code & data
Cloud Provider Client
You have to trust us as well
Cloud operators
Problem #1 Client code & data secrecy and integrity vulnerable to attack
11"
Systems and Internet Infrastructure Security Laboratory (SIIS) Page
Insider Threats• May trust the cloud vendor company
‣ But, do you trust all its employees?
• Insiders can control platform
‣ Determine what software runs consumers’ code
• Insiders can monitor execution
‣ Log instance operation from remote
• Insiders may have physical access
‣ Can monitor hardware, access physical memory, and tamper secure co-processors
23
Systems and Internet Infrastructure Security Laboratory (SIIS) Page
Co-Hosting Threats• An instance co-hosted on the same physical
platform could launch attacks against your instance
• Co-hosted instances share resources
‣ Computer
• CPU, Cache, Memory, Network, etc.
• Shared resources may be used as side channels to learn information about resource or impact its behavior
24
Systems and Internet Infrastructure Security Laboratory (SIIS) Page
Resource Freeing Attacks• Setup
• Victims
‣ One or more VMs with public interface
• Beneficiary
‣ VM whose performance we want to improve (contend over target resource)
• Helper
‣ Mounts attack using public interface
25
Helper&
VM#
VM#
Vic&m#
Beneficiary#
Systems and Internet Infrastructure Security Laboratory (SIIS) Page
Resource Freeing Attacks• Resource contention over the CPU
‣ Schedule beneficiary more frequently
• Attack: shift resource usage via public interface
‣ Normally, victim is scheduled and pollutes the cache
‣ Approach lower scheduling priority
• Make victim appear CPU-bound
26
RFA$intensi*es$–$*me$in$ms$per&second&
196%$slowdown$
86%$slowdown$
60%$Performance$Improvement$
Systems and Internet Infrastructure Security Laboratory (SIIS) Page
Preventing Vulnerabilities• How would you prevent these threats?
‣ Misconfigured instances
‣ Untrusted cloud services
‣ Insiders
‣ Side channels
‣ (Attacks to cloud platform also)
27
Systems and Internet Infrastructure Security Laboratory (SIIS) Page
Verifiable Computation
• Your services are black boxes - to the cloud!
‣ Send a program and encrypted data
‣ Program computes over encrypted data
‣ Scheme: KeyGen (for Program), Compute (Program), Verify
28
ServiceDataClient
Depends on heavy crypto - homomorphic encryption
Systems and Internet Infrastructure Security Laboratory (SIIS) Page
Pinocchio [Oakland 2013]• New cryptographic protocol for general-purpose public verifiable
computation with support for zero-knowledge arguments
• Big advance: Performance
‣ History: PCP (2007) = 72 trillion years, GGP (2010) = 37 centuries, Pepper/Ginger (2012) = 6 oom improvement, Pinocchio = 7 oom improvement (often ~10ms)
• Encoding in “quadratic programs”; signature depends only on security constant
‣ Idea behind quadratic arithmetic programs: each multiplication gate is a “small expression”. Construct polynomials that encode the equations, such that if the evaluation is correct, then D(z) / P(z). Then the protocol just checks divisibility randomly
• Beats local C execution (for verification)
29
Systems and Internet Infrastructure Security Laboratory (SIIS) Page
Service
Integrity Monitor Concept
• Integrity monitor similar to a reference monitor
‣ Mediate access to service based on integrity criteria
• Challenges
‣ Where do we measure integrity-relevant events?
‣ How do we verify ongoing integrity?
‣ How can we deploy this in a cloud environment?
30
IntegrityMonitorClient Data ServiceData
Systems and Internet Infrastructure Security (SIIS) Laboratory Page
Excalibur• Policy-sealed data [USENIX Sec 2012b]
‣ Do not release my data to the cloud until that cloud satisfies my requirements
‣ Customer-chosen policy
• How to ensure that only nodes that satisfy customer-chosen policy get data?
‣ Attribute-based encryption
‣ Encrypt data using ABE description of load-time configuration
‣ A verifiable monitor is trusted to delegate correct credentials to nodes (using hardware-based attestations - e.g., via TPM)
31
Systems and Internet Infrastructure Security (SIIS) Laboratory Page
Excalibur Approach
32
4/19/13 Nuno Santos
Excalibur Architecture!
13
! Check node configurations ! Monitor attests
nodes in background
! Scalable policy enforcement ! CP-ABE
operations at client-side lib
Monitor
Customer Policy-Sealed Data
+
seal
unseal attest & send
credential
Datacenter
From Nuno Santos’ slides
Systems and Internet Infrastructure Security (SIIS) Laboratory Page
Runtime Monitoring• Excalibur does not address runtime issues with instance
‣ Customers may want to ensure that clients of their services only receive communications from satisfactory instances
‣ Customer may want to take remediative actions
33
Systems and Internet Infrastructure Security Laboratory (SIIS) Page
Client
Cloud Node
VM
ChannelMediator
Integrity Verification Proxy(1) Register criteria
(2) Verify Monitor / Node
(3) Verify VM
(4) Connect
(5) Report ViolationMeasureFramework
ModulesMonitor VM
Integrity Verification Proxy• Clients specify criteria to be enforced by a channel
mediator [TRUST 2012]
• Set of measurement modules verifies the criteria
‣ Loadtime modules measure VM components
‣ VM Introspection to examine runtime criteria
• E.g., Binaries/data loaded, enforcement disabled, policy changes, kernel data (binary handler), etc.
34
Systems and Internet Infrastructure Security (SIIS) Laboratory Page
Cloud Verifier Overview
35
NodeClient Cloud
InstanceNodeCloudInstance
CloudVerifier
IVPClient
Client monitorsCV and cloud criteria
IVP monitorscloud instance
Client provides criteria
Client criteriasent to IVP
CV monitors cloud node
Block connection at the Cloud Node
DisableCloud Node
Client stopsusing Cloud
Cloud Anchor [CCSW 2010, TrustCom 2012]+IVP in OpenStack [CSAW 2013]
Systems and Internet Infrastructure Security (SIIS) Laboratory Page
Customer-Driven Monitoring• CV/IVP Limitation
‣ IVP must be trusted by cloud vendor
‣ Part of management VM
• What if you need to perform monitoring that the cloud vendors will not support?
36
Systems and Internet Infrastructure Security (SIIS) Laboratory Page
Self-Service Clouds• Customizable cloud platform stack [CCS 2012]
37
Why do these problems arise?
Hardware
Hypervisor
Management$VM$(dom0)$
Work"VM"
Work"VM"
Work"VM"
14"
Slides courtesy of Vinod Ganapathy
Systems and Internet Infrastructure Security (SIIS) Laboratory Page
Self-Service Clouds• Customizable cloud platform stack [CCS 2012]
38
Our solution
Hardware
Hypervisor
Management$VM$ Client’s$VMs$
19"
SSC: Self-service cloud computing
Systems and Internet Infrastructure Security (SIIS) Laboratory Page
Self-Service Clouds• Customizable cloud platform stack [CCS 2012]
‣ UDom0 boots customer-defined Service VMs
39
An SSC platform
Hardware
SSC Hypervisor
25"
SDom0$
Work$VM$
Work$VM$UDom0$
Client’s$metaBdomain$
Service$VM$
Equipped$with$a$Trusted$Plaiorm$Module$(TPM)$chip$
Systems and Internet Infrastructure Security Laboratory (SIIS) Page
Take Away• Cloud computing is here to stay
‣ In some form
• May be a solution or a problem or both
‣ Introduces new types of vulnerabilities into systems we ran on data centers - which had vulnerabilities to begin with
• Ultimately, have to improve service providers’ jobs
‣ Make it easy to ensure that systems perform as expected
• Two possible methods
‣ Verifiable computation and instance monitoring
40