Security as a Service Model for Cloud Environment

60 IEEE TRANSACTIONS ON NETWORK AND SERVICE MANAGEMENT, VOL. 11, NO. 1, MARCH 2014

Security as a Service Model for Cloud EnvironmentVijay Varadharajan, Senior Member, IEEE, and Udaya Tupakula, Member, IEEE

Abstract—Cloud computing is becoming increasingly impor-tant for provision of services and storage of data in the Internet.However there are several significant challenges in securing cloudinfrastructures from different types of attacks. The focus of thispaper is on the security services that a cloud provider can offer aspart of its infrastructure to its customers (tenants) to counteractthese attacks. Our main contribution is a security architecturethat provides a flexible security as a service model that a cloudprovider can offer to its tenants and customers of its tenants. Oursecurity as a service model while offering a baseline security tothe provider to protect its own cloud infrastructure also providesflexibility to tenants to have additional security functionalitiesthat suit their security requirements. The paper describes thedesign of the security architecture and discusses how differenttypes of attacks are counteracted by the proposed architecture.We have implemented the security architecture and the paperdiscusses analysis and performance evaluation results.

Index Terms—Cloud security, security architecture, securityand privacy.

I. INTRODUCTION

CLOUD computing [1]–[3] has become an importanttechnology where cloud services providers provide com-

puting resources to their customers (tenants) to host theirdata or perform their computing tasks. Cloud computing canbe categorized into different service deliver models such asSoftware as a Service (SaaS), Platform as a Service (PaaS),and Infrastructure as a Service (IaaS). Virtualisation [4] is oneof the key technologies used in the IaaS cloud infrastructures.For instance, virtualisation is used by some of the major cloudservice providers such as Amazon [2] and Microsoft [3] inthe provision of cloud services. We will use the term tenantto refer to cloud customers who wish to access services fromcloud providers. Tenants can themselves be using their virtualmachines to provide services to their own customers; we willrefer to customers (or users) as those who use the servicesof the tenants. Hence customers in our architecture are thecustomers of the tenants.

In general, the tenants in the cloud can run differentoperating systems and applications in their virtual machines.As the operating systems and applications of the tenants canbe potentially large and complex, they may contain securityvulnerabilities. Furthermore, there can be several tenants onthe same physical platform sharing resources in a cloudinfrastructure. The vulnerabilities in operating systems and

Manuscript received December 25, 2012; revised November 13, 2013 andFebruary 24, 2014. The associate editor coordinating the review of this paperand approving it for publication was G. Martinez Perez.

The authors are with the Advanced Cyber Security Research Cen-tre, Faculty of Science, Macquarie University, Sydney, Australia (e-mail:{vijay.varadharajan, udaya.tupakula}@mq.edu.au).

This work was supported by the Australian Research Council under GrantDP140100410.

Digital Object Identifier 10.1109/TNSM.2014.041614.120394

applications can be potentially exploited by an attacker togenerate different types of attacks. These attacks can betargeted against the cloud infrastructure as well as againstother virtual machines belonging to other tenants. So there isa need to design security architecture and develop techniquesthat can be used by the cloud service provider for securing itsinfrastructure and tenant virtual machines.

However there are several issues that arise when developingsecurity as a service for cloud infrastructures. In the currentenvironment, the cloud service providers do not generally offersecurity as a service to their tenants. For example, in [5]Amazon mentions that security of tenant virtual machines isthe responsibility of the tenants since they are free to run anyof the operating systems or applications1 (though it claimsto secure the underlying infrastructure). Hence tenants needto make their own arrangements for securing their virtualmachines that are hosted in the cloud. Although tenants canuse different security tools such as anti-virus and host basedintrusion detection systems to secure their virtual machines,the limitations arise [6] due to these tools residing in the samesystem as the one being monitored and hence are vulnerableto attacks. Also some tenants may not be capable of securingtheir tenant virtual machines. Hence there is a need for thecloud service provider to offer security as a service to suchtenants.

Furthermore, security requirements for tenants may varyand some tenants may opt for more security services fromthe cloud provider while others may opt for the baselinedefault security. For example, a tenant who is running financialservices on its virtual machines is likely to need more securitymeasures compared to a tenant who is providing basic webhosting. However, greater the level of security measures takenup by the tenant from the provider, greater is the possibilityfor the cloud provider to get to know more about the tenant’ssystem. That is, the security mechanisms and tools offeredby the cloud provider (as part of its security as a service)can gather more information about the operating system andapplications running in the tenant’s virtual machines. This inturn may lead to greater privacy concerns for the tenant. Hereprivacy concerns refer to the ability of the provider to finddetails about the services and applications’ data in a tenant’smachine.2

Our main contribution in this paper is a security architecturethat provides a flexible security as a service model that acloud provider can offer to its tenants and customers of itstenants. Our security as a service model while offering a

1Google goes on to say that it reserves the right to reviewthe tenant’s applications including the tenant customer data.(https://developers.google.com/storage/docs/terms)

2A tenant requiring such privacy has to employ techniques such ashomomorphic encryption for protecting application layer information.

1932-4537/14/$31.00 c© 2014 IEEE

VARADHARAJAN and TUPAKULA: SECURITY AS A SERVICE MODEL FOR CLOUD ENVIRONMENT 61

baseline security to the provider to protect its own cloudinfrastructure also provides flexibility to tenants to determinehow much control they wish to have over their own virtualmachines. The baseline security is needed by the providerto ensure that malicious tenants are not attacking the cloudinfrastructure or even hosting malicious software. Every tenanthas to have the security functionalities that forms part of thesecurity baseline, which offers basic security guarantees in itsdefault mode of operation. However there will be other tenantswho would require additional security services (on top of thebaseline) from the cloud provider to meet their requirementsas well as to protect them from other malicious tenants. Henceour security as a service model provides options to haveadditional security functionalities that suit tenants’ securityrequirements. These additional security functionalities canrequire the tenant to reveal more information about its servicesand applications, which may create privacy concerns for thetenant.3 Our approach offers a choice to the tenant to managingthis tension between the privacy concerns and the securitycontrols offered by the cloud provider. An important featureof our model is that it makes this trade-off between securityand privacy explicit. Furthermore, the choice by a tenant toopt in for additional security services can provide the cloudprovider to develop a framework for charging the tenants forthese additional security services.

The paper is organized as follows. We describe the threatmodel in Section II and consider the different types ofattackers and attacks that can occur in the infrastructureas a service cloud environment. Then we summarize thecapabilities of the security architecture that is proposed inthis paper. Section III describes the security as a servicemodel for cloud infrastructure. It describes the design ofthe security architecture and discusses how different typesof attacks are counteracted by the proposed architecture.Section IV describes the implementation and analysis of thesecurity architecture, and discusses the performance evaluationresults. Section V describes relevant related work and providesa comparison with the capabilities of our security architecture.Finally, Section VI concludes the paper.

II. THREAT MODEL

Our system model involves cloud service provider whichincludes cloud system administrators, tenant administrators (oroperators) who manage the tenant virtual machines, and tenantusers (or tenant’s customers) who use the applications and ser-vices running in the tenant virtual machines. Cloud providersare entities such as Amazon EC2 and Microsoft Azure whohave a vested interest in protecting their reputations. Thecloud system administrators are individuals from these cor-porations entrusted with system tasks and maintaining cloudinfrastructures, who will have access to privileged domains.We assume that as cloud providers have a vested interest inprotecting their reputations and resources, the adversaries from

3In a separate piece of work, we have developed a role-based encryption(RBE) scheme for cloud [35], which allows the tenant to store his data in anencrypted form in the cloud and to grant access to that data for users withspecific roles.

Fig. 1. Threat model.

the cloud provider perspective are malicious cloud systemadministrators.4

Consider a typical configuration of our system architectureshown in Fig. 1. In determining the threat model we needto look at the different types of attacks that are possiblein such a configuration. The circle in the figure shows thesource of the attack and the arrow head shows the target ofthe attack. We identify three domains in our architecture thatare relevant to the threat model. There is the tenant domaincomprising tenant administrators and tenant users. Each tenanthas its own tenant domain. There is the cloud system domainwhich consists of cloud system administrators and the VMMplatform (with its privileged domain and hardware). Then thereis the cloud cluster domain comprising cloud system domainsthat constitute the cloud infrastructure.

There can be attacks from tenant administrators on thetenant virtual machines. That is, the tenant administrators canexploit the vulnerabilities in the tenant virtual machine formalicious purposes. Such attacks can target both the cloud in-frastructure as well as co-located tenants. For example, attackssuch as VM escape [7] enable the virtual machine to exploitthe vulnerabilities in the virtual machine monitor (VMM) andaccess the privileged domain and the host operating system.As virtual machines that belong to different tenants can behosted on the same physical server, a malicious tenant whohas obtained access to the privileged domain can performdifferent attacks on the co-located tenant virtual machines [7]–[10]. For example, the malicious tenant can perform denialof service attacks by crashing the server or starving theresources to other tenant virtual machines or capture and/ortamper other tenant virtual machine traffic. If the resources

4We envisage the existence of malicious cloud system administrators; thisis despite the cloud providers having processes and methodologies to ensurethe trustworthiness and integrity of their cloud system administrators.


allocated to other virtual machines are significantly reduced,then this can result in tenant virtual machines not being ableto serve all the legitimate client requests, leading to violationof the Service Level Agreement (SLA) policies between thecloud service provider and the tenants. For example, [7],[8] presents vulnerabilities which enable the malicious tenantvirtual machine administrator to access the console of theprivileged domain and [9] presents the case of exploitingthe VMM scheduler to cause resource starvation to the co-located tenant virtual machines. The paper [10] proposedsignature wrapping and XSS techniques for exploiting thecontrol interfaces for Amazon and Eucalyptus cloud and havecomplete control on the victim. Such attacks cannot be easilydetected in the current state of art.

Then there can be attacks from tenant users (customers).Consider, for instance, a tenant which is a software develop-ment company making use of cloud resources. Although thetenant administrators have provided host based security toolsin their tenant virtual machines, a malicious tenant user (tenantemployee) may be able to circumvent such security tools.Consider Fig. 2 where the virtual machines which belong toa single tenant are hosted on multiple physical servers. Thiscan happen for efficiency reasons where the cloud serviceprovider hosts different customer virtual machines on a singlephysical server. In Fig. 2, although VMM11, VMM13 andVMM14 are hosting virtual machines that belong to singletenant, VMM12 is hosting virtual machines that belong todifferent tenants. In such cases, tenants may want to ensurethat data can be shared among their own virtual machines butat the same time may wish to protect their virtual machinesfrom other tenant’s virtual machines. For instance, in Fig. 2,if Tenant 1 has requested free communication between itsvirtual machines, and hence the cloud service provider decidesnot to monitor any of the communication between Tenant 1’svirtual machines; then a malicious tenant user will be able toexploit vulnerabilities in TVM13 to generate attacks on theother virtual machines (TVM11, TVM12). In this case, thetenant administrator can request the cloud service provider foradditional security mechanisms to deal with such maliciousinsider attacks within the tenant organization.

In a general situation, a malicious tenant user or moreimportantly a malicious tenant administrator can generateattacks against virtual machine belonging to another tenant(e.g. TVM21). The cloud service provider needs to providesecure isolation between the tenant virtual machines. Howeverthe cloud service provider may not be aware of the operatingsystems and applications running in a tenant virtual machine.Hence it is not an easy task for the cloud service provider toenforce security policies on the tenant virtual machines. Fur-thermore since the elastic nature of cloud allows the ability todynamically increase the resources allocated to tenant virtualmachines, the attacker can use this capability in compromisedtenant virtual machines to generate sophisticated attacks.

Alternatively, there can also be attacks from malicious cloudsystem administrators of the tenant virtual machines. Theseadministrators have the technical expertise and the monetarymotivation to misuse the privileges to examine the tenant dataand monitor the tenant’s services at will. For instance, theprivileged domain Dom0 in Xen [11] based cloud system has

access to runtime state information of tenants. Even if thetenant is using secure communications, Dom0 administratorscan obtain all the required information from the memoryallocated to the tenant when it is in unencrypted form. Ifthe VMM fails to provide proper isolation between the tenantvirtual machines then the co-located tenant can also accessthe victim tenant information. For example, [12] discussestechniques for retrieving the private key from the co-locatedtenant virtual machine. Hence the malicious party which hasaccess to the private key information of the tenant can easilyperform man in the middle attacks.

Even if the system administrators are themselves benign,there could be exploits against the privileged domain, Dom0,which can compromise the tenants’ data. Such attacks oncommodity VMMs can arise both out of the complexity ofthe VMM software as well as due to mis-configurations [7]–[10], [12].

Hence there can be: (i) insider attacks from the tenant do-main such as a malicious tenant user described above, and (ii)insider attacks from the cloud service provider administrators.

Finally, there can be attacks from the Internet targeting thecloud service infrastructure (including the tenants), and attacksfrom cloud infrastructure on external hosts in the Internet.The cloud has several attractive features for the attackers tocompromise the tenant virtual machines and use them forgenerating attacks. For instance, a tenant virtual machine canbe the victim of a denial of service attack or an attacker cancompromise the tenant machine and use it for the generationof the attack traffic [13]. As the tenant is often charged onusage, both these attacks can incur financial losses to thevictim tenant. Furthermore some of the attacks can also leadto disputes between the cloud service provider and its tenants.For example, if the tenant virtual machine is compromised andused for generating attack traffic with spoofed address, thiscould result in disputes between the cloud service providerand its tenants. The cloud service provider can charge thetenants for the generated attack traffic and the tenant candeny these charges since the attack traffic has spoofed identity.The study in Spoofer project [14] confirms that such spoofingattacks are easily possible.5 Furthermore, the attacker can usethe compromised tenant virtual machine to generate differenttypes of attack traffic such as ICMP flood, UDP flood, TCPSYN flood and Smurf attacks.

Our Approach: Our security architecture assumes that thecloud service provider provides a trusted VMM platform(for example, equipped with Trusted Platform Module (TPM)[15]). We also assume that the security components of ourarchitecture embedded within the VMM are trusted. The cloudprovider also provides controls and auditing procedures whichensure the physical security of the cloud infrastructure toovercome hardware based attacks such as cold-boot attacks.

Our security architecture allows a cloud provider to providea default baseline set of security measures even if the tenantsdo not require any security feature for their virtual machines.The rationale behind this design choice is as follows. Ifsecurity measures are not provided by the cloud provider, then

5Something of the order of 25% of the autonomous systems permits suchspoofing attacks.


Fig. 2. Cloud scenario.

the provider will not be able to detect any attack on its owninfrastructure itself as well as on other tenant virtual machinesand hosts on the Internet. For example, without any monitoringdone by the cloud provider, malicious tenants can use theirvirtual machines to flood other tenant virtual machines ortarget different services (such as DNS and web server) withinthe provider infrastructure. On the other hand, greater the levelof security measures, greater the possibility for the provider toget to know more details about the tenant’s operating systemand applications that are running in its virtual machines. It isup to the tenant to decide whether it is acceptable dependingon its privacy concerns.

Our security architecture protects the cloud infrastructurefrom attacks generated within a tenant virtual machine bythe tenant administrator and tenant users. It also protects co-located tenants from the attacks generated by such tenantentities. In our architecture, the baseline security mechanisms(in the SPAD component, Section III.D) allow the cloudservice provider to protect their infrastructure, legitimate ten-ants as well as external hosts from the attacks by malicioustenants. The security policies and mechanisms (in the securitycomponent TSAD, Section III.D) in our architecture deal withthe insider attacks from malicious tenant users.

Our security architecture also protects tenants from threatsposed by cloud system administrators who misuse their privi-leges and exploits against privileged domain. It enforces rolebased access control to partition the system administratorsto different roles which enables to restrict their privileges tospecified set of actions. This in turn helps to contain the harmof the insider attacks; in particular, it removes the possibilityof a single administrator (or a single role) having all theprivileges.

Our security architecture also provides mechanisms to dealwith some attacks on the VMM. This is done using a SecurityGateway component which specifies cluster wide policies andmechanisms to detect attacks on the VMM platforms.

Finally our security architecture provides the ability tocharge a tenant depending on the security services that are

required by the tenant. For example, a tenant virtual machinethat is running financial services may need more securitymeasures than a tenant that is running basic web hosting.

III. SECURITY AS A SERVICE MODEL

In this section, we describe our security as a servicemodel for cloud infrastructure. First we give an outline ofcloud architecture in Section A. We describe the assumptionsmade in the design of the security architecture in Section B.Section C gives an overview of the basic security architecture.Section D describes the details of the components in thesecurity architecture. Section E considers the different typesof attacks and how our architecture is able to deal with suchattacks. Finally Section F extends the security architectureto counteract further attacks on the VMM such as VMMcompromise.

A. Cloud Architecture Overview

Let us consider a generic cloud service provider architectureas shown in Fig. 2. Tenants (T1, T2, T3) are hosting one ormore virtual machines on the cloud service provider infrastruc-ture and remotely managing their virtual machines. The CloudController (CLC) is the main interface for the cloud tenantsand it is the top level management for the IaaS cloud. It canquery other controllers such as the Cluster Controllers (CC)and Node Controllers (NC), Storage Controller (SC) to makehigh level decision on the implementation of the tenant virtualmachines and storage of the data. CLC has policies required inthe IaaS infrastructure. It also handles the authentication ser-vice for the users. Storage Controller provides storage for theVM images, and user data. Node Controller is implementedon each physical server. Node Controller is responsible formanaging the tenant virtual machines hosted on each VMM.A group of Node Controllers report to the Cluster Controller.

The cloud service provider can provide prebuilt VM imageswith generic OS and applications (such as web server) or thetenants can transfer their specific VM that is currently running


in their environment into the cloud. Our security architectureextends the NC with the VMM security functionalities.

In our implementation, the tenant virtual machines arehosted on the Xen [11] VMM. We have used Xen becauseit is open source and we have prior experience in usingXen for system development. Furthermore, some commercialcloud service providers such as Amazon [2] use a modifiedversion of Xen for hosting the tenant virtual machines. InXen, there is a privileged domain called Dom0 which is usedfor administering the guest domains referred to as Dom U orunprivileged domains. Xen VMM has physical control of theresources used by the tenant virtual machines, and it is ableto monitor the interactions of its tenants.

The security architecture proposed in this paper focusmainly on the infrastructure-as-a-service (IaaS) platform.There are also other delivery models for cloud such as softwareas a service (SaaS) and platform as a service (PaaS). In thecase of SaaS or PaaS, the tenants have very limited access tothe cloud resources compared to the IaaS. Hence the attacksthat can be generated in SaaS or PaaS are limited to thespecific application software or platforms to which they haveaccess. For example, if an attacker can exploit the vulnerabilityin Gmail, the attacks are limited to the Gmail application. TheSaaS and PaaS providers can use security features available inthe operating system and traditional security tools to protectfrom such malicious tenants. Hence the proposed techniquescan be used as an additional layer of defence in SaaS andPaaS deployments. In the case of IaaS, the tenants havecomplete control on the virtualised systems (applications,operating system and the resources allocated to the virtualisedsystem). Hence the tenants can install any malicious softwareon their virtual machines to generate attacks. As mentioned in[16] there are significantly more security challenges in IaaScompared to SaaS and PaaS. Hence we address IaaS platformin this paper. Let us first consider the assumptions that ourarchitecture makes in Section B.

B. Assumptions

Let us now consider the assumptions made in our architec-ture. We assume that tenant virtual machines accept a securitybaseline (mentioned earlier) functionalities specified by thecloud service provider. If there are any special requirementsfor the tenant which do not comply with the baseline securityrequirements of the cloud service provider, then these need tobe resolved at the time of the registration. The security base-line is enforced by our architecture in the node controller. Withrespect to the applications running in the tenants, we assumethat the tenants are aware of the applications that are runningin their own machines. We also assume that tenants mayhave their own host based security tools (HBST) running intheir virtual machines. Furthermore, default security baselineprovides the best option for those tenants who are concernedabout the privacy of the applications and services running ontheir virtual machines. That is, in this case, the tenants do notreveal any additional information on their applications andservices to the cloud provider.

Alternatively, a tenant may choose to use the applicationsprovided by the cloud provider. For example, home users who

Fig. 3. Basic security architecture.

are temporarily using the cloud may use the default virtualmachine images provided by the cloud service provider. In thiscase, the cloud tenants may not have complete knowledge ofthe applications running in the default images. However thecloud provider will be providing the security baseline. Wedo not consider the situation of a malicious cloud providerproviding malicious applications to its tenants in this paper.As we mentioned earlier in the threat model section, the cloudservice provider has a vested interest in protecting his/herreputation, and hence does not deliberately provide maliciousapplications. Also a tenant can take some precautions inchoosing a suitable cloud provider; we have addressed thisissue of how a tenant can find a trusted cloud service providerin another paper [17].

As mentioned earlier, we assume that the VMM platformis trusted. That is, the VMM software is booted up in atrusted manner and is trusted to function correctly. Also theVMM software does not have any security vulnerabilities.Also the security components developed in this architecturewhich reside in the VMM are trusted.

C. Security Architecture Overview

Consider the basic security architecture diagram shown inFig. 3. As mentioned above, the tenants may wish to have theirown host based security tools (HBST) to run on the virtualmachines that they are obtaining from the cloud provider.Since host based security tools have good visibility into thesystem being monitored, this acts as a primary layer of defensein our security architecture. The other important componentsin our security architecture shown in Fig. 3 are the ServiceProvider Attack Detection (SPAD) and the Tenant SpecificAttack Detection (TSAD) components. First let us look theoperation of our architecture at a high level.

The tenant virtual machine traffic is received by the SPADcomponent. SPAD enforces the security baseline policiesrequired by the cloud service provider. If a tenant virtualmachine’s traffic violates any of the security policies in theSPAD, then the tenant virtual machine is isolated and an alertis generated to the tenant administrator and the cloud systemadministrator. In such cases, the tenant virtual machine canbe activated only after the issues are resolved by the tenantadministrator and the cloud system administrator.

The security policies enforced by the SPAD component areconcerned with the detection of spoofed source address and


associated attacks. The SPAD component also has policiesfor logging traffic from tenants. Since the SPAD securitypolicies are enforced on all the tenant virtual machines, theyare designed to be lightweight and provide basic security base-line with minimal privacy violations. As the SPAD securitymechanisms are needed for secure operation of the cloudprovider, we envisage that these services will be offered tothe tenant by the provider without any additional charges. InSection III.D we discuss the SPAD security mechanisms indetail and describe the types of attacks that these mechanismsare able to counteract.

The traffic which is validated by the SPAD componentis then forwarded to the actual destination or to the TSADdepending on the tenant’s service registration requirements.The traffic is forwarded to the actual destination if the tenanthas not requested for any additional security services (apartfrom the default security baseline) from the cloud provider. Ifthe tenant requires additional security from the cloud provider,then the traffic is forwarded to the TSAD component.

The TSAD enforces tenant specific security policies on thetenant traffic. The security policies in the TSAD are decided bythe tenants at the time of registration. As tenants’ requirementscan change, tenants are able to update the security policies inTSAD. However to minimize misuse, policies can be updatedwith the consent of the cloud provider. Now let us consider theoperation of the TSAD component in our security architecture.

If the traffic does not violate any of the security policiesin TSAD then it is forwarded to the actual destination. Ifthe traffic is violating any of the security policies in TSAD,then the traffic is dropped or rate limited according to thetenant requirements. For example, the tenants can specify thepolicies to drop the traffic if it matches with any of the attacksignatures. Alternatively, the policies can be configured to ratelimit the traffic in the case of sudden burst traffic. TSADcan also be used for pre-monitoring the traffic destined tothe tenant virtual machines. In this case, the incoming trafficis first received by the TSAD. This case is similar to the casein traditional networks where a security gateway such as afirewall monitors all the incoming traffic that is destined to theservers. The tenants are charged additional amounts dependingon the overhead of the TSAD security policies.

The TSAD component contains policies based on signaturesand anomalies to detect attacks. These include dropping trafficthat match with attack patterns, rate limiting traffic abovespecified thresholds (for instance, for ICMP, UDP and SYNpackets), dynamic allocation of resources to tenants as wellas identifying malicious communications using access policieson protocols and ports. TSAD policies also address runtimestate based mechanisms not only to detect suspicious processesrunning in the tenants but also for ensuring the specifiedsecurity related processes are running in the tenants, therebyvalidating security of tenants. We describe the mechanismsassociated with the TSAD policies in Section III.D. Specificexamples of policy specifications are given in the appendix.

D. Component Description

Service Provider Attack Detection (SPAD): SPAD isdesigned to enforce security policies in the baseline that is

offered by the cloud provider. They are intended to minimizethe attacks on the cloud provider infrastructure as well aspreventing attacks between the tenant virtual machines. NoteSPAD policies are enforced on all the tenant virtual machines.

In our architecture, the basic SPAD security policies preventattacks with spoofed source address from the compromisedtenant virtual machine and maintain traffic logs originatingfrom the tenant virtual machines for detecting anomalies.6

Spoofing is one of the fundamental challenges which makeit extremely difficult to deal with the attacks in the currentenvironment [14]. Hence one security objective of the SPADis to ensure that the tenant virtual machine will not sendmalicious traffic with spoofed traffic to external hosts. Thisis achieved by the SPAD monitoring all the traffic that isoriginating from the tenant virtual machines and dropping thetraffic with spoofed source addresses. The traffic with correctsource address is logged and forwarded to TSAD or actualdestination depending on tenant’s security requirements.

This mechanism in SPAD helps to prevent attacks from thetenant virtual machines with spoofed source addresses. Evenif the attacker was successful in exploiting the vulnerabilityin the operating system or applications in the tenant machine,the attacker is not successful in generating the attacks withspoofed source address on other cloud tenant virtual machinesor on the cloud infrastructure or external hosts on the Internet.

In practice, the cloud provider will have (in fact, has tohave) basic monitoring mechanisms for billing and charging.For example, if the cloud provider does not maintain a trafficlog, then it will not be possible to resolve the billing issues thatcan arise between the tenant and cloud providers. Furthermore,tenants may not even be aware of their virtual machinesmight have been compromised, it is important for the cloudprovider to maintain the traffic logs from the tenant virtualmachines. For example, consider a tenant who has downloadedand executed a confidential file on its virtual machine sinceit appears to have been sent from his manager or friend,but in fact has been sent by an attacker. This can be acommon occurrence. As soon as the file is executed, themalicious program in the file say makes copies of the fileand distributes it to other tenants and/or hosts in the Internet,with correct or spoofed source address. Not only the contentsof the file are compromised but this also increases the usageof network resources of the tenant, and hence is chargeableby the cloud provider. However since the tenant is not awareof the compromise of its virtual machine, the tenant mayrefuse to pay the additional charges incurred. These issues canbe further complicated if the malicious file was distributedwith spoofed source address. Hence there is a need for thecloud provider to resolve such issues which requires securelogging of traffic. Furthermore, note that the security enforcedby the SPAD is only at the network level. Those tenantswho are concerned about the privacy of their application dataneed to consider the use of encryption techniques such ashomomorphic encryption [18] to protect their data.

If the traffic originating from a tenant virtual machine hasspoofed source address, then SPAD isolates the tenant virtual

6Recently Amazon has started validating the IP address from all the tenantvirtual machines for EC2 environment.


Fig. 4. Security processes of Sophos Security tool.

machine and generates an alert to the tenant administratorand the cloud system administrator. Hence, SPAD enables thetenant administrator to become aware of the compromise ofits virtual machine.

Note that attacks are still possible from a compromisedtenant virtual machine with correct source address. Howeversince the traffic from the tenant virtual machine is loggedin SPAD, this provides a mechanism for the attacks to betraced and validated. For example, consider an attacker usinga compromised tenant virtual machine to flood other tenantsor the cloud infrastructure. If the victim is able to detectsuch an attack,7 then the victim can inform the cloud systemadministrator who can in turn determine the attacking sourcefrom the source address of the attack traffic and isolate thecompromised tenant virtual machine.

Tenant Specific Attack Detection (TSAD): TSAD compo-nent enforces tenant specific attack detection policies. In ourarchitecture, the tenant can request the cloud service providerto enforce signature based detection and/or anomaly baseddetection policies in the TSAD. First, the tenant is able tospecify application specific attack signatures depending on theapplications that are running in its virtual machines. This willvary with the tenant as different tenants can have differentapplications and services at different times. Furthermore,greater the information that a tenant is willing to revealabout the applications and services that are running in itsvirtual machines, more specific security mechanisms can beimplemented by the cloud service provider to detect serviceor application specific attacks. Though this provides a higherlevel of security, the tenant is revealing more information tothe provider and hence potentially its privacy can be reduced.

Let us now consider how such fine granular security can beenforced by the cloud service provider on the tenant virtualmachines with the process validation (Pro_Val) module in theTSAD. We will also discuss how different types of attacks canbe detected with the (Pro_Val) module.

Pro_Val is implemented in the anomaly detection part ofTSAD and it monitors the resources used by each tenant vir-

7However it may not be always easy for the victim to determine that it isbeing attacked (it depends on the type of attack) or where it is coming from(to identify the cloud system administrator who should be contacted).

tual machine such as memory, CPU, network, disk space andthe interactions of the tenant virtual machines. For example,Fig. 4 shows the security related processes for Sophos securitytool that are running in Windows tenant virtual machine.Pro_Val checks (i) if any of the required critical processes arenot running in the tenant virtual machines and (ii) if there isany suspicious process running in the tenant virtual machine.For example, attacks such as conficker and torpig disable anysecurity tool and critical services such as error reporting, autoupdates and windows defender services. In addition, there canbe malicious hidden processes running in the virtual machine.If Pro_Val observes any suspicious behaviour, then the tenantvirtual machine is isolated and further analysis is performedto detect if there is ongoing zero day attack.

Let us now consider the operation of the Pro_Val module.The Pro_Val queries the tenant virtual machine to report theprocesses running in the machine. When the tenant virtualmachine reports the running processes in the report TVM_Rep,Pro_Val obtains the list of processes that are actually runningin the memory assigned to the tenant virtual machine. The re-port from the tenant virtual machine is untrusted since it couldhave been compromised by an attacker and the report maynot include the processes that are controlled by the attacker.However the report obtained by Pro_Val is trusted since itcontains the list of all the processes that are actually running inthe memory allocated to the tenant virtual machine. Since theattacker does not have access to the Pro_Val component, theattacker cannot alter the process list observed by the Pro_Valcomponent. (We will mention later some attacks which canbypass the VMM itself).

Pro_Val first checks if the host based security tool relatedprocesses (see Fig. 4) are running in the tenant virtual ma-chine. If the tenant virtual machine is compromised, then theprocesses related to the security tool in the tenant virtual ma-chine will not be detected in the Pro_Val report. In such cases,the tenant virtual machine is considered to be compromisedwith malware. Hence attacks such as conficker and torpigwhich disable security tools in the tenant virtual machine aredetected by Pro_Val.

If security related processes are found to be running in thePro_Val report, then the processes listed in the TVM_Rep iscompared with the Pro_Val report. If there is any variation inthe list of processes reported by the tenant machine and the listof processes observed by Pro_Val, then the tenant machine isconsidered to be compromised or infected with malware. Forexample, the attacker can use rootkits such as URK and AFTto compromise the tenant virtual machines and ensure thatsuch processes are not detected by the security tool in thetenant virtual machine. Such attacks are detected by Pro_Val.

The query by Pro_Val is carried out at certain specifiedtimes. Hence there is the possibility of some new maliciousprocess being initiated and terminated between these suc-cessive Pro_Val query times. Hence to minimize the risk offalse positives and false negatives, this query step is repeatedseveral times, especially if there is a variation in the number ofprocesses. If the variation persists after repeated validations,then the tenant machine is considered to be malicious and thehidden processes are detected by comparing process reports.

On the other hand if security related processes are running


Fig. 5. TVM process validation.

in the tenant virtual machine and if no hidden processes aredetected, then the traffic is forwarded to the destination. Inour architecture, we not only analyze the processes but alsoanalyze the files that have been accessed by these processes.

However, as mentioned above, there can be situations wherethere can be attacks which bypass the VMM introspections.This can happen if the rootkits are able to modify the fieldsin the kernel data structures thereby manipulating the viewscreated by the VMM introspection [19]. In order to overcomethese attacks, it is necessary to remove the ability to manip-ulate kernel data structures such as global descriptor tables(GDTs) and interrupt descriptor tables (IDTs). Alternatively,hardware based mechanisms can be used to detect all the pro-cesses that are actually running including the rootkit processes.In the case of x86 based architecture, the CR3 register can beused to identify all the processes that are actually running, as itholds the address of the top-level page directory for the currentcontext. Hence these kernel level attacks can be detected aswrites to CR3 are trapped [20].

Furthermore, though Pro_Val allows us to detect differenttypes of attacks, it reveals information about the tenant ap-plications to the cloud service provider. For example, Fig. 5shows the output using VMM based introspection. From theprocess list, the cloud service provider can easily determinethat the tenant is running DNS application server in its virtualmachine. Hence the privacy of the tenant is reduced. Alsoas the number of security policies requested by the clientincreases, it will increase the overhead on the servers of thecloud services provider. The cloud service provider thereforehas the ability to charge additional amounts to the tenantdepending on the overhead cost of these policies.

E. Detection of Attacks

Insider Attack from Tenant DomainIn this section, we discuss how our architecture can be used

by the tenants to deal with insider attacks from their users(tenant users in their domain).

In our architecture, the tenant administrators can make useof the TSAD component to detect insider attacks. TSAD canbe used to log the activity of the users on their systems. Thelogs can help to extract the user behaviour, identify securitypolicies that need to be enforced on the user and also toanalyse the attacks if the malicious insider is successful inexploiting the vulnerabilities in the tenant virtual machines.However monitoring user activity may not be effective againstmalicious insiders who have joined recently and who donot have much history. Hence TSAD detects the attacks by

monitoring the user activity and system state for suspiciousbehaviour.

TSAD maintains logs of all activities of users and processeswithin the system. For example, the execve system call corre-sponds to the commands typed by the user. The user activity isextracted by filtering the execve system calls and relating theactivity based on the timestamps and events such as login andchdir. In addition, the system state is extracted by monitoringthe applications or processes running in the system, from theusage of resources by different processes, from the privilegesof users accessing the system, and from the traffic that isgenerated.8

At any time, if the user activity or the system state isfound to be suspicious or if the traffic from the user system isfound to be matching with the attack signatures, then TSADgenerates alerts to the tenant and cloud system administrators.For instance, TSAD is able to detect if a user installs anapplication that is not permitted (according to the tenant’spolicies) or disables a mandatory application (such as asecurity tool). This may happen, for instance, if a programdeveloper with admin privileges installs a Skype application,which is not permitted by the tenant organisation’s policies.This enables our architecture to detect the insider attacks bytenant users by monitoring user activities, system state andtraffic originating from the system for suspicious behaviour.

Insider Attacks from Cloud Service ProviderIn this paper, as mentioned earlier, we consider the cloud

service provider to be a trusted entity. However there canbe cloud system administrators involved in the managementof the cloud infrastructure performing a range of activitiessuch as software development, support, testing, and systemand network administration. These system administrators willneed different levels of access to the resources in the cloudto perform their tasks. However the privileged domains in thecurrent VMMs do not support fine granular access controlfor the cloud administrators. An important principle in thedesign of secure systems is the notion of least privilege; that is,system administrators will need to have only those privilegesthat are needed for the tasks at hand.

Accordingly, in our current system, we have developed asimple role based access control model for users of the cloudinfrastructure from the cloud provider domain. The roles inour model are similar to the roles in any of the traditionalrole based access control techniques in that each role is acollection of privileges related to tasks in our system. In thecurrent system, we have identified four types of cloud systemadministrators in different domains, namely Cloud TenantOperator (CTO), Cloud Tenant Administrator (CTA), CloudSystem Administrator (CSA) and Cloud Cluster Administrator(CCA). The roles CTO and CTA have privileges that interfacewith the administration of tenant virtual machines, whereas theroles CSA and CCA correspond to the more traditional rolesin the cloud infrastructure, which are not directly related tothe tenants. In this paper, we will treat the CSA role to be asingle role, though this role can be decomposed into severalother roles. We will not address role decomposition in this

8Techniques such as the one described in [21] can be used to determinethe user behaviour by analysing the commands typed by the users.


paper.The CTO role has access privileges related to the SPAD

component in our architecture. CTO has read level access tothe alerts from the SPAD. In case of SPAD policy violation,the CTO can stop the tenant virtual machine that is generatingattack traffic with spoofed source address. CTOs require thislevel of access on the tenant virtual machines since theviolation of SPAD policies can lead to attacks on the cloudprovider infrastructure and other tenant virtual machines. Alsoall actions by the CTOs are logged and monitored by theCTAs. In terms of the role hierarchy, CTA’s inherit all theprivileges of CTOs.

The CTAs have access privileges on the TSAD compo-nent. At the time of registration, the tenants discuss theirrequirements with the CTAs. Since TSAD supports techniquessuch as process validation, the CTAs have access to finegranular information on the application layer information inthe tenant virtual machines (except when the data is encryptedby the tenant such as homomorphic encryption). The CTA hasprivileges to specify TSAD policies in conjunction with theCSA, to read SPAD and TSAD alerts as well as the privilegeto respond to TSAD alerts.

The Cloud System Administrator (CSA) has the privilegeto access to SPAD and TSAD alerts to be able to respond tothem (e.g. to rate limit or drop the traffic). The CSA role’sprivileges also include the ability to configure TSAD policiesand to manage them for consistency.

Such a role based model allows us to partition the privilegesinto various categories thereby ensuring that only users inappropriate roles can perform administrative operations andaccess sensitive information related to tenants and cloudinfrastructure. This not only gives the benefits of a role basedsystem such as flexible security management and the abilityto handle better the dynamic changes in user population, butalso helps to make the system more accountable.

Denial of Service AttacksNot only a tenant machine can be subjected to a denial

of service attack, but also the compromised tenant virtualmachine can be used by the attacker to generate further attacktraffic. For instance, a tenant virtual machine can be usedto flood the victims with attack traffic such as ICMP, UDPand TCP SYN floods. Also the attackers can make use ofthe elastic nature in the cloud to allocate more resources tothe compromised tenant virtual machines and use them in thegeneration of the attack traffic. This could lead to seriousattacks in the Internet. Hence there is a need for techniquesto minimise denial of service attacks originating from tenantvirtual machines.

Our architecture counteracts the denial of service attacksin the following manner. The default security policies in theSPAD minimize different types of denial of service attacksfrom the compromised tenant virtual machines. For example,since SPAD validates all the tenant traffic for correct sourceaddress, it helps to eliminate Smurf attack and ICMP, UDP,TCP SYN attacks that are generated with spoofed sourceaddress.

In a Smurf attack, the attacker can command the com-promised tenant virtual machine to spoof the source addresswith that of the victim machine and send a flood of request

messages to a broadcast address. Since the source address ofthe attack traffic is spoofed with the address of the victim,all the machines that receive this request will send a replymessage to the victim machine. Also the reply packets willbe generated with the amplification factor N, where N isthe number of hosts present in the broadcast domain. Thiscan result in a severe reply flood at the victim’s machineor network. Since SPAD eliminates any spoof address basedtraffic, this attack is minimized.

Let us now consider how flooding with correct sourceaddress can be minimized. In the case of ICMP and UDPtraffic, the tenant specifies maximum threshold (λ) for ICMPand UDP traffic that can be generated from tenant virtualmachines. The tenant can request the cloud service providerto drop the traffic if the ICMP or UDP traffic exceeds thethreshold and raise an alarm to the tenant administrator. Sucha policy in TSAD will be negotiated at the time of registration.

In the case of TCP SYN flooding attacks, the attackerexploits the weaknesses present in the three-way handshakeprocess. The attacker floods the victim machine with SYNpackets and does not respond with the final acknowledgementpackets. This leads to half-open connections at the victimmachine. The victim has a data structure in its memorydescribing all the half-open connections. If the attacker cancause overflow at the victim with the half-open connections,then the victim cannot accept any new incoming connections.In our architecture, a tenant virtual machine can be the victimor it can itself be the attacker. There are several scenariosin this attack, all of which we will not go through here.TCP SYN flooding is minimized by setting the threshold (

∑

SYN -∑

Final ACK ≥ γ). The threshold is enforced onlywhen the tenant virtual machine is not responding with thefinal acknowledgement after receiving the SYN-ACK packet.Hence this will not have any impact on the legitimate caseswhere the tenant virtual machine is legitimately respondingwith final acknowledgements.

F. Extended Security Architecture

Let us now extend our basic security architecture for VMMplatforms to the scenario shown in Fig. 2 with cloud clusters.The extended security architecture has an additional securitycomponent namely Cloud Cluster Security Gateway (CCSG)that is responsible for a clusters wide security policies andmechanisms (see Fig. 6). The main objectives of this extendedsecurity architecture are to provide mechanisms that can helpto detect attacks on a VMM itself as well as to detect attacksacross multiple VMM platforms. Hence CCSG can be thoughtof as providing an additional layer of defense. The extendedarchitecture introduces another role namely Cloud ClustersAdministrator (CCA) who is responsible for managing se-curity policies at the cluster level. CCSG is implemented asvirtual machine on the cluster controller and only CCA hasaccess to this privileged domain.

Consider now for instance a VMM attack that compromisesthe security policies in the VMM (in SPAD and TSAD com-ponents). Such a compromise can lead to attackers creatingsubtle attacks of giving more resources to a particular tenantand starving the other co-located tenants. CCSG has policies


and mechanisms to detect such attacks. We outline the CCSGpolicies in the following four different categories.

In the category related to tenants, there is the policy onsingle sign-on for users of tenants. A tenant’s users mayneed to access services provided by several of the virtualmachines in different VMM cloud platforms and clusters. Inthis context, it is efficient to perform authentication once at thecloud clusters domain and this is achieved at the CCSG. Alsowhen a tenant requests for a default virtual machine anywherein a cluster, it is the policy in the CCSG which determinesas to what services are offered in a default virtual machineconfiguration (such as VM with Microsoft Windows, VM withLinux/Unix) as well as how often these services need to havesecurity updates.

Then there is the category of policies which relate to attackshappening over multiple VMM platforms in clusters. Forinstance, CCSG has policies on when migration of virtualmachines can occur between different VMM platforms in theclusters. For migration, though the policy in the source TSAD(from where the migration is happening) and the policy inthe destination TSAD (to where the migration occurs) needto be satisfied, there are also meta-policies in the CCSG(such as Chinese wall policies that ensure there is no conflict)which determine when migration can occur. CCSG also has animportant role to play in the detection of attacks that rely oncorrelating traffic from multiple VMM cloud platforms. In ourarchitecture, CCSG has policies related to logging traffic frommultiple platforms and analyzing them for detecting end to endattacks and denial of service attacks where each compromisedmachine may only contribute to small amount of attack traffic.

Then there is a category of policies that are related tooverall Service Level Agreements (SLAs) with the tenants.It is normal to expect such SLAs to have conditions that canonly be observed at the clusters level in our architecture ratherthan at an individual platform level. These conditions varywith the services offered (such as DNS and web services), andthey can be affected by attacks in any one of the platformsbut the policy violation is only detected at the CCSG level.For instance, attacks increasing the response times above thespecified threshold in the SLA or fall in the traffic levelbelow specified minimum threshold level in the SLA. Considerthe situation where as part of SLA, a tenant has requesteda response time of <1 sec for 100 connections. CCSG hasmonitoring policies to ensure such requirements from a tenantare satisfied. So if CCSG observes that the number of externalconnections is less than 100 and the response time from theserver is >1sec, then an appropriate alert can be triggered bythe policies in the CCSG. Such a situation might arise due toan attacker compromising a VMM and starving resources tocertain co-located tenant virtual machines.

The final category of policies in CCSG refers to meta-policies related to specific security policies already enforcedin the VMM platform (by the SPAD and TSAD components).For instance, policies such as process validation in a tenantvirtual machine clearly need to be specified in TSAD andenforced by TSAD on the VMM platform. However CCSGhas meta-policies on the frequency with which TSAD needsto perform process validation of its tenant virtual machines.Similarly CCSG has meta-policies to detect traffic with IP

Fig. 6. Implementation setup.

address not belonging to the cluster (just like SPAD in VMMdetects traffic with IP address not belonging to the VMMplatform).

IV. ANALYSIS

In this section, we discuss the implementation and analysisof our security architecture. Section A presents the imple-mentation setup. Section B describes the implementation ofour security architecture. Section C discusses performanceevaluation results.

A. Implementation Setup

We have used the open source based system Xen hypervisorto implement our architecture. However it is to be notedthat our security architecture can be implemented using otherVMM based systems such as VMWare or HyperV.

Fig. 6 shows the basic implementation of our security archi-tecture at a single VMM platform level using Xen hypervisor.A tenant hosts its services on virtual machines that are runningon Xen hypervisor, which belongs to the cloud provider. Wehave used different subnets for the cloud provider network,tenant domain, the tenant users (who are the customers of thetenant), and the attack domain.

The tenant admin manages the tenant virtual machines thatare hosted in the cloud. The tenant users are the customers ofthe tenant. RIP protocol is used between the routers R1 andR2. The SPAD and TSAD functionalities are implemented inthe privileged domain Dom0. The attacking sources in theattack domain generate the different types of attack traffic onthe tenant virtual machine that we have experimented with.

First we present an overview of Xen that is relevant forour architecture and then discuss the implementation of thecomponents of our security architecture. We then discuss howour architecture deals with different attack scenarios describedabove in Section III.

B. Implementation of Security Architecture

In Xen, the privileged VM Dom 0 [11] is used for hostingand the management of the guest virtual machines. In our


Fig. 7. Attacks with spoofed traffic from tenant virtual machine.

implementation, we are using two guest virtual machinesbased on Linux and one based on Windows running on top ofXen hypervisor version 3.1.2.

We have extended Dom0 to provide the functionalities ofour architecture. To minimize the code in the hypervisor, thedesigners of Xen have opted to include the device driversin the privileged domain. Dom0 manages the actual deviceand exports to the guest virtual machines a generic class ofdevice driver interface. The front end drivers are installed inthe virtual machines and the back end driver is installed inDom0 which has privileged access to the physical resources.The backend driver is also responsible for ensuring the fairaccess the usage of the devices by multiple guest virtualmachines. XenStore is a database for sharing the configurationinformation and is used as a mechanism for controlling devicesin guest domains. Xend daemon is a special process whichruns as root in Dom0 and is responsible for providing ad-ministrative interface to the hypervisor allowing the CustomerTenant Administrator (CTA) to define security policies forvirtual machines.

Our implementation uses Dom0 as the Node Controller forall administrative functions of the cloud provider. When anew instance is created, Xen assigns its preconfigured networkinterfaces to the instance. Every interface has a uniform nameconsisting of a DomainID and interface ID. For example,vif1.2 is a second network interface in Domain1. An outgoingpacket from DomainU passes its sending network interface,enters ebtables filter, then virtual bridge “virbr0” and iptablesfilter and then finally the physical network interface connectedto an external network, peth0. Our architectural componentsSPAD and TSAD components are placed between the frontend drivers of the tenant virtual machines and back end driverof Dom0.

Let us now consider the implementation of certain specificmechanisms in SPAD and TSAD to illustrate how they dealwith the attack scenarios.

Let us first consider the implementation of the mechanismin SPAD that detects spoofed traffic from the tenant virtualmachine. The SPAD module captures network packets fromthe kernel using iptables in connection with libipq module (ipqueue kernel module) and validates the source address of thetraffic. However, one issue was that since packets from tenantVMs enter iptables filter after passing “virbr0,” informationabout the original sending interface is lost. We have usedebtables to encode the information in packet:mark. This isdone by patching Xen script which launches VMs. When a

Fig. 8. Process validation.

VM is launched, a patched Xen adds an ebtables rule, whichadds the information to every packet from the launched VM.SPAD then decodes the information and is able to determinereliably the sending VM regardless of the packet content (itmay be spoofed). We have used scapy for generating the attacktraffic with spoofed source address. Fig. 7 shows the alertswhen attack traffic with spoofed source address is detected bythe SPAD component.

Let us now consider the implementation of mechanisms inthe Pro_Val module in TSAD that detect rootkit attacks viaprocess validation. Process validation is designed as a client-server application to validate the runtime state of Dom U fromDom0. The server daemon runs in DomU and presents a listof processes in DomU on request. The list of processes isobtained by executing Linux utility ps. The client programruns in Dom0. It can query the server daemon for processesin DomU and also extract the list of processes that areactually running in the memory allocated to Dom U. Thetrusted view of DomU processes is obtained by traversingkernel memory of DomU with XenAccess [22]. The queryto the server daemon is realized via XenStore shared memorybetween Dom0 and DomU. XenAccess is a virtual machineintrospection library for Xen hypervisor. It allows a user inDom0 to view the runtime state of DomU. The memory accessallows for mapping arbitrary memory pages of DomU intothe user space of Dom0. The current implementation supportsmemory access and disk monitoring. We have extended thisimplementation for network based monitoring.

Fig. 8 shows different cases of process validation byPro_Val. First run shows the result for a legitimate scenario,where no hidden processes are detected in the tenant virtualmachine with Linux OS. Second run shows the result where


Fig. 9. Process validation time.

0

10

20

30

40

0 2 4 6 8 10

Without SPAD

SPAD +TSAD SPAD+TSAD+PV

Fig. 10. File transfer.

the tenant virtual machine was infected with URK rootkit andtwo hidden process with process id 2224 and 2226 are detectedby the Pro_Val component. In this case, the traffic generatedby the tenant VM is dropped and the tenant virtual machineis isolated for further analysis.

In the third case, we considered the situation where thetenant has registered for additional security services fromthe cloud service provider and has also requested for pre-monitoring of the incoming traffic to the tenant virtual ma-chine which is hosting a web server. In this case, the tenanthas also provided attack signatures for its virtual machinethat is running the web server. We have used signatures fromSnort IDS in this scenario. The tenant’s customer machine (seeFig. 6) is used for legitimate access of the web server on thetenant virtual machine. Now the attacking sources generatedifferent types of attacks on the tenant virtual machine. Sincethe tenant has registered for additional security services, theincoming traffic to the tenant web server is pre-monitored forattacks by the TSAD. All the traffic that matched with theattack signatures is dropped by TSAD.

Note that there is no false alarm when determining thedropping of the spoof traffic in the SPAD. However falsenegatives and positives are possible with the security policiesin TSAD, which are negotiated with the tenants.

C. Performance Evaluation

In building and evaluating this system, we have beencarrying out many sets of experiments with different types of

Fig. 11. TSAD policy updates.

malware attacks. In this paper, we present performance resultswith some of these experiments as well as those involving theuse of SPECjvm benchmarks [23].

Fig. 9 shows the average response time for 10 runs forPro_Val module for varying number of tenant virtual ma-chines. It shows the time taken for validating the processes inthe tenant virtual machine. Pro_Val ensures that the processesrelated to the host based security tool are running in thetenant virtual machine and also detects hidden processes. Theresponse time increases with the number of tenant virtualmachines hosted on the VMM. The response is faster whenall tenant virtual machines have the same operating system(and version) compared to tenant virtual machines havingdifferent operating systems. Furthermore process validation isperformed only once for each flow for the tenants who haverequested additional security services.

Fig. 10 shows the average transfer time for 10 runs fordifferent file sizes from the tenant virtual machine to tenantcustomers without our architecture and with different compo-nents of our architecture. There is negligible delay with SPADand a minor overhead with the inclusion of TSAD and processvalidation (PV) components. We have used a modified versionof Sophos based tool for TSAD with detection engine whichcan detect up to 3483207 types of attacks. The attacks detectedinclude denial of service attacks (e.g. TFN, Trin00 and LOIC),privilege escalation attacks, browser buffer overflow attacks,and worms (e.g. ADMw0rm). The detection component alsochecks for the signature updates every 10 minutes. Total of 104file types for different applications such as rar, zip, doc, docx,and html were monitored for intrusions. We have also mon-itored for anomalous behaviour such as rate limiting ICMPpackets and TCP SYN floods. In this case, process validationis performed once for each flow. Comparing Figs. 9 and 10,one can see that the overhead due to SPAD, TSAD and processvalidation seems to be negligible compared to the overheadcaused by the file transfer. The reason for this observationis that the network delay is a major factor compared to theprocessing delay due to the security components. Furthermore,the process validation is performed only once for each filetransfer.

Fig. 11 shows the average response time for 10 runsupdating the policies in the TSAD from CCSG for varyingnumber of tenant virtual machines hosted on the VMM. Thepolicy consists of default security policy, 50 attack signatures,3 statistical parameters for anomaly detection and time interval


Fig. 12. SPEC benchmark results.

for process validation of TVM. The response is slower asthe number of tenant virtual machines increases. The SPADcomponent remains the same for all the tenants. However theTSAD component varies for each tenant.

We have used the SPECjvm 2008 benchmarks to validatethe performance of our architecture. SPECjvm 2008 bench-marks consist of 38 applications grouped into 11 categories.We have also experimented with specific applications in thedifferent categories. The benchmarks use different types ofwork loads with well known applications such as compres-sion, compiler, scientific, serial and XML for validating theperformance. If a category has multiple benchmarks, then theresult for the benchmark is calculated using geometric mean.The composite score shows the geometric mean of all thebenchmarks. Higher scores represent better performance.

In Fig. 6, the SPECjvm benchmarks were installed on Dom0of Xen. Fig. 12 shows the benchmark results performed on asystem with Intel i7 2.2GHz processor, 6M cache with 8GBRAM with Xen 3.1.2 VMM and Centos 5.1 host operatingsystem. The virtual machines were running windows XP SP2and Linux operating system with 512 MB RAM.

In Fig. 2, the first series (“Without” in blue) shows theresults of the machine with Xen hypervisor, Dom0 and threetenant virtual machines. Different types of applications suchas Yahoo messenger, Quick Time Real Player, Apache Ser-vices, Skype Application, and auto refresh websites such asnews and cricinfo.com were running on the virtual machines.The applications generate different types of traffic at regularintervals, for example, to maintain the connection status, andto update the news. The composite result in this case is 45.13operations per minute (ops/m) without invoking any securitycomponents of our architecture and without any performancetuning.

The next series (S) shows the results with similar setup asthe base run but with SPAD component. In this case tenantvirtual machine traffic is monitored for spoofed source addressand logged before forwarding the traffic to the destination.The composite result in this case is 44.96 ops/m without anyperformance tuning. The overhead in this case is negligible.Hence, this can be included as a standard package even if thecustomers do not opt for any security requirements.

The third series (S+T) shows the results with SPAD andTSAD components with signature and anomaly detection. As

mentioned earlier we have used a modified version of Sophosbased tool with detection engine that can detect up to 3483207types of attacks and can also monitor anomalous behaviour inICMP and TCP SYN traffic. The composite result in this caseis 44.21 ops/m. Compared to Series 1 results, the overhead inthis case is of the order of 2%.

The final series (S+T+PV) shows the results of the operationof our architecture with SPAD, TSAD and process validationperformed for every 10 seconds. This includes rootkits suchas URK and AFT. The composite result in this case is 43.81ops/m. Hence compared to Series 1, the overhead in this caseis of the order of 3%. However, the overhead can vary fordifferent types of attacks. Furthermore, the overhead can bechanged by modifying the time interval for validation of theprocesses in the tenant virtual machines. The performancefigures will also vary with the speed of the hardware.

We have also carried out experimentations with specificapplications such as startup.helloworld, startup.crypto.aes,crypto.aes, compress.zip, mpegaudio, startup.mpegaudio andxml. In all our experiments with different types of attacks, wefound the overhead to be always less than 6% maximum.

We also observed that variation in the composite result dueto the addition of attack signatures on a daily basis has negli-gible impact on the performance of our architecture. Hence thecloud service provider can charge the tenants on a thresholdbasis (instead of every signature) as well as depending onthe interval for process validation. For example, the premiumcan be increased every say 1,000 attack signatures and as theinterval between process validation decreases. Furthermore,the tenants can use TSAD as an additional layer of securityfor their services. Hence our architecture provides benefits toboth the cloud service provider and the tenants.

V. RELATED WORK

During the course of the description of the security archi-tecture, we have already referenced many related works. Inthis section we consider additional relevant related works andcompare them with our architecture.

CloudVisor [24] uses nested virtualisation to deal with thecompromise of the hypervisor. In this technique a securehypervisor is introduced below the traditional hypervisor andthe interactions between the traditional VMM and virtualmachines are monitored by the secure hypervisor. Howeversince the resource management is still performed by thetraditional VMM, the compromise of VMM can impact theoperation of the virtual machines. Compared to CloudVisor themain focus of our work is securing the network interactionsof tenant virtual machines. The technique proposed in [25]allocates a separate privileged domain for each tenant. Thetenants can use this for the enforcement of VMM basedsecurity on their virtual machines. However the model canbecome more complex as different tenant virtual machinescan be hosted on the same physical server. Furthermore,such models cannot deal with the case of malicious tenantsthat misuse the cloud resources to generate attacks on otherhosts. Our architecture considers the case of malicious cloudadministrators and malicious tenants.

There have also been some prior works addressing privacyrelated issues in the cloud. Butt et al [26] proposed self service


cloud which splits the privileged domain into system widedomain (Sdom0) and privileged client domains. Each tenanthas their own privileged domain for enforcement of securitypolicies on their virtual machines. However, since severaltenant virtual machines can be implemented on the samephysical server, a separate client administrative domain has tobe created for each tenant. This makes the model considerablycomplex. Furthermore, an attacker who has control of Sdom0can cause resource starvation to tenant virtual machines. Inour architecture, we enforce different level of access to thecloud administrators using role based access control.

The techniques proposed in [27] consider making the cloudservices scalable to the dynamic changes in the runtimeenvironment. In the proposed architecture, the cloud serviceprovider monitors the load (active connections) on the tenantweb server and dynamically varies the number of virtualmachines allocated to the tenant. Attacks can lead to increaseof load on the tenant virtual machines. Our architecture isable to identify the increase in load caused by the attacktraffic. There have also been some prior works which makeuse of cloud for securing the traditional systems and networks.For instance, Beaty et al [28] proposed network based accesscontrol for different cloud deployments. In this technique, theadministrators from different cloud deployments report to aCloud Access Manager on their requirements. However, ourmodel also detects system level attacks such as rootkits.

There has been considerable research interest [29]–[32] todevelop security techniques to deal with spoofing attacks.Ingress filtering technique [29] has been proposed to validatethe source address of the IP packets. However these techniquesshould be universally deployed for them to be effective. Someauthors have suggested traceback through packet marking [30],logging [31], overlay networks [31] and pushback technique[32]. Although, several filtering and traceback techniques havebeen proposed, they can only detect the approximate domainfrom which the spoofed traffic is originating. Furthermore,Ingress filtering is not effective if the compromised host spoofsits source address with a valid address within the domain.On the other hand our architecture can efficiently identifythe actual attacking source (or tenant virtual machine) thatis generating the attack traffic with spoofed source address.

Currently there is significant interest to develop securitytools based on virtualization technology [6], [33], [34]. Dunlapet al., [33] proposed ReVirt architecture for secure logging byplacing the logging tool inside the VMM. ReVirt logs detailedinformation such as real time clock, keyboard, mouse events,user inputs and system calls, which enables the administratorto replay the execution of virtual machine. Garfinkel [6]proposed a Livewire intrusion detection system which makesuse of the virtual machine monitor to analyse the state ofvirtual machines and detect attacks. Lycosid [34] detectshidden process in the virtual machines by comparing theimplicit guest view with the VMM image. However attackscan be generated by non hidden processes.

However as already discussed, the virtualization techniquescannot be directly applied to the cloud environment due to thesemantic gap problem. As the semantic gap increases the num-ber of false alarms increases. This is a major issue as the cloudservice provider is not aware of the applications running in

the tenant virtual machines and privacy requirements preventthe cloud service provider to use the introspection techniqueswithout the consent of the tenants. In our architecture, thereis no need for the cloud provider to have information of theoperating system or applications in the tenant virtual machinefor enforcing the basic security policies using SPAD. Also,there are no false alarms with the security policies in SPAD.The security policies in TSAD are enforced only with theconsent of the tenants and hence the cloud service providersare not solely responsible for the false alarms due to securitypolicies in the TSAD. Also in the case of cloud, the virtualmachines belong to the tenants and the VMM belongs to thecloud service provider. Hence there is a need for justificationfor using VMM based security techniques in the cloud. Wehave provided a strong justification for using our architecturein practice and how our security as a service offers advantagesto the cloud provider, tenants and tenant customers.

VI. CONCLUSION

In this paper we have proposed a security architecture thatprovides a security as a service model that a cloud providercan offer to its multiple tenants and customers of its tenants.Our security as a service model while offering a baselinesecurity to the provider to protect its own cloud infrastructurealso provides flexibility to tenants to have additional securityfunctionalities that suit their security requirements. The paperdescribed the design of the security architecture and discussedhow different types of attacks are counteracted by the proposedarchitecture. We have described the implementation of thesecurity architecture and gave a detailed analysis of thesecurity mechanisms and performance evaluation results.

APPENDIX

Policy Specification Examples: Several policy languageshave been proposed over the years. We have chosen XACML[36] as it has the necessary constructs to specify the typesof policies even though it is generally regarded as an accesscontrol policy language. Also it is an international standard,and XML based schemas allow verification of the structure ofthe policy file; we have also developed an evaluation engine.Furthermore, Intrusion Detection Message Exchange Format(IDMEF) [37] which defines data formats and exchangeprocedures for sharing information of interest to intrusiondetection and response systems also makes use of XML.Hence we make use of the XACML since the tenants and cloudservice providers can be using different security tools for theenforcement of the security policies. Below we give examplesshowing the use of such a language; Fig. 13 shows an exampleof security policy specification, whereas Fig. 14 shows howinformation such as alerts can be exchanged between differentsecurity agents implemented in different physical servers.

Fig. 13 shows a sample specification of tenant policiesin XACML [37]. The Tenant-ID is used for identifying thetenants and retrieving the security registration details of thetenant. SPAD security policies are enforced by default for allthe tenants. TSAD is optional and the tenants can chooseone or more security enforcement such as signature basedand/or anomaly based and/or process validation. Fig. 13 shows


Fig. 13. Tenant policy specification in XACML.

the case for signature based detection of Slammer attack andanomaly based threshold for ICMP traffic to 3 messages persecond.

Fig. 14 shows the specification of SPAD policies whichraises alarms to the cloud system administrator. The alertmessage ID refers to the attack. The analyser identifies thecluster ID and VMM within the cluster. There are 12 definedcategories (such as DNS for domain name system, NT forwindows domain) defined by the IDMEF[37]. In this case, thecloud provider does not have to be aware of the services in thetenant virtual machine. Hence we use the category 0 whichrefers to “Domain unknown or not relevant.” Name identifiesthe specific sensor “SPAD” that detected the attack.

REFERENCES

[1] L. Youseff, M. Butrico, and D. Da Silva, “Towards a unified ontologyof cloud computing,” in Proc. 2008 Grid Computing EnvironmentsWorkshop.

[2] Amazon Inc., “Amazon elastic compute cloud (Amazon EC2),” 2011.Available: http://aws.amazon.com/ec2/

[3] “Windows Azure.” Available: http://www.windowsazure.com/en-us/[4] J. E. Smith and R. Nair, “The architecture of virtual machines,” IEEE

Internet Comput., May 2005.[5] “AWS security center.” Available: http://aws.amazon.com/security/

Fig. 14. Example of SPAD policy specifications.

[6] T. Garfinkel and M. Rosenblum, “A virtual machine introspection basedarchitecture for intrusion detection,” in Proc. 2003 Netw. Distrib. Syst.Security Symp.

[7] “VM escape.” Available: http://www.zdnet.com/blog/security/us-cert-warns-of-guest-to-host-vm-escape-vulnerability/12471

[8] “Xen security advisory 19 (CVE-2012-4411)–guest administrator canaccess QEMU monitor console.” Available: http://lists.xen.org/archives/html/xen-announce/2012-09/msg00008.html

[9] V. Varadarajan, et al., “Resource-freeing attacks: improve your cloudperformance (at your neighbor’s expense),” in Proc. 2012 ACM Comput.Commun. Security Conf.

[10] J. Somorovsky, et al., “All your clouds belong to us—security analysis ofcloud management interfaces,” in 2011 ACM Comput. Commun. SecurityConf.

[11] P. Barham, et al., “Xen and the art of virtualization,” in Proc. 2003ACM Symp. Operating Syst. Principles.

[12] Y. Zhang, et al., “Cross-VM side channels and their use to extract privatekeys,” in 2012 ACM Comput. Commun. Security Conf.

[13] J. Idziorek, M. F. Tannian, and D. Jacobson, “The insecurity of cloudutility models,” IEEE Cloud Comput., pp. 14–18, May–June 2013.

[14] R. Beverly, R. Koga, and K. C Claffy, “Initial longitudinal analysisof IP source spoofing capability on the Internet,” July 2013. Avail-able: http://www.internetsociety.org/doc/initial-longitudinal-analysis-ip-source-spoofing-capability-internet

[15] B. Balacheff, et al., Trusted Computing Platforms — TCPA Technologyin Context. Hewlett-Packard Books, 2003.

[16] H. Takabi, J. B. D. Joshi, and G. J. Ahn, “Security and privacychallenges in cloud computing environments,” IEEE Security Privacy,vol. 8, no. 6, Nov.–Dec. 2010.

[17] S. M. Habib, V. Varadharajan, and M. Muhlhauser, “A framework forevaluating trust of service providers in cloud marketplaces,” in Proc.2013 ACM Symp. Applied Comput.

[18] C. Gentry, “Fully homomorphic encryption using ideal lattices,” in Proc.2009 ACM Symp. Theory Comput.

[19] S. Bahram, et al., “DKSM: subverting virtual machine introspection forfun and profit,” in Proc. 2010 IEEE Symp. Reliable Distrib. Syst.

[20] J. Pfoh, C. Schneider, and C. Eckert, “Exploiting the x86 architectureto derive virtual machine state information,” in Proc. 2010 Int. Conf.Emerging Security Inf., Syst. Technol.

[21] J. Ryan and M. J. Lin, “Intrusion detection with neural networks,” inProc. 1998 Advances Neural Inf. Process. Syst.

[22] “XenAccess library.” Available: http://code.google.com/p/xenaccess/[23] “Standard performance evaluation corporation.” Available: http://www.

spec.org/download.html


[24] F. Zhang, et al., “CloudVisor: retrofitting protection of virtual machinesin multi-tenant cloud with nested virtualization,” in Proc. 2011 Symp.Operating Syst. Principles.

[25] C. Yu, et al., “Protecting the security and privacy of the virtual machinethrough privilege separation,” in Proc. 2013 Int. Conf. Comput. Sci.Electron. Eng.

[26] S. Butt, et al., “Self-service cloud computing,” in Proc. 2012 ACMComput. Commun Security Conf.

[27] T. C. Chieu, et al., “Dynamic scaling of web applications in a virtualizedcloud computing environment,” in Proc. 2009 IEEE Int. Conf. e-BusinessEng.

[28] K. Beaty, et al., “Network-level access control management for thecloud,” in Proc. 2013 IEEE Int. Conf. Cloud Eng.

[29] P. Ferguson and D. Senie, Network Ingress Filtering: Defeating Denialof Service Attacks Which Employ IP Source Address Spoofing, RFC2267, Jan. 1998.

[30] S. Savage, D. Wetherall, A. Karlin, and T. Anderson, “Network supportfor IP traceback,” ACM/IEEE Trans. Netw., vol. 9, no. 3, pp. 226–237,June 2001.

[31] R. Stone, “CenterTrack: an IP overlay network for tracking DoS floods,”in Proc. 2000 Usenix Security Symp.

[32] R. Mahajan, et al., “Controlling high bandwidth aggregates in thenetwork,” ACM Comput. Commun. Rev., vol. 32, no. 3, pp. 62–73, July2002.

[33] G. W. Dunlap, et al., “ReVirt: enabling intrusion analysis throughvirtual-machine logging and replay,” in Proc. 2002 Operating Syst. Des.Implementation.

[34] S. T. Jones, et al., “VMM-based hidden process detection and identifica-tion using lycosid,” in Proc. 2008 ACM Virtual Execution Environments.

[35] L. Zhou, V. Varadharajan, and M. Hitchens, “Enforcing role-based

access control for secure data storage in the cloud,” Comput. J., vol.54 , no. 10, pp. 1675–1687, 2011.

[36] eXtensible Access Control Markup Language (XACML), Version 3.0,OASIS Standard, Jan. 22, 2013.

[37] H. Debar, D. Curry, and B. Feinstein, The Intrusion Detection MessageExchange Format, RFC 4765, Mar. 2007.

Vijay Varadharajan is the Microsoft Chair Profes-sor at Macquarie University. He is also the Directorof the Advanced Cyber Security Research Centre.Vijay has published more than 300 papers in inter-national journals and conferences. Vijay has been/ison the Editorial Board of several journals includingACM TISSEC, IEEE TDSC, IEEE TIFS, and IEEETCC.

Udaya Tupakula is a Research Fellow at the Ad-vanced Cyber Security Research Centre at Mac-quarie University. In 2006, he completed his Ph.D.under the supervision of Prof. Varadharajan of Mac-quarie University. Uday has 50 publications in dif-ferent research areas such as network security, denialof service attacks, MANET security, and securevirtual systems.