November 19, 2008 CSC 682 Use of Virtualization to Thwart Malware Written by: Ryan Lehan Presented by: Ryan Lehan Directed By: Ryan Lehan Produced By:

November 19, 2008 CSC 682

Use of Virtualization to Thwart Use of Virtualization to Thwart MalwareMalware

Written by: Ryan LehanWritten by: Ryan LehanPresented by: Ryan LehanPresented by: Ryan LehanDirected By: Ryan LehanDirected By: Ryan LehanProduced By: Ryan LehanProduced By: Ryan Lehan

Starring: Ryan LehanStarring: Ryan Lehan

IntroductionIntroduction

MalwareMalware McAfee Avert LabsMcAfee Avert Labs

Prediction of nearly 800,00 security threats Prediction of nearly 800,00 security threats for the year 2008for the year 2008

300% growth rate over 2007 300% growth rate over 2007 99% fall in 3 categories99% fall in 3 categories

Identity TheftIdentity Theft Data TheftData Theft System CompromiseSystem Compromise

Attack TechniqueAttack Technique

Attack OS directlyAttack OS directly Direct access to OS via OS Application Direct access to OS via OS Application

Programming Interface (API)Programming Interface (API) Attach other applicationsAttach other applications

Exploiting vulnerability pointsExploiting vulnerability points Directly via the application’s APIDirectly via the application’s API Indirectly via specifically formed filesIndirectly via specifically formed files MS 2008 Security Intelligence ReportMS 2008 Security Intelligence Report

33rdrd party applications are killing their security party applications are killing their security

Due to the openness of the OSDue to the openness of the OS

Modern Operating Modern Operating SystemSystem

HardwareCPU Memory NIC Disk

Hardware Abstraction Layer

Operating System

OS API

Application ApplicationApplication

Figure 1. Traditional OS Approach

Problems Thwarting Problems Thwarting MalwareMalware

ExpensiveExpensive TimeTime MoneyMoney ResourcesResources

Thousands of applications each with the Thousands of applications each with the possibility of tens or even hundreds of possibility of tens or even hundreds of vulnerability pointsvulnerability points

Not a single defenseNot a single defense Ant-Virus, Spyware, Adware, FirewallAnt-Virus, Spyware, Adware, Firewall

Good tactics require above average computer Good tactics require above average computer skillsskills

Non-IntuitiveNon-Intuitive

Current Malware Defensive Current Malware Defensive TechniquesTechniques

Signature BaseSignature Base Malicious code recognition by patterns Malicious code recognition by patterns

in code (signatures)in code (signatures) Signatures created by security vendor Signatures created by security vendor

and then downloaded to computer userand then downloaded to computer user Problem areaProblem area

Obfuscation – cryptographic technique to Obfuscation – cryptographic technique to masquerade a random code signaturemasquerade a random code signature

Zero Day Attack – window of time from Zero Day Attack – window of time from when a vulnerability exists to the time when when a vulnerability exists to the time when the security vendors release a patch.the security vendors release a patch.

Current Malware Defensive Current Malware Defensive Techniques (cont.)Techniques (cont.)

Behavior BlockingBehavior Blocking Malicious code recognition based upon Malicious code recognition based upon

user configurable policiesuser configurable policies Monitors the code as it runs in real timeMonitors the code as it runs in real time

Code attempts a function that violates a Code attempts a function that violates a predefined policy then action is takenpredefined policy then action is taken

Can thwart zero day attackCan thwart zero day attack Problem areaProblem area

Policies too tight can cause high false positivesPolicies too tight can cause high false positives Policies too loose can allow malicious code to Policies too loose can allow malicious code to

runrun

Current Malware Defensive Current Malware Defensive Techniques (cont.)Techniques (cont.)

Virtual MachinesVirtual Machines Isolates guests operating systems from host Isolates guests operating systems from host

operating systemoperating system Allows user to run within a clean environmentAllows user to run within a clean environment Contains malicious code to guest environment Contains malicious code to guest environment

onlyonly Problem areaProblem area

Requires above average computer skillsRequires above average computer skills Does not recognize malicious codeDoes not recognize malicious code Malicious code can still run within the guest Malicious code can still run within the guest

environmentenvironment

VirtualizationVirtualization Definition: technique of isolation systems, Definition: technique of isolation systems,

applications, or end users from the physical applications, or end users from the physical characteristics of computer resourcescharacteristics of computer resources

IsolationIsolation Fundamental conceptFundamental concept

Process IsolationProcess Isolation Data IsolationData Isolation

Virtualized environment should guarantee that any Virtualized environment should guarantee that any action performed inside the virtual environment cannot action performed inside the virtual environment cannot interfere outside that environmentinterfere outside that environment

Break-In: situation when an external process enters into Break-In: situation when an external process enters into the same environment as another processthe same environment as another process

Break-Out: situation when an internal process escapes Break-Out: situation when an internal process escapes from its confined environmentfrom its confined environment

VirtualizationVirtualization(cont.)(cont.)

Shared ResourcesShared Resources Just like operating systemsJust like operating systems Each isolated environment views the Each isolated environment views the

shared resource as an object for its sole shared resource as an object for its sole useuse

Data storage exampleData storage example Single physical resource appear as multiple Single physical resource appear as multiple

logical resourceslogical resources Multiple physical resources appear as a Multiple physical resources appear as a

single logical resourcesingle logical resource

Current Virtualization Current Virtualization TechniquesTechniques

Virtual Machines and EmulatorsVirtual Machines and Emulators Software that emulates a physical Software that emulates a physical

computercomputer CPU, Hard Disk, Video, Network card, MemoryCPU, Hard Disk, Video, Network card, Memory

Run modified and unmodified guest Run modified and unmodified guest operating systemsoperating systems

Guest OS does not know that it is running Guest OS does not know that it is running within a host OSwithin a host OS

Good for isolating host OSGood for isolating host OS Requires above average computer skillsRequires above average computer skills

Virtual Machine and Virtual Machine and EmulatorEmulator

HardwareCPU Memory NIC Disk

Hardware Abstraction Layer

Host Operating System

OS API

Figure 2. Heavy-Weight Virtual Container Approach(Virtual Machine)

“Heavy-Weight” Virtualization Layer

Emulated Hardware

App AppApp

Guest Operating System

OS API

Emulated Hardware

App AppApp

Guest Operating System

OS API

Current Virtualization Current Virtualization TechniquesTechniques

Language Dependent Virtual EnvironmentsLanguage Dependent Virtual Environments Some computer languages are designed to run Some computer languages are designed to run

only within a virtual environment (sandbox)only within a virtual environment (sandbox) JavaJava

Does not emulate hardware but creates a set of Does not emulate hardware but creates a set of APIs from which the application interfaces withAPIs from which the application interfaces with

Security advantages over complete virtual Security advantages over complete virtual machines, but only for that specific computer machines, but only for that specific computer languagelanguage

If a vulnerability exists, patch the environment not If a vulnerability exists, patch the environment not the applicationsthe applications

One area not thousandsOne area not thousands Only works with specific computer languagesOnly works with specific computer languages

Current Virtualization Current Virtualization TechniquesTechniques

Application PackagingApplication Packaging Builds upon the use of Virtual MachinesBuilds upon the use of Virtual Machines Applications are pre-built into a ready Applications are pre-built into a ready

made virtual environmentmade virtual environment If package becomes infected, just re-If package becomes infected, just re-

download itdownload it Does not prevent the user from Does not prevent the user from

installing other softwareinstalling other software

Current Virtualization Current Virtualization TechniquesTechniques

Virtual MemoryVirtual Memory Used by modern operating systemsUsed by modern operating systems Gives an application the impression that Gives an application the impression that

it contiguous working memory all to it contiguous working memory all to itselfitself

Not designed to thwart malicious codeNot designed to thwart malicious code

What We NeedWhat We Need

Strength and security of isolationStrength and security of isolation Seamless operation for all levels of Seamless operation for all levels of

computer skillscomputer skills IntuitiveIntuitive

Anti-Virus vs Spyware vs AdwareAnti-Virus vs Spyware vs Adware Single area for defenseSingle area for defense

Computer userComputer user Vendor maintenanceVendor maintenance

Virtualization Technique to Virtualization Technique to Thwart MalwareThwart Malware

Light-weight Virtual EnvironmentLight-weight Virtual Environment Process and Data isolation happens at Process and Data isolation happens at

the application levelthe application level No guest OS is neededNo guest OS is needed Malicious code runs isolated from other Malicious code runs isolated from other

applications and OSapplications and OS Seamless operation for the userSeamless operation for the user

Pure isolation can be counter productivePure isolation can be counter productive Provides an API or a secure communication Provides an API or a secure communication

channel to the OS or other applicationschannel to the OS or other applications

Light-weight Virtual Light-weight Virtual EnvironmentEnvironment

HardwareCPU Memory NIC Disk

Hardware Abstraction Layer

Operating System

OS API

Figure 3. Light-Weight Virtual Application Approach

“Light-Weight” Virtualization Layer

App&

Data

App&

Data

App&

Data

App&

Data

App&

Data

App&

Data

Virtualization Technique to Virtualization Technique to Thwart Malware (cont.)Thwart Malware (cont.)

Layered SecurityLayered Security Policy based, similar to Behavior Blocking.Policy based, similar to Behavior Blocking.

Allows for vendor and user configurationsAllows for vendor and user configurations Layered Layered

To combat the attack, not just recognizeTo combat the attack, not just recognize To reduce code complexity To reduce code complexity Separation of dutySeparation of duty

3 Layers3 Layers Process Level Security PoliciesProcess Level Security Policies

Dictate level of isolation including Trusted and StatelessDictate level of isolation including Trusted and Stateless Inter Process Communication Security PoliciesInter Process Communication Security Policies

Dictate if and how applications communicate with each otherDictate if and how applications communicate with each other Auto ConfigurableAuto Configurable

OS API Security PoliciesOS API Security Policies Dictate if and how application communicate with the OSDictate if and how application communicate with the OS Auto ConfigurableAuto Configurable

Layered SecurityLayered Security

App&

Data

App&

Data

TrustedApp&

Data

State-lessApp&

Data

TrustedApp&

Data

App&

Data

Security Policies for OS API

Process-level and Isolation Security Policies

Figure 4. Layered Security

Inter Process Communication Security Policies

Inter Process Communication Channel

“Light-Weight” Virtualization Layer

Working in TandemWorking in Tandem

Identity TheftIdentity Theft To thwart phishing attacks, many To thwart phishing attacks, many

techniques rely on a trusted 3techniques rely on a trusted 3rdrd party party 33rdrd Party applications will be isolated Party applications will be isolated

and can be marked as Trustedand can be marked as Trusted Insures the safety of the trusted Insures the safety of the trusted

application as well as enhance the application as well as enhance the security of applications that use itsecurity of applications that use it

Working in TandemWorking in Tandem

Data TheftData Theft Data is isolatedData is isolated Malicious code will not have access to Malicious code will not have access to

other applications’ dataother applications’ data Access to other data areas will need to Access to other data areas will need to

pass through the security policiespass through the security policies

Working in TandemWorking in Tandem

System CompromiseSystem Compromise Process is isolatedProcess is isolated Malicious code will have a difficult time Malicious code will have a difficult time

infecting other applicationsinfecting other applications Removal of direct communication Removal of direct communication

between processes and OSbetween processes and OS If an application is exploited, that If an application is exploited, that

application itself is contain within an application itself is contain within an isolated environmentisolated environment

Dealing with Dealing with VulnerabilitiesVulnerabilities

Fix the environmentFix the environment No need to fix thousands of applications, just No need to fix thousands of applications, just

the environment (sandbox)the environment (sandbox) Language Dependent Virtual Environment (Java)Language Dependent Virtual Environment (Java)

Focused AttentionFocused Attention Only 4 areas that need to be looked atOnly 4 areas that need to be looked at

Security policiesSecurity policies ConfigurableConfigurable

Virtualization LayerVirtualization Layer OS APIOS API OS itselfOS itself

ConclusionConclusion

Currently, many tools and techniques Currently, many tools and techniques for combating malware but they are for combating malware but they are lacking in one form or anotherlacking in one form or another

Virtualization is a proven method for Virtualization is a proven method for strong process and data isolationstrong process and data isolation Combined with layered security can Combined with layered security can

defeat many forms of malwaredefeat many forms of malware Many benefits for both users and Many benefits for both users and

vendors alikevendors alike