Top Banner
Studying Botnets Using BotLab Arvind Krishnamurthy Wednesday, February 17, 2010
87

Studying Botnets Using BotLab - courses.cs.washington.edu · 2010-02-18 · Coaxing Bots to Run • Some bots send spam using webservices (such as HotMail) • C&C servers are setup

Jun 18, 2020

Download

Documents

dariahiddleston
Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Page 1: Studying Botnets Using BotLab - courses.cs.washington.edu · 2010-02-18 · Coaxing Bots to Run • Some bots send spam using webservices (such as HotMail) • C&C servers are setup

Studying Botnets Using BotLabArvind Krishnamurthy

Wednesday, February 17, 2010

Page 2: Studying Botnets Using BotLab - courses.cs.washington.edu · 2010-02-18 · Coaxing Bots to Run • Some bots send spam using webservices (such as HotMail) • C&C servers are setup

Botnets: a Growing Threat

Wednesday, February 17, 2010

Page 3: Studying Botnets Using BotLab - courses.cs.washington.edu · 2010-02-18 · Coaxing Bots to Run • Some bots send spam using webservices (such as HotMail) • C&C servers are setup

Botnets: a Growing Threat

Wednesday, February 17, 2010

Page 4: Studying Botnets Using BotLab - courses.cs.washington.edu · 2010-02-18 · Coaxing Bots to Run • Some bots send spam using webservices (such as HotMail) • C&C servers are setup

Botnets: a Growing Threat

Wednesday, February 17, 2010

Page 5: Studying Botnets Using BotLab - courses.cs.washington.edu · 2010-02-18 · Coaxing Bots to Run • Some bots send spam using webservices (such as HotMail) • C&C servers are setup

Botnets: a Growing Threat

Wednesday, February 17, 2010

Page 6: Studying Botnets Using BotLab - courses.cs.washington.edu · 2010-02-18 · Coaxing Bots to Run • Some bots send spam using webservices (such as HotMail) • C&C servers are setup

What do Bots do?

• Steal personal information, install keyloggers

• Participate in “distributed denial of service” attacks

• Send spam

• Infect other machines

• Perform click fraud

• ...

Wednesday, February 17, 2010

Page 7: Studying Botnets Using BotLab - courses.cs.washington.edu · 2010-02-18 · Coaxing Bots to Run • Some bots send spam using webservices (such as HotMail) • C&C servers are setup

Botnets still a mystery...

• Increasing awareness, but there is a dearth of hard facts especially in real-time

• Meager network-wide cumulative statistics

• Sparse information regarding individual botnets

• Most analysis is post-hoc

Wednesday, February 17, 2010

Page 8: Studying Botnets Using BotLab - courses.cs.washington.edu · 2010-02-18 · Coaxing Bots to Run • Some bots send spam using webservices (such as HotMail) • C&C servers are setup

Inconsistent Information

Wednesday, February 17, 2010

Page 9: Studying Botnets Using BotLab - courses.cs.washington.edu · 2010-02-18 · Coaxing Bots to Run • Some bots send spam using webservices (such as HotMail) • C&C servers are setup

Inconsistent Information

Wednesday, February 17, 2010

Page 10: Studying Botnets Using BotLab - courses.cs.washington.edu · 2010-02-18 · Coaxing Bots to Run • Some bots send spam using webservices (such as HotMail) • C&C servers are setup

Inconsistent Information

Wednesday, February 17, 2010

Page 11: Studying Botnets Using BotLab - courses.cs.washington.edu · 2010-02-18 · Coaxing Bots to Run • Some bots send spam using webservices (such as HotMail) • C&C servers are setup

Research Agenda

To build a botnet monitoring platform that can track the activities of the most significant spamming botnets currently operating in real-time

Wednesday, February 17, 2010

Page 12: Studying Botnets Using BotLab - courses.cs.washington.edu · 2010-02-18 · Coaxing Bots to Run • Some bots send spam using webservices (such as HotMail) • C&C servers are setup

Botnet Lifecycle (Traditional View)

InfectingMachine

Wednesday, February 17, 2010

Page 13: Studying Botnets Using BotLab - courses.cs.washington.edu · 2010-02-18 · Coaxing Bots to Run • Some bots send spam using webservices (such as HotMail) • C&C servers are setup

Botnet Lifecycle (Traditional View)

InfectingMachine

Wednesday, February 17, 2010

Page 14: Studying Botnets Using BotLab - courses.cs.washington.edu · 2010-02-18 · Coaxing Bots to Run • Some bots send spam using webservices (such as HotMail) • C&C servers are setup

Botnet Lifecycle (Traditional View)

Bot

Bot

Bot

Command & ControlServer (C&C)

InfectingMachine

IRC Message

s

Wednesday, February 17, 2010

Page 15: Studying Botnets Using BotLab - courses.cs.washington.edu · 2010-02-18 · Coaxing Bots to Run • Some bots send spam using webservices (such as HotMail) • C&C servers are setup

Tools for MonitoringHoneypot Infecting

Machine

Snooper

Bot

Bot

Command & ControlServer (C&C)

IRC

Mes

sage

s

Wednesday, February 17, 2010

Page 16: Studying Botnets Using BotLab - courses.cs.washington.edu · 2010-02-18 · Coaxing Bots to Run • Some bots send spam using webservices (such as HotMail) • C&C servers are setup

Botnet Operators’ Response

• Use social engineering techniques for infection

• Cleverly crafted emails/websites induce users to download malicious programs

Wednesday, February 17, 2010

Page 17: Studying Botnets Using BotLab - courses.cs.washington.edu · 2010-02-18 · Coaxing Bots to Run • Some bots send spam using webservices (such as HotMail) • C&C servers are setup

Botnet Operators’ Response

• Use social engineering techniques for infection

• Cleverly crafted emails/websites induce users to download malicious programs

Wednesday, February 17, 2010

Page 18: Studying Botnets Using BotLab - courses.cs.washington.edu · 2010-02-18 · Coaxing Bots to Run • Some bots send spam using webservices (such as HotMail) • C&C servers are setup

Botnet Operators’ Response

• Use social engineering techniques for infection

• Cleverly crafted emails/websites induce users to download malicious programs

Wednesday, February 17, 2010

Page 19: Studying Botnets Using BotLab - courses.cs.washington.edu · 2010-02-18 · Coaxing Bots to Run • Some bots send spam using webservices (such as HotMail) • C&C servers are setup

Botnet Operators’ Response

• Use social engineering techniques for infection

• Cleverly crafted emails/websites induce users to download malicious programs

• Detect virtualization

• Use customized protocols over HTTP

• Use dynamic adaptation

• Malware binaries morph every few minutes

• FastFlux DNS allows for fast redirection to new C&C

• Change C&C protocols as well

• Serve malware/phishing from compromised websites

Wednesday, February 17, 2010

Page 20: Studying Botnets Using BotLab - courses.cs.washington.edu · 2010-02-18 · Coaxing Bots to Run • Some bots send spam using webservices (such as HotMail) • C&C servers are setup

• How are vulnerable servers found?

• Brute force -- not very feasible

• Use search to narrow scope

• Lots of known bugs in php, asp, etc.

• Underground sites post such vulnerabilities

Finding vulnerable servers

Wednesday, February 17, 2010

Page 21: Studying Botnets Using BotLab - courses.cs.washington.edu · 2010-02-18 · Coaxing Bots to Run • Some bots send spam using webservices (such as HotMail) • C&C servers are setup

One such hacker site

Wednesday, February 17, 2010

Page 22: Studying Botnets Using BotLab - courses.cs.washington.edu · 2010-02-18 · Coaxing Bots to Run • Some bots send spam using webservices (such as HotMail) • C&C servers are setup

A malicious query

Wednesday, February 17, 2010

Page 23: Studying Botnets Using BotLab - courses.cs.washington.edu · 2010-02-18 · Coaxing Bots to Run • Some bots send spam using webservices (such as HotMail) • C&C servers are setup

Detecting Vulnerability Searches

Attackers'

queries +

results

Seed

queries

Seed

queries

Seed

queries

Suspicious Query Expansion

Framework

Wednesday, February 17, 2010

Page 24: Studying Botnets Using BotLab - courses.cs.washington.edu · 2010-02-18 · Coaxing Bots to Run • Some bots send spam using webservices (such as HotMail) • C&C servers are setup

Detecting Vulnerability Searches

Attackers'

queries +

results

Seed

queries

Seed

queries

Seed

queries

Suspicious Query Expansion

Framework

• 70 seed queries• From milw0rm

Wednesday, February 17, 2010

Page 25: Studying Botnets Using BotLab - courses.cs.washington.edu · 2010-02-18 · Coaxing Bots to Run • Some bots send spam using webservices (such as HotMail) • C&C servers are setup

Detecting Vulnerability Searches

Attackers'

queries +

results

Seed

queries

Seed

queries

Seed

queries

Suspicious Query Expansion

Framework

• 70 seed queries• From milw0rm

• 1.2M searches• 16k unique queries• 436 IPs

Wednesday, February 17, 2010

Page 26: Studying Botnets Using BotLab - courses.cs.washington.edu · 2010-02-18 · Coaxing Bots to Run • Some bots send spam using webservices (such as HotMail) • C&C servers are setup

An attacker’s view

Search for vulnerability

Compromise site

Host phishing/malware page

Propagate through spam

Domain added to blacklist

Wednesday, February 17, 2010

Page 27: Studying Botnets Using BotLab - courses.cs.washington.edu · 2010-02-18 · Coaxing Bots to Run • Some bots send spam using webservices (such as HotMail) • C&C servers are setup

An attacker’s view

Search for vulnerability

Compromise site

Host phishing/malware page

Propagate through spam

Domain added to blacklist

Detection

Wednesday, February 17, 2010

Page 28: Studying Botnets Using BotLab - courses.cs.washington.edu · 2010-02-18 · Coaxing Bots to Run • Some bots send spam using webservices (such as HotMail) • C&C servers are setup

Defender’s view

Search for vulnerability

Compromise site

Host phishing/malware page

Propagate through spam

Domain added to blacklist

Possible detection

Wednesday, February 17, 2010

Page 29: Studying Botnets Using BotLab - courses.cs.washington.edu · 2010-02-18 · Coaxing Bots to Run • Some bots send spam using webservices (such as HotMail) • C&C servers are setup

Defender’s view

Search for vulnerability

Compromise site

Host phishing/malware page

Propagate through spam

Domain added to blacklist

Possible detection

• Can proactively inform administrators

• Can predict which servers might be attacked

Wednesday, February 17, 2010

Page 30: Studying Botnets Using BotLab - courses.cs.washington.edu · 2010-02-18 · Coaxing Bots to Run • Some bots send spam using webservices (such as HotMail) • C&C servers are setup

BotLab Design

• Active as opposed to passive collection of binaries

• Attribution: run actual binaries and monitor behavior without causing harm

• Scalably identify duplicate binaries

• Correlate incoming spam with outgoing spam

Wednesday, February 17, 2010

Page 31: Studying Botnets Using BotLab - courses.cs.washington.edu · 2010-02-18 · Coaxing Bots to Run • Some bots send spam using webservices (such as HotMail) • C&C servers are setup

1. Malware Collection

Incoming Spam

URLs

Message Summary DB

Relay IPs

Headers

Subject

Malware Crawler

UR

Ls

Archival Storage

Internet

TOR

Wednesday, February 17, 2010

Page 32: Studying Botnets Using BotLab - courses.cs.washington.edu · 2010-02-18 · Coaxing Bots to Run • Some bots send spam using webservices (such as HotMail) • C&C servers are setup

1. Malware Collection• Augment honeypots with

active crawling of spam URLsIncoming Spam

URLs

Message Summary DB

Relay IPs

Headers

Subject

Malware Crawler

UR

Ls

Archival Storage

Internet

TOR

Wednesday, February 17, 2010

Page 33: Studying Botnets Using BotLab - courses.cs.washington.edu · 2010-02-18 · Coaxing Bots to Run • Some bots send spam using webservices (such as HotMail) • C&C servers are setup

1. Malware Collection• Augment honeypots with

active crawling of spam URLs

• 100K unique URLs/day; 1% malicious

Incoming Spam

URLs

Message Summary DB

Relay IPs

Headers

Subject

Malware Crawler

UR

Ls

Archival Storage

Internet

TOR

Wednesday, February 17, 2010

Page 34: Studying Botnets Using BotLab - courses.cs.washington.edu · 2010-02-18 · Coaxing Bots to Run • Some bots send spam using webservices (such as HotMail) • C&C servers are setup

1. Malware Collection• Augment honeypots with

active crawling of spam URLs

• 100K unique URLs/day; 1% malicious

• Most URLs hosted on legitimate (compromised) webservers

Incoming Spam

URLs

Message Summary DB

Relay IPs

Headers

Subject

Malware Crawler

UR

Ls

Archival Storage

Internet

TOR

Wednesday, February 17, 2010

Page 35: Studying Botnets Using BotLab - courses.cs.washington.edu · 2010-02-18 · Coaxing Bots to Run • Some bots send spam using webservices (such as HotMail) • C&C servers are setup

2. Network Fingerprinting

New Bot

Binary

Incoming Spam

URLs

Message Summary DB

Relay IPs

Headers

Subject

Malware Crawler

NetworkFingerprinting

UR

Ls

New VM-aware

Bot

Bot

VM

Bot

VM

Virtual Machines

Execution Engine

Archival Storage

Internet

TOR

Bot

Bare-metal

Bot

Wednesday, February 17, 2010

Page 36: Studying Botnets Using BotLab - courses.cs.washington.edu · 2010-02-18 · Coaxing Bots to Run • Some bots send spam using webservices (such as HotMail) • C&C servers are setup

2. Network Fingerprinting• Goal: find new bots while

discarding duplicates

New Bot

Binary

Incoming Spam

URLs

Message Summary DB

Relay IPs

Headers

Subject

Malware Crawler

NetworkFingerprinting

UR

Ls

New VM-aware

Bot

Bot

VM

Bot

VM

Virtual Machines

Execution Engine

Archival Storage

Internet

TOR

Bot

Bare-metal

Bot

Wednesday, February 17, 2010

Page 37: Studying Botnets Using BotLab - courses.cs.washington.edu · 2010-02-18 · Coaxing Bots to Run • Some bots send spam using webservices (such as HotMail) • C&C servers are setup

2. Network Fingerprinting• Goal: find new bots while

discarding duplicates

• Simple hash is insufficient

New Bot

Binary

Incoming Spam

URLs

Message Summary DB

Relay IPs

Headers

Subject

Malware Crawler

NetworkFingerprinting

UR

Ls

New VM-aware

Bot

Bot

VM

Bot

VM

Virtual Machines

Execution Engine

Archival Storage

Internet

TOR

Bot

Bare-metal

Bot

Wednesday, February 17, 2010

Page 38: Studying Botnets Using BotLab - courses.cs.washington.edu · 2010-02-18 · Coaxing Bots to Run • Some bots send spam using webservices (such as HotMail) • C&C servers are setup

2. Network Fingerprinting• Goal: find new bots while

discarding duplicates

• Simple hash is insufficient

• Execute binaries and generate a fingerprint, which is a sequence of flow recordsNew Bot

Binary

Incoming Spam

URLs

Message Summary DB

Relay IPs

Headers

Subject

Malware Crawler

NetworkFingerprinting

UR

Ls

New VM-aware

Bot

Bot

VM

Bot

VM

Virtual Machines

Execution Engine

Archival Storage

Internet

TOR

Bot

Bare-metal

Bot

Wednesday, February 17, 2010

Page 39: Studying Botnets Using BotLab - courses.cs.washington.edu · 2010-02-18 · Coaxing Bots to Run • Some bots send spam using webservices (such as HotMail) • C&C servers are setup

2. Network Fingerprinting• Goal: find new bots while

discarding duplicates

• Simple hash is insufficient

• Execute binaries and generate a fingerprint, which is a sequence of flow records

• Each flow record defined by (DNS, IP, TCP/UDP)

New Bot

Binary

Incoming Spam

URLs

Message Summary DB

Relay IPs

Headers

Subject

Malware Crawler

NetworkFingerprinting

UR

Ls

New VM-aware

Bot

Bot

VM

Bot

VM

Virtual Machines

Execution Engine

Archival Storage

Internet

TOR

Bot

Bare-metal

Bot

Wednesday, February 17, 2010

Page 40: Studying Botnets Using BotLab - courses.cs.washington.edu · 2010-02-18 · Coaxing Bots to Run • Some bots send spam using webservices (such as HotMail) • C&C servers are setup

2. Network Fingerprinting• Goal: find new bots while

discarding duplicates

• Simple hash is insufficient

• Execute binaries and generate a fingerprint, which is a sequence of flow records

• Each flow record defined by (DNS, IP, TCP/UDP)

• Execute both inside and outside of VM to check for VM detection

New Bot

Binary

Incoming Spam

URLs

Message Summary DB

Relay IPs

Headers

Subject

Malware Crawler

NetworkFingerprinting

UR

Ls

New VM-aware

Bot

Bot

VM

Bot

VM

Virtual Machines

Execution Engine

Archival Storage

Internet

TOR

Bot

Bare-metal

Bot

Wednesday, February 17, 2010

Page 41: Studying Botnets Using BotLab - courses.cs.washington.edu · 2010-02-18 · Coaxing Bots to Run • Some bots send spam using webservices (such as HotMail) • C&C servers are setup

2. Network Fingerprinting• Goal: find new bots while

discarding duplicates

• Simple hash is insufficient

• Execute binaries and generate a fingerprint, which is a sequence of flow records

• Each flow record defined by (DNS, IP, TCP/UDP)

• Execute both inside and outside of VM to check for VM detection

• Execute multiple times as some bots issue random flows (e.g., Google searches)

New Bot

Binary

Incoming Spam

URLs

Message Summary DB

Relay IPs

Headers

Subject

Malware Crawler

NetworkFingerprinting

UR

Ls

New VM-aware

Bot

Bot

VM

Bot

VM

Virtual Machines

Execution Engine

Archival Storage

Internet

TOR

Bot

Bare-metal

Bot

Wednesday, February 17, 2010

Page 42: Studying Botnets Using BotLab - courses.cs.washington.edu · 2010-02-18 · Coaxing Bots to Run • Some bots send spam using webservices (such as HotMail) • C&C servers are setup

3. Monitor Running Bots

• Execute bots and trap all spam they send

• But need to manually tweak bots to get them to run

Bot

VM

Bot

VM

Virtual Machines

Execution Engine

OutgoingSpam

Bot

Bare-metal

Bot spamhole

Internet

TOR

C&C Traffic

Wednesday, February 17, 2010

Page 43: Studying Botnets Using BotLab - courses.cs.washington.edu · 2010-02-18 · Coaxing Bots to Run • Some bots send spam using webservices (such as HotMail) • C&C servers are setup

Manual Adjustments

• SMTP verification

• One bot sent email to special server, which is verified later by the C&C server

C&C server

Special mail server

Wednesday, February 17, 2010

Page 44: Studying Botnets Using BotLab - courses.cs.washington.edu · 2010-02-18 · Coaxing Bots to Run • Some bots send spam using webservices (such as HotMail) • C&C servers are setup

Manual Adjustments

• SMTP verification

• One bot sent email to special server, which is verified later by the C&C server

C&C server

Special mail serverTest Email

Wednesday, February 17, 2010

Page 45: Studying Botnets Using BotLab - courses.cs.washington.edu · 2010-02-18 · Coaxing Bots to Run • Some bots send spam using webservices (such as HotMail) • C&C servers are setup

Manual Adjustments

• SMTP verification

• One bot sent email to special server, which is verified later by the C&C server

C&C server

Special mail serverTest Email

Message code #$#@

Wednesday, February 17, 2010

Page 46: Studying Botnets Using BotLab - courses.cs.washington.edu · 2010-02-18 · Coaxing Bots to Run • Some bots send spam using webservices (such as HotMail) • C&C servers are setup

Manual Adjustments

• SMTP verification

• One bot sent email to special server, which is verified later by the C&C server

C&C server

Special mail serverTest Email

Message code #$#@

Code $%@@

Wednesday, February 17, 2010

Page 47: Studying Botnets Using BotLab - courses.cs.washington.edu · 2010-02-18 · Coaxing Bots to Run • Some bots send spam using webservices (such as HotMail) • C&C servers are setup

Manual Adjustments

• SMTP verification

• One bot sent email to special server, which is verified later by the C&C server

C&C server

Special mail serverTest Email

Message code #$#@

Code $%@@

Wednesday, February 17, 2010

Page 48: Studying Botnets Using BotLab - courses.cs.washington.edu · 2010-02-18 · Coaxing Bots to Run • Some bots send spam using webservices (such as HotMail) • C&C servers are setup

Manual Adjustments

• SMTP verification

• One bot sent email to special server, which is verified later by the C&C server

C&C server

Special mail serverTest Email

Message code #$#@

Code $%@@

Wednesday, February 17, 2010

Page 49: Studying Botnets Using BotLab - courses.cs.washington.edu · 2010-02-18 · Coaxing Bots to Run • Some bots send spam using webservices (such as HotMail) • C&C servers are setup

Coaxing Bots to Run

Bot

VM

Bot

VM

Virtual Machines

Execution Engine

OutgoingSpam

Bot

Bare-metal

Bot spamhole

Internet

TOR

C&C Traffic

Wednesday, February 17, 2010

Page 50: Studying Botnets Using BotLab - courses.cs.washington.edu · 2010-02-18 · Coaxing Bots to Run • Some bots send spam using webservices (such as HotMail) • C&C servers are setup

Coaxing Bots to Run

Bot

VM

Bot

VM

Virtual Machines

Execution Engine

OutgoingSpam

Bot

Bare-metal

Bot spamhole

Internet

TOR

C&C Traffic

Wednesday, February 17, 2010

Page 51: Studying Botnets Using BotLab - courses.cs.washington.edu · 2010-02-18 · Coaxing Bots to Run • Some bots send spam using webservices (such as HotMail) • C&C servers are setup

Coaxing Bots to Run

• Some bots send spam using webservices (such as HotMail)

Bot

VM

Bot

VM

Virtual Machines

Execution Engine

OutgoingSpam

Bot

Bare-metal

Bot spamhole

Internet

TOR

C&C Traffic

Wednesday, February 17, 2010

Page 52: Studying Botnets Using BotLab - courses.cs.washington.edu · 2010-02-18 · Coaxing Bots to Run • Some bots send spam using webservices (such as HotMail) • C&C servers are setup

Coaxing Bots to Run

• Some bots send spam using webservices (such as HotMail)

• C&C servers are setup to blacklist suspicious IP ranges

Bot

VM

Bot

VM

Virtual Machines

Execution Engine

OutgoingSpam

Bot

Bare-metal

Bot spamhole

Internet

TOR

C&C Traffic

Wednesday, February 17, 2010

Page 53: Studying Botnets Using BotLab - courses.cs.washington.edu · 2010-02-18 · Coaxing Bots to Run • Some bots send spam using webservices (such as HotMail) • C&C servers are setup

Coaxing Bots to Run

• Some bots send spam using webservices (such as HotMail)

• C&C servers are setup to blacklist suspicious IP ranges

• Bots with 100% email delivery rate are considered suspicious

Bot

VM

Bot

VM

Virtual Machines

Execution Engine

OutgoingSpam

Bot

Bare-metal

Bot spamhole

Internet

TOR

C&C Traffic

Wednesday, February 17, 2010

Page 54: Studying Botnets Using BotLab - courses.cs.washington.edu · 2010-02-18 · Coaxing Bots to Run • Some bots send spam using webservices (such as HotMail) • C&C servers are setup

Coaxing Bots to Run

• Some bots send spam using webservices (such as HotMail)

• C&C servers are setup to blacklist suspicious IP ranges

• Bots with 100% email delivery rate are considered suspicious

• Fortunately only O(10) botnets; so manual tweaking possible

Bot

VM

Bot

VM

Virtual Machines

Execution Engine

OutgoingSpam

Bot

Bare-metal

Bot spamhole

Internet

TOR

C&C Traffic

Wednesday, February 17, 2010

Page 55: Studying Botnets Using BotLab - courses.cs.washington.edu · 2010-02-18 · Coaxing Bots to Run • Some bots send spam using webservices (such as HotMail) • C&C servers are setup

4. Clustering/Correlation Analysis

• Two sources of information:

• Spam sent by bots running in BotLab (Outgoing Spam)

• Spam received by UW (Incoming Spam)

Wednesday, February 17, 2010

Page 56: Studying Botnets Using BotLab - courses.cs.washington.edu · 2010-02-18 · Coaxing Bots to Run • Some bots send spam using webservices (such as HotMail) • C&C servers are setup

4. Clustering/Correlation Analysis

• Two sources of information:

• Spam sent by bots running in BotLab (Outgoing Spam)

• Spam received by UW (Incoming Spam)

URLs

Message Summary DB

Relay IPs

Headers

Subject

Bot

VM

Bot

VM

Virtual Machines

ClusteringDNS

MonitoringHostnames

Subjects, Relays

Resolved

IP addresses

CorrelationAnalysis

Execution Engine

Result Storage

OutgoingSpam

Bot

Bare-metal

Bot spamhole

Outgoing Spam

Incoming Spam

Wednesday, February 17, 2010

Page 57: Studying Botnets Using BotLab - courses.cs.washington.edu · 2010-02-18 · Coaxing Bots to Run • Some bots send spam using webservices (such as HotMail) • C&C servers are setup

Measurements

• Analysis of outgoing spam feed

• Analysis of incoming spam feed

• Correlation of outgoing and incoming spam feeds

Wednesday, February 17, 2010

Page 58: Studying Botnets Using BotLab - courses.cs.washington.edu · 2010-02-18 · Coaxing Bots to Run • Some bots send spam using webservices (such as HotMail) • C&C servers are setup

Behavioral Characteristics

BotnetC&C

Discovery

C&C servers contacted

over lifetimeC&C protocol

spam send rate

(msgs/min)

Grum

Kraken

Pushdo

Rustock

MegaD

Srizbi

Storm

Wednesday, February 17, 2010

Page 59: Studying Botnets Using BotLab - courses.cs.washington.edu · 2010-02-18 · Coaxing Bots to Run • Some bots send spam using webservices (such as HotMail) • C&C servers are setup

Behavioral Characteristics

BotnetC&C

Discovery

C&C servers contacted

over lifetimeC&C protocol

spam send rate

(msgs/min)

Grum static IP 1

Kraken algorithmic DNS 41

Pushdo set of static IPs 96

Rustock static IP 1

MegaD static DNS name 21

Srizbi set of static IPs 20

Storm p2p (Overnet) N/A

Wednesday, February 17, 2010

Page 60: Studying Botnets Using BotLab - courses.cs.washington.edu · 2010-02-18 · Coaxing Bots to Run • Some bots send spam using webservices (such as HotMail) • C&C servers are setup

Behavioral Characteristics

BotnetC&C

Discovery

C&C servers contacted

over lifetimeC&C protocol

spam send rate

(msgs/min)

Grum static IP 1 encrypted HTTP

Kraken algorithmic DNS 41 encrypted HTTP

Pushdo set of static IPs 96 encrypted HTTP

Rustock static IP 1 encrypted HTTP

MegaD static DNS name 21 encrypted custom protocol (port 80)

Srizbi set of static IPs 20 unencrypted HTTP

Storm p2p (Overnet) N/A encrypted custom

Wednesday, February 17, 2010

Page 61: Studying Botnets Using BotLab - courses.cs.washington.edu · 2010-02-18 · Coaxing Bots to Run • Some bots send spam using webservices (such as HotMail) • C&C servers are setup

Behavioral Characteristics

BotnetC&C

Discovery

C&C servers contacted

over lifetimeC&C protocol

spam send rate

(msgs/min)

Grum static IP 1 encrypted HTTP 344

Kraken algorithmic DNS 41 encrypted HTTP 331

Pushdo set of static IPs 96 encrypted HTTP 289

Rustock static IP 1 encrypted HTTP 33

MegaD static DNS name 21 encrypted custom protocol (port 80)

1638

Srizbi set of static IPs 20 unencrypted HTTP 1848

Storm p2p (Overnet) N/A encrypted custom 20

Wednesday, February 17, 2010

Page 62: Studying Botnets Using BotLab - courses.cs.washington.edu · 2010-02-18 · Coaxing Bots to Run • Some bots send spam using webservices (such as HotMail) • C&C servers are setup

Botnet Mailing Lists

• Random fetch model allows us to estimate botnet mailing list sizes

• As we see more of the spam feed, there will be more duplicates in recipient email addresses

• If mailing list size is N and if bot obtains C addresses for each C&C query, then probability that an email address will appear again in the next K emails is

• Some mailing list sizes: MegaD’s is 850 million, Rustock’s is 1.2 billion, Kraken’s is 350 million

• Overlap between mailing lists is small (less than 28%)

1 - (1 - C/N)K/C

Wednesday, February 17, 2010

Page 63: Studying Botnets Using BotLab - courses.cs.washington.edu · 2010-02-18 · Coaxing Bots to Run • Some bots send spam using webservices (such as HotMail) • C&C servers are setup

Outgoing Spam Characteristics

• Bots are stateless

• List of recipients downloaded from C&C server is randomly chosen

• Bots can be periodically restarted to quickly obtain information on ongoing spam campaigns

• Some bots are buggy

• C&C servers change infrequently

• Some botnets are partitioned

Wednesday, February 17, 2010

Page 64: Studying Botnets Using BotLab - courses.cs.washington.edu · 2010-02-18 · Coaxing Bots to Run • Some bots send spam using webservices (such as HotMail) • C&C servers are setup

Correlation Analysis

• Combine our sources of data:

• Outgoing spam from BotLab

• Incoming spam at UW

Wednesday, February 17, 2010

Page 65: Studying Botnets Using BotLab - courses.cs.washington.edu · 2010-02-18 · Coaxing Bots to Run • Some bots send spam using webservices (such as HotMail) • C&C servers are setup

Combining our spam sources

The Internet

2.5 million emails per day

• Incoming spam provides a different perspective

• Spam is received from almost every bot out in the world

• Local view of spam produced

• Global view of spam producers

Wednesday, February 17, 2010

Page 66: Studying Botnets Using BotLab - courses.cs.washington.edu · 2010-02-18 · Coaxing Bots to Run • Some bots send spam using webservices (such as HotMail) • C&C servers are setup

Combining our spam sources

The Internet

BotLab

Global view of spam produced

Global view of spam producers

+

Wednesday, February 17, 2010

Page 67: Studying Botnets Using BotLab - courses.cs.washington.edu · 2010-02-18 · Coaxing Bots to Run • Some bots send spam using webservices (such as HotMail) • C&C servers are setup

Combining our spam sources

The Internet

BotLab

Challenge: create mapping between incoming spamand bot generated spam

Global view of spam produced

Global view of spam producers

+

Wednesday, February 17, 2010

Page 68: Studying Botnets Using BotLab - courses.cs.washington.edu · 2010-02-18 · Coaxing Bots to Run • Some bots send spam using webservices (such as HotMail) • C&C servers are setup

Combining our spam sources

Wednesday, February 17, 2010

Page 69: Studying Botnets Using BotLab - courses.cs.washington.edu · 2010-02-18 · Coaxing Bots to Run • Some bots send spam using webservices (such as HotMail) • C&C servers are setup

• Observation:

• Spam subjects are carefully chosen

• NO overlap in subjects sent by different botnets (489 subjects/day per botnet)

• Solution: Use subjects to attribute spam to particular botnets

Combining our spam sources

Wednesday, February 17, 2010

Page 70: Studying Botnets Using BotLab - courses.cs.washington.edu · 2010-02-18 · Coaxing Bots to Run • Some bots send spam using webservices (such as HotMail) • C&C servers are setup

Who is sending all the spam?

21%

1%3%4%

16%20%

35%Srizbi

RustockMegaD

Kraken

Unknown

PushdoStorm

The Internet

Average over 50 days

Wednesday, February 17, 2010

Page 71: Studying Botnets Using BotLab - courses.cs.washington.edu · 2010-02-18 · Coaxing Bots to Run • Some bots send spam using webservices (such as HotMail) • C&C servers are setup

Who is sending all the spam?

21%

1%3%4%

16%20%

35% 79% of the spam came from just 6 botnets!

Srizbi

RustockMegaD

Kraken

Unknown

PushdoStorm

The Internet

Average over 50 days

Wednesday, February 17, 2010

Page 72: Studying Botnets Using BotLab - courses.cs.washington.edu · 2010-02-18 · Coaxing Bots to Run • Some bots send spam using webservices (such as HotMail) • C&C servers are setup

Botnets and spam campaigns

• We define a spam campaign by the contents of the webpage the spam URL points to

Wednesday, February 17, 2010

Page 73: Studying Botnets Using BotLab - courses.cs.washington.edu · 2010-02-18 · Coaxing Bots to Run • Some bots send spam using webservices (such as HotMail) • C&C servers are setup

Botnets and spam campaigns

• We define a spam campaign by the contents of the webpage the spam URL points to

Wednesday, February 17, 2010

Page 74: Studying Botnets Using BotLab - courses.cs.washington.edu · 2010-02-18 · Coaxing Bots to Run • Some bots send spam using webservices (such as HotMail) • C&C servers are setup

Botnets and spam campaigns

• We define a spam campaign by the contents of the webpage the spam URL points to

Wednesday, February 17, 2010

Page 75: Studying Botnets Using BotLab - courses.cs.washington.edu · 2010-02-18 · Coaxing Bots to Run • Some bots send spam using webservices (such as HotMail) • C&C servers are setup

Botnets and spam campaigns

• We define a spam campaign by the contents of the webpage the spam URL points to

• We found the mapping between botnets and spam campaigns to be many-to-many

Wednesday, February 17, 2010

Page 76: Studying Botnets Using BotLab - courses.cs.washington.edu · 2010-02-18 · Coaxing Bots to Run • Some bots send spam using webservices (such as HotMail) • C&C servers are setup

Where are campaigns hosted?

• How does the Web hosting infrastructure relate to the botnets?

Web servers

1

Botnets2 43

Wednesday, February 17, 2010

Page 77: Studying Botnets Using BotLab - courses.cs.washington.edu · 2010-02-18 · Coaxing Bots to Run • Some bots send spam using webservices (such as HotMail) • C&C servers are setup

Where are campaigns hosted?

• How does the Web hosting infrastructure relate to the botnets?

Web servers

1

Botnets2 43

Wednesday, February 17, 2010

Page 78: Studying Botnets Using BotLab - courses.cs.washington.edu · 2010-02-18 · Coaxing Bots to Run • Some bots send spam using webservices (such as HotMail) • C&C servers are setup

Where are campaigns hosted?

• How does the Web hosting infrastructure relate to the botnets?

Web servers

1

Botnets2 43

Wednesday, February 17, 2010

Page 79: Studying Botnets Using BotLab - courses.cs.washington.edu · 2010-02-18 · Coaxing Bots to Run • Some bots send spam using webservices (such as HotMail) • C&C servers are setup

Where are campaigns hosted?

• How does the Web hosting infrastructure relate to the botnets?

• Does all spam sent from one botnet point to a single set of web servers?

Web servers

1

Botnets2 43

Wednesday, February 17, 2010

Page 80: Studying Botnets Using BotLab - courses.cs.washington.edu · 2010-02-18 · Coaxing Bots to Run • Some bots send spam using webservices (such as HotMail) • C&C servers are setup

Where are campaigns hosted?

• How does the Web hosting infrastructure relate to the botnets?

• Does all spam sent from one botnet point to a single set of web servers?

Web servers

1

Botnets2 43

Wednesday, February 17, 2010

Page 81: Studying Botnets Using BotLab - courses.cs.washington.edu · 2010-02-18 · Coaxing Bots to Run • Some bots send spam using webservices (such as HotMail) • C&C servers are setup

• How does the Web hosting infrastructure relate to the botnets?

• Our data shows a many-to-many mapping

• Suggests hosting spam campaigns is a 3rd party service and not tied to botnets

Web servers

1

Botnets2 43

Where are campaigns hosted?

Wednesday, February 17, 2010

Page 82: Studying Botnets Using BotLab - courses.cs.washington.edu · 2010-02-18 · Coaxing Bots to Run • Some bots send spam using webservices (such as HotMail) • C&C servers are setup

• How does the Web hosting infrastructure relate to the botnets?

• Our data shows a many-to-many mapping

• Suggests hosting spam campaigns is a 3rd party service and not tied to botnets

Web servers

1

Botnets2 43

Where are campaigns hosted?

• 80% of spam points to just 57 Web server IPs

Wednesday, February 17, 2010

Page 83: Studying Botnets Using BotLab - courses.cs.washington.edu · 2010-02-18 · Coaxing Bots to Run • Some bots send spam using webservices (such as HotMail) • C&C servers are setup

Botnet Membership

• What fraction of the botnet members can we identify in a single day at a given location?

• Again use probabilistic analysis based on the random recipient address model

• Let P is the probability that a given spam message is sent to an UW email address

• Let N be the number of email messages sent by a bot over a given period

• Then probability of UW receiving a spam message:

1 - e-N*P

Wednesday, February 17, 2010

Page 84: Studying Botnets Using BotLab - courses.cs.washington.edu · 2010-02-18 · Coaxing Bots to Run • Some bots send spam using webservices (such as HotMail) • C&C servers are setup

Botnet Membership

• Even the most gentle bots send N = 48K messages per day

• UW receives 2.4M messages of a total world-wide estimate of 110B messages; P = 2.2*10-5

• Over a 24-hour uptime, probability of identifying a botnet participant is 0.65

Wednesday, February 17, 2010

Page 85: Studying Botnets Using BotLab - courses.cs.washington.edu · 2010-02-18 · Coaxing Bots to Run • Some bots send spam using webservices (such as HotMail) • C&C servers are setup

Conclusions

• BotLab is an engineering exercise that pulls together many of the ideas proposed earlier

• Key components: active crawling, executing captive bots, network fingerprinting, correlation

• Enables a rich set of measurements. Results include:

• Small number of botnets generate most of the spam

• Complex (not one-to-one) relationships between botnets, spam campaigns, and hosting infrastructures

• BotLab also promises better defenses (safe browsing, spam filtering, bot detection, etc.)

Wednesday, February 17, 2010

Page 86: Studying Botnets Using BotLab - courses.cs.washington.edu · 2010-02-18 · Coaxing Bots to Run • Some bots send spam using webservices (such as HotMail) • C&C servers are setup

Conclusion

• Botnets pose serious security challenges

• Requires greater understanding

• BotLab is an engineering exercise that pulls together many of the ideas proposed earlier

• Key components: active crawling, executing captive bots, network fingerprinting, correlation

• Potentially enables better defenses (safe browsing, spam filtering, bot detection, etc.)

Wednesday, February 17, 2010

Page 87: Studying Botnets Using BotLab - courses.cs.washington.edu · 2010-02-18 · Coaxing Bots to Run • Some bots send spam using webservices (such as HotMail) • C&C servers are setup

• More questions? Just toss me an email (arvind@cs) or stop by my office (CSE 544).

Wednesday, February 17, 2010