Background Knowledge course/cs402 Peng Bo School of EECS, Peking University 6/26/2008 Refer to Aaron Kimball’s slides.

Background Knowledge

http://net.pku.edu.cn/~course/cs402

Peng BoSchool of EECS, Peking University

6/26/2008

Refer to Aaron Kimball’s slides

Background Topics

• Parallelization & Synchronization

• Fundamentals of Networking

• Search Engine Technology– Inverted index– PageRank algorithm

Parallelization & Synchronization

Parallelization Idea

• Parallelization is “easy” if processing can be cleanly split into n units:

work

w1 w2 w3

Partition problem

Parallelization Idea (2)

w1 w2 w3

thread thread thread

Spawn worker threads:

In a parallel computation, we would like to have as many threads as we have processors. e.g., a four-processor computer would be able to run four threads at the same time.

Parallelization Idea (3)

thread thread thread

Workers process data:

w1 w2 w3

Parallelization Idea (4)

results

Report results

thread thread threadw1 w2 w3

Parallelization Pitfalls

But this model is too simple!

• How do we assign work units to worker threads?• What if we have more work units than threads?• How do we aggregate the results at the end?• How do we know all the workers have finished?• What if the work cannot be divided into

completely separate tasks?

What is the common theme of all of these problems?

Parallelization Pitfalls (2)

• Each of these problems represents a point at which multiple threads must communicate with one another, or access a shared resource.

• Golden rule: Any memory that can be used by multiple threads must have an associated synchronization system!

What is Wrong With This?

Thread 1:

void foo() {

x++;

y = x;

}

Thread 2:

void bar() {

y++;

x++;

}

If the initial state is y = 0, x = 6, what happens after these threads finish running?

Multithreaded = Unpredictability

• When we run a multithreaded program, we don’t know what order threads run in, nor do we know when they will interrupt one another.

Thread 1:

void foo() {

eax = mem[x];

inc eax;

mem[x] = eax;

ebx = mem[x];

mem[y] = ebx;

}

Thread 2:

void bar() {

eax = mem[y];

inc eax;

mem[y] = eax;

eax = mem[x];

inc eax;

mem[x] = eax;

}

• Many things that look like “one step” operations actually take several steps under the hood:

Multithreaded = Unpredictability

This applies to more than just integers:

• Pulling work units from a queue• Reporting work back to master unit• Telling another thread that it can begin the

“next phase” of processing

… All require synchronization!

Synchronization Primitives

• synchronization primitive – Semaphore / mutex– Condition variable– Barriers

Semaphores

• A semaphore is a flag that can be raised or lowered in one step

• Semaphores were flags that railroad engineers would use when entering a shared track

Set: Reset:

The “Corrected” Example

Thread 1:

void foo() {

sem.lock();

x++;

y = x;

sem.unlock();

}

Thread 2:

void bar() {

sem.lock();

y++;

x++;

sem.unlock();

}

Global var “Semaphore sem = new Semaphore();” guards access to x & y

Condition Variables

• A condition variable notifies threads that a particular condition has been met

• Inform another thread that a queue now contains elements to pull from (or that it’s empty – request more elements!)

The final example

Thread 1:

void foo() {

sem.lock();

x++;

y = x;

fooDone = true;

sem.unlock();

fooFinishedCV.notify();

}

Thread 2:

void bar() {

sem.lock();

while(!fooDone) fooFinishedCV.wait(sem);

y++;

x++;

sem.unlock();

}

Global vars: Semaphore sem = new Semaphore(); ConditionVar fooFinishedCV = new ConditionVar(); boolean fooDone = false;

Barriers

• A barrier knows in advance how many threads it should wait for. Threads “register” with the barrier when they reach it, and fall asleep.

• Barrier wakes up all registered threads when total count is correct

• Pitfall: What happens if a thread takes a long time?

Too Much Synchronization? Deadlock

Synchronization becomes even more complicated when multiple locks can be used

Can cause entire system to “get stuck”

Thread A:Thread A:semaphore1.lock();semaphore2.lock();/* use data guarded by semaphores */semaphore1.unlock(); semaphore2.unlock();

Thread B:semaphore2.lock();semaphore1.lock();/* use data guarded by semaphores */semaphore1.unlock(); semaphore2.unlock();

(Image: RPI CSCI.4210 Operating Systems notes)

And if you thought I was joking…

The Moral: Be Careful!

• Synchronization is hard– Need to consider all possible shared state– Must keep locks organized and use them

consistently and correctly

• Knowing there are bugs may be tricky; fixing them can be even worse!

• Keeping shared state to a minimum reduces total system complexity

Fundamentals of Networking

Sockets: The Internet = tubes?

• A socket is the basic network interface• Provides a two-way “pipe” abstraction

between two applications• Client creates a socket, and connects to

the server, who receives a socket representing the other side

Ports

• Within an IP address, a port is a sub-address identifying a listening program

• Allows multiple clients to connect to a server at once

Example: Web Server (1/3)

1) Server creates a socket attached to port 80

80

The server creates a listener socket attached to a specific port. 80 is the agreed-upon port number for web traffic.

Example: Web Server (2/3)

The client-side socket is still connected to a port, but the OS chooses a random unused port number

When the client requests a URL (e.g., “www.google.com”), its OS uses a system called DNS to find its IP address.

2) Client creates a socket and connects to host

80Connect: 66.102.7.99 : 80

(anon)

Example: Web Server (3/3)

Server chooses a randomly-numbered port to handle this particular client

Listener is ready for more incoming connections, while we process the current connection in parallel

3) Server accepts connection, gets new socket for client

80

(anon) (anon)

4) Data flows across connected socket as a “stream”, just like a file

What makes this work?

• Underneath the socket layer are several more protocols

• Most important are TCP and IP (which are used hand-in-hand so often, they’re often spoken of as one protocol: TCP/IP)

Your dataTCP header

IP header

Even more low-level protocols handle how data is sent over Ethernet wires, or how bits are sent through the air using 802.11 wireless…

IP: The Internet Protocol

• Defines the addressing scheme for computers

• Encapsulates internal data in a “packet”

• Does not provide reliability

• Just includes enough information for the data to tell routers where to send it

TCP: Transmission Control Protocol

• Built on top of IP• Introduces concept of “connection”• Provides reliability and ordering

Your dataTCP header

IP header

Why is This Necessary?

• Not actually tube-like “underneath the hood”• Unlike phone system (circuit switched), the

packet switched Internet uses many routes at once

you www.google.com

Networking Issues

• If a party to a socket disconnects, how much data did they receive?

• … Did they crash? Or did a machine in the middle?

• Can someone in the middle intercept/modify our data?

• Traffic congestion makes switch/router topology important for efficient throughput

Search Engine Technology

This presentation © Michael CafarellaRedistributed under the Creative Commons Attribution 3.0

license.

Doug Cutting

Meta-details

• Built to encourage public search work– Open-source, w/pluggable modules– Cheap to run, both machines & admins

• Goal: Search more pages, with better quality, than any other engine– Pretty good ranking– Has done ~ 200M pages, more possible

• Hadoop is a spinoff

WebDB

Fetcher 2 of NFetcher 1 of N

Fetcher 0 of N

Fetchlist 2 of NFetchlist 1 of N

Fetchlist 0 of N

Update 2 of NUpdate 1 of N

Update 0 of N

Content 0 of NContent 0 of N

Content 0 of N

Indexer 2 of NIndexer 1 of N

Indexer 0 of N

Searcher 2 of N

Searcher 1 of N

Searcher 0 of N

WebServer 2 of MWebServer 1 of M

WebServer 0 of M

Index 2 of NIndex 1 of N

Index 0 of N

Inject

Moving Parts

• Acquisition cycle– WebDB– Fetcher

• Index generation– Indexing– Link analysis (maybe)

• Serving results

WebDB

• Contains info on all pages, links– URL, last download, # failures, link score,

content hash, ref counting– Source hash, target URL

• Must always be consistent• Designed to minimize disk seeks

– 19ms seek time x 200m new pages/mo = ~44 days of disk seeks!

• Single-disk WebDB was huge headache

Fetcher• Fetcher is very stupid. Not a “crawler”

• Pre-MapRed: divide “to-fetch list” into k pieces, one for each fetcher machine

• URLs for one domain go to same list, otherwise random– “Politeness” w/o inter-fetcher protocols– Can observe robots.txt similarly– Better DNS, robots caching– Easy parallelism

• Two outputs: pages, WebDB edits

2. Sort edits (externally, if necessary)

WebDB/Fetcher Updates

URL: http://www.flickr/com/index.html

LastUpdated: Never

ContentHash: None

URL: http://www.cnn.com/index.html

LastUpdated: Never

ContentHash: None

URL: http://www.yahoo/index.html

LastUpdated: 4/07/05

ContentHash: MD5_toewkekqmekkalekaa

URL: http://www.about.com/index.html

LastUpdated: 3/22/05

ContentHash: MD5_sdflkjweroiwelksd

Edit: DOWNLOAD_CONTENT

URL: http://www.cnn.com/index.html

ContentHash: MD5_balboglerropewolefbag

Edit: DOWNLOAD_CONTENT

URL: http://www.yahoo/index.html

ContentHash: MD5_toewkekqmekkalekaa

Edit: NEW_LINK

URL: http://www.flickr.com/index.html

ContentHash: None

WebDB Fetcher edits

1. Write down fetcher edits3. Read streams in parallel, emitting new database4. Repeat for other tables

URL: http://www.cnn.com/index.html

LastUpdated: Today!

ContentHash: MD5_balboglerropewolefbag

URL: http://www.yahoo.com/index.html

LastUpdated: Today!

ContentHash: MD5_toewkekqmekkalekaa

Indexing

• How to retrieve some information from a large document set efficiently?

Document Collection

User Information Need

• Search inside this news site for articles talks about Culture between China and Japan, and doesn’t talk about students abroad.

• QUERY ：– “ 中国 日本 文化 - 留学生”

How to do it?

• Could grep all of Web Pages for “ 中国” ,“ 文化” and “ 日本” , then strip out pages containing “ 留学生” ?– Slow (for large corpora)– NOT “ 留学生” is non-trivial– Other operations (e.g., find “ 中国” NEAR

“ 日本” ) not feasible

Document Representation

• Bag of words model• Document-term incidence matrix

中国 文化 日本 留学生

教育 北京 …

D1 1 1 0 0 1 1

D2 0 1 1 1 0 0

D3 1 0 1 1 0 0

D4 1 0 0 1 1 0

D5 1 1 1 0 0 1

D6 0 0 1 0 0 1

…

1 if page contains word, 0 otherwise

Incidence Vector

D1 D2 D3 D4 D5 D6…

中国 1 0 1 1 1 0

文化 1 1 0 0 1 0

日本 0 1 1 0 1 1

留学生

0 1 1 1 0 0

教育 1 0 0 1 0 0

北京 1 0 0 0 1 1

…

• Transpose the Document-term incidence matrix• So we have a 0/1 vector for each term.

Retrieval

• Search inside this news site for articles talks about Culture between China and Japan, and doesn’t talk about students abroad.

• To answer query: – take the vectors for “ 中国” ,“ 文化” ,“ 日本” ,

“ 留学生” (complemented) bitwise AND– 101110 AND 110010 AND 011011 AND 100011

= 000010

D5

Let’s build a search system!

• Consider N = 1million documents, each with about 1K terms.

• Avg 6 bytes/term include spaces/punctuation – 6GB of data in the documents.

• Say there are M = 500K distinct terms among these.

Can’t build the matrix

• 500K x 1M matrix has half-a-trillion 0’s and 1’s.

• But it has no more than one billion 1’s.– matrix is extremely sparse.

• What’s a better representation?– We only record the 1 positions.

Why?

Inverted index

• For each term T: store a list of all documents that contain T.

• Do we use an array or a list for this?中国文化留学生

1 2 3 5 8 13 21 34

2 4 8 16 32 64128

13 16

What happens if the word 中国 is added to document 14?

Inverted index

• Linked lists generally preferred to arrays– Dynamic space allocation– Insertion of terms into documents easy– Space overhead of pointers

中国文化留学生

2 4 8 16 32 64 128

2 3 5 8 13 21 34

13 16

1

Dictionary Postings

Sorted by docID (more later on why).

Inverted index construction

Tokenizer

Token stream. Friends Romans Countrymen

Linguistic modules

Modified tokens. friend roman countryman

Indexer

Inverted index.

friend

roman

countryman

2 4

2

13 16

1

Documents tobe indexed.

Friends, Romans, countrymen.

• Sequence of (Modified token, Document ID) pairs.

I did enact JuliusCaesar I was killed

i' the Capitol; Brutus killed me.

Doc 1

So let it be withCaesar. The noble

Brutus hath told youCaesar was ambitious

Doc 2

Term Doc #I 1did 1enact 1julius 1caesar 1I 1was 1killed 1i' 1the 1capitol 1brutus 1killed 1me 1so 2let 2it 2be 2with 2caesar 2the 2noble 2brutus 2hath 2told 2you 2

caesar 2was 2ambitious 2

Indexer steps

Term Doc #I 1did 1enact 1julius 1caesar 1I 1was 1killed 1i' 1the 1capitol 1brutus 1killed 1me 1so 2let 2it 2be 2with 2caesar 2the 2noble 2brutus 2hath 2told 2you 2caesar 2was 2ambitious 2

• Sort by terms. Term Doc #ambitious 2be 2brutus 1brutus 2capitol 1caesar 1caesar 2caesar 2did 1enact 1hath 1I 1I 1i' 1it 2julius 1killed 1killed 1let 2me 1noble 2so 2the 1the 2told 2you 2was 1was 2with 2

Core indexing step

• Multiple term entries in a single document are merged.

• Frequency information is added.Term Doc # Freqambitious 2 1be 2 1brutus 1 1brutus 2 1capitol 1 1caesar 1 1caesar 2 2did 1 1enact 1 1hath 2 1I 1 2i' 1 1it 2 1julius 1 1killed 1 2let 2 1me 1 1noble 2 1so 2 1the 1 1the 2 1told 2 1you 2 1was 1 1was 2 1with 2 1

Term Doc #ambitious 2be 2brutus 1brutus 2capitol 1caesar 1caesar 2caesar 2did 1enact 1hath 1I 1I 1i' 1it 2julius 1killed 1killed 1let 2me 1noble 2so 2the 1the 2told 2you 2was 1was 2with 2

• The result is split into a Dictionary file and a Postings file.

Term Doc # Freqambitious 2 1be 2 1brutus 1 1brutus 2 1capitol 1 1caesar 1 1caesar 2 2did 1 1enact 1 1hath 2 1I 1 2i' 1 1it 2 1julius 1 1killed 1 2let 2 1me 1 1noble 2 1so 2 1the 1 1the 2 1told 2 1you 2 1was 1 1was 2 1with 2 1

Doc # Freq2 12 11 12 11 11 12 21 11 12 11 21 12 11 11 22 11 12 12 11 12 12 12 11 12 12 1

Term N docs Tot Freqambitious 1 1be 1 1brutus 2 2capitol 1 1caesar 2 3did 1 1enact 1 1hath 1 1I 1 2i' 1 1it 1 1julius 1 1killed 1 2let 1 1me 1 1noble 1 1so 1 1the 2 2told 1 1you 1 1was 2 2with 1 1

Why split?

The index we just built

• How do we process a Boolean query?

Query processing

• Consider processing the query:中国 AND 文化– Locate 中国 in the Dictionary;

• Retrieve its postings.

– Locate 文化 in the Dictionary;• Retrieve its postings.

– “Merge” the two postings:

128

34

2 4 8 16 32 64

1 2 3 5 8 13 21

中国文化

34

1282 4 8 16 32 64

1 2 3 5 8 13 21

The merge

• Walk through the two postings simultaneously, in time linear in the total number of postings entries

128

34

2 4 8 16 32 64

1 2 3 5 8 13 21

中国文化2 8

If the list lengths are x and y, the merge takes O(x+y)operations.Crucial: postings sorted by docID.

Indexing in Nutch

• Iterate through all k page sets in parallel, constructing inverted index

• Creates a “searchable document” of:– URL text– Content text– Incoming anchor text

• Other content types might have a different document fields– Eg, email has sender/receiver– Any searchable field end-user will want

• Uses Lucene text indexer

Link analysis• A page’s relevance depends on both

intrinsic and extrinsic factors– Intrinsic: page title, URL, text– Extrinsic: anchor text, link graph

• PageRank is most famous of many

• Others include:– HITS– OPIC– Simple incoming link count

Web Graph

http://www.touchgraph.com/TGGoogleBrowser.html

PageRank Algorithm

Link analysis in Nutch

• Nutch performs analysis in WebDB– Emit a score for each known page– At index time, incorporate score into inverted index

• Extremely time-consuming– In our case, disk-consuming, too (because we want to

use low-memory machines)

• Link analysis is sexy, but importance generally overstated, so fast and easy:– 0.5 * log(# incoming links)

“britn

ey

”

Query ProcessingDocs 0-1M Docs 1-2M Docs 2-3M Docs 3-4M Docs 4-5M

“ britney”“

britney

”

“bri

tney

”

“ britney

”“ britney

”

Ds 1, 29

Ds 1.2M,

1.7M

Ds 2

.3M

, 2.9

M

Ds 3.1M,

3.2MDs 4.4M,

4.5M

1.2

M, 4

.4M

, 29

, …

Administering Nutch• Admin costs are critical

– It’s a hassle when you have 25 machines– Google has >100k, probably more

• Files– WebDB content, working files– Fetchlists, fetched pages– Link analysis outputs, working files– Inverted indices

• Jobs– Emit fetchlists, fetch, update WebDB– Run link analysis– Build inverted indices

Administering Nutch (2)• Admin sounds boring, but it’s not!

– Really– I swear

• Large-file maintenance– Google File System (Ghemawat, Gobioff, Leung)– Nutch Distributed File System

• Job Control– Map/Reduce (Dean and Ghemawat)– Pig (Yahoo Research)

• Data Storage (BigTable)

Nutch Distributed File System• Similar, but not identical, to GFS

• Requirements are fairly strange– Extremely large files– Most files read once, from start to end– Low admin costs per GB

• Equally strange design– Write-once, with delete– Single file can exist across many machines– Wholly automatic failure recovery

NDFS (2)

• Data divided into blocks

• Blocks can be copied, replicated

• Datanodes hold and serve blocks

• Namenode holds meta info– Filename block list– Block datanode-location

• Datanodes report in to namenode every few seconds

NDFS File Read

Namenode

Datanode 0 Datanode 1 Datanode 2

Datanode 3 Datanode 4 Datanode 5

1. Client asks datanode for filename info2. Namenode responds with blocklist,

and location(s) for each block3. Client fetches each block, in

sequence, from a datanode

“ crawl.txt”(block-33 / datanodes 1, 4)(block-95 / datanodes 0, 2)(block-65 / datanodes 1, 4, 5)

NDFS Replication

Namenode

Datanode 0(33, 95)

Datanode 1(46, 95)

Datanode 2(33, 104)

Datanode 3(21, 33, 46)

Datanode 4(90)

Datanode 5(21, 90, 104)

1. Always keep at least k copies of each blk

2. Imagine datanode 4 dies; blk 90 lost3. Namenode loses heartbeat,

decrements blk 90’s reference count. Asks datanode 5 to replicate blk 90 to datanode 0

4. Choosing replication target is tricky

(Blk 90 to dn 0)

Map/Reduce• Map/Reduce is programming model from

Lisp (and other places)– Easy to distribute across nodes– Nice retry/failure semantics

• map(key, val) is run on each item in set– emits key/val pairs

• reduce(key, vals) is run for each unique key emitted by map()– emits final output

• Many problems can be phrased this way

Map/Reduce (2)

• Task: count words in docs– Input consists of (url, contents) pairs– map(key=url, val=contents):

• For each word w in contents, emit (w, “1”)

– reduce(key=word, values=uniq_counts):• Sum all “1”s in values list• Emit result “(word, sum)”

Map/Reduce (3)

• Task: grep– Input consists of (url+offset, single line)– map(key=url+offset, val=line):

• If contents matches regexp, emit (line, “1”)

– reduce(key=line, values=uniq_counts):• Don’t do anything; just emit line

• We can also do graph inversion, link analysis, WebDB updates, etc

Map/Reduce (4)

• How is this distributed?1. Partition input key/value pairs into chunks,

run map() tasks in parallel

2. After all map()s are complete, consolidate all emitted values for each unique emitted key

3. Now partition space of output map keys, and run reduce() in parallel

• If map() or reduce() fails, reexecute!

Map/Reduce Job Processing

JobTracker

TaskTracker 0TaskTracker 1 TaskTracker 2

TaskTracker 3 TaskTracker 4 TaskTracker 5

1. Client submits “grep” job, indicating code and input files

2. JobTracker breaks input file into k chunks, (in this case 6). Assigns work to ttrackers.

3. After map(), tasktrackers exchange map-output to build reduce() keyspace

4. JobTracker breaks reduce() keyspace into m chunks (in this case 6). Assigns work.

5. reduce() output may go to NDFS

“ grep”

Nutch & Hadoop

• NDFS stores the crawl and indexes

• MapReduce for indexing, parsing, WebDB construction, even fetching– Broke previous 200M/mo limit– Index-serving?

• Required massive rewrite of almost every Nutch component

Summary

• Parallelization & Synchronization

• Fundamentals of Networking

• Search Engine Technology– Inverted index– PageRank algorithm

Readings

• [Brin and Page,1998] S. Brin and L. Page, "The anatomy of a large-scale hypertextual Web search engine," presented at Proceedings of the 7th Internation World Wide Web Conference/Computer Networks, Amsterdam, 1998.

Q&A