ORGANIC SEARCH
CRAWL
INDEX
RANK
CRAWLING
KnownWeb
pages
Index Servers
CrawlerMachines
Googlebot
Doc Servers
DEVELOPING THE LIST OF KNOWN WEB PAGES
KnownWeb
pages
Prior Crawls
/addurl
PEREMECH
CRAWLING FOR PEREMECH
usm.maine.edu/index.html
/academic/index.html
/sb/index.html
/sb/facultystaff.html
/sb/zsuleiman.html
/~suleiman/index.html
/~suleiman/peremech/index.html
other linked pages
other linked pages
other linked pages
CRAWLERS IDENTIFY THEMSELVES
ROBOTS.TXT – EXCLUDES SITES FROM CRAWLING
whitehouse.gov/robots.txt
CRAWLERS SHOULD NOT HOG RESOURCES
CRAWLING PROBLEM: THE WEB IS VAST
CRAWLING PROBLEM: THE HIDDEN WEB
INDEXING
RANKING
Closely guarded, proprietary algorithm
RANKING
Organic Search Results
Keyword Phrase
WHY IS RED LOBSTER RANKED HIGHER?Can you find the word Lobster?
HOW CAN I IMPROVE MY RANK?
BASIC SEO CONCEPTS –TEXTKEYWORD PROXIMITY
We sell live Maine lobster
Home of Maine’s best lobster
All of our lobsters come from Maine
BASIC SEO CONCEPTS –TEXTKEYWORD DENSITY
What percentage of the words on this page are lobster.
BASIC SEO CONCEPTS –TEXTKEYWORD FREQUENCY
The keyword lobster is visible 32 times and occurs 182 times on the page
BASIC SEO CONCEPTS –TEXTKEYWORD PROMINENCE
LobsterLobster
Lobster
Lobster
Lobster
Lobster
BASIC SEO CONCEPTS – LINKINGLINK EXCHANGE AND PAID PLACEMENT
BASIC SEO CONCEPTS – LINKING
An article written at thefreelibrary.com
BASIC SEO CONCEPTS – LINKINGPAID PLACEMENT
BASIC SEO CONCEPTS – LINKINGLINK EXCHANGE - RELEVANT
BASIC SEO CONCEPTS – LINKINGLINK EXCHANGE – NON RELEVANT
BASIC SEO CONCEPTS – LINKINGLINK SPAM
BASIC SEO CONCEPTS – LINKINGCOMMENT/ FORUM SPAM
BASIC SEO CONCEPTS – LINKINGCOMMENT/ FORUM SPAM
LINK EXCHANGE IS CONTROVERSIAL
USER RELEVANCE
LOCATION
USER ID
130.111.115.24
USER EXPERTISE VARIES
PHOTO ATTRIBUTIONSSlide 10: 'OINK! OINK!’ www.flickr.com/photos/49968232@N00/51679641
Slide 11: ‘The Vast Web’ wikicommons.com
Slide 13: 'Royal Guards’ www.flickr.com/photos/23697065@N05/2356109302
Slide 17: 'Yoda (IMG_3308)‘ www.flickr.com/photos/73531322@N00/194828922
Slide 30: ‘Vera Cruz’ BAM/Photofest
