Ismétlés 1. Gyakorlat Felkészülés a web programozásra Kecskeméti Gábor A/1 336-os szoba – fizika tanszék Konzultációs idő: Csütörtök 9-11 Előadás: XX.

Ismétlés 1. GyakorlatFelkészülés a web programozásra

Kecskeméti GáborA/1 336-os szoba – fizika tanszékKonzultációs idő: Csütörtök 9-11

Előadás: XX. Előadó, Csütörtök 14-16Gyakorlat: 207-es terem, Csütörtök 12-14

2

The Universal Resource Locator (URL)

Each page of information on the web has a unique address called the URL at which it can be found

http://faculty.uscupstate.edu/atzacheva/lecture1.html

The document can be obtained

using the Hypertext

Transfer Protocol (HTTP)

Host Name - The Name of Web Server

Path to the Web Page

Denotes that the File is Written in HTMLHyperText Markup

Language

File Name

1 2 3

Protocol Host Name File Name

Önálló feladat

• Készítsünk egy programot ami feldolgozza a parancssori paraméterként kapott URLt és darabjaira bontja

• Az URL alábbi részeit minimum tudja felolvasni a program:– 1. protokoll (séma)– 2. Domain név (hostazonosító)– 3. Port (nem protokollnak megfelelő portszám)– 4. Elérési útvonal– 5. Ha Queryt is tartalmaz az URL akkor a paraméterek nevei– 6. Ha Queryt is tartalmaz az URL akkor a paraméterek értékei

• Ha valaki elkészült a feladattal ellenőrzését és tesztelését én végzem, jelezze mindenki ha mehetek

Példák

• urlparser.exe http://www.hwsw.hu– Protokoll (séma): http– Domain név (hostazonosító): www.hwsw.hu

• urlparser.exe x://y:1/a?b=c&d=e– Protokoll (séma): x– Domain név (hostazonosító): y– Port (nem protokollból következő): 1– Elérési útvonal: a?b=c&d=e– Query paraméterek: b,d– Query értékek: c,e

A HTTP protokoll

Kecskeméti GáborA/1 336-os szoba – fizika tanszékKonzultációs idő: Csütörtök 9-11

Előadás: XX. Előadó, Csütörtök 14-16Gyakorlat: 207-es terem, Csütörtök 12-14

EMLÉKEZTETŐ

Internet protocol stack• application: supporting network applications

– FTP, SMTP, HTTP• transport: process-process data transfer

– TCP, UDP• network: routing of datagrams from source to

destination– IP, routing protocols

• link: data transfer between neighboring network elements

– PPP, Ethernet• physical: bits “on the wire”

application

transport

network

link

physical

Ports

• Each IP address is subdivided into ports, each port assigned to a single program

• You can browse the Internet using one port while receiving e-mail using another port, with a single IP address

Protocols• Sets of rules or standards that let computers

communicate over the Internet• Type or size or brand of computer doesn’t

interfere with communication if the protocol is used

HÁTTÉR

CS 640 11

WWW Background

• 1989-1990 – Tim Berners-Lee invents the World Wide Web at CERN– Means for transferring text and graphics simultaneously– Client/Server data transfer protocol

• Communication via application level protocol• System ran on top of standard networking infrastructure

– Text mark up language• Not invented by Bernes-Lee• Simple and easy to use• Requires a client application to render text/graphics

About: Tim Bernes Lee• Born: 1955• Native City: London• Education: B.A. in physics, Queen's College, Oxford, 1976 .• Hobbies: Windsurfing• Misc Facts:

– Knighted by the British Empire in 1998.– While naming the “World Wide Web” he also considered the names “The Information Mine” and

“Information Mesh”– In October 1999, Berners-Lee published the book Weaving the Web: The Original Design and

Ultimate Destiny of the World Wide Web by Its Inventor, which told the personal story of how he created the World Wide Web and discussed the future of the mass medium.

The Story of Lee’s Invention• Tim Bernes Lee attended college at Oxford in 1976. 1980 he went to go work for a software

company called CERN.

• Always been fascinated by computers, it was at Oxford that he struggled with a problem.

• This problem was of integrating and exchanging information held on different computers in often widely scattered places.

• He found a solution to his problem and named it Enquire.

• This method incorporated the use of hypertext, a system that links documents from different

sources, forming an electronic path that users follow to obtain related information.

Tim Berners-Lee

Tim Berners-Lee was knighted by Queen Elizabeth for his invention of the World Wide Web. He is shown here, along with the first picture posted on the Web and a screen shot from an early version of his Web browser.

CS 640 15

WWW History contd.• 1994 – Mark Andreesen invents MOSAIC at National Center for

Super Computing Applications (NCSA)– First graphical browser– Internet’s first “killer app”– Freely distributed– Became Netscape Inc.

• 1995 (approx.) – Web traffic becomes dominant– Exponential growth– E-commerce– Web infrastructure companies– World Wide Web Consortium

• Reference: “Web Protocols and Practice”, Krishnamurthy and Rexford

No royalties • While the component ideas of the World Wide Web are

simple, Berners-Lee's insight was to combine them in a way which is still exploring its full potential. Perhaps his greatest single contribution, though, was to make his idea available freely, with no patent and no royalties due.

• In 1994 he founded World Wide Web Consortium (W3C) at the MIT Laboratory for Computer Science in Cambridge, Massachusetts, and in 2003, the organization decided that all standards must contain royalty-free technology, so they can be easily adopted by anyone.

Taken from: http://en.wikipedia.org/wiki/Tim_Berners-Lee#No_royalties

CS 640 17

WWW Components• Structural Components

– Clients/browsers – to dominant implementations– Servers – run on sophisticated hardware– Caches – many interesting implementations– Internet – the global infrastructure which facilitates data transfer

• Semantic Components– Hyper Text Transfer Protocol (HTTP)– Hyper Text Markup Language (HTML)

• eXtensible Markup Language (XML)– Uniform Resource Identifiers (URIs)

18

World Wide Web

• WWW comprises software (Web server and browser) and data (Web sites)

Client Side

JavaScriptVBScriptDHTML

Java Applets

Server Side

CGIASP

Java Servlets

HTML, XML, ...

CS 640 19

Quick Aside – Web server use

Source: Netcraft Server Survey, 2001

CS 640 20

WWW Structure

• Clients use browser application to send URIs via HTTP to servers requesting a Web page

• Web pages constructed using HTML (or other markup language) and consist of text, graphics, sounds plus embedded files

• Servers (or caches) respond with requested Web page– Or with error message

• Client’s browser renders Web page returned by server– Page is written using Hyper Text Markup Language (HTML)– Displaying text, graphics and sound in browser– Writing data as well

• The entire system runs over standard networking protocols (TCP/IP, DNS,…)

21

How HTML is Displayed

Browser Command

HTML Display

Http protocol(HyperText Transfer Protocol)

Text & binary data

render

HTML URL:http://www.google.com

22

How HTML is Displayed – from remote site

HTML Display

User

Browser CommandURL:http://www.yahoo.com

RemoteWeb Server

Client Site

DB

HTMLCGIASPPHP…

http request

http response

23

How HTML is Displayed – from client site

HTML Browser Command

URL:c:\my_page.html

HTML Display

User

Client Site

What is HTTP?• HyperText Transfer Protocol, the underlying protocol used

by the World Wide Web. HTTP defines how messages are formatted and transmitted, and what action Web servers and browsers should take in response to various commands. For example, when you enter a URL in your browser, this actually sends an HTTP command to the Web server directing it to fetch and transmit the requested Web page. www.wmo.ch/web/www/WDM/Guides/Internet-glossary.html

• Basically HTTP is a protocol used by computers to transmit data from one computer to another. It is a universal format that is understood most all computers now-a-days and is the backbone of the World Wide Web (WWW).

25

HTTP Basics• Protocol for client/server communication

– The heart of the Web– Very simple request/response protocol

• Client sends request message, server replies with response message– Stateless– Relies on URI naming mechanism

• Three versions have been used– 09/1.0 – very close to Berners-Lee’s original

• RFC 1945 (original RFC is now expired)– 1.1 – developed to enhance performance, caching, compression

• RFC 2068– 1.0 dominates today but 1.1 is catching up

• HTTP defines how Web pages are requested and served on the Internet

• Early servers and browsers used an ad-hoc approach

• A standardized protocol, called HTTP/1.0, was derived from this

• The earlier approach is now called HTTP/0.9• Later, HTTP/1.0 was extended to HTTP/1.1• The protocol versions are upwardly compatible

– servers and browsers which can handle HTTP/1.1 can also handle HTTP/1.0 and HTTP/0.9

History: “HTTP/0.9”• HTTP/0.9 was very simple:

– A browser would send a request like this to a server: GET /hobbies.html

– In response, the server would send the contents of the requested file.

– Only GET requests were supported– Only a file path and name could appear in a GET

request– The response had to be a HTML document.

History (contd.)• Different browsers/servers soon extended this basic

scheme in various ways• To achieve some standardization, the HTTP/1.0 protocol

was specified, in 1996, in a document called RFC1945– (for historical reasons, an Internet standard spec is called a

Request for Comment or RFC)

• This was soon extended to HTTP/1.1, in RFC2068, released in January 1997

• An update to RFC2068 was produced in June 1999, as RFC2616

• Various other protocols, based on HTTP, have been produced from time-to-time– we will see a “cookie” protocol, based on HTTP, which was

specified in February 1997, in RFC2109

Both were invented at the same time by the same person

HTTP vs HTML

• HTML: hypertext markup language– Definitions of tags that are added to Web documents to

control their appearance

• HTTP: hypertext transfer protocol– The rules governing the conversation between a Web client

and a Web server

PROTOKOLL ALAPOK

HTTP is an application layer protocol

• The Web client and the Web server are application programs• Application layer programs do useful work like retrieving Web pages,

sending and receiving email or transferring files• Lower layers take care of the communication details• The client and server send messages and data without knowing anything

about the communication network

Web communication

Internet

Get http://www.google.com/index.html

HTTP Request:

HTTP Reply

Client: browser (Firefox) on local computer

Web server: apache on www.google.com

Overall Operation of HTTP

• The HTTP protocol is a request/response protocol.• request

– An HTTP message sent by a client to a server• response

– An HTTP message sent by a server to a client which has made a request.

• client– A program that establishes connections for the purpose of

sending requests. • server

– A program that accepts connections in order to service requests by sending back responses.

• As we shall see, a program may act as both a client and a server.

HTTP – HyperText Transfer Protocol

• together with HTML forms the base of WWW• is standardized by IETF (rfc 2616)• is a request-response protocol• it is stateless (does not maintain a state of a session) and

asynchronous (an html document is loaded asynchronous by the browser, as soon as parts of it are available)

• latest version is HTTP/1.1• runs on top of TCP on the standardized port 80

• I would like to open a connection

• GET <file location>

• Display response• Close connection

• OK

• Send page or error message

• OK

Client Server

HTTP is the set of rules governing the format and content of the conversation between a Web client and server

An HTTP conversation

HTTP messages

• HTTP is the language that web clients and web servers use to talk to each other– HTTP is largely “under the hood,” but a basic

understanding can be helpful• Each message, whether a request or a

response, has three parts:1. The request or the response line2. A header section3. The body of the message

A KLIENS OLDALA http kérés – egy böngészőtől (pl. Firefox, IE, Chrome)

Message from a client:

A client sends, over a connection, to a server • a request line in the form of

– a request method, – a URI (Uniform Resource Identifier), and – a protocol version,

• possibly followed by a message containing – request modifiers,– information about the client,

• and (possibly) body content.

What the client does, part I

• The client sends a message to the server at a particular port (80 is the default)

• The first part of the message is the request line, containing:– A method (HTTP command) such as GET or POST– A document address, and– An HTTP version number

• Example:– GET /index.html HTTP/1.0

HTTP Request Packets

• Sent from client to server• Consists of HTTP header

– header is hidden in browser environment– contains:

• content type / mime type• content length• user agent - browser issuing request• content types user agent can handle

• and a URL

HTTP - methods

• Methods– GET

• retrieve a URL from the server– simple page request– run a CGI program– run a CGI with arguments attached to the URL

– POST• preferred method for forms processing• run a CGI program• parameterized data in sysin• more secure and private

HTTP Request• has the form:

Request-Method SP Request-URL SP HTTP-Version <CR><LF>(generic-header | request-header | entity-header <CR><LF>)<CR><LF>[message body]

• Request-Method is:– GET – request whatever information is identified by the Request-URL

– POST – request that server accepts the entity enclosed in the request

– OPTIONS - request for information about communication options

– PUT – request that the enclosed entity be stored under the Request-URL

– DELETE – request that the server delete the resource identified by Request-URL

– TRACE – invoke a remote, application-layer loopback of the request message

– CONNECT – used by proxies in SSL connections

– HEAD – identical to GET, but server must not return a message body in response

– LINK: Request header information be associated with a document on the server

– UNLINK: Request to undo a LINK request

HTTP - methods

• Methods (cont.)– PUT

• Used to transfer a file from the client to the server

– HEAD• requests URLs status header only• used for conditional URL handling for performance

enhancement schemes– retrieve URL only if not in local cache or date is more recent than

cached copy

What the client does, part II

• The second part of a request is optional header information, such as:– What the client software is– What formats it can accept

• All information is in the form Name: Value• Example:

User-Agent: Mozilla/2.02Gold (WinNT; I)Accept: image/gif, image/jpeg, */*

• A blank line ends the header

HTTP Request Headers

• Precede HTTP Method requests• headers are terminated by a blank line• Header Fields:

– From - Email address of user of client program– Accept - MIME types of resources accepted by browser– Accept-Encoding - encoding accepted by browser– Accept Language - language accepted by browser

(Preferred language (For example: English - en, French - fr, German - de))

– Accept-Charset : charset accepted by browser

From:

• In internet mail format, the requesting user• Does not have to correspond to requesting host

name (might be a proxy)• should be a valid e-mail address

Accept:

– List of schemes which will be accepted by client– <field> = Accept: <entry> * [,<entry>]– <entry> = <content type> *[;<param>]– <param> = <attr> = <float>– <attr> = q / mxs / mxb

– <float> = <ANSI-C floating point > – Accept: text/html– Accept: audio/basic q-1– if no Accept is found; plain/text is assumed– may contain wildcards (*)

Accept-Encoding

• Like Accept but list is a list of acceptable encoding schemes

• Ex– Accept-Encoding: x-compress;x-zip

HTTP Request Headers (cont.)

– Referer - the URL of document refering this URL– Authorization - user-agent wishes to authenticate

itself with a server– Host : the host Request-URL points to– Charge-To– If-Modified-Since– Pragma – User-Agent : The browser or other client program

sending the request– Cookies: Information persistence between requests

Referer

• For Server’s benefit, client lists URL od document (or document type) from which the URL in request was obtained.

• Allows server to generate back-links, logging, tracing of bad links…

• Ex.– Referer: http:/www.w3.com/xxx.html

Authorization:

• For Password and authentication schemes• Ex.

– Authorization: user fred:mypassword– Authorization: kerberos kerberosparameters

ChargeTo:

• Accounting information• Accounting system dependent

Modified-Since:

• Used with GET to make a conditional GET• if requested document has not been modified

since specified date a Modified 304 header is sent back to client instead of document– client can then display cached version

Pragma:

• Same format as accept• for servers• should be passed through proxies, but used by

proxy• only pragma currently defined is no-cache;

proxy should get document from owning server rather than cache

User-Agent

• Software product used by original client• <field> = User-Agent: <product>• <product> = <word> [/<version>]• <version> = <word>• Ex.

– User-Agent: IBM WebExplorer DLL /v960311

Cookies

• Cookie: name=value– Information about a cookie for that URL– Multiple cookies can be separated by commas

What the client does, part III

• The third part of a request (after the blank line) is the entity-body, which contains optional data– The entity-body part is used mostly by POST

requests– The entity-body part is always empty for a GET

request

HTTP Request exampleGet http://www.google.com HTTP/1.1Host: www.google.comUser-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US;

rv:1.9.1.3) Gecko/20090824 Firefox/3.5.3 (.NET CLR 3.5.30729)Accept:

text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8

Accept-Language: en-us,en;q=0.5Accept-Encoding: gzip,deflateAccept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7Keep-Alive: 300Connection: keep-aliveCookie:PREF=ID=141ca2d4581746b4:U=f22e9e94ccc4a56f:FF=4:LD=en:N

R=10:CR=2:TM=1249567334:LM=1251146058:GM=1:S=qWowBrte7hrXniGp; NID=27=n9Khexo85YHnovw93wK4qC2lZtGa1DnzVQEB6iul9tn62fsJ7gFuMVK252ceLCD3iS54r-nHD6kWDdD1JP77akDhMl0EWzoTbPt3cM5g8mapG9SskdRSyEyLWcJK1LrX

Cache-Control: max-age=0

SZERVER OLDALA HTTP válasz – pl egy Apache webszervertől

Response Packets

• Sent by server to client browser in response to a Request Packet

What the server does, part I

• The server response is also in three parts• The first part is the status line, which tells:

– The HTTP version– A status code– A short description of what the status code means

• Example: HTTP/1.1 404 Not Found• Status codes are in groups:

100-199 Informational 200-299 The request was successful 300-399 The request was redirected 400-499 The request failed 500-599 A server error occurred

Common status codes• 200 OK

– Everything worked, here’s the data• 301 Moved Permanently

– URI was moved, but here’s the new address for your records• 302 Moved temporarily

– URL temporarily out of service, keep the old one but use this one for now

• 400 Bad Request– There is a syntax error in your request

• 403 Forbidden– You can’t do this, and we won’t tell you why

• 404 Not Found– No such document

• 408 Request Time-out, 504 Gateway Time-out– Request took too long to fulfill for some reason

Status Codes

• 201 created• 202 accepted• 204 no content• 304 not modified

• 401 unauthorized• 500 int. server error• 501 not impl.• 502 bad gateway• 503 svc not avail

HTTP Response• has the form:

Http-Version SP Status-Code SP Reason-Phrase<CR><LF>(generic-header | response-header | entity-header <CR><LF>)<CR><LF>[message body]

• Response-header has the following fields (selection):– Accept-Ranges : server indicates its acceptance of range requests for resource

– Age : sender’s estimate of the amount of time since the response was generated by server

– Location : redirect the client to a location other than Request-URL for completion of the request

– Retry-After : indicate to client how long the service is expected to be unavailable

– Server : information about software used by the server to handle the request

What the server does, part II• The second part of the response is header information,

ended by a blank line• Example:• Content-Length: 2532

Connection: CloseServer: GWS/2.0Date: Sun, 01 Dec 2002 21:24:50 GMTContent-Type: text/htmlCache-control: privateSet-Cookie: PREF=ID=05302a93093ec661:TM=1038777890:LM=1038777890:S=yNWNjraftUz299RH; expires=Sun, 17-Jan-2038 19:14:07 GMT; path=/; domain=.google.com

All on one line

There is a nice header viewer at http://www.delorie.com/web/headers.html

Server response headers

• Server: NCSA/1.3– Name and version of the server

• Content-Type: type/subtype– Should be of a type and subtype specified by the

client’s Accept header• Set-Cookie: name=value; options

– Requests the client to store a cookie with the given name and value

Set-Cookie Options• Domain, path:

– define the scope of the cookie– cookies should only be sent back to the server for the given

domain and path– If not specified, they default to the domain and path of the

object that was requested.• Expires

– Exact date– When to delete the cookie from the browser

• Max-Age– interval of seconds in the future, relative to the time the

browser received the cookie

What the server does, part III

• The third part of a server response is the entity body

• This is often an HTML page– But it can also be a jpeg, a gif, plain text, etc.--

anything the browser (or other client) is prepared to accept

EGYSZERŰ PÉLDA

Connecting to the HTTP server on student.cs.ucc.ie

• On any machine, say interzone, specify the address and port in a telnet command:

interzone.ucc.ie> telnet student.cs.ucc.ie 80• You will get the following response:

Trying 143.239.211.125...Connected to student.cs.ucc.ie.Escape character is '^]'.

• The HTTP server is now listening

Requesting the home page• Issue the following HTTP/1.0 request, noting that you

must type two carriage returns:GET / HTTP/1.0 [RETURN] [RETURN]

• The response consists of – a status line, – a sequence of headers and – the requested home page

• Then you are told that the telnet connection was closed by the server,

as you will see on the next slide

The reply to your request:

• The server’s response:HTTP/1.1 200 OK...Content-Type: text/html <HTML>... </HTML>

• Then your local telnet program tells you that the connection was closed by the server:

Connection closed by foreign host.interzone.ucc.ie>

Getting a different page:• Consider the page whose URL is

http://student.cs.ucc.ie/cs1064/jabowen/• Telnet to the server:

interzone.ucc.ie> telnet student.cs.ucc.ie 80• When the server is listening, ask for the page like this:

GET /cs1064/jabowen/ HTTP/1.0 [RETURN] [RETURN]

What was going on above:

• Once connected to a HTTP server, we can – send a HTTP request line, – optionally followed by request headers.

• In the cases above,GET / HTTP/1.0and GET /cs1064/jabowen/ HTTP/1.0were request lines

• Each request line was terminated by pressing [RETURN]• In each case, the second [RETURN] marked the end of an

empty list of request headers

ÁLLAPOTKEZELÉS

Statelessness

• Because of the Connect, Request, Response, Disconnect nature of HTTP it is said to be a stateless protocol– i.e. from one web page to the next there is

nothing in the protocol that allows a web program to maintain program “state” (like a desktop program).

– “state” can be maintained by “witchery” or “trickery” if it is needed

Maintaining program “state”

• Hidden variables (<input type=hidden>• Sessions

– Special header tags interpreted by the server• Used by ASP, PHP, JSP

– Implemented at the language api level

PROTOKOLL ÖSSZEHASONLÍTÁS

CS 640 79

HTTP/1.0 Network Interaction

• Clients make requests to port 80 on servers– Uses DNS to resolve server name

• Clients make separate TCP connection for each URL– Some browsers open multiple TCP connections

• Netscape default = 4

• Server returns HTML page– Many types of servers with a variety of implementations– Apache is the most widely used

• Freely available in source form

• Client parses page– Requests embedded objects

CS 640 80

HTTP/1.1 Performance Enhancements• HTTP/1.0 is a “stop and wait” protocol

– Separate TCP connection for each file• Connect setup and tear down is incurred for each file• Inefficient use of packets• Server must maintain many connections in TIME_WAIT

• Mogul and Padmanabahn studied these issues in ’95– Resulted in HTTP/1.1 specification focused on performance

enhancements• Persistent connections• Pipelining• Enhanced caching options• Support for compression

Simple communication

• Involves single connection between user agent (UA) and origin server (O)

• This connection is denoted, in diagrams on this and future slides, by -------

====request chain ==========> UA -----------------------------------O <=========response chain====

More complicated case • Intermediaries present in request/response chain.

====request chain =======================> UA ----------- A ----------- B ----------- C ----------- O

<======================response chain==== • Above, 3 intermediaries (A, B, and C) lie between user

agent and origin server. • Intermediaries act as both clients and servers• Request or response message that travels the whole chain

passes through 4 separate connections: UA-A connection; A-B connection; B-C connection;

C-O connection

Simple versus complicated

• Distinction is important because some HTTP options may apply – only to the connection with the nearest

neighbour, – only to the end-points of the chain, – or to all connections along the chain.

3 forms of intermediary

• proxy, an agent which– receives a request for a resource whose URI is in its absolute form

and,– if necessary, rewrites all or part of the message and forwards the

reformatted request toward the server identified by the URI.

• gateway, an agent which– acts as a translation interface to a server for another protocol,

such as WAP, etc.

• tunnel, an agent which – acts as a relay point between two connections without changing

messages; – tunnels are used, for example, in security firewalls

CACHING

Caching

• User agents, proxies and gateways (but not tunnels) may use a local cache to handle requests, instead of forwarding them on to an origin server

• A request/response chain is shortened if one of the parties along the chain has a cached response applicable to the request.

Example Network topology

The example caching scenarios in the next few slides will use this network:

UA3____________D | UA2_____ | | | | | UA1_____A______B________C_________O

Caching Example 1

====request chain ====================>UA1 ----------- A ----------- B -------- C --------- O <==================response chain=====

• In the example above:– the user has made a request for a resource on origin server O– neither UA1 nor any of the proxies A, B or C has an appropriate

cached response– so the request has been forwarded all the way to O– Four connections are involved in servicing the request

Caching Example 2

request chainUA1…………….... A ……... B …….. C …… O response chain

• In the example above:– the user has repeated the same request for a resource on O– UA1 has a cached response to the earlier request and gives this to

the user without sending the request anywhere– No connection is involved in servicing the request

Caching Example 3

===request chain =>UA2 ----------------- UA1 …..……...... A …….. B …….. C ……... O<=response chain==

• In the example above:– the user at UA2 has requested the same resource on origin server

O that was earlier requested by the user at UA1– UA2 has forwarded the request to proxy A– proxy A has an appropriate cached response, from when it

serviced the earlier request from UA1– Only one connection is involved in servicing the request

Caching Example 4===request chain ====>UA3 ---------- D -------- |UA1 …..…... A …….. B …….. C ……... O <===response chain===

• In the example above:– the user at UA3 has requested the same resource on origin server

O that was earlier requested by the user at UA1– UA3 has forwarded the request to proxy D, which has forwarded it

to proxy B– proxy B has an appropriate cached response, from when it

serviced the earlier request from UA1 – Two connections are involved in servicing the request

To cache or not?

• Not all responses are usefully cacheable• As we will see later, some requests may contain modifiers

which place special requirements on cache behavior.

Caching/Proxy architectures

• A wide variety of cache and proxy architectures/configurations exist, including:– national hierarchies of proxy caches to save inter-national and/or

inter-continental bandwidth,– systems that broadcast or multicast cache entries, – organizations that distribute subsets of cached data via CD-ROM, – and so on.

CONNECTIONS

Temporary Connections• In most implementations of HTTP/1.0, a server closed a connection

after it had serviced the request received on that connection: – We saw this earlier, when the server on student.cs.ucc.ie closed

the telnet connection that we had established, after it had sent its response to the HTTP/1.0 GET request we had sent

• The use of inline images, sound files, etc., in web pages often requires a client to make multiple requests of the same server when loading one document

• Thus the temporary connections provided by HTTP/1.0 meant that loading even one web page required many separate TCP connections (one to to fetch each inline image, each sound file etc.)

• This imposed a significant unnecessary load on HTTP servers and caused congestion on the Internet.

Advantages of Persistent Connections

Persistent HTTP connections offer a number of advantages: – By opening and closing fewer TCP connections, CPU time is saved– HTTP requests and responses can be pipelined on a connection,

allowing a client to make multiple requests without waiting for each response

– Network congestion is reduced by reducing the number of packets caused by TCP opens,

– Latency on subsequent requests is reduced since there is no time spent in TCP's connection-opening handshake.

Persistent Connections in HTTP/1.1 • Unlike HTTP/1.0 and earlier, persistent connections are

the default behavior of any HTTP/1.1 connection.• This means that, in HTTP/1.1, when a connection has been

opened to service a request, it is kept open for further possible requests from the same client

• This is true even if the initial request triggered an error response from the server

• But, when no further request has been received after some time-out period, the server may close the connection

• However, a client can indicate, when making a request, that it wants the connection closed after the request is serviced

Connection Persistency Negotiation

• HTTP/1.1 provides a mechanism by which a client and a server can signal the close of a TCP connection. – the Connection: header field.

• If a HTTP/1.1 client wants a connection closed after it receives a response to its request, it should include, in the request, a Connection: header containing the token "close" .

• Similarly, if a HTTP/1.1 server intends to close a connection closed after it sends a response to a request, it should include, in the response, a Connection: header containing the token "close" .

• If either the client or the server sends the close token in a Connection: header, that request becomes the last one for the connection.

Example 1: Introduction

• A human, using a telnet client, sends a HTTP/1.0 request to a HTTP/1.1 server

• The server assumes that the client, because it is using HTTP/1.0, cannot handle persistent connections and, in its response, signals its intention to close the connection

• After printing the response, the telnet client says that the connection was closed by the foreign host

Example 1interzone.ucc.ie> telnet student.cs.ucc.ie 80Trying 143.239.211.125...Connected to student.cs.ucc.ie.Escape character is '^]'.HEAD /cs1064/jabowen/ HTTP/1.0 HTTP/1.1 200 OKDate: Sat, 06 Jan 2001 17:56:44 GMTServer: Apache/1.3.14 (Unix) PHP/4.0.3pl1Last-Modified: Wed, 20 Dec 2000 11:34:46 GMTETag: "2160-2dee-3a409956"Accept-Ranges: bytesContent-Length: 11758Connection: closeContent-Type: text/html Connection closed by foreign host.

Example 2: Introduction

• A human, using a telnet client, sends a HTTP/1.1 request to a HTTP/1.1 server

• The server assumes that the client, because it is using HTTP/1.1, wants a persistent connection– thus, there is no Connection: header in the response

• The telnet client prints the response for the human to see• After a significant delay (the time-out period), the server

realizes the client has no further request and closes the connection

• The telnet client then tells the human that the connection was closed by the foreign host

Example 2:interzone.ucc.ie> telnet student.cs.ucc.ie 80Trying 143.239.211.125...Connected to student.cs.ucc.ie.Escape character is '^]'.HEAD /cs1064/jabowen/ HTTP/1.1Host: student.cs.ucc.ie HTTP/1.1 200 OKDate: Sat, 06 Jan 2001 17:57:08 GMTServer: Apache/1.3.14 (Unix) PHP/4.0.3pl1Last-Modified: Wed, 20 Dec 2000 11:34:46 GMTETag: "2160-2dee-3a409956"Accept-Ranges: bytesContent-Length: 11758Content-Type: text/html

A time-out period elapses before server closes connection

Connection closed by foreign host.

Example 3: Introduction• A human, using a telnet client, sends a HTTP/1.1 request

to a HTTP/1.1 server• The client knows that, because it is using HTTP/1.1, the

server will think it wants a persistent connection• Since the client does not want a persistent connection it

sends a Connection: header with a close token in the request

• Seeing this, the server indicates its intention to close the connection immediately, by including a Connection: header with a close token in its response

• The telnet client prints the response for the human to see and, immediately thereafter, tells the human that the connection was closed by the foreign host

Example 3:

interzone.ucc.ie> telnet student.cs.ucc.ie 80Trying 143.239.211.125...Connected to student.cs.ucc.ie.Escape character is '^]'.HEAD /cs1064/jabowen/ HTTP/1.1Host: student.cs.ucc.ieConnection: close HTTP/1.1 200 OKDate: Sat, 06 Jan 2001 17:57:58 GMTServer: Apache/1.3.14 (Unix) PHP/4.0.3pl1Last-Modified: Wed, 20 Dec 2000 11:34:46 GMTETag: "2160-2dee-3a409956"Accept-Ranges: bytesContent-Length: 11758Connection: closeContent-Type: text/html Connection closed by foreign host. (No time-out delay before this from telnet client)

Pipelining Requests • A client that supports persistent connections may

"pipeline" its requests (i.e., send multiple requests without waiting for each response).

• A server must send its responses to those requests in the same order that the requests were received.

Example 4: Introduction• A human, using a telnet client, sends two HTTP/1.1

requests to a HTTP/1.1 server, sending the second request before it even receives a response to the first request

• Since he has only two requests, the client sends a Connection: header with a close token in the second request

• The server responds to both requests and, because of the close token in the 2nd request, indicates its intention to close the connection immediately, by including a Connection: header with a close token in its response to the 2nd request.

• The telnet client prints the responses for the human to see and, immediately thereafter, tells the human that the connection was closed by the foreign host

Example 4: the pipelined requestsinterzone.ucc.ie> telnet student.cs.ucc.ie 80Trying 143.239.211.125...Connected to student.cs.ucc.ie.Escape character is '^]'.HEAD http://student.cs.ucc.ie/cs1064/jabowen/ HTTP/1.1Host: student.cs.ucc.ie HEAD http://student.cs.ucc.ie/cs4400/jabowen/ HTTP/1.1Host: student.cs.ucc.ieConnection: close

Example 4: the sequence of responses

HTTP/1.1 200 OKDate: Wed, 31 Jan 2001 20:01:41 GMTServer: Apache/1.3.14 (Unix) PHP/4.0.3pl1Last-Modified: Thu, 25 Jan 2001 13:26:32 GMTETag: "2160-2e25-3a702988"Accept-Ranges: bytesContent-Length: 11813Content-Type: text/html HTTP/1.1 200 OKDate: Wed, 31 Jan 2001 20:01:41 GMTServer: Apache/1.3.14 (Unix) PHP/4.0.3pl1Last-Modified: Wed, 20 Dec 2000 12:42:39 GMTETag: "13d3a-2b60-3a40a93f"Accept-Ranges: bytesContent-Length: 11104Connection: closeContent-Type: text/html Connection closed by foreign host. (No time-out delay before this message from telnet client)

Pipelining Requests (contd.)

• Clients which assume persistent connections and pipeline immediately after connection establishment should be prepared to retry their connection if the first pipelined attempt fails.

• If a client does such a retry, it must NOT pipeline before it knows the connection is persistent.

• Clients must also be prepared to resend their requests if the server closes the connection before sending all of the corresponding responses.

Pipelining Requests (contd.)• Care must be taken when pipelining

– because some requests (called non-idempotent requests) may change the state of the server (for example, by changing a database used by the server)

• Clients should NOT pipeline such requests• Otherwise, a premature termination of the transport

connection could lead to indeterminate results.• A client wishing to send a non-idempotent request should

wait to send that request until it has received the response status for the previous request.

Summary• HTTP is a fairly straightforward protocol with a lot of

possible kinds of predefined header information– More kinds can be added, so long as client and server agree

• A request from the client consists of three parts:1. A header line2. A block of header information, ending with a blank

line3. The (optional) entity body, containing data

• A response from the server consists of the same three parts• HTTP headers are “under the hood” information, not

normally displayed to the user