May 20, 2015 Networking Part I May 20, 2015 JDP Networking Part I1 of 67.

May 20, 2015

Networking Part INetworking Part I

May 20, 2015

JDP Networking Part I 1 of 67

May 20, 2015

Administrivia: IntroductionsAdministrivia: IntroductionsDennis Mumaugh

Undergraduate: BSEE - University of California, BerkeleyMS Computer Science - University of MarylandPh.D. Studies - University of MarylandTeaching at DePaul since September 2000

WorkSenior Engineer - National Security Agency

ARPANet Pioneer, Unix™ Technology TransferMember of the Technical Staff - Bell Labs/Lucent Technologies

Unix Development - Current EngineeringIS&R Systems - Knowledge Based SystemsSoftware Tools and OO Technology

InterestsOperating Systems and System ProgrammingSoftware Productivity, Compilers and Software MetricsSoftware Engineering


May 20, 2015

Administrivia: contact detailsAdministrivia: contact details• Contact Information:

Email: [email protected] Phone: 630-983-1221 (10:00 am - 11:00 pm) except just

before classes Office: CDM 432 Office Hours: Monday, 4:00-5:30


May 20, 2015

Administrivia: reading materialsAdministrivia: reading materials• Course home page: http://condor.depaul.edu/dmumaugh/JDPcontains

reading assignments, lectures, homework, pointers to API documentation, more reading material, sample source code

• Textbooks Core Java, Volume I – Fundamentals, Eighth Edition, Cay S.

Horstmann and Gary Cornell, ISBN: 978-0132354769 • Other books you may want to read

Core Java, Volume II – Advanced Features, Eighth Edition, Cay S. Horstmann and Gary Cornell, ISBN:978-0132354790

Java Network Programming, Harold, Elliotte Rusty, O'Reilly, 3rd edition, 2004, ISBN 0-596-00217-3

Fundamental Networking in Java, Pitt, Esmond, Springer, 2006, ISBN 1-84628-030-3


http://condor.depaul.edu/dmumaugh/JDP

http://condor.depaul.edu/~dmumaugh/JDP

May 20, 2015

Thought for the DayThought for the Day

(In 1970) The original Internet (called the ARPANet) was intended to be used to provide distributed and collaborative systems. It was actually used primarily

for sending email.


May 20, 2015

This LectureThis Lecture• Topic: Network programming• Reading:

Harold: Chapters 2-3, 9 and 10; pp. 525-542 Pitt: Chapters 1-3, 13 Core Java, Vol. 2: pp. 1-64, 169-217 Core Java, Vol. 1: pp. 516-539 Sun's [really Oracle] API javadoc documentation on the

Socket and ServerSocket classes. Technology Briefs (see class page for URL)

TCP/IP HTTP

Article on “Reading Data from the Internet”


May 20, 2015

Java Networking Java Networking


May 20, 2015

Introduction to NetworkingIntroduction to Networking• What is a network? Why do we care?

Need to understand what we are working with.• Terms

LAN WAN Internet


May 20, 2015

NetworksNetworks


May 20, 2015

NetworksNetworks


May 20, 2015

Introduction to NetworkingIntroduction to Networking• What is a hub? • A switch? • A router?• A WAP?


May 20, 2015

NetworksNetworks

RouterFirewall

RouterFirewall


May 20, 2015

The Internet and network programmingThe Internet and network programmingWhat is the Internet?

Hosts Interfaces Routers LANS Gateways Protocols SOFTWARE


May 20, 2015

Network Programming Network Programming


May 20, 2015

The Notion of an Internet ProtocolThe Notion of an Internet Protocol• How is it possible to send bits across incompatible LANs

and WANs?• Solution: protocol software running on each host and router

smooths out the differences between the different networks.

• Implements an internet protocol (i.e., set of rules) that governs how hosts and routers should cooperate when they transfer data from network to network.• TCP/IP is the protocol for the global IP Internet.


May 20, 2015

What Does an Internet Protocol Do?What Does an Internet Protocol Do?1. Provides a naming scheme

An internet protocol defines a uniform format for host addresses.

Each host (and router) is assigned at least one of these internet addresses that uniquely identifies it.

2. Provides a delivery mechanism An internet protocol defines a standard transfer unit

(packet) Packet consists of header and payload

Header: contains info such as packet size, source and destination addresses.

Payload: contains data bits sent from source host.


May 20, 2015

Global IP InternetGlobal IP Internet• Most famous example of an internet.• Based on the TCP/IP protocol family

Level 3 - IP (Internet protocol) : Provides basic naming scheme and unreliable delivery

capability of packets (datagrams) from host-to-host. Level 4 - UDP (User Datagram Protocol)

Uses IP to provide unreliable datagram delivery from process-to-process.

Level 4 - TCP (Transmission Control Protocol) Uses IP to provide reliable byte streams from process-

to-process over connections.• Accessed via a mix of Java file I/O and functions from the

sockets interface.


May 20, 2015

Transferring Data Over an internetTransferring Data Over an internet

(1)

(2)

(3)

LAN2

FH2

protocolsoftware

client

LAN1adapter

Host A

data

data PH FH1

data PH

data PH FH2

LAN1

data

data PHdata PH

(4) (5)

(6)

(7)

(8)

internet packet

LAN2 frame

protocolsoftware

LAN1adapter

LAN2adapter

Router

FH1

LAN1 frame

data PHprotocolsoftware

server

LAN2adapter

Host B


May 20, 2015

Introduction to Network ProgrammingIntroduction to Network Programming• What is IP? • What is a firewall?• A proxy? • Network Address Translation?• What is an RFC?

Request For Comment


May 20, 2015

Introduction to Network ProgrammingIntroduction to Network Programming• What is a lossy network? • What is an unordered network? TCP versus UDP• What is TCP? What does it provide?

TCP (Transmission Control Protocol) guarantees that all segments will arrive at the destination and in the right order. (It makes no guarantees about how long it will take.)

• What is UDP? Why use it?

UDP (User Datagram Protocol) makes no such guarantees. So UDP is mostly like IP, but with the extension that it gets data from source process to destination process. UDP is used where performance is needed and occasional loss of data is not critical. E.g. audio and video streaming, short protocols such as DNS, name services, etc.

UPD is a faster protocol. It has less overhead. But the trade off is loss and unordered.


May 20, 2015

Introduction to Network ProgrammingIntroduction to Network ProgrammingQuestions to answer:• What is the client-server model? • What is the difference between the client and the server? • What are example client/server pairs? • Are there other models?


May 20, 2015

Client-Server ModelClient-Server Model

Clientprocess

Serverprocess

1. Client sends request

2. Server handlesrequest

3. Server sends response4. Client handles

response

Resource

• Every network application is based on the client-server model: A server process and one or more client processes Server manages some resource. Server provides service by manipulating resource for clients.

Note: clients and servers are processes running on hosts (can be the same or different hosts).


May 20, 2015

ClientsClients• Examples of client programs

Web browsers, ftp, telnet, ssh• How does a client find the server?

The IP address in the server socket address identifies the host (more precisely, an adapter on the host)

The (well-known) port in the server socket address identifies the service, and thus implicitly identifies the server process that performs that service.

Examples of well known ports Port 7: Echo server Port 23: Telnet server Port 25: Mail server Port 80: Web server


May 20, 2015

ServersServers• Servers are long-running processes (daemons).

Created at boot-time (typically) by the init process (process 1)

Run continuously until the machine is turned off.• Each server waits for requests to arrive on a well-known

port associated with a particular service. Port 7: echo server Port 23: telnet server Port 25: mail server Port 80: HTTP server

• A machine that runs a server process is also often referred to as a “server.”


May 20, 2015

Server ExamplesServer Examples• Web server (port 80)

Resource: files/compute cycles (CGI programs) Service: retrieves files and runs CGI programs on behalf of the client

• FTP server (20, 21) Resource: files Service: stores and retrieve files

• Telnet server (23) Resource: terminal Service: proxies a terminal on the server machine

• Mail server (25) Resource: email “spool” file Service: stores mail messages in spool file

See /etc/services for a comprehensive list of the services available on a Linux machine.


May 20, 2015

A ProgrammerA Programmer’’s View of the Internets View of the Internet1. Hosts are mapped to a set of 32-bit IP addresses.

128.2.203.179

2. The set of IP addresses is mapped to a set of identifiers called Internet domain names. [A host name]. 128.2.203.179 is mapped to www.cs.cmu.edu How do hostnames get matched to IP addresses?

What is /etc/hosts? What is a DNS?

3. A process on one Internet host can communicate with a process on another Internet host over a connection. What is special about addresses 127.0.0.*?

1.Each host has a locally defined domain name localhost which always maps to the loopback address 127.0.0.1

192.168.*.*? (See notes page for answer).


May 20, 2015

Internet ConnectionsInternet Connections• Clients and servers communicate by sending streams of bytes over

connections: Point-to-point, full-duplex (2-way communication), and reliable.

• A socket is an endpoint of a connection Socket address is an IPaddress:port pair

• A port is a 16-bit integer that identifies a process: Ephemeral port: Assigned automatically on client when client

makes a connection request Well-known port: Associated with some service provided by a

server (e.g., port 80 is associated with Web servers)• A connection is uniquely identified by the socket addresses of its

endpoints (socket pair) (cliaddr:cliport, servaddr:servport)


May 20, 2015

Using Ports to Identify ServicesUsing Ports to Identify Services

Web server(port 80)

Client host

Server host 128.2.194.242

Echo server(port 7)

Service request for128.2.194.242:80

(i.e., the Web server)

Web server(port 80)

Echo server(port 7)

Service request for128.2.194.242:7

(i.e., the echo server)

Kernel

Kernel

Client

Client


May 20, 2015

Introduction to Network ProgrammingIntroduction to Network Programming• Are there other models?

Peer-to-peer [Napster] Three tiered systems Service Oriented Architecture CORBA Distributed Services


May 20, 2015

Domain Naming System (DNS)Domain Naming System (DNS)• The Internet maintains a mapping between IP addresses and

domain names in a huge worldwide distributed database called DNS.

mil edu gov com

cmu berkeleymit

cs ece

unnamed root

amazon

www208.216.181.15

First-level domain names

Second-level domain names

Third-level domain names


May 20, 2015

Querying DNS from the Command LineQuerying DNS from the Command Line• Domain Information Groper (dig) provides a scriptable

command line interface to DNS.

linux> dig +short kittyhawk.cmcl.cs.cmu.edu 128.2.194.242 linux> dig +short -x 128.2.194.242 KITTYHAWK.CMCL.CS.CMU.EDU. linux> dig +short aol.com 205.188.145.215 205.188.160.121 64.12.149.24 64.12.187.25 linux> dig +short -x 64.12.187.25 aol-v5.websys.aol.com.


May 20, 2015

The Eight Fallacies of Distributed ComputingThe Eight Fallacies of Distributed ComputingEssentially everyone, when they first build a distributed application, makes

the following eight assumptions. All prove to be false in the long run and all cause big trouble and painful learning experiences.

1. The network is reliable

2. Latency is zero

3. Bandwidth is infinite

4. The network is secure

5. Topology doesn't change

6. There is one administrator

7. Transport cost is zero

8. The network is homogeneous


May 20, 2015

Network Programming Network Programming


May 20, 2015

Introduction to Network ProgrammingIntroduction to Network ProgrammingProtocols

• What is FTP, Telnet, and SMTP?

• What is NNTP?

• What is HTTP?

• Why is Telnet useful (other than for command line interfaces)?

Markup Languages

• What is SGML?

• HTML?

• XML?


May 20, 2015

TopicsTopics• Sockets interface• Writing clients and servers

How to use the sockets interface to establish Internet connections between clients and servers

How to copy data from one host to another over an Internet connection.


May 20, 2015

Sockets InterfaceSockets InterfaceCreated in the early 80’s as part of the original Berkeley

distribution of Unix that contained an early version of the Internet protocols.

• Provides a user-level interface to the network.• Underlying basis for all Internet applications.• Based on client/server programming model.

The key to network programming is understanding the answers to the following questions:

• A socket?• What is a port?

How do we know who to talk to? What is a well-known port?


May 20, 2015

SocketsSockets• Sockets as an abstraction provide a conduit through which a

process can send data out onto a network to another process. (Both processes could be on the same machine.)

• Sockets can be used with both the TCP and the UDP transport layer protocols.

• Remember that TCP and UDP sockets need IP addresses and port numbers.

• Conceptually this is all that is needed to specify a socket, although the details vary somewhat depending on the programming language and environment used.


May 20, 2015

SocketsSockets• What is a socket?

To the kernel, a socket is an endpoint of communication. To an application, a socket is a file descriptor that lets the

application read/write from/to the network. Remember: All I/O devices, including networks, are

generally modeled as files.• Clients and servers communicate with each other by

reading from, and writing to, socket descriptors.• The main distinction between regular file I/O and socket I/O

is how the application “opens” the socket descriptors.


May 20, 2015

The Java Socket ClassThe Java Socket ClassSockets can1. Connect to a remote machine2. Send data3. Receive data4. Close a connection5. Bind to a port6. Listen for incoming connection7. Accept connections from remote machines on a bound

port


May 20, 2015

The Java Socket ClassThe Java Socket ClassThe Socket class supports the

1. Connect to a remote machine [socket = new Socket(…)]

2. Send data [socket.write()]3. Receive data [socket.read()]4. Close a connection [socket.close()]

What is the kind of data that can be transmitted?1. What is a byte stream?

Normally a socket is encapsulated in a InputStream class or a Reader class. [More next lecture].


May 20, 2015

The Java ServerSocket ClassThe Java ServerSocket ClassThe ServerSocket class additionally supports the

5. Bind to a port [server_socket.bind()]

6. Listen for incoming connection [server_socket.listen()]

7. Accept connections from remote machines on a bound port [server_socket.accept()]


May 20, 2015

Network ProgrammingNetwork Programming• General theory

Create and/or open a socket Convert a socket to a standard Java I/O class

Input stream Output stream

Use standard Java I/O for all operations• Works for "normal" TCP connections


May 20, 2015

BasicsBasics• Create a new socket with the Socket() constructor

Provide a host name and port.• Socket attempts to contact remote host• After connection exchange data

Connection is full-duplex• After interactions are done close the connection• Normally a socket is encapsulated in an InputStream

class and possibly a Reader class, and an OutputStream class and possibly a Writer class. [More next lecture].


May 20, 2015

AddressesAddresses• Connect to a remote machine• Must have fully qualified domain name

Can have short form • How are names known

Local files may have name: /etc/host System may have an NIS database (Solaris) Otherwise you must a priori know the name

• Must have a port number How are port numbers found?


May 20, 2015

Well Known PortsWell Known Ports# Network services, Internet style

# WELL KNOWN PORT NUMBERS

daytime 13/tcp

daytime 13/udp

qotd 17/tcp quote #Quote of the Day

qotd 17/udp quote #Quote of the Day

ftp 21/tcp #File Transfer [Control]

ssh 22/tcp #Secure Shell Login

telnet 23/tcp

smtp 25/tcp mail #Simple Mail Transfer

http 80/tcp #World Wide Web HTTP

nntp 119/tcp usenet #Network News Transfer Protocolhttps 443/tcp #Secure World Wide Web HTTPsyslog 514/udp


May 20, 2015

TelnetTelnetUsing telnet to experiment• The telnet program can be used to debug a server• Examples:

telnet condor.depaul.edu 23 telnet condor.depaul.edu 80

• You can try out being a client process of many protocols that use TCP by using telnet.

telnet condor.depaul.edu 80• But what does a protocol say about the interchange

between client and server? • See next slide for an example of a transaction.


telnet://condor.depaul.edu:23/

telnet://condor.depaul.edu:80/

May 20, 2015

Using Telnet to Simulate a HTTP SessionUsing Telnet to Simulate a HTTP Session$ telnet condor.depaul.edu 80GET /~dmumaugh/index.html HTTP/1.1Host: condor.depaul.edu<blank line>HTTP/1.1 200 OKDate: Wed, 02 Apr 2003 20:35:34 GMTServer: Apache/2.0.39 (Unix) PHP/4.2.1Last-Modified: Wed, 02 Apr 2003 05:06:49 GMTETag: "29261-103d-e21d0c40"Accept-Ranges: bytesContent-Length: 4157Content-Type: text/html; charset=ISO-8859-1<blank line>Blah…Blah…Blah…


May 20, 2015

ExamplesExamples


Network I/ONetwork I/O• Remember: once a socket is open we have two major

operations: read() write()

• We may also convert a socket into a byte stream using getInputStream() getOutputStream()

• We can then “wrap” a byte stream into Reader and Writer classes and use more powerful methods: readLine() – for input streams println() – for output streams

• We will discuss more on network I/O next time

May 20, 2015 JDP Networking Part I 49 of 67

Network I/ONetwork I/OSocket s = new Socket( host, port);• Consider the method:

String BufferedReader.readLine ()• And:

BufferedReader reader = new BufferedReader (new InputStreamReader

(s.getInputStream(), "UTF-8")); String request = reader.readLine ();• Consider the method:

PrintWriter.println (String msg)• And: PrintWriter writer = new PrintWriter (new OutputStreamWriter (s.getOutputStream(), "UTF-8")); writer.println ("GET /index.html HTTP/1.1"); writer.flush ();

May 20, 2015 JDP Networking Part I 50 of 67

May 20, 2015

ExamplesExamples• Examples

Port Scanner on arbitrary host Time of Day Client Second Time of Day Client Port Scanner on current host Reverse Client Reverse Server Reverse Server Multi-threaded Serialize Client Serialize Server


May 20, 2015

Example: Port ScannerExample: Port ScannerExamine code for the simple port scanner

http://condor.depaul.edu/dmumaugh/JDP/examples/lect01/PortProbe.java

• This will scan the ports on the specified host to see what are available.

Note this is considered to be an unfriendly act by most network administrators and can have unpleasant consequences.


http://condor.depaul.edu/dmumaugh/JDP/examples/lect01/PortProbe.java.html



May 20, 2015

Example: Connecting to a Time ServerExample: Connecting to a Time ServerWhat is a time server?• Let’s look at how to talk with one

http://condor.depaul.edu/dmumaugh/JDP/examples/lect01/DayTime.java

• Another version (slightly different code) http://condor.depaul.edu/dmumaugh/JDP/examples/lect0

1/DaytimeClient.java


http://condor.depaul.edu/dmumaugh/JDP/examples/lect01/DayTime.java.html

http://condor.depaul.edu/dmumaugh/JDP/examples/lect01/DayTime.java.html

http://condor.depaul.edu/dmumaugh/JDP/examples/lect01/DaytimeClient.java.html

http://condor.depaul.edu/dmumaugh/JDP/examples/lect01/DaytimeClient.java.html

May 20, 2015

Servers ClassServers ClassServerSocket class• Listens on a designated port for a connection• Accepts a connection

Uses getInputStream() and getOutputStream()• Communicates with client• Closes connection


May 20, 2015

Servers ClassServers ClassOnce a connection is made client and server must

communicate• Protocol• Traditional protocol by the server is

Message number or type Message text Additional data


May 20, 2015

Example: A Port ScannerExample: A Port ScannerCheck to see what ports on the local host are in use• Connects as a server locally

http://condor.depaul.edu/dmumaugh/JDP/examples/lect01/LocalPortScanner.java


http://condor.depaul.edu/dmumaugh/JDP/examples/lect01/LocalPortScanner.java.html

http://condor.depaul.edu/dmumaugh/JDP/examples/lect01/LocalPortScanner.java.html

May 20, 2015

Palindrome DetectorPalindrome Detector• Reverse Client

http://condor.depaul.edu/dmumaugh/JDP/examples/lect01/ReverseClient.java

• Reverse Server http://condor.depaul.edu/dmumaugh/JDP/examples/lect0

1/ReverseServer.java• Reverse Server Multi-threaded

http://condor.depaul.edu/dmumaugh/JDP/examples/lect01/ReverseServerMT.java


http://condor.depaul.edu/dmumaugh/JDP/examples/lect01/ReverseClient.java.html

http://condor.depaul.edu/dmumaugh/JDP/examples/lect01/ReverseClient.java.html

http://condor.depaul.edu/dmumaugh/JDP/examples/lect01/ReverseServer.java.html

http://condor.depaul.edu/dmumaugh/JDP/examples/lect01/ReverseServer.java.html

http://condor.depaul.edu/dmumaugh/JDP/examples/lect01/ReverseServerMT.java.html

http://condor.depaul.edu/dmumaugh/JDP/examples/lect01/ReverseServerMT.java.html

May 20, 2015

SerializationSerialization


May 20, 2015

Object SerializationObject SerializationTo serialize an object:

OutputStream out = ...;

MyObject anObject = ...;

ObjectOutputStream oOut = new ObjectOutputStream (out);

oOut.writeObject (anObject);

To unserialize an object:

InputStream in = ...;

ObjectInputStream oIn = new ObjectInputStream(in);

MyObject anObject = (MyObject)(oIn.readObject());


May 20, 2015

Object SerializationObject SerializationObject output streams provide:

writeObject (Object); writeInt (int); writeBoolean (boolean); ...

Object input streams provide:

Object readObject (); int readInt (); boolean readBoolean (); ...

By some `magical process', data is sent down the line.


May 20, 2015

Object SerializationObject Serialization• Not all objects can be serialized. Why not?

• Objects which can be serialized are called serializable.

• Serializable objects are instances of classes:

class MyClass implements Serializable { ... } • Note that the Serializable interface is empty! This is a really

bad hack!

• Internally the Serializable interface provides version control.

• Many container classes are already marked as Serializable. See note.


May 20, 2015

Object SerializationObject SerializationWhat happens if we write:

class Unserializable { stuff you can't send across the network}class OhDear implements Serializable { Unserializable stupid = new Unserializable();}

then try:

Serializable oh = new OhDear (); oo.writeObject (oh);

Not one of the best bits of Java!


May 20, 2015

Object SerializationObject Serialization Consider class Example implements Serializable { String sendMe = "hello"; transient String dontSendMe = "world"; } At the sender:

Example test = new Example (); test.sendMe = "fred"; test.dontSendMe = "wilma"; oOut.writeObject (test); What gets printed by the receiver:

Example copy = (Example)(oIn.readObject ()); System.out.println (copy.sendMe); System.out.println (copy.dontSendMe);


May 20, 2015

Object SerializationObject SerializationWhat are transient fields good for?• Suppress special information

Passwords• Hold process dependent information

Idents (PID) File names


May 20, 2015

Object SerializationObject SerializationUnder the hood, the objects are converted into a stream of bytes.

For most classes, this is easy:

• To serialize ints, booleans, etc. just send the binary data (doubles are a bit trickier but not much).

• To serialize an object, send the name of the class, then recursively serialize all the fields.

• Other metadata is written as well: Inheritance tree, version ID, number of fields, description of all fields, etc.; enough to recreate the object exactly.

For example, what is sent when:

class Foo { String a = "AString"; int b = 42; }

class Bar { Foo c = new Foo (); }

oOut.writeObject (new Foo ());

oOut.writeObject (new Bar ());


May 20, 2015

Serialize ExampleSerialize Example• Serialize Server

http://condor.depaul.edu/dmumaugh/JDP/examples/lect01/SerialS.java

• Serialize Client http://condor.depaul.edu/dmumaugh/JDP/examples/lect0

1/SerialC.java


http://condor.depaul.edu/dmumaugh/JDP/examples/lect01/SerialS.java.html

http://condor.depaul.edu/dmumaugh/JDP/examples/lect01/SerialS.java.html

http://condor.depaul.edu/dmumaugh/JDP/examples/lect01/SerialC.java.html

http://condor.depaul.edu/dmumaugh/JDP/examples/lect01/SerialC.java.html

May 20, 2015

Next TimeNext TimeTopic: I/O streams, Applets, More Networking: UDP and Multi-

cast

Reading: Core Java, Vol. 2: pp. 1-64, 169-217 Sun(Oracles)'s API Javadoc Java Streams Basics


May 20, 2015 Networking Part I May 20, 2015 JDP Networking Part I1 of 67.

Documents

java networking jdp

networksjdp networking

fundamental networking

network programming

java network programming

readcore java

network programmingreading

network programmingwhat