Mint706.Part.7.Smtp.2014.Pdf0s

8/9/2019 Mint706.Part.7.Smtp.2014.Pdf0s

1/43

!2005-2014

(portions) Paul Lu

1

SMTP Overview

Paul Lu

!

2005-2014 (portions)with material from Indrajit Ray and various RFCs


2/43

!2005-2014

(portions) Paul Lu

2

Goals

Understand a common and important

Internet application: Email using SMTP

Understand theprinciples and

concepts, not necessarily the gory

details of SMTP, ESMTP, MIME, etc.


3/43

!2005-2014

(portions) Paul Lu

3

Relevant RFCs

Original SMTP (821): Postel, J., "Simple Mail Transfer Protocol", STD 10, RFC

821, USC/Information Sciences Institute, August 1982.

Original Mail headers (822): Crocker, D., "Standard for the Format of ARPA

Internet Text Messages", STD 11, RFC 822, UDEL, August 1982.

Original MIME (1521): Borenstein, N., and N. Freed, "Multipurpose Internet

Mail Extensions", RFC 1521, Bellcore, Innosoft, September 1993.

Moore, K., "Representation of Non-ASCII Text in Internet Message Headers",

RFC 1522, University of Tennessee, September 1993.

RFC 1123, published in 1989. A few corrections to RFC 821.

RFC 1425, published in 1983. Introduced the EHLO verb.

RFC 1651, published in 1994. Replaced RFC 1425.

RFC 1869 (ESMTP), published in 1995. Replaced RFC 1651.

RFC 2821, published in 2001, code-named Klensin. Replaced 821/1123/1869.

Others: 3501 (IMAP)


4/43

!2005-2014

(portions) Paul Lu

4

Another view of RFCs

RFC 821 (Simple Mail Transfer Protocol) [Obsoleted by RFC 2821]

RFC 822 (Internet Mail Header Format) [Obsoleted by RFC 2822]

RFC 1123 (Internet Host Requirements)

RFC 1869 (SMTP Service Extensions)

RFC 1891 (SMTP Delivery Status Notifications)

RFC 1892 (Multipart/Report)

RFC 1893 (Mail System Status Codes)

RFC 1894 (Delivery Status Notifications)

RFC 1985 (SMTP Service Extension for Remote Message Queue Starting)

RFC 2034 (SMTP Service Extension for Returning Enhanced Error Codes)

RFC 2045 (MIME)

RFC 2476 (Message Submission)

RFC 2554 (SMTP Service Extension for Authentication)

RFC 2821 (Simple Mail Transfer Protocol)

RFC 2822 (Internet Mail Header Format)


5/43

!2005-2014

(portions) Paul Lu

5

Internet e-mail system

User AgentsMail Server

Mail Server

sendmail, procmail etc.

Mail Server

User Agents

User Agents

User Agents

Outlook, Eudora, Pine etc

User Agents

User Agents

SMTP POP3 / SMTP

IMAP / SMTP

HTTP / SMTP


6/43

!2005-2014

(portions) Paul Lu

6

Concepts

SMTP is for host-to-host mail transfer

End users access their mailbox with tools

(e.g., mutt, pine) using other protocols (e.g.,IMAP, IMAPS, POP)

Zmailer and Sendmail are popularimplementations of SMTP

Mail transfer agent (MTA)

SMTP/MIME is the basis of contemporaryemail


7/43


8/43

!2005-2014

(portions) Paul Lu

8

Concepts: Mail User Agent

(MUA)

A client program to send and receive

email

One could refer to a mail retrieval agent

(MRA) as one part of the MUA


9/43

!2005-2014

(portions) Paul Lu

9

Concepts: Mail Transfer

Agent (MTA)

A server program to transfer mail from one

machine to another (I.e., host to host)

SMTP is the de factoMTA protocol andstandard

Receiver of the email needs to have an

account on the target/destination MTAs host

aka receiving MTA

aka the maildrop


10/43

!2005-2014

(portions) Paul Lu

10

User Agents

a.k.a. mail reader

Composing, editing, reading mailmessages

e.g., Eudora, Outlook, elm, Netscape

Messenger

Outgoing, incoming messages storedon server


11/43

!2005-2014

(portions) Paul Lu

11

Concepts: Component-based

software (the Unix/Internet way)

What we know as email is implemented as acollection of

Protocols and standards, as defined by RFCs Different software components (I.e., MTA, MUA,

MRA)

Different implementations of the components (e.g.,sendmail, zmailer, mutt, pine, Outlook)

Protocols and standards are what make thecomponents inter-operable

State and data are often distributed


12/43

!2005-2014

(portions) Paul Lu

12

Mail Servers mailboxcontains incoming

messages for user

messagequeueof outgoing

(to be sent) mail messages SMTP protocolbetween mail

servers to send emailmessages

Sender side mail serveris the client

Receiving side mailserver is the server

mailserver

useragent

useragent

user

agent

mail

server

useragent

useragent

mailserver

useragent

SMTP

SMTP

SMTP

user mailbox

outgoingmessage queue


13/43

!2005-2014

(portions) Paul Lu

13

Alice sends message to Bob

Alice composes

email message

Provides Bobs

email address to

her user-agent

Alices mail

server

Bobs mail

server

Alices user-agent

uses SMTP client connection

to push message to a SMTP

server on Alices mail server

Alices mail server

queues up message

for a suitable time

to deliver

Alices email server creates

a TCP based SMTPclient connection

to an SMTP server running on Bobs

mail server. Sends Alices email to

Bobs mail server.

Bobs mail server

queues up message

to be picked up by

Bob at a suitable time

Bob uses his

user-agent to

retrieve email

message

Bobs user-agent uses

a client POP3/IMAP/

HTTP connection to

Bobs mail server


14/43

!2005-2014

(portions) Paul Lu

14

Email message formatSMTP: protocol for exchanging

email msgs

RFC 822: standard for text

message format: header lines, e.g.,

To:

From:

Subject:

differentfrom SMTP

commands! body

the message, ASCIIcharacters only

header

body

blankline


15/43


16/43

!2005-2014

(portions) Paul Lu

16

Email headerDate: Wed, 16 Jun 2004 12:34:49 +0200

From: Marta Oliva

To: Dr. Indrajit Ray

Subject: Re: Registration to the 18th Annual IFIP WG 11.3 WC on Data and

Application Security, 2004


17/43

!2005-2014

(portions) Paul Lu

17

Email header (full)Received: from mailr3.udl.es (mailr3.udl.es [193.144.10.36])by chico.cs.colostate.edu (8.12.10/8.12.9) with ESMTP id i5GAYmvN008288

for ; Wed, 16 Jun 2004 04:34:50 -0600 (MDT)

Received: from eps.udl.es (fermat.udl.net [10.50.54.28])

by mailr3.udl.es (8.11.6/8.11.6) with ESMTP id i5GAYga31371

for ; Wed, 16 Jun 2004 12:34:42 +0200

Received: from eps.udl.es by eps.udl.es (8.8.8+Sun/SMI-SVR4)id MAA22736; Wed, 16 Jun 2004 12:34:40 +0200 (MET DST)

Message-ID:

Date: Wed, 16 Jun 2004 12:34:49 +0200

From: Marta Oliva

User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.4) Gecko/20030624 Netscape/7.1 (ax)

X-Accept-Language: en-us, en

MIME-Version: 1.0

To: "Dr. Indrajit Ray" Subject: Re: Registration to the 18th Annual IFIP WG 11.3 WC on Data and Application

Security, 2004

References:

In-Reply-To:

Content-Type: text/plain; charset=us-ascii; format=flowed

Content-Transfer-Encoding: 7bit


18/43

!2005-2014

(portions) Paul Lu

18

Displaying email headers

You can instruct most email programs todisplay the full header

In Eudora: Click the Blah Blah Blahbutton. In Netscape: Select: View->Headers->All

In Outlook: Select: View->Options

In Mutt and Pine: Type H. (Requires the enable-full-header-cmdfeature.)

In WebMail: Click the Optionsbutton, then select"Show message headers in body of message" andclick OK.


19/43

!2005-2014

(portions) Paul Lu

19

Generation of email headers (1)

salieri.cs.colostate.edu chico.cs.colostate.edu mailhost.isse.gmu.edu pinky.isse.gmu.edu

From: [email protected] (Alice The Great)

To: [email protected]

Date: Fri, 18 Jun 2004 10:22:55 -0600 (MDT)X-Mailer: Pine v2.32

Subject: Conference call today?

Header generated by Alices

user agent and handed off tochico.cs.colostate.edu


20/43

!2005-2014

(portions) Paul Lu

20


salieri.cs.colostate.edu

chico.cs.colostate.edu mailhost.isse.gmu.edu pinky.isse.gmu.edu

Received: from salieri.cs.colostate.edu (salieri.cs.colostate.edu [129.82.45.76] by

chico.cs.colostate.edu (8.12.10/8.12.9) id i5IGMtv0004345

From: [email protected] (Alice The Great)To: [email protected]

Date: Fri, 18 Jun 2004 10:22:55 -0600 (MDT)

Message-ID:

X-Mailer: Pine v2.32Subject: Conference call today?

Header fields added by chico.cs.colostate.edu as it transmits the

message to mailhost.isse.gmu.edu


21/43

!2005-2014

(portions) Paul Lu

21


Received: from chico.cs.colostate.edu (chico.cs.colostate.edu [129.82.45.30]) by

mailhost.isse.gmu.edu (8.8.5/8.7.2) with ESMTP id LAA20869 for

; Fri, 18 Jun 2004 12:24:24 -0400 (EDT)Received: from salieri.cs.colostate.edu (salieri.cs.colostate.edu [129.82.45.76] by



To: [email protected]

Date: Fri, 18 Jun 2004 10:22:55 -0600 (MDT)Message-ID:

X-Mailer: Pine v2.32


salieri.cs.colostate.educhico.cs.colostate.edu mailhost.isse.gmu.edu pinky.isse.gmu.edu

Added by mailhost.isse.gmu.edu after it has received and finished

processing the email for Bob to pickup


22/43

!2005-2014

(portions) Paul Lu

22

Examining email headers

The most important header field for email

tracing purposes is the Receivedheader

line(s)

Syntax

Received: from ? by ? via ? with ? id ?

for ? ; date-time

where from, by, via, with, id, and forare token with values within a single header value

Not all tokens will have values all the times


23/43

!2005-2014

(portions) Paul Lu

23

Examining Received header

We are interested in the from and bytokens

in the Received header field

from name(dns-name [ip-address])Received:fromchico.cs.colostate.edu (chico.cs.colostate.edu [129.82.45.30])

This piece of mail was received from a machine calling itself (name)

chico.cs.colostate.edu

which is really named (dns-name) chico.cs.colostate.eduand has the IP address ([ip-address]) 129.82.45.30

Single most important piece of information for tracing email


24/43

!2005-2014

(portions) Paul Lu

24

Examining Received headers

(2)

bymailhost.isse.gmu.edu (8.8.5/8.7.2)

by receiving-host-name(software version number)

The machine that received the email was (receiving-host-name) mailhost.isse.gmu.eduIts running a software with version (software version number) 8.8.5/8.7.2

by default the software is sendmail


25/43

!2005-2014

(portions) Paul Lu

25

Examining Received headers (3)

withESMTP IDLAA20869

with(protocol) ID(server-assigned-id)

The machine that received the mail was running (protocol) ESMTP

The machine assigned the identifier number (server-assigned-id) LAA20869

the system administrator needs to have this ID number to look up the message

in the machines log files no other use for this ID number


26/43

!2005-2014

(portions) Paul Lu

26


for;

for(


27/43

!2005-2014

(portions) Paul Lu

27


Every time an email moves through a new mailtransfer agent (a mail server or a mail relay), a newReceived header line is added to the beginningof the

headers list This means that as we read the Received headers in an

email message from top to bottom, we are gradually movingcloser to the machine/person that sent the email.

Received: from chico.cs.colostate.edu (chico.cs.colostate.edu [129.82.45.30]) by


; Fri, 18 Jun 2004 12:24:24 -0400 (EDT)



From: [email protected] (Alice The Great)To: [email protected]: Fri, 18 Jun 2004 10:22:55 -0600 (MDT)

Message-ID: X-Mailer: Loris v2.32Subject: Conference call today?

closest to Bob

one hop away


28/43

!2005-2014

(portions) Paul Lu

28

Examining other portions of

email header

From: [email protected] (Alice TheGreat)

This mail was sent by [email protected],who gives her real name as Alice The Great

To: [email protected] The mail was addressed to [email protected]

Date: Fri, 18 Jun 2004 10:22:55 -0600 (MDT) The email was composed on Friday 18 June 2004

at 10:22:55 Mountain Daylight Time which is 6hours behind GMT


29/43

!2005-2014

(portions) Paul Lu

29

Examining other portions of email

header Message-ID:

The email was provided with this number bychico.cs.colostate.edu to identify it.

This ID is different from the ESMTP / SMTP ID numbers in theReceived: headers

It is attached to the message for life

Sometimes this ID may provide valuable clue, most of the time

it is un-intelligible information about senders email address

information about the machine on which the email was composed

Email program used to compose email


30/43

!2005-2014

(portions) Paul Lu

30

Examining other portions of email

header X-Mailer: Pine v2.32

The message was sent using a program called Pine, version2.32

Subject: Conference Call Today? Subject matter for the email

There can be many other header fields in the email header,

like Bcc, Cc etc. For the most part these do not contribute

for email tracing purposes. For complete list of header

fields please see RFC 2076


31/43


32/43

!2005-2014

(portions) Paul Lu

32

Transcript of SMTPconnectionbetween

Alice's mail server and Bob's

Client SMTP running on sendingmail server host, establishesTCP connection on port 25 toserver SMTP running on

receiving email server host. TCP guarantees error-free

delivery of email message

ASCII texts prefaced with C:/S:are exactly the lines the client/server send

Client issued 5 commands.Server replied to eachcommand with each replyaccompanied by a reply-code

S: 220 mailhost.isse.gmu.edu ESMTP Sendmail8.8.5/1.4/8.7.2/1.13; Fri, 18 Jun 2004 12:24:24 -0400 (EDT)

C: HELOmailhost.isse.gmu.edu

S: 250 Hello chico.cs.colostate.edu, pleased to meet you

C: MAIL FROM:

S: 250 [email protected] Sender ok

C: RCPT TO: [email protected]

S: 250 [email protected] Recipient ok

C: DATA

S: 354 Enter mail, end with . on a line by itself

C: Received: from salieri.cs.colostate.edu (salieri.cs.colostate.edu[129.82.45.76] by .

C:

C: Subject: Conference Call Today?

C: Are we having the conference call today?

C: .

S: 250 LAA20869 Message accepted for delivery

C: QUIT

S: 221 hamburger.edu closing connection


33/43

!2005-2014

(portions) Paul Lu

33

Understanding SMTP commands

HELO

Identifies the sending machine

The sender can lie Nothing, in principle, prevents

chico.cs.colostate.edu from saying HELOabc.freebie.com

Receiver can find out the sending machinesreal identity, using reverse DNS lookup, forexample

Most modern email servers do this


34/43

!2005-2014

(portions) Paul Lu

34

Understanding SMTP commands MAIL FROM Initiates email processing

Address need not be the same as the senders own address

Turns into the fromaddress in the Received header

RCPT TO

Dual of MAIL FROM

Specifies the intended recipient (the one to which the email will bedelivered regardless of whatever is specified in the To: line in the

message)

One mail can be sent to multiple recipients by including multiple

RCPT TO command

Turns into the foraddress in the Received header


35/43

!2005-2014

(portions) Paul Lu

35

Understanding SMTP commands

DATA Starts the actual mail entry. Everything following it

is considered the message No restrictions on its form

Lines at the beginning of the message that startwith a single word followed by a colon isconsidered part of message header

Line consisting only of a period terminates themessage

QUIT Terminates the SMTP connection


36/43

!2005-2014

(portions) Paul Lu

36

Effect of firewalls on email

headers Introduces one extra hop in the e-mail's passage.

Firewall acts as just one more machine that forwards email

Adds Received: line for each extra hop

salieri.cs.colostate.edu chico.cs.colostate.edu mailhost.isse.gmu.edu pinky.isse.gmu.edu

firewall.cs.colostate.edu firewall.isse.gmu.edu


37/43

!2005-2014

(portions) Paul Lu

37

Effect of firewall on email

headersReceived: from firewall.isse.gmu.edu (firewall.isse.gmu.edu [129.174.142.12]) by


; Fri, 18 Jun 2004 12:24:24 -0400 (EDT)

Received: from firewall.cs.colostate.edu (firewall.cs.colostate.edu [129.82.45.35]) by

firewall.isse.gmu.edu (8.8.3/8.7.1) with ESMTP id LAA20869 for

; Fri, 18 Jun 2004 12:23:54 -0400 (EDT)Received: from chico.cs.colostate.edu (chico.cs.colostate.edu [129.82.45.30]) by

firewall.cs.colostate.edu (8.12.10/8.12.9) with ESMTP id i5IGMtv0004345 for

; Fri, 18 Jun 2004 10:23:56 -0600 (MDT)




To: [email protected]: Fri, 18 Jun 2004 10:22:55 -0600 (MDT)

Message-ID:

X-Mailer: Pine v2.32



38/43

!2005-2014

(portions) Paul Lu

38

Email relays

SMTP allows messages to be relayed toother SMTP servers towards a destination

Historically this was the way SMTP was meant tobe

Currently, only unethical spammers use SMTPrelaying to conceal the source of their messages

This way spammers hope to deflect complaints to the(innocent) relay site rather than the spammers own ISP


39/43

!2005-2014

(portions) Paul Lu

39

Email relaysReceived: from unwilling.intermediary.com (unwilling.intermediary.com [98.134.11.32]) bymailhost.isse.gmu.edu (8.8.5/8.7.2) ID 004B32 for ; Fri, 18 Jun 200416:39:50 -0400 (EDT)

Received: from galangal.org ([104.128.23.115]) by unwilling.intermediary.com (8.6.5/8.5.8) with SMTP ID

LAA12741; Fri, 18 Jun 2004 16:36:28 -0400 (EDT)

From: Anonymous Spammer

To: (recipient list suppressed)

Message-Id:

X-Mailer: Massive Annoyance

Subject: WANT TO MAKE ALOT OF MONEY???

Message originated at galangal.org, was passed from there

to unwilling.intermediary.comand from there to mailhost.isse.gmu.edu


40/43

!2005-2014

(portions) Paul Lu

40

How did that happen? (Most

likely scenario) galangal.org simply connected to the port 25 at

unwilling.intermediary.com

Told unwilling.intermediary.com to send message [email protected] RCPT TO: [email protected]

unwilling.intermediary.com handed off the email tomailhost.isse.gmu.edu in the usual manner One thing to note is that Message-ID: line was filled in not by

the sending machine but by the relayer: Message-Id:

One way to confirm relayed mail


41/43

!2005-2014

(portions) Paul Lu

41

Mail Access Protocols

Used by Email reader programs to pull

stored email messages from the mail

server to the recipients machine.

For the most part do not add anything extra

to the email header

May format the email header


42/43

!2005-2014

(portions) Paul Lu

42

POP3 Protocol Authentication/authorization

phase

client commands:

user:declare username

pass:password

server responses

+OK

-ERR

transaction phase,client:

list:list message numbers

retr:retrieve message bynumber

dele:delete

quit

C: list

S: 1 498

S: 2 912

S: .

C: retr 1

S:

S: .

C: dele 1

C: retr 2S:

S: .

C: dele 2

C: quit

S: +OK POP3 server signing off

S: +OK POP3 server ready

C: user bob

S: +OK

C: pass hungry

S: +OKuser successfully logged on


43/43

!2005-2014

(portions) Paul Lu

43

POP3 (more) and IMAP Previous example uses

download and deletemode.

Bob cannot re-read e-mail if he changes client

Download-and-keepmode: copies ofmessages on different

clients POP3 is stateless

across sessions

Keep all messages in

one place: the server

Allows user to organizemessages in folders

IMAP keeps user state

across sessions:

names of folders and

mappings betweenmessage IDs and folder

name

Mint706.Part.7.Smtp.2014.Pdf0s

Documents