CT 320: Network and System Administration Fall 2014 * Dr. Indrajit Ray Email: [email protected]@cs.colostate.edu Department of Computer.

CT 320: Network and System AdministrationFall 2014*

Dr. Indrajit RayEmail: [email protected]

Department of Computer ScienceColorado State University

Fort Collins, CO 80528, USA

* Thanks to Dr. James Walden, NKU and Russ Wakefield, CSU for contents of these slides

mailto:[email protected]

E-mail

Dr. Indrajit Ray, Computer Science Department CT 320 – Network and Systems Administration, Fall 2014

Topics

1. Anatomy of a Mail Message2. Components of an E-mail System3. SMTP4. IMAP & POP5. E-mail Addresses6. Mail Policies


Internet E-mail System

User Agents

Mail Server

Mail Server sendmail, procmail etc.

Mail Server

User Agents

User Agents

User AgentsOutlook, Eudora, Pine etc

User Agents

User Agents

SMTP

SMTP

SMTP

POP3 / SMTP

IMAP / SMTP

HTTP / SMTP


Components of a Mail System

TA

Sendmail

TA

Sendmail

UA

Eudora

UA

Outlook

UA

mutt

DA

mail.localMsgStore

AA

imapdUA

mutt


Message Store

• Communication– Receives data from MDA (mail.local, procmail)– Provides data to MAA (IMAP, POP, NFS, web)

• Types of stores– Files (all messages for a user in one file)– Directories (directory per user)– Databases


Mail Access Agents

• Older systems directly accessed mail files.• Modern systems use network– POP: Post Office Protocol• Simple download protocol for offline reading.

– IMAP: Internet Mail Access Protocol• Online and offline modes of reading.• Partial message fetch (headers, attachments,

etc.)• Message state stored on server, not client.• Multiple mailbox and multiple client support.


IMAP

• IMAP Servers– Cyrus– UW

• IMAP Features– Message store types– Authentication– Security (SSL)


Mail User Agents

• Text clients– mail– mutt– pine

• GUI clients– Eudora– Mozilla Thunderbird– MS Outlook

• Web clients– Run on remote web server.


Mail Addressing

• Relative Addresses– mcvax!uunet!ucbvax!hao!boulder!air!evi

• Absolute Addresses– user@domain

• MX Records– Mail clients use MX records, not A records.– Lowest preference # = highest priority.– Permits failover if server down.


Aliases

• Allow mail to be rerouted.– Sysadmin: files (/etc/mail/aliases), local db, NIS,

LDAP– Personal: ~/.forward

• Alias destinations– Local: address– Remote: address@domain– File: :include:pathname– Program: |pathname

• Required aliases– postmaster, abuse, root


Email Header

• Header Format– Header-name: Header-data

• Common headers– From:– To:, CC:, Reply-To:– Date:– Message-ID:– Subject:

• Multiple headers– Received: for each mail server handling message.


Body

• Separated from header by blank line.• Contains 7-bit ASCII text by default.• Any non-ASCII text must be encoded:– uuencode– MIME


Envelope

• Headers aren’t the full story– Recipient isn’t necessarily on To: or CC:– Sender isn’t necessarily given on From: header.

• Envelope specifies sender/receiver– Specified via SMTP commands.– Envelope recipient used for BCC:– Envelope recipient used by mail lists.– Envelope facilities used by spammers too.


MTAs

• Mail Transport Agents– Receive mail from MUAs.– Route mail across internet.

• MTA Protocol: SMTP• MTA Examples– sendmail– postfix– qmail


Alice sends message to Bob

Alice composes email message

Provides Bob’s email address toher user-agent

Alice’s mailserver

Bob’s mailserver

Alice’s user-agentuses SMTP client connectionto push message to a SMTPserver on Alice’s mail server

Alice’s mail serverqueues up messagefor a suitable timeto deliver

Alice’s email server createsa TCP based SMTP client connectionto an SMTP server running on Bob’smail server. Sends Alice’s email toBob’s mail server.

Bob’s mail serverqueues up messageto be picked up by Bob at a suitable time

Bob uses hisuser-agent toretrieve emailmessage

Bob’s user-agent usesa client POP3/IMAP/HTTP connection toBob’s mail server


Email header

• Every received email message will have a header

• Header lines are added by entities (email tools, user-agents, email servers) as they store and forward and email messages

• The header lines are a series of text lines– Syntax Header-Name: Header-Value– If a line starts with a “tab” character or a “space”

then that line is a continuation of previous header-value

Email (envelope) header

Date: Wed, 16 Jun 2004 12:34:49 +0200From: Marta Oliva <[email protected]>To: Dr. Indrajit Ray <[email protected]>Subject: Re: Registration to the 18th Annual IFIP WG 11.3 WC on Data and Application Security, 2004


Email header (full)


Received: from mailr3.udl.es (mailr3.udl.es [193.144.10.36])by chico.cs.colostate.edu (8.12.10/8.12.9) with ESMTP id i5GAYmvN008288for <[email protected]>; Wed, 16 Jun 2004 04:34:50 -0600 (MDT)

Received: from eps.udl.es (fermat.udl.net [10.50.54.28])by mailr3.udl.es (8.11.6/8.11.6) with ESMTP id i5GAYga31371for <[email protected]>; Wed, 16 Jun 2004 12:34:42 +0200

Received: from eps.udl.es by eps.udl.es (8.8.8+Sun/SMI-SVR4)id MAA22736; Wed, 16 Jun 2004 12:34:40 +0200 (MET DST)

Message-ID: <[email protected]>Date: Wed, 16 Jun 2004 12:34:49 +0200From: Marta Oliva <[email protected]>User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.4) Gecko/20030624 Netscape/7.1 (ax)X-Accept-Language: en-us, enMIME-Version: 1.0To: "Dr. Indrajit Ray" <[email protected]>Subject: Re: Registration to the 18th Annual IFIP WG 11.3 WC on Data and Application Security, 2004References: <[email protected]> <[email protected]>In-Reply-To: <[email protected]>Content-Type: text/plain; charset=us-ascii; format=flowedContent-Transfer-Encoding: 7bit

Displaying email headers

• You can instruct most email programs to display the full header – In Netscape: Select: View->Headers->All – In Outlook: Select: View->Options – In Pine: Type H. (Requires the enable-full-header-

cmd feature.) – In WebMail: Click the Options button, then select

"Show message headers in body of message" and click OK.



Generation of email headers (1)

salieri.cs.colostate.edu chico.cs.colostate.edu mailhost.isse.gmu.edu pinky.isse.gmu.edu

From: [email protected] (Alice The Great)To: [email protected]: Fri, 18 Jun 2004 10:22:55 -0600 (MDT)X-Mailer: Pine v2.32Subject: Conference call today?

Header generated by Alice’s user agent and handed off to chico.cs.colostate.edu



Received: from salieri.cs.colostate.edu (salieri.cs.colostate.edu [129.82.45.76] by chico.cs.colostate.edu (8.12.10/8.12.9) id i5IGMtv0004345

From: [email protected] (Alice The Great)To: [email protected]: Fri, 18 Jun 2004 10:22:55 -0600 (MDT)Message-ID: <[email protected]>X-Mailer: Pine v2.32Subject: Conference call today?

Header fields added by chico.cs.colostate.edu as it transmits the message to mailhost.isse.gmu.edu



Received: from chico.cs.colostate.edu (chico.cs.colostate.edu [129.82.45.30]) bymailhost.isse.gmu.edu (8.8.5/8.7.2) with ESMTP id LAA20869 for<[email protected]>; Fri, 18 Jun 2004 12:24:24 -0400 (EDT)




Added by mailhost.isse.gmu.edu after it has received and finished processing the email for Bob to pickup


Examining email headers

• The most important header field for email tracking purposes is the Received header line(s)

• Syntax –Received: from ? by ? via ? with ? id ? for ? ; date-time

– where from, by, via, with, id, and for are token with values within a single header value

– Not all tokens will have values all the times


Examining ‘Received’ header

• Tip – Break a single Received line into multiple lines

Received: from chico.cs.colostate.edu (chico.cs.colostate.edu [129.82.45.30]) by mailhost.isse.gmu.edu (8.8.5/8.7.2) with ESMTP id LAA20869 for <[email protected]>; Fri, 18 Jun 2004 12:24:24 -0400 (EDT)

Received: from chico.cs.colostate.edu (chico.cs.colostate.edu [129.82.45.30])by mailhost.isse.gmu.edu (8.8.5/8.7.2) with ESMTP id LAA20869for <[email protected]>;Fri, 18 Jun 2004 12:24:24 -0400 (EDT)



Examining ‘Received’ header (2)

• For tracking purposes, we are interested in the from and by tokens in the Received header field– from name (dns-name [ip-address])

Received: from chico.cs.colostate.edu (chico.cs.colostate.edu [129.82.45.30])

This piece of mail was received from a machine calling itself (name) chico.cs.colostate.eduwhich is really named (dns-name) chico.cs.colostate.eduand has the IP address ([ip-address]) 129.82.45.30

Single most important piece of information for tracing email


Examining ‘Received’ headers (3)

by mailhost.isse.gmu.edu (8.8.5/8.7.2)

by receiving-host-name (software version number)

The machine that received the email was (receiving-host-name) mailhost.isse.gmu.eduIt’s running a software with version (software version number) 8.8.5/8.7.2

by default the software is sendmail


with ESMTP ID LAA20869

with (protocol) ID (server-assigned-id)

The machine that received the mail was running (protocol) ESMTP

The machine assigned the identifier number (server-assigned-id) LAA20869

The system administrator needs to have this ID number to look up the message in the machine’s log files – no other use for this ID number



for <[email protected]>;

for (<recipient's email address>);

The email was addressed to (<recipient’s email address>) [email protected] – This header is not related to the email address provided in the To: header line

date-time

Fri, 18 Jun 2004 12:24:24 -0400 (EDT)

This mail transfer (from chico.cs.colostate.edu to mailhost.isse.gmu.edu) occurred on Friday, 18 June, 2004 at 12:24:24 Eastern Daylight Time which is 4 hours behind Greenwich Mean Time


Examining Received headers (6)• Every time an email moves through a new mail transfer

agent (a mail server or a mail relay), a new Received header line is added to the beginning of the headers list– This means that as we read the Received headers in an email

message from top to bottom, we are gradually moving closer to the machine/person that sent the email.

Received: from chico.cs.colostate.edu (chico.cs.colostate.edu [129.82.45.30]) bymailhost.isse.gmu.edu (8.8.5/8.7.2) with ESMTP id LAA20869 for<[email protected]>; Fri, 18 Jun 2004 12:24:24 -0400 (EDT)


From: [email protected] (Alice The Great)To: [email protected]: Fri, 18 Jun 2004 10:22:55 -0600 (MDT)Message-ID: <[email protected]>X-Mailer: Loris v2.32Subject: Conference call today?

closest to Bob

one hop away


Examining other portions of email header

• From: [email protected] (Alice The Great)– This mail was sent by [email protected], who

gives her real name as Alice The Great

• To: [email protected]– The mail was addressed to [email protected]

• Date: Fri, 18 Jun 2004 10:22:55 -0600 (MDT)– The email was composed on Friday 18 June 2004 at

10:22:55 Mountain Daylight Time which is 6 hours behind GMT




• Message-ID: <[email protected]>– The email was provided with this number by

chico.cs.colostate.edu to identify it. • This ID is different from the ESMTP / SMTP ID

numbers in the Received: headers• It is attached to the message for life• Sometimes this ID may provide valuable clue, most

of the time it is un-intelligible – information about sender’s email address– information about the machine on which the email was

composed– Email program used to compose email



• X-Mailer: Pine v2.32– The message was sent using a program called Pine,

version 2.32

• Subject: Conference Call Today?– Subject matter for the email

There can be many other header fields in the email header, like Bcc, Cc etc. For the most part these do not contributefor email tracing purposes. For complete list of header fields please see RFC 2076


Simple Mail Transfer Protocol (RFC 2821)• Principal application layer protocol for Internet

electronic mail. • Runs over TCP (port 25)• It is used to “push” email messages from one

mail server to another or from an user agent to a mail server

Application Layer

Physical Layer

Network Layer

TCPUDP

Application Layer

TCP UDP

Network Layer

Physical Layer

SMTP SMTP


Transcript of SMTP connection between Alice’s mail server and Bob’s

• Client SMTP running on sending mail server host, establishes TCP connection on port 25 to server SMTP running on receiving email server host.– TCP guarantees error-free

delivery of email message

• ASCII texts prefaced with C:/S: are exactly the lines the client/server send

• Client issued 5 commands. Server replied to each command with each reply accompanied by a reply-code

S: 220 mailhost.isse.gmu.edu ESMTP Sendmail 8.8.5/1.4/8.7.2/1.13; Fri, 18 Jun 2004 12:24:24 -0400 (EDT)C: HELO mailhost.isse.gmu.eduS: 250 Hello chico.cs.colostate.edu, pleased to meet youC: MAIL FROM: <[email protected]>S: 250 [email protected] … Sender okC: RCPT TO: [email protected]: 250 [email protected] … Recipient okC: DATAS: 354 Enter mail, end with “.” on a line by itselfC: Received: from salieri.cs.colostate.edu (salieri.cs.colostate.edu [129.82.45.76] by ……. C: ……C: Subject: Conference Call Today?C: Are we having the conference call today?C: .S: 250 LAA20869 Message accepted for deliveryC: QUITS: 221 hamburger.edu closing connection


SMTP Commands

HELO hostnameEHLO hostnameMAIL FROM: addrRCPT TO: addrVRFY addrEXPN addrDATAQUITRSETHELP

Understanding SMTP commands• HELO– Identifies the sending machine– The sender can lie• Nothing, in principle, prevents

chico.cs.colostate.edu from saying “HELO abc.freebie.com”• Receiver can find out the sending machine’s real

identity, using reverse DNS lookup, for example– Most modern email servers do this



Understanding SMTP commands

• MAIL FROM– Initiates email processing– Address need not be the same as the sender’s own

address– Turns into the from address in the Received header

• RCPT TO– Dual of MAIL FROM– Specifies the intended recipient (the one to which the

email will be delivered regardless of whatever is specified in the To: line in the message)

– One mail can be sent to multiple recipients by including multiple RCPT TO command

– Turns into the for address in the Received header

Understanding SMTP commands• DATA– Starts the actual mail entry. Everything following it

is considered the message– No restrictions on its form– Lines at the beginning of the message that start

with a single word followed by a colon is considered part of message header

– Line consisting only of a period terminates the message

• QUIT– Terminates the SMTP connection


POP3 / IMAP / HTTP Protocols• Used by Email reader programs to “pull” stored

email messages from the mail server to the recipient’s machine.– For the most part do not add anything extra to the

email header– May format the email header



Effect of firewalls on email headers

• Introduces one extra “hop” in the e-mail's passage.– Firewall acts as just one more machine that

forwards email– Adds Received: line for each extra hop

salieri.cs.colostate.educhico.cs.colostate.edu mailhost.isse.gmu.edu

pinky.isse.gmu.edu

firewall.cs.colostate.edu firewall.isse.gmu.edu


Effect of firewall on email headers

Received: from firewall.isse.gmu.edu (firewall.isse.gmu.edu [129.174.142.12]) bymailhost.isse.gmu.edu (8.8.5/8.7.2) with ESMTP id LAA20869 for<[email protected]>; Fri, 18 Jun 2004 12:24:24 -0400 (EDT)

Received: from firewall.cs.colostate.edu (firewall.cs.colostate.edu [129.82.45.35]) byfirewall.isse.gmu.edu (8.8.3/8.7.1) with ESMTP id LAA20869 for<[email protected]>; Fri, 18 Jun 2004 12:23:54 -0400 (EDT)

Received: from chico.cs.colostate.edu (chico.cs.colostate.edu [129.82.45.30]) byfirewall.cs.colostate.edu (8.12.10/8.12.9) with ESMTP id i5IGMtv0004345 for<[email protected]>; Fri, 18 Jun 2004 10:23:56 -0600 (MDT)




Effect of firewall on email headers

Received: from firewall.openuniversity.edu (firewall.openuniversity.edu [203.174.142.12]) by mailhost.openuniversity.edu (8.8.5/8.7.2) with ESMTP id LAA20987 for <[email protected]>; Fri, 18 Jun 2004 12:26:24 -0400 (EDT) Received: from mailfilter.newsadhost.com (mailfilter.newsadhost.com [73.82.45.30]) by

firewall.openuniversity.edu (8.8.5/8.7.2) with ESMTP id LAA20869 for<[email protected]>; Fri, 18 Jun 2004 10:24:24 -0600 (MDT)

Received: from mail.newsadhost.com (mail.newsadhost.com [73.82.45.35]) bymailfilter.newsadhost.com (8.8.3/8.7.1) with ESMTP id i5IGMtv0004387 for<[email protected]>; Fri, 18 Jun 2004 10:23:57 -0600 (MDT)

Received: from mailfilter.newsadhost.com (mailfilter.newsadhost.com [73.82.45.30]) bymail.newsadhost.com (8.12.10/8.12.9) with ESMTP id i5IGMtv0006734 for<[email protected]>; Fri, 18 Jun 2004 10:23:56 -0600 (MDT)

Received: from 127.0.0.1 (mail-131-73.eak.fdj.bestadonline.com [205.214.131.73] by mailfilter.newsadhost.com (8.12.10/8.12.9) with ESMTP id i5IGMtv0004345

From: Anonymous Spammer (Alice The Great)To: [email protected]: Fri, 18 Jun 2004 10:22:55 -0600 (MDT)Message-ID: <[email protected]>X-Mailer: Pine v2.32Subject: Want to make a lot of money?


Email relays

• SMTP allows messages to be relayed to other SMTP servers towards a destination– Historically this was the way SMTP was meant to be– Currently, only unethical spammers use SMTP

relaying to conceal the source of their messages• This way spammers hope to deflect complaints

to the (innocent) relay site rather than the spammers’ own ISP


Email relaysReceived: from unwilling.intermediary.com (unwilling.intermediary.com [98.134.11.32]) by

mailhost.isse.gmu.edu (8.8.5/8.7.2) ID 004B32 for <[email protected]>; Fri, 18 Jun 2004 16:39:50 -0400 (EDT)

Received: from galangal.org ([104.128.23.115]) by unwilling.intermediary.com (8.6.5/8.5.8) with SMTP ID LAA12741; Fri, 18 Jun 2004 16:36:28 -0400 (EDT)

From: Anonymous Spammer <[email protected]>

To: (recipient list suppressed)

Message-Id: <[email protected]>

X-Mailer: Massive Annoyance

Subject: WANT TO MAKE ALOT OF MONEY???

Message originated at galangal.org, was passed from there to unwilling.intermediary.comand from there to mailhost.isse.gmu.edu


How did that happen? (Most likely scenario)

• galangal.org simply connected to the port 25 at unwilling.intermediary.com

• Told unwilling.intermediary.com to send message to [email protected]– RCPT TO: [email protected]

• unwilling.intermediary.com handed off the email to mailhost.isse.gmu.edu in the usual manner– One thing to note is that Message-ID: line was filled in

not by the sending machine but by the relayer: Message-Id:

<[email protected]>» One way to confirm relayed mail

Example of suspicious headerHELO galangal.org250 mailhost.isse.gmu.edu Hello turmeric.com [104.128.23.115], pleased to meet youMAIL FROM: [email protected] [email protected]... Sender okRCPT TO: [email protected] [email protected]... Recipient OKDATA354 Enter mail, end with "." on a line by itselfFrom: [email protected]: (your address suppressed for stealth mailing and annoyance).250 OAA08757 Message accepted for delivery

From [email protected]: from galangal.org ([104.128.23.115]) by mailhost.isse.gmu.edu (8.8.5) for <[email protected]>...From: [email protected]: (your address suppressed for stealth mailing and annoyance)

By reverse DNS lookup on IP address

Mail server may not always provide dns-nameCan rely on this IP address



Things to be aware of

• Do not take any domain (host) name or user name or email address in the email header at their face values.– They can be easily forged by compromising the

sending SMTP server

• Pay attention to the trail of ip-addresses in the from tokens– These are directly gathered by the receivers from IP

packets

• The topmost IP address in the email header is the IP address of the computer that last forwarded the email.


Things to be aware of

• False header information– Spammers may try to introduce fake Received: header lines in

the message• Introduced as part of data

– Follow the trail through the Received: header fields and use common sense

• False IP Address– The IP address may have been that of an naïve relay not the

actual sender

• Dynamic IP address– Sender’s machine may not have a fixed IP address– However mail server used by sender almost invariably has one– Solicit the help of the ISP who can trace back the sender from

DHCP logs


Mailing List Aliases

• Mailing List Aliases– mylist: :include:/etc/mail/include/mylist– owner-mylist: mylist-request– mylist-request: me– owner-owner: postmaster

• Purpose– owner: Messages appear to be from owner.

Receives bounces, list management mail.– request: Indirection ensures owner’s real address

doesn’t appear on Return-Path.– owner-owner: Receives errors from messages

destined for owner-* aliases.


Mailing List Software

• Automate list management.– E-mail interface.– Web interface.

• Packages– Mailman– Majordomo– Listserv

• List Archiving– Mailman– MHonArc


Mail Policies

1. Privacy Policy2. Namespaces3. Reliability4. Scaling5. Security


Privacy Policy

• Personal Use Policy– Personal v. commercial use.– When may employee e-mail be read?• By whom• Under what circumstances

– Automatic monitoring

• Retention Policy– Legal requirements.


Namespaces

• Avoid first.last format addresses.– There will be duplicates: John.Smith.– Use middle initials?– Append numbers?

• Create unique organization-wide namespace.– Use directory to lookup addresses.


Reliability

• Customers expect same reliability as power.– Failures generate many support calls.

• Reliability measures– Redundant servers.– Backup MX hosts.– RAID arrays.– Multiple NICs, power supplies, processors, etc.


Scalability

• Types of scability– To address growth in avg messages/day.– To address spikes in mail traffic.

• Number of messages grows– faster than linearly with number of users.– with time, even if user base is constant.– due to spam too.

• Size of messages grows– due to technology: more + larger attachments.


Security

• Mail server as a target– Complexity of mail leads to vulnerabilities.– Mail is an asset attackers want to take.

• E-mail as a conduit– Brings viruses and trojans into organization.– Leaks confidential information outward.– ex (2005): Apple sues bloggers over releasing data

about upcoming products.

• E-mail relaying• Intercepting e-mail

CT 320: Network and System Administration Fall 2014 * Dr. Indrajit Ray Email: [email protected]@cs.colostate.edu Department of Computer.

Documents

mail message

systems administration

computer science department

mail files

mail policies

mda mail

indrajit ray email

email slide