Blocking spam relaying and junk mail - University of Leeds  · Web viewJanusz Lukasiak University of Manchester Blocking spam relaying and junk mail. Janusz Lukasiak University of

JTAPJISC Technology Applications Programme

Blocking spam relaying and junk mail

Janusz LukasiakUniversity of Manchester

Report: 40 JISC TechnologyApplications Programme

Joint Information Systems Committee

October 1999

JISC Technology Applications Programme

Blocking spam relaying and junk mail

Janusz LukasiakUniversity of Manchester

The JISC Technology Applications Programme is an initiative of the Joint Information Systems Committee of the Higher Education Funding Councils.

For more information contact:Tom FranklinJTAP Programme ManagerComputer BuildingUniversity of ManchesterManchesterM13 9PL

email: [email protected]: http://www.jtap.ac.uk/

Table of contents

1. Introduction...................................................................................................... 1

2. Scope of this document....................................................................................1

3. Background...................................................................................................... 1

4. Definitions....................................................................................................... 24.1. Spam.....................................................................................................................24.2. Relaying................................................................................................................2

5. Objectives........................................................................................................ 25.1. Mail relaying.........................................................................................................25.2. Incoming spam......................................................................................................45.3. Locally generated spam.........................................................................................5

6. Local anti-spamming policy.............................................................................56.1. Unacceptable use (internally generated spam)........................................................56.2. Dealing with incoming spam.................................................................................66.3. Restricting visibility of email addresses.................................................................6

7. Blocking email relaying: strategy outline.........................................................87.1. Define policy........................................................................................................87.2. Disseminate information........................................................................................97.3. Seek user & system managers co-operation.........................................................107.4. Define which systems to protect.........................................................................10

8. Is my system open to relaying?.......................................................................11

9. Blocking email relaying: Implementation steps..............................................129.1. Prepare detailed policy and timetable...................................................................129.2. Inform users and system administrators, emphasising benefits.............................129.3. Add MX records..................................................................................................129.4. Configure firewall with SMTP holes for mailrouters............................................139.5. Directing outgoing email via mailrouters.............................................................131.6. Configure MTA on mailrouters...........................................................................141.7. Special considerations for POP3 and IMAP users................................................14

10. Other (address-based) antispam measures.......................................................1510.1. RBL and DUL.................................................................................................1510.2. ORBS and IMRSS..........................................................................................1510.3. Unregistered addresses....................................................................................16

11. Securing common mailers against relaying.....................................................1611.1. exim................................................................................................................1611.2. Sendmail.........................................................................................................1711.3. MS Exchange Server.......................................................................................1711.4. Mercury..........................................................................................................17

12. User-level filters............................................................................................. 18

13. Overview of Manchester mailrouters..............................................................18

14. Conclusions.................................................................................................... 19

15. Bibliography.................................................................................................. 19

1. IntroductionThis document, prepared under the JISC/JTAP programme, describes methods of dealing with, and reducing the amount of, junk email. The solution described here is based on the work done by Manchester Computing (part of the University of Manchester) while securing the 130.88.0.0 class B network. Particular emphasis is put on the prevention of illicit email relaying, which, as a side effect, reduces the number of incoming unsolicited messages. Steps to minimise the incidence and possible impact of locally generated spam are also described.

Some facts, figures and general definitions: The 130.88.0.0 network serves the University of Manchester and UMIST, i.e. the

[man|mbs|mcc|umist].ac.uk

domains. For the purpose of this document, this network will be referred to simply as ‘Manchester’, although it obviously does not cover the whole city or even all its higher education institutions. There are about 23,000 registered email users, with accounts on approximately 400 mail hosts (mainly Netware fileservers and assorted Unix systems). These are owned and managed mainly by strongly independent faculties, with only a few under direct control of Manchester Computing. In brief, this is a huge and heterogeneous collection of users and email-enabled machines, with no central authority over end systems.

2. Scope of this documentThe document describes reasons for, and technical issues of, spam prevention. It does not discuss legal implications of spamming or computer viruses spread via email. The text has been intended primarily for postmasters and email administrators.

3. BackgroundJunk email has gained notoriety in recent years due to two factors:

(i) increased use of email as a common form of interpersonal and corporate communication

(ii) deliberate exploitation of weaknesses of the underlying protocol (SMTP).

The former means that more and more ordinary people (as opposed to computer experts) have access to, and regularly use, email, making it a potentially very attractive advertising medium. The protocol has had assorted loopholes since the very beginning, but in the ‘good old days’ they were either left conscientiously unexploited or put to good use by Internet developers.

In particular, in the days of restricted connectivity, it was a common practice to pass on a ‘difficult’ message to a third party with better resources, more comprehensive routing tables etc., which would deliver it to the final destination. Network administrators used this trick to debug mail connectivity and to route around known mail problems.

- 1 -

Nowadays, virtually the same technique is frowned upon, as it is used almost exclusively to steal the third party’s resources and conceal the identity of the real sender. The concealment is made even easier by the fact that SMTP, as originally designed, was assumed to be used between honest sites and users, thus making no attempts at the verification of sender’s identity or location.

The situation reached the point where it was considered desirable to issue an RFC document (RFC 2505) describing the best current practice for Mail Transfer Agents (MTAs) to make them less vulnerable to spam attacks.

4. Definitions

4.1. SpamThe electronic equivalent of junk paper mail is quite often referred to as ‘spam’, or more technically as ‘UCE’ (Unsolicited Commercial Email’) or ‘UBE’ (Unsolicited Bulk Email). Strictly speaking, these terms are not really interchangeable, but in practice they are treated as synonymous. Some mass mailings advertise genuine products, but most spam messages, sent to thousands of random addresses, describe various 'get rich quick' (usually multi-level marketing) schemes, services of dubious taste and legality etc. There are various products on the market facilitating distribution of such messages, using addresses obtained without owners’ knowledge or consent, these addresses being bought and sold in massive lists. Usually the mass-mailers use diverse tricks to hide the real identity of the sender.

Unfortunately, spam costs the sender very little - most of the charges are paid for by the recipient or the carrier rather than by the sender.

4.2. RelayingMail relaying (also known as ‘relay rape’) occurs when a mail server processes a mail message from an unauthorised external source with neither the sender nor the recipient being a local user. The mail server is an entirely unrelated third party to this transaction and the message has no business passing via the server.

5. ObjectivesThe site management should realise that while it is possible to make its mail service relay-proof, making it completely spam-free is probably not worth the effort (leaving aside technical and legal issues, see section ). Therefore, these two matters will be dealt with separately.

5.1. Mail relayingWhy mail relaying is bad? There are quite a few good reasons:

a) unauthorised use of computer and network resources.Spammers don’t use a ‘third party’ to distribute just a few dozen messages, they throw tens of thousands on an unprotected host. This can easily amount to a denial of service attack: before the Manchester network was secured, it was not uncommon to see mail machines

- 2 -

distributing only external spam for a good few hours, leaving no resources for any legitimate transactions.

b) damage to reputationWhen an item of junk mail is relayed via a third party, email headers seem to indicate that the relay machine is the source. This, predictably, doesn’t endear the recipients to the institution they see as a culprit, particularly if the apparent source is a university. Some recipients and Internet Service Providers feel very strongly against spammers, so any system perceived to let them use its resources is likely to become a subject of retaliatory actions, see below.

It is worth pointing out in this context that the full set of email headers contains sufficient information to prove the relaying site innocent. Technical knowledge required for analysing the headers is however beyond the need and skills of the vast majority of email users. This isn’t helped by the fact that the majority of popular email interfaces display only the bare minimum of headers (From:, To:, Date:, Subject:) and either do not provide easily accessible information on how to show the full set, or even totally hide additional headers.

c) risk of blacklisting As mentioned above, some institutions, network managers and ISP’s take a strong line against spammers and anybody perceived as helping (even unwillingly) to distribute spam. There are voluntary groups keeping lists of known spamming and relaying sites, for example RBL (Realtime Blackhole List) maintained by the MAPS (the Mail Abuse Prevention System), see

http://maps.vix.com/rbl/

This information is used by many sites to automatically block any email, or all Internet connectivity, from the offending site, or even worse, from the whole class B or C network (see section for details). It is worth adding that such a drastic action is perceived by those implementing it as being incompatible with the free flow of information over the Internet, and usually the blockade is promptly revoked as soon as steps to block relaying are taken by the offending site. For the blacklisted site this means that, apart from taking hasty steps to remedy the situation, there is an additional work to inform blacklist maintainers when the site is made relay-proof

d) dealing with the aftermath The spam tolerance level of more and more individuals and network managers is rapidly decreasing. Therefore any case of a site being used for mail relaying causes a flood of protests, warnings about possible retaliatory action, and notices of imminent or actual blacklisting, with only an occasional piece of practical advice about putting the house in order. Protests may be ignored (this is not to be taken as a recommendation!) due to the lack of time needed to send apologies and explanations, but warnings and blockage notices should

- 3 -

be treated seriously. In our experience, dealing with the RBL blacklist is simple in the sense that the maintainers act fast on a proof of remedial action. More annoying and time consuming are cases where an institution blocks incoming email using a local blacklist, without checking the up-to-date RBL: quite often legitimate mail from the once-guilty site is still bounced weeks or months after relaying has been stopped.

e) increased risk of mail forgeryWhile nobody claims that email is a secure communication medium, relaying opens yet another way of forging mail headers in such a way that the end product looks quite genuine to a non-technical recipient. An example is given in section .It should be noted in this context that the subject of secure email is outside the scope of this document.

The reasons given above should be sufficient to convince the site management about the necessity to protect the site against email relaying. This protection will benefit not only one’s site, but also the Internet community as a whole. In addition, many MTA’s with anti-relaying provisions can do more stringent checking of incoming email headers or SMTP connections, thus reducing the number of spam messages reaching the users’ mailboxes.

5.2. Incoming spamIt is worth keeping in mind (and continuously repeating it to end-users) that there is very little one can do to stop all spam reaching email accounts. The only practical goals are

§ educate users how to deal with it

§ educate users how to reduce visibility of their email addresses

§ prevent spammers from using legitimate mailing lists

These steps are discussed in some details in section .

It is worth noting one deliberate omission: scanning all incoming email for undesirable contents (including spam) is not discussed here. This possibility has never been seriously considered in Manchester due to:

a) practical difficulty in defining what constitutes spam.While an objectionable character of many cases is immediately obvious, a good proportion of unsolicited messages may be ‘innocent’, or even of real interest to recipients. This is particularly true in an academic environment, where all sorts of information may be legitimately exchanged via email. In these circumstances, any automated rejection mechanism would be either too blunt and restrictive or too lax and ineffective. Therefore ‘borderline’ messages would have to be ‘quarantined’ and later assessed for acceptability by a human being. This is unworkable due to the two reasons listed below.

b) sheer volume of traffic (a few millions messages per week). In the author’s experience, spam constitutes at most 10% of his incoming email. With 2 million messages processed in Manchester per

- 4 -

week and assuming 5% ‘spam rate’, some 100,000 spam items are expected per week. If an automated ‘spam detector’ leaves only 1% of those for a human censor to approve or reject, this amounts to 200 messages to analyse per working day.

c) privacy and academic freedom issues. Users conscientiously and voluntarily (at least in theory) accept restrictions imposed by the JANET acceptable use policy or local regulations. Nobody however would want his or her email regularly read by a third party with the power to make decisions about which messages are or are not to be delivered.

5.3. Locally generated spamIn practice there is very little one can do to prevent a user sending a spam (or otherwise objectionable) message to a large number of recipients. However the site should have a clear policy describing correct use of email, with references to the JANET acceptable use policy, local rules and regulations etc, and all users should be required to confirm that they are familiar with the policy. This, plus well-publicised cases of dealing with transgressors (in Manchester: disciplinary action by the user’s department plus a temporary suspension of email access) seems to keep the local spamming activity at a minimal level.

6. Local anti-spamming policy

6.1. Unacceptable use (internally generated spam)Local users should be informed that the following actions are contrary to the institutional acceptable computer use policy (it is assumed that each academic site already has one) and will not be tolerated:

a) sending unsolicited advertising email

b) sending unsolicited bulk email (including chain letters)

c) attempts to conceal sender’s identity or forge mail headers

d) subscribing anyone to a mailing list without their permission

e) sending to mailing lists material irrelevant to the purpose of the list

f) setting the ‘Reply-To:’ field to somebody else’s address without their permission

g) denial of service attacks, e.g. sending multiple very large emails with the purpose of overloading the recipient’s resources

h) sending email designed to damage the receiver’s system (e.g. containing viruses)

i) sending email with illegal content, as defined by the law of the land.

While some of these acts (particularly the last two) are not considered spamming in the strict meaning of the word, it makes sense to keep the list of prohibited activities together.

- 5 -

To underline the seriousness of these points it should be made clear that most of them are not only disciplinary offences, but also contravene the JANET acceptable use policy and criminal law (e.g. the Computer Misuse Act 1990).

6.2. Dealing with incoming spamUsers should be advised

a) that ignoring spam is the best policy in most cases

b) that one cannot expect, on grounds of the cost-to-benefit ratio, to eliminate all unsolicited messages

c) not to overreact: it takes only one keystroke or mouse click to delete an unwanted message

d) not to reply to spam, even when a procedure for removal from a mailing list is given (spammers use this ruse to confirm that the address is still valid and active, making it more valuable in future mass mailings)

e) not to take any action implied in the spam message (e.g. distribute virus warning) without contacting computing services first

f) report serious cases of spam attacks (e.g. deeply offensive or illegal contents, denial of service attacks) to computing services.

An address for reporting email abuse should be set up for use by external and internal users. RFC 2142 recommends that [email protected] ([email protected]) be used for this purpose, although quite often complaints are sent to postmaster@site. It is a good practice to acknowledge, and give a reference number to, each complaint. This can be done by an automated process, with further actions, possibly followed by a brief report to the complainer, taken by a human being. In Manchester, these actions are taken by the postmaster or computer security officer, depending on the seriousness and complexity of each individual case.

JANET offers a central spam reporting point, [email protected], which obviously could be approached by the person dealing with complaints sent to [email protected]. It should however be kept in mind that there are many hosts providing free and unauthenticated email accounts. The management of these sites has no real recourse against spammers and other abusers of their terms and conditions. All the managers can do is to close the account (which they usually do in response to a well-documented complaint), but are obviously unable to act against an anonymous owner. In any case, the offender is free to open another account with the same, or another, host.

6.3. Restricting visibility of email addressesThe vast majority of spam is sent to addresses surreptitiously gathered from various publicly available sources on the Internet. It therefore a good idea to make this address harvesting process as difficult as possible. Here are some hints, given with the full understanding that they are contrary to what the Internet is supposed to be about, namely easy location and exchange of information. However, in the author’s opinion, address harvesting for spamming is not a legitimate use of the Internet,

- 6 -

therefore these steps are fully justified, even if they sometimes make life slightly more difficult for bona fide users:

a) do not reveal email addresses in USENET postingsUSENET is the main source of information for address harvesters. Steps commonly taken to frustrate this abuse are controversial, as they break the USENET standard (RFC 1036). In the author’s (highly contentious) opinion, this is a clear case where the end justifies the means. When posting news, either do not include your email address (this makes it impossible to contact you individually on matters related to your article), or disguise your address in such a way that it is understandable to humans, but not to an automated harvester.Examples of the latter technique are: jl at fs3 dot mcc dot ac dot ukjl @ fs3.mcc.ac.uk with a comment ‘remove

spaces around @ when replying’[email protected] with a comment ‘remove

NOSPAM when replying’

b) make exploitation of local mailing lists as difficult as possibleThese are very tempting targets for any spammer, as they provide up to date, ready-made and free lists of people with some common element (location, professional or personal interests, etc). Local lists can be protected from being abused by imposing certain restrictions:

· make the lists closed (only members can submit) or moderated (each message has to be approved by the list moderator)

· disallow obtaining the list of lists from the list server

· refuse requests to reveal the list membership (particularly to people not on the list)

· restrict the domains allowed to post to a well-focussed list (for example allow only mcc.ac.uk, man.ac.uk, mbs.ac.uk and umist.ac.uk addresses to submit messages to a list dealing with local networking issues in Manchester).

It is worth mentioning here that all these restrictions can be easily implemented in majordomo: http://hpc.uh.edu/majordomo/

a free mailing list manager extensively used in Manchester.

a) avoid revealing email addresses in WWW documents. Consider using a CGU form for feedback instead of the mailto: tag. When this is inconvenient, specify your email address in a form correctly interpreted by a browser, but not by an automated mailer. One possible method is to replace some characters on the right hand side (or both) of the @ sign by HTML entities: a is ‘a’......{ is ‘z’ (the numbers are decimal values of corresponding ASCII character codes). For example the author’s address could be written as

- 7 -

mailto:J.Lukasiak@mcc.ac.uk

Note that %<hex> could be used here instead of &#<dec>; with the hexadecimal values in the former and decimal in the latter.

7. Blocking email relaying: strategy outline

7.1. Define policyThe first choice is do decide whether to secure all email-capable systems on the local network, or to go for a ‘rampart’ model.

Securing all email-capable systems is practical only in a limited set of circumstances:

a) there is a relatively small number of mail machines

b) new systems can be installed only with the knowledge and approval of some central authority

c) this central authority has powers and expertise to vet and approve the way each systems under its control is configured

d) local system administrators cannot change email configuration

These conditions are unlikely to be fulfilled in a typical decentralised campus environment, with faculties controlling their own IT budgets and no central computer purchasing authority. Users, specialists in their research fields but usually no computer experts, install email-capable systems on their desks, with the mailers usually left in the ‘out-of-box’ state, which often means no anti-spam protection.

The alternative is to create an SMTP ‘rampart’1 separating protected systems from the outside world by forcing all SMTP connections via a central mailrouter. Advantages are as follows:

a) full protection is needed only on the mailrouter

b) usually the mailrouter already is under central control and run by skilled administrators

c) such a mailrouter may already exist in order to provide some other functionality, e.g. mapping of ‘real name’ email addresses to actual mailboxes.

Note that, for simplicity, a single mailrouter is assumed here. Of course the same applies to multiple mailrouters, whether they are duplicated for resilience, improved throughput, administrative convenience or any other reason.

In order for this model to operate, all incoming SMTP connections have to be forced to go via the mailrouter. This in turn implies the existence of a firewall at the boundary of the protected area. The firewall can be either dedicated equipment or just a set of filtering rules in a router connecting the campus to the Internet (JANET, MAN, commercial ISP).

1 ‘rampart’ is deliberately use here instead of the expected ‘firewall’. The latter forms a part of the former, but there is more to anti-spam protection than blocking certain types of TCP connections on certain ports.

- 8 -

This also is an appropriate moment to decide whether outgoing SMTP connections should equally be channelled via a mailrouter. While this does nothing to protect users from incoming spam, or mailrouters from mail relaying, it offers a single choking point for any internally generated spam (see section ). Consider a situation where massive spam is generated internally: how to stop thousands of outgoing messages? Local system administrators are not always responsive, even if they can be contacted quickly... In such a situation, a single choking point is needed, allowing stopping all outgoing email from the offending system and sorting the situation out afterwards. Such an action is obviously totally indiscriminate and unfair to legitimate users of the blocked system, therefore it should be used with extreme caution. It is however justified and effective in an emergency as a means of protecting the whole Internet from a spammer, and is recommended in the ‘best current practice’ RFC 2505. Obviously if the MTA allows choking outgoing email from a specific user, that’s even better.

7.2. Disseminate informationIt is extremely important to keep administrators of campus systems well informed about the strategy, and allow them to have input at the planning stage. Of course, this is a common sense approach to any major change in existing working practices. But in this case the author’s experience shows that a surprisingly large number of system administrators are fairly ignorant of SMTP and DNS subtleties, quite often resulting in dramatic pleas along the lines “but my users do receive email from outside the campus, and you are saying SMTP connections to their machines will be blocked... that’s unacceptable!”. It is therefore important not only to tell what will be done and why, but to reassure users and system administrators that the changes are to their benefit, see section .

In view of some common misunderstandings of practical implications of anti-relaying measures it is worth stressing the following points:

a) all external email will be arriving as usual (more technical users can be told: ‘via a mailrouter, but you will see no difference’)

b) there is no need to change any email addresses

c) mailing lists with, and automatic forwarding to, external addresses will not be affected (again some users may appreciate an explanation: list expansion and mail forwarding consist of two separate activities - mail is first received by a local address and then sent out from the same address, unlike relaying where no local address is involved)

These points may seem trivial, but it is surprising how many times this has had to be explained in Manchester.

An article in the MC newsletter describing the changes, and reasons behind themhttp://www.mcc.ac.uk/newsletters/Local/issue66/junk.html

may serve as an example of information distributed to the user community.

- 9 -

7.3. Seek user & system managers co-operationIt is important to present anti-spam measures not as a managerial whim, but as something of benefit to users and system administrators. It is probably easier to enumerate disadvantages of doing nothing. These are listed in some detail is section and are briefly summarised for quick reference here:

a) spam relaying misuses resources

b) spam relayed through a local site causes embarrassment

c) explaining and apologising to spam victims is an unproductive use of one’s time

d) risk of the whole campus network being denied email access to networks implementing ban on spamming sites (if real life cases are available, they should be widely publicised)

e) anti-relaying measures produce, as a side effect, a reduction in the number of locally received spam messages

It should also be stressed that imposing a centrally managed ‘rampart’ relieves most system administrators from doing antispamming work themselves. The “if you are sure you can properly configure your sendmail.cf, that’s fine by me” argument rarely fails to convince....

7.4. Define which systems to protectTo implement an anti-relaying policy, ‘inside’ and ‘outside’ have to be defined (remember: relaying is passing email from outside to outside). It is therefore necessary to define a single ‘border crossing point’ which should be made as secure as appropriate. An obvious solution is to declare the router between the local network and JANET as the boundary. For a large local network (like Manchester’s) there may be multiple routers connecting it to JANET, but the principle remains the same.

For a relatively small site, ‘inside’ is defined simply as everything in its own domain (e.g. salchester.ac.uk), and outside is everything else. Larger sites may have to consider additional points

a) some systems have multiple unrelated names (of the type system.domain.ac.uk and service.ac.uk), apparently in different domains. For example the system with the IP address of 130.88.203.18 is known as both cs6400.mcc.ac.uk and midas.ac.uk. Obviously they all have to be ‘on the same side’, i.e. almost certainly ‘inside’

b) the presence of additional sites (on sponsored or secondary connections) warrants analysis on the case by case basis. Some of them may handle their email independently (and professionally) in which case they could be left ‘outside’. Others depend for both incoming and outgoing email on the main site, so should be considers as ‘insiders’

c) some departments (i.e. subdomains) insist, for various reasons, on handling their email themselves, usually via an existing departmental mailhub. If they have sufficient expertise to secure their mailhub

- 10 -

against relaying, there is no technical reason why this should be refused

d) decide which systems will be allowed SMTP connections through the firewall. The central mailrouter is an obvious choice, but if any other systems are deemed capable of securely handling their email, they should not be forgotten when setting up firewall filters.

When all these preliminary steps are completed and users’ approval secured, one can start practical implementation steps, as detailed in section .

8. Is my system open to relaying?This question should be asked after making changes intended to secure a mail system. The same method can also be used to demonstrate dangers of leaving the system unprotected.

The trick is simple: connect on port 25 to the system under test and instigate a dialogue similar to the following (other end’s responses are indented for clarity):

telnet victim.sys.ac.uk 25 Connecting to victim.sys.ac.uk ...

<<< 220- victim.sys.uk Sendmail 8.6.11/8.6.12 ready >>> HELO hacker.naughtysys.ac.uk

<<< 250 ******.***.ac.uk Hello hacker.naughtysys.ac.uk [130.88.x.x]>>> MAIL FROM:<[email protected]>

<<< 250 <[email protected]>... Sender ok>>> RCPT TO:<[email protected]>

<<< 250 :<[email protected]>... Recipient ok>>> DATAcan contain anything …..terminated by a line with only a single dot

<<< 250 OAA14072 Message accepted for delivery>>> QUIT

<<< 221 victim.sys.ac.uk closing connection

There is a web site provided by the Mail Abuse Protection System’s Transport Security Initiative:

http://maps.vix.com/tsi/ar-test.html

where this simple procedure is automated. You enter the address of the site to be tested and a perl script does the rest. The underlying script, rlytest, can be downloaded via a link given on the same page.

One trivial but easily overlooked piece of advice: when testing a system that receives email via a mail exchanger make sure you use the mail exchanger’s address.

Keep in mind that this technique is not foolproof, occasionally giving false positive results (i.e. declaring a site as open to relaying whereas it isn’t). This happens with mailers running in an ‘application firewall’, which initially accept anything thrown at them, and only later decide whether to make the delivery. If such a mailer detects a relay attempt, the sender may or may not get a failure report, and the manager of the site under test may or may not get informed about the attempt.

- 11 -

9. Blocking email relaying: Implementation steps

9.1. Prepare detailed policy and timetableThis step is a logical continuation of the policy definition (section ). All decisions have to be documented and distributed to personnel responsible for the implementation of each step. For practical reasons, the number of systems to be secured should be kept to a minimum (systems inside the ‘SMTP rampart’ may be left unsecured against relaying, although it would be a good idea to improve their security all the same).

The timetable of changes should allocate a few relatively short test periods (e.g. a couple of hours twice weekly during normal working hours), when system administrators and firewall managers can monitor the new mode of operation and make corrections rapidly, or even abandon the test, if things start going badly wrong.

It is highly recommended to turn the firewall logging on during tests to make sure that connection to or from various addresses are accepted or rejected as intended. Due to overheads and storage requirements imposed by logging it probably is a good idea to turn it off outside the test time.

9.2. Inform users and system administrators, emphasising benefitsInformation about what will be done, why and when (see sections and ) should be distributed to all interested parties, in particular to system managers and support staff. There is no need to keep the information away from end users, but on the other hand there is no compelling reason to explicitly inform everybody, as the changes will not affect them directly (POP3 users are an exception, see section ). The method of dissemination will of course depend on the local practice and available communication channels. In Manchester, we have used various mailing lists (e.g. support staff, LAN managers), web pages, and articles in the computer centre newsletter, see section for an example.

9.3. Add MX recordsBefore you change (or ask somebody to change) any DNS details, be prepared for many questions. You will be surprised how many system administrators don’t know what the MX record is and what it does. So, when they ask, reply that ‘X is a mail exchanger for Y’ means that email for Y should initially be sent to X, which will take care of the final delivery to Y, and that the DNS entry for Y has to have an extra line like this:

<Y’s DSN name> <ttl> MX <priority> <X’s DNS name>

In practice this step means adding to the existing DNS entry, for example:

nessie.mcc.ac.uk 86400 A 130.88.200.20

one or more MX records, pointing to the mailrouter(s) handling the incoming email:nessie.mcc.ac.uk. 86400 MX 7 mailrouter4.mcc.ac.uk.nessie.mcc.ac.uk. 86400 MX 4 mailrouter1.mcc.ac.uk.nessie.mcc.ac.uk. 86400 MX 5 mailrouter2.mcc.ac.uk.nessie.mcc.ac.uk. 86400 MX 6 mailrouter3.mcc.ac.uk.

- 12 -

Of course, if more than one MX record is present, ordering depends on local considerations and does not have to be the same for all ‘inside’ systems. In Manchester, the order in general is such that the mailrouter physically nearest to the end system’s location is tried first (it has the lowest ‘priority’ in the MX record).

This method works in all cases, however it implies that all messages to all systems from all systems go via a mailrouter. In some circumstances, for example where a clear majority of messages are exchanged between machines on the same subnet, this may not be the most efficient way of working. In this case, it makes sense to try a local mailhub first:

sys.dept.umist.ac.uk. 86400 MX 3 mailhub.dept.umist.ac.uksys.dept.umist.ac.uk. 86400 MX 7 mailrouter1.umist.ac.uk.sys.dept.umist.ac.uk. 86400 MX 8 mailrouter2.umist.ac.uk.sys.dept.umist.ac.uk. 86400 MX 9 mailrouter3.umist.ac.uk.sys.dept.umist.ac.uk. 86400 MX 10 mailrouter4.umist.ac.uk.

In this case, intradepartmental mail will be handled in the first instance by mailhub.dept.umist.ac.uk, at the expense of a slightly less efficient way of treating incoming external SMTP connections. They will try mailhub.dept.umist.ac.uk first, fail because it is behind the firewall, and succeed only on a subsequent connection to one of the mailrouters.

9.4. Configure firewall with SMTP holes for mailroutersThe firewall, as mentioned earlier can be either a dedicated unit, or a set of filtering rules in a router located on the boundary between ‘inside’ and ‘outside’, as defined in the policy document. Incoming (JANET to local network) SMTP connections should be allowed only to secure mailrouters. If outgoing email is forced to pass through mailrouters, a similar set of rules should control outgoing SMTP connectivity. The exact format of filter rules obviously depends on the type of firewall or router.

Managers of local networks with multiple connections to JANET will obviously have to ensure that the same level of protection is implemented on each connection.

9.5. Directing outgoing email via mailroutersIf the policy is to implement this recommendation, the following hints on how to make some popular MTAs conform may prove useful.

On Unix systems running sendmail a one-line change in the sendmail.cf file is necessary. The DS macro, defining the ‘smart’ relay host and normally null, needs to point to the mailrouter, so it now should read DSmailrouter.site.ac.uk.

If the majority of email traffic is sent to machines in the same domain, it is worth considering having these messages routed directly, and send only messages to other destinations via the mailrouter. In this case, use a mailertable instead of ‘smart’ host. Here is an example:

dept.umist.ac.uk esmtp:dept.umist.ac.uk.dept.umist.ac.uk esmtp:%1.dept.umist.ac.uk

which will deliver mail in the local domain and any subdomain of that locally. This has to be enabled by

- 13 -

FEATURE(mailertable, dbm -o /etc/mail/mailertable)

For MS Exchange Server, follow the this series of clicks from the console: Configuration ? Connections ? Internet mail servers ? Connections ? Message delivery ?Forward all messages to host and enter the mailrouter DNS name in the box.

Mercury always forwards all outgoing email to a single machine – simply make sure it is one on the ‘authorised’ list.

9.6. Configure MTA on mailroutersThis is mentioned here only as a ‘placeholder’ in the list of implementation steps. More information is given in section .

9.7. Special considerations for POP3 and IMAP usersImposition of a restriction on addresses allowed to make incoming SMTP connections to end systems has implications for some POP3 or IMAP users. Differences between POP3 and IMAP are irrelevant for the purpose of this discussion, and ‘POP3’ will be used as a generic name for both. Only users sending email from locations outside the firewall are affected.

POP3-based MUA’s (‘mail programs’, e.g. Eudora, Pegasus, Netscape Communicator, Outlook Express) use two separate ports and hosts for their operation. Mail is read on port 110 from the site mail is stored on (usually called ‘incoming mail server’), and sent out on port 25 via an ‘SMTP host’, alternatively called ‘outgoing mail server’. It is of course possible to use the same host for both purposes.

Reading is obviously unaffected by any firewall SMTP filters, and POP3 users with IP addresses inside the firewall can send mail out unaffected. But what if a POP3 user moves off site and is allocated a ‘foreign’ IP address, appropriate for the network his machine is currently on?

First of all, the connection to the user’s usual SMTP host on port 25 may not be get through the firewall. Secondly, even if the ‘SMTP host’ is set to be the mailrouter (which, by definition, accepts all incoming SMTP connections) it will see an incoming mail from the outside (as determined from the IP address) with the destination address being likely also on the outside. This is indistinguishable from an illegitimate attempt to relay, so the message will be discarded (with or without a warning). Email to ‘inside’ destinations will however get through as before, adding to the poor user’s confusion.

The only way to offer POP3 users a working solution it to educate them about

a) the need for changing the ‘SMTP host’ setting according to where they connect to the Internet from. The advice is to set it to whatever is recommended by the network administrator of the site their machine is plugged in to (e.g. when visiting another university) or by the ISP they use for a dial-up connection to the Internet.

b) how to do it. This may sound trivial, but a sizeable majority of POP3 users have had their setup done by somebody else, and often have no clue how to change it.

- 14 -

c) reasons behind this inconvenience. In Manchester’s experience users easily accept antispamming measures, even at the expense of a minor reconfiguration of their favourite mail interface according to their location.

10. Other (address-based) antispam measures

10.1. RBL and DULAs mentioned in section it is unrealistic to filter spam out acting on the message contents. On the other hand, massive spam attacks tend in practice to come from, or through, a relatively few sites, thus making a total ban on all email traffic from them justifiable.

As an aside, it is worth adding here that some domains (e.g. aol.com, or hotmail.com) have a bad reputation as a source of objectionable mass mailings. However, they also have too many sensible, legitimate and well-behaved users, to warrant a total blockade of these domains.

The approach taken by the Mail Abuse Prevention System (MAPS, mentioned in section ) is that they maintain a list of spamming and spammer-friendly (e.g. allowing email relaying) sites, and make it freely available to anybody willing to use it. MAPS stress they do not impose any blocking of mail traffic, they simply provide information for others to filter out undesirable sites, and nobody is forced to act on the information contained in their list.

MAPS provide two types of lists, which can be used jointly or separately:

a) Real Blackhole List (RBL), with specific addresses known as a source of spam, or used for relaying spam.

b) Dial Up List (DUL), listing ranges of dynamic IP addresses of co-operating ISPs. Email from these addresses should normally be routed via ISP’s recommended mailservers with fixed addresses. Spammers tend to bypass these to avoid detection, therefore email coming directly from a dynamic address is unlikely to be legitimate. For more information on MAPS DUL see

http://maps.vix.com/dul/

Latest versions of sendmail and exim include provisions for making use of RBL and/or DUL. Sample configuration entries for exim are listed in section .

It should be kept in mind that both RBL and DUL use DNS records for storing blacklisted sites, therefore checking addresses for all incoming messages imposes an additional load on the DSN server (and obviously on the MTA itself).

10.2. ORBS and IMRSSThese are other volunteer organisations maintaining lists of sites open to relaying. The main difference seems to be that MAPS maintains its lists manually based on reports of abuse, whereas ORBS and IMRSS do automated scanning.

See http://www.imrss.org/ and http://www.orbs.org/ for more information.

- 15 -

10.3. Unregistered addressesIt is expected that all well-configured bone-fide systems on the Internet have their names registered in the DNS, with either static or dynamic address allocation, the latter being almost universal with dial-up ISPs. Not having a DNS name offers no legitimate advantage, and quite often is a sign of either incompetent management or an attempt to conceal the site’s identity (usually both). It is therefore a reasonable assumption that email coming from an unregistered site is spam, and in any case, any reply or complaint sent to the (apparent) sender is likely to be undeliverable. In brief, it makes a good sense to reject an incoming SMTP connection from an IP address with no corresponding DNS name, and some MTAs allow that as a configurable option.

11. Securing common mailers against relayingIt is not the purpose of this section to replace existing documentation. It simply gives a brief overview of facilities offered by each mailer, and provides pointers to sources of detailed information. For mailers not listed here, see

http://maps.vix.com/tsi/ar-fix.html

Some MTA’s have effectively only a simple ‘relay/do not relay’ configuration option. Others are more flexible when it comes to rejecting spam: e.g. the rejection may come at different stages of the SMTP dialogue and may use different return codes. If the MTA allows that, and if the mail administrator is up to the task, RFC 2505 discusses various options available, and warns against potential pitfalls.

11.1. eximFull exim documentation is available from:

http://www.exim.org/

The exim specification document:

http://www.exim.org/exim-html-3.00/doc/html/spec.html

has a specific section on relaying, with all options fully documented: http://www.exim.org/exim-html-3.00/doc/html/spec_39.html#SEC711

Note that by default, exim prohibits any relaying (because the value of relay_domains in the configuration file is null), which is probably sufficient for a small site. For more detailed controls, the following example, taken from the ‘Main Configuration Settings’ section of the configuration file used on Manchester mailrouters, may serve as a starting point for local modifications:

sender_verifysender_verify_reject

The first line activates the ‘unregistered address’ check (section ). The second line rejects connections from addresses failing the check (without it only an ‘X-warning’ line would be added to message headers)rbl_domains = local.rbl.mcc.ac.uk:rbl.maps.vix.com:dul.maps.vix.comrbl _reject_receipients = truereceipients_reject_except = [email protected]:[email protected]

- 16 -

The first line here lists three sources of ‘undesirable’ addresses: a local table, the RBL list and the DUL list. The second line says we reject messages from these addresses instead of accepting them with an ‘X-warning’ header inserted. The third line modifies the second, allowing messages from ‘undesirable’ sites to specific recipients.

relay_domains = partial-dbm;/var/spool/exim/db/relay-domainssender_host_accept_relay = partial-dmb;/var/spool/exim/db/relay-domains

These lines define the source of the lists of domains we legitimately relay to and from.

The values given above are those used in Manchester, other sites may wish to take more or less stringent measures, see the full documentation for details.

11.2. Sendmail Note that version 5 is obsolete and there is no known solution to prevent relaying. More importantly, this version has a number of serious security holes, therefore it should be upgraded to version 8 (or later) as soon as possible.

Full information about sendmail is available from its official site:http://www.sendmail.org/

Sendmail prohibits relaying by default in version 8.9. This can be controlled and fine-tuned by a number of parameters described in the Anti-Spam Configuration Control section of the sendmail.cf and README files.

11.3. MS Exchange ServerVersions up to 5.0 are vulnerable to relay if they permit any local SMTP users.

Provisions have been made to prevent unauthorised relaying in Exchange Server from version 5.5 onwards. Procedure on how to secure Exchange Server is given in

http://www.nvt.net/antispam.html

which in turn is taken straight from the "README.DOC" file, coming with the full release of Microsoft Exchange 5.5.

11.4. MercuryDetailed Mercury documentation is available from

http://www.pegasus.usa.com/

Mercury is not in the same class as ‘heavy’ MTA’s (e.g. sendmail or exim) and is mentioned here only because it may run on a machine not hidden behind a firewall and therefore potentially open to relaying.

The Mercury mailer for Netware servers has provisions to prevent unauthorised relaying in versions from 1.40 onwards. Mercury/32, the 32-bit code redesigned for Windows 95 and NT, incorporates similar provisions in versions 2.11 or later.

To disable relaying, the following lines should be added to the [MercuryS] section of mercury.ini:

Relay : 0

- 17 -

Strict_Relay : 1

12. User-level filtersWhile the idea of rejecting spam in the MUA (e.g. Pegasus, Eudora, Outlook Express, Netscape) looks superficially attractive, practical considerations make it much less so.

User-level filtering can be done on the basis of email contents (including the Subject: line). This is however fraught with the same difficulties as mentioned in section , namely defining what constitutes undesirable contents. Of course this decision is much easier for an individual and getting it wrong does not affect anybody else. Filtering on the basis of incoming addresses is risky, because sites and userids used for spamming tend to come and go, on the other hand some domains seem to be populated by spammers and legitimate users in roughly equal numbers (see the beginning of section ).

In the author’s experience, user-level filtering is much more useful for separating incoming email from well-known legitimate addresses into different folders, or for disposing of nuisance messages from individual persistent senders (the latter are guilty of harassment rather than spamming).

Of course with the decentralised nature of user-level filtering it is nearly impossible to say how widespread this practice is, how well it works, and how many innocent messages are lost due to filter misconfiguration. Anecdotal evidence seems to suggest that filters are used, for whatever purpose, predominantly by more sophisticated users, who rarely seek advice or complain about mistakes they make. The general advice to Manchester users, as detailed in section is ‘delete manually and ignore’.

13. Overview of Manchester mailroutersThis section gives, just for reference, some information about software and hardware used by central Manchester mailrouters.

There are five mailrouters on the 130.88.0.0 network. All of them are custom-built PC’s, with processors ranging from P133 to PII 400, running FreeBSD and exim. The software has been found to be reliable and requiring very little day-to-day attention. Email traffic statistics (number of messages delivered, delayed, failed etc.) are gathered by each system and reports produced weekly.

Although in theory each mailrouter is dedicated to serve a particular (sub)domain, they are configured in such a way that they can deliver email to, and receive it from, any machine of the campus. This, together with a judicious choice of MX records for each end systems, illustrated in section , means that the mailrouters do not create a single point of failure for incoming email to any end system.

All mailrouters are administered by Manchester Computing, with one located on the UMIST campus and four on the University of Manchester campus. Of the latter, three are in the computer centre and the fourth in a faculty building. This positioning has been chosen as a (fairly crude) method of reducing unnecessary traffic on the local network. One of the mailrouters is the recommended ‘SMTP host’ for POP3 users on the Manchester network (see section ), but in practice this is not enforced and any mailrouter can be used for this purpose.

- 18 -

14. ConclusionsWork done in the last 18 month or so by Manchester Computing (MC) on securing Manchester network against mail relaying and on reducing the amount of junk mail has produced the desired results. No cases of mail relaying by Manchester machines have been discovered or reported since the enhancements are in place, whereas previously they were happening at least once a week.

The success is due to two factors: a large pool of technical expertise available within MC and a co-operative approach to necessary changes. The impact of the former is probably the most valuable lesson to be learned from this project. Technical issues were non-trivial, but resolvable with the right personnel in the right places, which MC is lucky to have.

Of course one doesn’t really need 18 months to implement all the technical steps described in this document. The length of time to complete the work is almost entirely due to superficially unproductive periods spent on distributing information, gathering comments, answering questions and in general on persuading users that the world will not come to an end because of the planned changes. Explaining the rationale for the ‘SMTP firewall’, and its benefits for end users and system administrators ensured co-operation and understanding between MC and its user community.

This delay has ultimately paid off: there has been no complaints about the way the implementation of relay blocking took place, and quite a few users have taken trouble to inform MC that they noticed a drop in the number of spam messages in their mailboxes.

15. BibliographyThere is a huge number of documents (mostly on-line) about use and abuse of email, including spamming and its prevention. This section lists only those explicitly referred to in the text.

JANET acceptable use policy: http://www.ja.net/documents/use.html

Request For Comments (RFC): They are mirrored at various sites around the world, for example at http://src.doc.ic.ac.uk/computing/internet/rfc/ (a searchable index is available e.g. at http://www.nexor.com/info/rfc/index/rfc.htm )

RFC 1036 M. Horton and R. Adams: “Standard for Interchange of USENET Messages”, December 1987

RFC 2142 D. Crocker: “Mailbox Names for Common Services, Rolesand Functions”, May 1997

RFC 2505 G. Lindberg: “Anti-Spam Recommendations for SMTP MTAs”, February 1999

Sample information to the user community about anti-spamming measures:http://www.mcc.ac.uk/newsletters/Local/issue66/junk.html

- 19 -