Blog

How to avoid spam - what every company and its employees should know

30 September 2005

Email is a valuable tool few companies can do without these days. However, the ease, speed and low cost with which messages can be sent has inevitably resulted in email being widely abused as a communications medium too. The most widespead abuse of email is undoubtedly 'spamming' - the act of sending bulk mail to those who did not request it.

As a company, receiving spam can be costly even if you pay a fixed price for your connectivity. Spam also occupies disk space, and this can be significant for a large organization. But perhaps most costly of all is the time taken for employees to sift messages and delete the spam before they can deal with their real mail. It may take just a few seconds per message, but add them up over the year and you're talking a significant amount of time and money wasted. JHD ('just hit delete') is not a practical solution these days.

Eliminating all the spam you receive is unlikely to be a realistic prospect for most companies, but reducing spam to a level where it is not a notable disruption to employees is certainly possible. Below are some steps a company can take to reduce the level of spam.

Use a contact form

Remove all email addresses from your web site and instead have a contact form that uses the web server to send a mail to the correct department. Spammers trawl web sites just like search engines do. If they can't find your email addresses, they can't add them to their lists.

Some people suggest you can obscure email addresses from automated 'spiders' with javascript or other such methods. But if a human can see the email address you still risk having it added to lists by a person. A contact form is the safest way to allow anyone browsing your site to make contact by email while not revealing email addresses. Once you reply to the initial contact, the person will have your email address and can email you without a problem.

Educate your employees

Spammers also get hold of email addresses entered on various web sites. Most reputable sites will treat information according to a privacy policy and you will expressly be able to opt not to receive any promotional mail or newsletters. But less reputable sites might not be so trustworthy. Educate employees to avoid entering their company email address on such sites and to use a free webmail address such as Hotmail or Gmail that can be easily discarded if overwhelmed with spam.

An email address is worth more to a spammer if it is known to be active, and the spam sent to it is actually read by the recipient. Make sure employees never click a 'remove' link or any other link in spam they do receive. This will only confirm to the spammer that the message reached a human target and will result in more spam than before.

Blocking mail based on IP address

There are a variety of means to block spam from reaching employee inboxes. DNSBLs (block lists) such as the Spamhaus SBL/XBL, SPEWS and Spamcop can be used to check if the mail originates from a part of the internet known to be used by spammers. Your mailserver can be configured to check the sending IP addresses of mail messages and reject the delivery attempt if the IP is listed as a suspected spammer on any of the DNSBLs. We've found these to be extremely effective with very few false positives (where legitimate mail is blocked) - it is important to note that they are looking at the IP address (the unique number of the computer's connection to the internet) rather than the 'from' email address which can be easily forged.

There are also various packages like SpamPal that can use these DNSBLs to filter mail downloaded from a POP or IMAP4 mailbox such as those provided by most ISPs. Messages still get downloaded, but a special header tag identifying them as spam is added so your mail reader can be configured to easily recognize such spams and move them to a 'junk' folder for you. This way, the job of going through and sorting your messages is largely handled automatically. Some DNSBLs are more aggressive than others; each has a different listing policy and removal policy. You may need to experiment with which combination gives you the best protection with minimal false positives.

Content filtering

Another useful approach to spam protection is message content filtering. Rather than looking at whether the message comes from parts of the internet known to host spammers, content filtering instead scans the message for signs that it is spam. Perhaps the most effective method of content filtering we've come across is 'Bayesian Filtering'. It assesses the probability of a mail being a spam message based on what it has learned from previous messages. Messages over a certain probability that you set can be sent to a junk folder or deleted. The best thing about Bayesian filtering is that it bases its analysis on what you consider spam and what you consider real mail. If it wrongly identifies a message, you can easily correct it and it will do better next time. Many mail clients like Outlook and Thunderbird have built in filtering, but we have found Robin Keir's K9 to be more effective. It sits as a proxy between a pop box and mail client, and best of all, it is free.

The drawbacks with Bayesian and other filtering are firstly that it requires the full message to be downloaded in order to analyze it, and secondly that it is quite processor intensive. For this reason, if you run your own SMTP mailserver rather than using a popbox at an ISP, it is far preferable to use DNSBLs as a first line of defensive before using content filtering. They only need the message headers and so can reject much of your spam without downloading it.

Challenge-response

Any method claiming to offer a 100% spam-free inbox is most likely using some variant of 'challenge-response'. This works by only forwarding mail directly to your inbox if the sender's email address is on your 'white-list'. Anyone who emails you for the first time will have their mail put in a holding area. They are then sent a link on which they must click to confirm they are a real person. Since spams are invariably sent in vast quantities, with forged 'from' addresses, spam will not get confirmed and hence only real mail will get through.

While this is very effective at reducing spam, we would strongly advise not using C-R systems for a number of reasons. Firstly, they only add to the general noise on the internet. The vast majority of mail flying about is spam, so for each legitimate mail you send a challenge mail to, you will send 10 or 20 or more to forged addresses. You are effectively sending out mails in response to every spam you get, and all of these will end up either being bounced, or worse, filling the inbox of an innocent individual whose address was forged by a spammer. So to protect your inbox, you are effectively spamming others with large numbers of challenges, only a few of which will be responded to.

Secondly, C-R systems will block mail from automated systems that might be legitimate. Order confirmations from online stores and confirmations of online support tickets you may have opened will all be blocked. Furthermore, if another user uses challenge-response, both parties' automatic challenges may go unheeded and hence contact will fail.

Ensuring you don't spam

Steps should be taken to ensure you don't inadvertently become a spammer. The most common problem is maintaining mailing lists. You must take care to ensure those on your mailing list are genuinely the owners of the addresses they are subscribing, and that you can prove this should your ISP ever get a complaint. To do this, you need to operate a 'confirmed opt-in' system. Many mail servers provide such tools and it can be implemented in web scripts too.

It works similarly to challenge-response in that any sign-up is not immediately added to your list, but is instead set as 'pending' and has a random token assigned to the record. An email containing this token is sent to the email address in question, and the user must click a link or reply to the email to confirm their sign-up. Only then will the email address be added to the mailing list.

While it is true that this type of system could potentially spam innocent users, in practice this is less likely because it responds to sign-up requests on your web site rather than bulk mail you receive. Your system should record the time of the request and/or the confirmation as well as the user's IP address. We generally put the IP address of the person who entered the sign-up request in the confirmation request too. Furthermore it is good practice to have the ability to block addresses or complete domains from being sent confirmation mails, so if someone does decide to try to flood someone with sign-up confirmation mails from your site, you can easily stop this abuse.

The networks

Ultimately the spam problem persists because the big networks who form the basis of the internet will do deals with spammer that allow them to break the posted AUP (Acceptable Use Policy). Virtually all networks have rules that prevent spamming, but some less scrupulous networks will sign 'pink contracts' - sleazy deals that let spammers pushing anything from drugs to porn to reside on the network and spam with impunity.

Spamhaus maintains a list of active spammers and their web sites and MCI is without any doubt the spammers' biggest friend.

Top 10

MCI was previously known as Worldcom, a company that collapsed in spectacular fashion a couple of years back with huge debts. Several senior executives were charged with criminal acts including CEO Bernie Ebbers who was found guilty of cooking the books. The name may have changed, but the poor business ethics appear to be the same. Worldcom was a major spam support service, and MCI continues as the world's leading spam-support ISP.

At the time of writing this, MCI has 266 known spam issues on their network compared to the second placed SBC with 100. Every one of those 266 issues could be turned off instantly if MCI wanted, since all are violations of their posted AUP. Unfortunately the money is just too good. The spammers pay them for a 'bullet-proof' connection and MCI ignores abuse reports. Bernie Ebbers may be in prison, but there is still a morality deficit at MCI.

The solution?

Whether a technical solution will ever be found is debatable, but a legal solution is certainly possible. Proper laws are needed, not weak 'opt-out' laws like CAN-SPAM that effectively legalize spamming and require you to opt-out of every single spam you receive. Of course, spammers never have respected opt-outs and never will.

We would propose that spam be dealt with similarly to money laundering. Banks are under an obligation to have an internal compliance programme where they actively look out for suspicious activity and must do their best to prevent it. There can be severe penalties if they permit their services to be used by money launderers. The big networks like MCI should be put under a similar obligation to prevent spammers using their services. And in the same way that some countries have been isolated because their banks don't do enough to prevent money laundering, networks would be forced to block and filter traffic coming from countries known to be a major source of spam.

Sadly legislation like this looks unlikely. Until it comes, we will all be forced to defend ourselves and also make business decisions about the ISPs we use. Boycotting the likes of MCI is not just a moral stand, it is good business. Spam-friendly ISPs like MCI are widely blocked, so you may well find your legitimate mail is blocked if you send it from MCI's network.