If you do not have access to a UNIX server, a number of client side tools are also available. The advantage to server-side filtering is that the mail is filtered before you ever need to spend the time to download the message.

User configurable list of sites (or patterns) which are to be rejected.

The software comes with a used configurable list that includes by default prolific spammers such as:

• cyberpromo.com
• shoppingplanet.com
• any many many many more..

As well as the easily abused

• earthlink.net
• aol.com
• netcom.com
• and others...

Headers examined for traces of taint from known spam sites

The From, Reply-to, Authenticated Sender, and Received-by, Message-ID, and other headers are all examined for traces of known spam sites or commonly used techniques in mail forgery that the spammers use. Sites that are popularly used by spammers to bounce their mail off can also be easily filtered. Other bulk mailers can leave tell traces in the message headers. These are also looked for.

Headers scanned for commonly used junk emailers

The script can be configured to reject mail based on the X-Mailer. The X-Mailer header from of a number of commonly used junk mailers are included in the defaults.

Sends back response mail.

The script can send mail back to the sender and/or the postmaster of the originating site. The message sent (which could easily be customized by the user), states the reason the message was returned, points out that spam is not only rude but may potentially be illegal, and then returns the message along with instructions for how the user or system administrator may get through the filter to get word to you.

Forges mail headers.

The mail headers of the response message are forged so that the from: and reply-to: fields have your name with their email address. This will protect the user from autoresponders since the return address is either their own or nonexistent.

Support for multiple SMTP ports.

If one of the SMTP ports is down, the script will try the next in line until a sucessful transaction occurs.

User configurable list of exceptions to the filtering.

Email addresses (or patterns) of users can be configured to pass through the filter. Two seperate lists are maintained: one is matched against the from: header and the other against the to: header. The former is used to allow people from filtered sites like aol.com to mail you. The latter is used to ensure that the filter does not reject mail sent to you from mailing lists you subscribe to.P>

User configurable list of subjects to be filtered.

The software comes with a default of words like: money, sex, profit, $$, and many more.

Support for special subject fields.

User configurable subjects can be set up which will allow mail from spam sites through. This is an important feature since it will allow legitimate users a way to contact you so you can add them as an exception. Another special subject can be configured which will tag the mail as SPAM mail. This is useful for testing.

NOTE: TYPE THE SUBJECT IN CORRECTLY!!!!

-->

Protection from infinite looping

All mail generated by the NAGS Spam Filter adds a special header in the message. It scans incoming messages that have been tagged as Spam Mail for this header. If it finds it, then it knows that the message it has received was a reply (most likely automated) to one of its generated complaints. By ignoring the mail instead of issuing another complaint, infinite recursion is avoided.

Extensive transaction logging.

All received mail is time stamped and the activities of the script are logged. Any errors that occur in the script are also written to the log. A user defined directory contains copies of all spam mail and ignored mail that was not passed along to the user. This should be purged periodically so as not to take up excessive space.

• UNIX Shell Account Access (DEC machines may not work)
• Access to an SMTP server (don't worry everyone should have this)
• PERL v5 (v4 will *NOT* work)

WARNING: I just rewrote this section and repackaged the distribution so there is a good chance that things may not work right. If that is the case, send me mail and I will try to fix it up.

Step 0: Download Script

The NAGS Spam filter distribution comes with a number of files. The easiest way to get the package is to download the NAGS TAR into your home directory then enter:

tar xf nags_distrib.tar
rm nags_distrib.tar


This should create a directory called "nags" and create a number of subdirectories and files underneath it.

If you prefer you may download the file individually. This is generally the best thing to do if you are just looking to upgrade. These files should be put in a directory called "nags" that branches from your home directory. If you are using a version of the NAGS Filter older than v3.B2, you should redownload and configure the entire package.

• NAGS package (TAR) - The whole enchilada
• nags_filter.pl - Main program (rename from nags_filter.pl.txt to nags_filter.pl)
• nags_config.pl - Configuration (rename from nags_config.pl.txt to nags_config.pl)
• spam_sites - List of spammers (or patterns to match against site names)
• spam_subjects - List of subjects which should be rejected
• junkmailers - List of X-Mailers to be rejected
• from_exceptions - List of people whose mail is automatically accepted
• to_exceptions - List of address when mail is sent to that are automatically accepted

In addition to these files, there are a number of "test" files that should be put in ~/nags/tests. You will find links to these later in the documentation.

Don't forget to set the executable bit on the script. This can be done by entering the following at your UNIX prompt:

chmod u+x nags/nags_filter.pl

Step 1: Configuration

A number of parameters must be customized for your site. These can be set by loading nags_config.pl in your favorite text editor. The code describes each of these parameters and suggests values. Make sure that you change the following:

• $your_name
• $your_email
• $home_dir
• $mailbox
• @smtp_hosts

There are a number of other variables that you should also configure, but the script should work properly if you leave these alone.

It is strongly recommended that you examine the "spammers" and "exceptions" files. The "spammers" file defaults to rejecting email from some of the large and often abused service providers like AOL. You must decide whether you want to leave AOL tagged as a spammer. If you do leave AOL in the "spammers" file, you should add the email addresses of any friends you have into the "exceptions" file so email from them is not rejected.

Step 2: Command line testing

Perform all these tests from your home directory

Step 2a: Check PERL Version

The first thing to do is to make sure that you are running PERL v5.0. Enter the following at your command prompt:

perl -v

If you have PERL v5.x, go to the next step. If you have PERL v4.x, you will need to ask your system administrators to upgrade their PERL version.

Step 2b: Check PERL Syntax

The first thing to do is to have PERL verify the scripts code. Enter the following at your command prompt:

perl -c nags/nags_filter.pl
The result should be:

./nags/nags_filter.pl syntax OK

If it says "perl" command not found, then mail your system administrator and ask them where PERL is installed, and if it isn't installed.. WHY NOT?!

If it lists some error, pay attention to the first line number listed and check your entry. You probably are missing a quote (or added an extra quote) or something like that.

Step 2c: Parse test messages

Test 1: Normal mail
./nags/nags_filter.pl -justparse <nags/tests/okay.test
The results should be:

NAGS Spam Filter v2.D - Diagnostic Mode
----------------------------------------
From: ian@axxis.com
To: whoever@youare.com
Subject: This mail should go to your mail box
X-Mailer: Mail*Link SMTP-QM 3.0.2
Messade-ID: 

Test 2: Spam mail
./nags/nags_filter.pl -justparse <nags/tests/spam.test
The results should be:

NAGS Spam Filter v2.D - Diagnostic Mode
----------------------------------------
From: isuck@cyberpromo.com
To: ian@axxis.com
Subject: I will make you so rich it is unbelievable
X-Mailer: Mail*Link SMTP-QM 3.0.2
Messade-ID: 

This mail is spam mail (cyberpromo.com)
        A complaint would be sent:
                To: isuck@cyberpromo.com
                From: Ian Leicht 
        A complaint would be sent:
                To: postmaster@cyberpromo.com
                From: Ian Leicht                  

Test 3: Spam mail that should be ignored
./nags/nags_filter.pl -justparse <nags/tests/ignore.test
The results should be:

NAGS Spam Filter v2.D - Diagnostic Mode
----------------------------------------
From: isuck@cyberpromo.com
To: ian@axxis.com
Subject: Even though this is spam it should be ignored
X-Mailer: Mail*Link SMTP-QM 3.0.2
Messade-ID: 

This mail is spam mail (cyberpromo.com)
This spam mail will be ignored

Test 4: Spam mail subject test
./nags/nags_filter.pl -justparse <nags/tests/spam_subject.test
The results should be:

NAGS Spam Filter v2.D - Diagnostic Mode
----------------------------------------
From: YOUR_ADDRESS@HERE.COM
To: ian@axxis.com
Subject: THIS IS A SPAM MAIL TEST
X-Mailer:
Messade-ID: <01BBABCD.D1A7C8A0@zog.axxis.com>

This mail is spam mail (Autoreject Subject)
        A complaint would be sent:
                To: YOUR_ADDRESS@HERE.COM
                From: Ian Leicht 

Test 5: Mail from a spam site with the magic subject that should be let through
./nags/nags_filter.pl -justparse <nags/tests/okay_subject.test
The results should be:

NAGS Spam Filter v2.D - Diagnostic Mode
----------------------------------------
From: normal@cyberpromo.com
To: ian@axxis.com
Subject: I AM NOT SPAM
X-Mailer: Mail*Link SMTP-QM 3.0.2
Messade-ID: 

This mail is okay

Test 6: See if the X-Mailer based detection is working
./nags/nags_filter.pl -justparse <nags/tests/xmailer.test
The results should be:

NAGS Spam Filter v2.D - Diagnostic Mode
----------------------------------------
From: someone@somewhere.com
To: ian@axxis.com
Subject: FloodGate email test
X-Mailer: FloodGate
Messade-ID: 

This mail is spam mail (FloodGate)
        A complaint would be sent:
                To: someone@somewhere.com
                From: Ian Leicht 
        A complaint would be sent:
                To: postmaster@somewhere.com
                From: Ian Leicht 

Step 2d: Run script with test messages

If you want to watch the progress of this test enter the following:

tail -f nags/mail_log&

Note: if that file doesn't already exist, create an empty one.

The mail_log files contains a complete log of the NAGS filter's activity including a listing of mail it kept, ignored, and rejected. When appropriate the reason that a piece of mail was tagged as spam is also logged. check this file often. This is the best way to see which mail messages are being rejected. It will let you know if you are rejecting the wrong pieces of mail or how good of a job it is doing.

WARNING: My test messages may not be in the same format as real email messages that will be passed to the script. As a result they may corrupt your mailbox. If this happens, remove your mailbox. This does not mean the filter will not work, it does mean that after running these tests you will need to reset your mailbox.

Then have the filter process a test of a good message:

./nags/nags_filter.pl <nags/tests/okay.test

This should result in a new mail message being delivered to your mailbox. To have the filter send you a REJECTED mail letter form the command line, edit the spam_subject.test file and replace YOUR_ADDRESS@HERE.COM with your email address. Then enter the following:

./nags/nags_filter.pl <nags/tests/spam_subject.test

If you are watching your mail_log file, this should create an entry similar to this:

9-28-96 16:29:25: Spam mail recieved.
        From: [your email address] received
        Subject: THIS IS A SPAM MAIL TEST
        Mail sent to: [your email address] from: [your name] <[your email address]> via [your SMTP host]

Depending on who you use as your SMTP host, the mail will either be accepted:

9-28-96 16:21:39: Kept mail from: [your email address]

or it will be ignored (if your SMTP host is a known spam site):

9-28-96 16:30:13: Ignored mail from: [your email address]
        Subject: REJECTED MAIL

If the mail is rejected, you can find it in the junk-mail directory with its filename being the same as the time stamp

Step 3: Installation

WARNING: If the above tests have completed sucessfully, then the script is probably functioning correctly. However, there is always the possiblity that something is wrong. From your home directory:

cat >.forward
|/your/home/directory/nags/nags_filter.pl
^D [hold down the control key then press d]


Most people use their .forward file to send their mail to another address. By using the pipe symbol and the full path to the script, all incoming mail is piped through the NAGS mail filter.

Step 4: Final Testing

WARNING: If any of the following tests fail, immediately remove (or rename) your .forward file. If you do not do this, it is possible you may lose email that is sent to you.

Step 4a: Send yourself a piece of normal mail

Send yourself a piece of mail. It is better if you can do this from another account so you can see what the error message is if it doesn't work. Unless your ISP is really slow, you should receive it within seconds. The mail should be accepted and sent to your mailbox and an entry should be made in the mail_log file.

If this does not work, you need to have mail sent to your account from a different address. The sender should receive a piece of email that says that there was a delivery problem and then describe what the reason is. If thr explanation looks like:

sh: /your/home/directory/nags/nags_filter.pl: No such file or directory
554 |/your/home/directory/nags/nags_filter.pl... unknown mailer error 126

Then you will need to change your .forward file to read:

"|perl /home/cds/nags/nags_filter.pl"

Make sure you include the ""s! After doing this, try repeating this test. If it still doesn't work, try: "|/usr/local/bin/perl /home/cds/nags/nags_filter.pl"

If that doesn't work -- please let me know.

Step 4b: Send yourself a spam mail test

Assuming you have not changed the value of $autoreject_subject, send yourself a piece of mail with the subject:

THIS IS A SPAM MAIL TEST

After a short delay you should see an entry in the mail_log like this:

9-28-96 16:29:25: Spam mail recieved.
        From: [your email address] received
        Subject: THIS IS A SPAM MAIL TEST
        Mail sent to: [your email address] from: [your name] <[your email address]> via [your SMTP host]

After about 30 seconds, you should see another entry that will depend on whom you chose as your SMTP host. The mail will either be accepted:

9-28-96 16:21:39: Kept mail from: [your email address]

or it will be ignored (if your SMTP host is a known spam site):

9-28-96 16:30:13: Ignored mail from: [your email address]
        Subject: REJECTED MAIL

If the mail is rejected, you can find it in the junk-mail directory with its filename being the same as the time stamp

• Juan Cabanela (too much to list)
• Scott Keszler (too much to list)
• Gail Gurman (too much to list)
• Russ Allbery (taught me the right way to do socket code)
• Markus Baumeister (found socket code bugs)
• Stephen Mack (found typo's in my rejection letter)
• John Deters (ID spam trap: empty to: addresses)
• Mark Baushke (ID spam trap: empty message-id)
• Kevin Franden (ID spam trap: X-Shock*, to: address checking)
• Art Yaffee (detect current directory better)
• Zack Weinberg (suggestions about using reverse-DNS to spam trap)
• Bill Stewart-Cole (educating me about private networks)
• Timity Coldenhoff (reminding me to make spam_sites case insensitive)
• Scott Emmons (ID spam trap: numeric email addresses)