A client identified high server system load averages and a mail queue that was inaccessible through Plesk.

The first thing we should do is identify the MTA we’re dealing with–this will determine the tools we will use to view and interact with the mail queue.

1
2
3
4
5

$ telnet 0 25
Trying 0.0.0.0...
Connected to 0.
Escape character is '^]'.
220 foo.bar.fqdn ESMTP

Experience tells us this is qmail. You can verify this through other means, such as netstat.

So what’s the status of mail on the server? We’ll use a tool called qmHandle. If the MTA banner returns Postfix, use pfHandle.

1
2
3
4
5
6
7

# -s; show some statistics
$ qmHandle -s
Total messages: 677217
Messages with local recipients: 0
Messages with remote recipients: 554681
Messages with bounces: 37086
Messages in preprocess: 230073

550,000+ messages in the remote queue–definitely a spammer.

Although possible, asking qmHandle to list the contents of the remote mail queue (option -R) would take far too long to dump and scan.

Instead, we’ll catch a random spam message directly from the maillog–we’ll try the Google ISP and see if we get lucky:

1
2
3

$ tail -n1000 /usr/local/psa/var/log/maillog | grep gmail
Feb  7 01:22:54 foo.bar.fqdn qmail: 1391757774.265378 starting delivery 98301: msg 16451571 to remote [email protected]
Feb  7 01:22:54 foo.bar.fqdn qmail-remote-handlers[5840]: to=[email protected]

Bam. First try we get a valid result. Try different destination mail providers if you didn’t get anything–AOL, Hotmail, Yahoo!, etc.

What’s important from that above string? The message ID number: 16451571

Using qmHandle -m, we can view the contents of this message:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17

# -mN; display message number N
$ qmHandle -m16451571

 --------------
MESSAGE NUMBER 16451571
 --------------
Received: (qmail 6032 invoked from network); 6 Feb 2014 07:13:57 -0600
Received: from 93-79-80-154.sumy.volia.net (HELO ogrjdxj) (93.79.80.154)
  by foo.bar.fqdn with ESMTPA; 6 Feb 2014 07:13:57 -0600
Subject: TO `P) SIT ~ES )
From: [email protected]
Date: Thu, 6 Feb 2014 05:00:42 -0700
To: <[email protected]>
Mime-Version: 1.0
Content-Type: text/plain; charset="utf-7"

http://asgolfvalgarde.net/video.htm?fyqimeha

What’s important from that block? Two things:

• The first line ("invoked from network") indicates the spam message was accepted and delivered via SMTP--this user will therefore appear as an authorized user.
• The IP address it originated from: 93.79.80.154.

Armed with that data, we’re ready to identify the compromised mailbox:

1
2
3
4

$ grep 93.79.80.154 /usr/local/psa/var/log/maillog
Feb  6 05:30:59 foo.bar.fqdn /var/qmail/bin/relaylock[21592]: /var/qmail/bin/relaylock: mail from 93.79.80.154:2321 (93-79-80-154.sumy.volia.net)
Feb  6 05:31:00 foo.bar.fqdn smtp_auth: SMTP connect from 93-79-80-154.sumy.volia.net [93.79.80.154]
Feb  6 05:31:00 foo.bar.fqdn smtp_auth: SMTP user [email protected] : logged in from 93-79-80-154.sumy.volia.net [93.79.80.154]

The spammer logged in using the credentials for “[email protected]”.

You will need to disable this mailbox entirely (or change the mailbox password) and kill off any open, established SMTP auth connections, so that an active spammer (still connected) will be forced to authenticate yet again (and now fail).

If you don’t care about losing valid, legitimate messages from the mail queue, flush the mail queue–warning, you will lose the entire contents of the queue:

1
2

# -D; delete all messages in the queue (local and remote)
$ qmHandle -D

Add the spammer’s IP to your iptables INPUT chain and mark DROP:

1

$ sudo iptables -A INPUT -s 93.79.80.154 -j DROP

Hope this helps! Educate your users about strong mailbox passwords! Until next time.