It came to my attention today that we were having some mail delivery issues. I took a look at the mail logs, and saw a ton of authenticated connections trying to send email from email@example.com to around 700,000 different email addresses.
It turns out that about a year ago, an account was created with a weak password, and yesterday, a brute-forcer managed to stumble upon it.
The mail queue on our Astaro firewall appliance was holding around 700,000 emails. Given the 415k figure from above, it means that over 1 million messages would have been sent out. Ouch. On our backend mail server, I cleared the queue by bringing down courier-mta and running the following from the msgs/msgq directories under PREFIX/var/
grep -ilr "firstname.lastname@example.org" | xargs rm
Since this has thoroughly wrecked our mail server's reputation for the day, we needed a way to stop any more mail from going out from this sender. Luckily they did nothing to try and randomize their return agrep -ilr "email@example.com" | xargs rm ddress, so I blocked all mail from 'firstname.lastname@example.org' in courier-mta's BOFH configuration.
psql -U postgres -d smtp smtp=# update m set action = 'delete' where sender = 'email@example.com';
The query has been running for about an hour at this point, and in addition to that, I have set the max connections to around 2000 in courier's config to help speed up rejection of the backlogged mail.