So I recently managed to make my Self-Hosted mailserver an Open Relay. This is bad.
My mailserver (dockerized mailcow) currently runs on a little NUC under my stairs. It has worked well with only minor problems over the 3 or so years I've had it running; I got spamhaused once, etc.
The problem all started with me trying to patch a percieved security hole. See, docker doesn't respect firewalls like UFW or firewalld (all based on iptables of course), instead opting to allow ports through iptables as you add -p flags to your containers in spite of any other rules you may have.
Now I thought this was rather terrible. I don't want to have to look both at my firewall and at all my docker port bindings to check if something is open. So as many of us would do, I started trawling the internet for solutions and started to learn about why this behaviour existed.
According to some articles/stackoverflows/etc. the way to stop docker messing with iptables and creating its own rules is to disable the feature in the daemon.json. Seems simple enough. The only caveat that I found mentioned was that container networking would break (in terms of internet reachability) but that's ok because I just had to add a firewalld rule to allow masquerading and that problem was solved.
Now the problem I failed to see was that of NAT changing. Prior to disabling the iptables flag, the mailserver would see connections' IPs as their real public ones. However afterwards, every single IP was that of the internal docker network default route.
I didn't think much of it at the time, merely that it would be more annoying to see who was connecting but that was fine because I had what I wanted. Firewalld was now the sole controller of my ports ๐
Little did I know (or maybe I did and just forgot) that postfix has a trusted list of IPs and it will relay anything from them without question. These IPs include internal IPs such as that of the default route...
So essentially every SMTP request was being NATed to have a sender address of 172.22.1.1 and postfix started sending EVERYTHING ๐ตโ๐ซ
It wasn't long before a plethora of bots had saturated my poor NUC with HUNDERDS OF THOUSANDS of emails.
I got home this evening to lag spikes in Tarkov which prompted me to check the server where I found this mess.
After taking everything down, re-enabling the iptables and flushing all the postfix queues, I was able to spin back up and not have the whole thing start spiralling again.
Some tips for those hosting mailservers:
- Use a mail server checker like https://mxtoolbox.com/SuperTool.aspx
- Setup monitoring and alerts for server CPU usage/spikes in requests/etc
- Don't fix what ain't broke
I'm gonna go cry myself to sleep now and pray that the big mail hosts like Google and Microsoft take pity on me and my screw up. (We all know I'll never be able to send another email to Microsoft again, who am I kidding)
Spectrum in the US allows the customer to replace their provided router with their own equipment, so all they provided to me is the modem, after that I have my own router, so I can pretty much do whatever I want. Never had an issue hosting a mail server, sending/receiving works perfect.
This has got nothing to do with hosting your own equipment like routers (with recent regulatory changes, all ISPs in the US are now required to allow this - i.e. be able to run your own equipment without paying a monthly fee to the ISP).
But this is more about ISPs blocking sending on specific ports like port 25 for SMTP. Instead you need to use some mail relay to send your email, who in turn will send the mail over SMTP on port 25.
Some ISPs in the US do allow port 25 SMTP, but very few and none of the large ones like Comcast, AT&T, Verizon.
Spectrum is the second largest ISP in the US and I have zero issues sending or receiving on my mail server, no relay in place