As already mentioned here, SPF/DKIM/DMARC are important, as are realizing the implications of each (especially if you’re forwarding mail), but equally important is the rate of sending.
Google and others will tempfail your messages if you start sending faster than usual (this happened when @doctorow started his most recent book tour and started announcing each stop on his mailing list).
Additionally, working forward and reverse DNS that resolves to real hostnames is also critical.
Using a service like mxtoolbox.com to monitor your IP addresses on RBL’s is also wise.
Lastly, make sure you have TLS enabled with a real certificate (letsencrypt.com is your friend here), and make sure you are configured to allow modern crypto (TLS 1.2, etc.)
SPF in combination with forwarding is nasty. You can go for SRS (ugly) or simply forbid forwarding to a external server (unpopular eventhough the alternative, the pop-fetch feature in Gmail, is better then forwarding in many ways).
Either way, it’s smart to think about this because spam sent to the first address (on your server) that gets forwarded to Gmail, gets attributed to your server. So any spam not filtered out by your filter that gets picked up by gmails filter will influence your servers reputation.
It is hard to imagine for me. What hoops would you have to jump through to get on that whitelist that a scammer could not?
Like @fuzzyfungus mentions, a lot of the spam actually comes from hacked websites (mostly Wordpress) so those are legit email senders that took time to get added to a whitelist (besides a whole heap of other configuration steps that already take time and effort) that then suddenly start spamming, do they get removed from the whitelist? Isn’t the damage already done then?
So, in summary, the article says that google will block newsletter sent from home servers.
Well, yes. Google, as a company, has no incentive to deliver your advertisements. Unless you pay.
They deliver some mail, but only because they need to pretend that gmail is normal email. Compare with Facebook. FB chose to be more obvious about it: their message system is not called email. If you want to deliver a newsletter via FB, you need to pay.
Google has exactly the same business model. They are more covert about it, that is the only difference.
I run a web and mail server for a recreational club, with mailing lists for ~200 members. We use SPF, which is great but does not help with gmail. We can not use DMARC because it does not play well with mailing lists. I have not looked into DKIM yet.
“Normal” blacklists like Spamhaus have never been a problem for us, because we keep our shit in order. In the past, we had issues with members who had outlook dot com mail addresses, we apparently got blacklisted because someone in the same IP range (it is a virtual server at a large European hoster) did shady stuff. To get off that list (twice) was a lengthy bureaucratic process not any less infuriating than to be expected from Microsoft, but ultimately successful. With gmail, however, we’re f*cked, and I have repeatedly recommended to members who could not receive our newsletters to change their mail provider, which most of them did.
Well, I run a small mail site, including some mailing lists, and I’m sure we are as hard as Gmail on enforcing DKIM and SPF for inbound mail. Because, since we are not Google, we just cannot rely on content analysis to successfully drop spam.
We are in 2019, exchanging mail without DKIM should be considered as bad as serving web content over http.
There’s certainly no justice in it(nor would one expect there to be, heuristic spam spam filtering is pretty much machine assisted stereotyping deployed to hassle and/or terminate the undesirables at the border); but one particularly ironic touch is how abjectly awful a lot of institutional/enterprise report/status email starts life as:
Something like a rackmount PDU is pretty much just a power strip with delusions of grandeur and a price tag to match, so rarely thrown out and replaced just because the vendor stops doing firmware updates(not that anyone was applying them to notice); so stuff like that often encapsulates an email alert generator wholly ignorant of any good ideas in email security from the last decade(sometimes substantially longer, though the example I’m thinking of didn’t even do DHCP, static or BOOTP, kids! So it might have had a few other issues…)
However, that kind of stuff is usually laundered through the real mail server(set to ‘trust with childish innocence’ for mail from select hosts on the management VLAN(assuming nobody is really screwing up that day) and relay it on with appropriate sanity and reputation upgrades to bring it to parity with messages generated by normal authenticated users of the mail infrastructure.)
I suspect that this is the other half of exchange server distrust of PHP and other atypical origins. Your average exchange shop likely has some mailer agents whose awfulness would make your PHP software run screaming(embedded devices, finest Enterprise IIS 6 line of business mission critical applications…); but practice is always to route that stuff through the real mail server for cleanup.
This likely reinforces the impression that anyone with atypical mail origins leaking out is either just a spammer on a compromised system or a grossly misconfigured sender who isn’t running all their oddball email through the exchange server before releasing it into the wild.
(Ironically, of course, while dodgy mailer agents on hapless server instances contribute a lot of the bulk spam, at least in my limited personal experience it’s the stuff from reputable corporate servers that contributes most of the danger, since it goes right through the usual filters, DKIM and DMARC in order, often a frequent correspondent; but one of their employees has fallen for a phishing attack or malicious attachment whose signatures aren’t yet recognized and their credentials are now being used to bombard their address book with the same phishing attack or bugged PDF or whatever. Spam is mostly a nusiance; but that can really get out of hand unless you detect and clamp down fast…)
Setting up a mail server has got to be one of the more complex and arcane things I’ve had to do in recent years. It really shouldn’t have to be so difficult.
Setting up a basic mail server is easy. Hardening it is difficult and that’s really because there’s been an arms race with spammers for almost 20 years by now, so you need to set up reverse PTRs, SPF records and DKIM and then establish a reputation…
