Gmail's automated spam-filtering is making it much harder to run an independent mail-server

Originally published at: https://boingboing.net/2019/05/01/algorithmic-discrimination.html

As someone who runs a small, independent low-volume mail server, the only problems I’ve seen with Gmail (so far) have been with messages forwarded via our service to Gmail users. Gmail blocks these messages on the grounds that SPF rules don’t authorize my server to send mail with [otherdomain] addresses in the From line. Which is fair enough.

My biggest bugbear currently is an ISP in the UK (no names, no packdrill) that uses rDNS checks to verify the existence of the sending domain, but has some kind of bug in their resolver so that it fails about 1 time in 100. This results in a hard “sending domain does not exist” rejection (rather than a more appropriate 451 temporary rejection). I have not yet succeeded in getting said ISP to take an interest in fixing this.

As a rule, the bigger a provider is, the less of a shit they have to give if their implementation is causing anyone else problems. Which really does make Gmail the Thousand-Pound Gorilla of Indifference, so if you do have problems delivering to them, you’re basically screwed.

13 Likes

Unless what he had to do is inappropriate to talk about in a public forum, get we junior/wannabe sysadmins get some deets?

10 Likes

I wonder if SPF, DKIM, and DMARC would solve this. My small provider sadly doesn’t provide DKIM yet. But I think it will be a requirement soon.

3 Likes

I mean, the problem is basically:

“We’re the phone company. We don’t care. We don’t have to.”

It’s hard to see how a technical solution will fix this social / political issue.

1 Like

I can see how Gmail’s filtering can be a problem for self-hosted email servers. But for me the aggressive spam filtering is what made my domain email usable. I have a very short domain name, consisting of all home row characters. So, thanks to millions of monkeys, well, people in this case, the domain and many, many aliases have been used as randomly typed throw away addresses by thousands of people. Before Google management of my domain email I got thousands of spams a day. After, it was a trickle.

5 Likes

FeatureBug

16 Likes

That fact is why newsletters are on the rise:

Wait. So this aggressive spam management may also render newsletters less useful and thus eventually decrease the likelihood that when I visit a site I get a subscription-shaming dark-pattern newsletter popup? Frankly, that sounds like a win-win.

5 Likes

Agreed. With proper DKIM and DMARC setup, gmail doesn’t seem to have an issue. Obviously sending mail with a from address not on your domain to that domain gets you blocked by pretty much anyone using SPF. I have a harder time dealing with exchange servers than anything else (they seem to hate PHPs mail function)

3 Likes

Outlook is the real bad one. Outlook has the nasty tendency to permaban you for no reason, so many independent, smallish sites I’ve seen just straight up say “sorry, if you have an outlook email you can’t sign up with it.” You’re also perma-blacklisted if you ever send any spam with no recourse iirc. Least that’s how it was last time I joint-ran an email server.

1 Like

I’ve had no problems with GMail, Comcast, or Charter, but AT&T has its own internal blocklist that has a couple of times bitten me when sending email. If anything, they’re at least as difficult to deal with as Google.

Can confirm this. Especially the “they don’t make it easy” part.

They aren’t wrong, though.

2 Likes

I wrote about this last year:

I heard from a google muckamuck, but it didn’t change anything.

It’s not hard to imagine how a whitelist could work.

3 Likes

I don’t know if there is an irrational bias, or merely an evidence-based one; but in my capacity as a part-time shepherd of a bunch of spam and phishing targets it has been my observation that mail with any visible traces of PHP in its provenance has an unfortunate likelihood of having been generated by some mailer script on a compromised website; rather than a legitimate mailserver.

I suspect WordPress being heavy on PHP, tepid on security; and massively beloved of people who are rather out of their depth actually running their own website has a lot to do with this; because it assures a bountiful supply of free-as-in-stolen and low risk spam boxes all conveniently preconfigured to run PHP applications. Also because, of the cases where I’ve checked up on what the host the mail came from is supposed to be doing, people’s hapless WordPress sites do show up rather a lot.

5 Likes

He hangs out here on the BBS so just give him a mention and he will show up.
Hey @orenwolf you got anything about this you can share in public for the more technical minded geeks/nerds here?

5 Likes

Maybe, maybe not. Its Google. Sometimes it might, other times, no.

Nope. Mechanisms like that have a habit of living long after their purpose ceases being fruitful. Nobody notices their 1% hit rate dropping to 0%.

2 Likes

meanwhile, here I am with a hand coded PHP application that sends legit mail with all proper DKIM and DMARC setup and all I want to do is send someone a report when they ask for it.

2 Likes

After setting up dkim and dmarc on top of SPF I can receive a hand-submitted fakemail in gmail. Another year, another layer of protections to implement. Work harder to stay in the same place, amirite