Architectural principles for creating a large-scale email server that doesn't get blacklisted

Question

I am considering what it takes to implement an email server. Google Cloud basically doesn't allow you to send emails at scale (they block the email ports pretty much), though it sounds like you could receive email. AWS on the other hand allows you to send emails for about $1 per 10k. This sums up some other SMTP services like SendGrid, and the costs involved.

I am aware (vaguely) that there are lots of problems Internet Service Providers (ISPs) want to prevent, like email spam. It sounds like they have IP blacklists, and somehow intercept the emails and can figure out if they are spam by checking their content. Somehow also they get access to abandoned email accounts and check who is emailing there (I have no idea how this works, but if there are some helpful links I'd love to know, though not relevant for the question). Basically, the ISP uses all kinds of techniques to figure out if your email service is spammy, so they can block your IP and shut it off. I don't see why this needs to happen at the ISP level, but that's beside the point.

What I'm wondering about is how to architect an email server so as to not get black listed, and to have it "work" 24/7, for years and years without interruption. Say I want to implement a service like Gmail or SendGrid. I'm wondering what measures you should take to architect an email server. That is, what the best practices are architecture-wise to create a successful email server.

Specifically where I'm at currently is, it seems using Amazon SES is the best option. It is the cheapest by far and doesn't have any bells and whistles. Otherwise you would have to buy your own hardware and build your own cloud if you wanted to get any cheaper or lower level I'd imagine, and buy your own IP addresses. But short of that, using AWS SES sounds like a good option.

They give you the ability to use dedicated IP addresses, and as they state:

most email certification programs require you to have dedicated IPs because they demonstrate your commitment to managing your email reputation.

So email server architecture principle 1, have dedicated IP addresses. But I don't want to do this just yet and then get blacklisted for an unknown reason, which brings me to the crux of the question. How not to get blacklisted. Given this is a service like Gmail or SendGrid, which could be sending millions of marketing emails and millions of personal emails, from millions of different email accounts, every day. I don't see how to tell if I am putting the right things in place for the email server to be top quality and to potentially be "certified" (not sure what email server certification really is or if it's a thing, Google search doesn't reveal anything, but AWS mentions it). That is, what the high level things are that you should put in place to guarantee that all emails will always get delivered (or all emails from all "good" email accounts on your system get delivered). If it's not possible to guarantee this, then I'd like to know why not, and the answer could just be tailored to whatever is closest to a guarantee that we can get.

Basically, the architectural measures to put in place for an email server to consistently deliver email without being blocked.

I am not (for this question) considering anything about scaling the email server or building the email server itself, just the architectural best practices to prevent being blacklisted.

From my understanding so far, some of the initial principles are:

1. Have a dedicated IP address. (Not sure if you should just have one, or if you can have 2 or 3, or 100).
2. Don't send spam.

That's all I can think of. For (2), this means you have to have good spam filters in place, and other security measures such as verifying that there is a person behind the email account, etc. But for (2) as well, I am unsure how to handle the problem of false negatives. That is, some users might send 100+ individuals a day, maybe even a few mass marketing emails like on those "get rich with adwords" marketing sites with email lists in the 10's of thousands. I would like to know if purely the volume of emails causes a red flag, and how to get around that. Then the content, just want to make sure this is purely based on in-house spam filters, and that the ISP wouldn't block that kind of stuff.

If this is a broad topic, I would like to keep it narrowly focused. I imagine one part of this is to learn more about email spam prevention, which I will do. So this question doesn't need to cover the spam stuff in any detail. To keep it narrowly focused, I'm wondering what architectural measures should be put in place not to be blacklisted. This might include (just making this up):

• Have a fixed number of dedicated IP addresses, less than x number.
• Contact some ISP providers and tell them manually (on the phone even) about your business goals.
• Implement spam filters to prevent spam going out in the first place.
• If you have geographically distributed email servers, perhaps something there as well.
• Programmatically send the abandoned accounts or closed accounts to the ISP for checking.
• Give access of ISP to some other stuff perhaps, by manually creating an API integration and partnership or something.
• Associate phone numbers with the accounts.
• etc.

I can understand how to implement an email/SMTP server, and send/receive messages at scale. So architecturally that makes sense. What's missing in the picture is the architectural components to prevent being blacklisted at this sort of scale.

To put it succinctly, I'd be interested to know how Gmail and SendGrid avoid getting blacklisted, but that's probably proprietary :)

Answer 1

As the comments have said, there is no technical solution to this. Your server needs to build a reputation as being trustworthy. As a mail admin at a small/medium sized church mail server (so we're not offering accounts to the public, all users are known) I'm on both sides - we want mail from our users to get through, and we want to curb incoming spam as much as possible. Here are some things that we do on the outgoing side:

• Static IP at a hosting provider, reverse DNS, non-anonymous registration, websites with proper info as required by law (i.e. whoever wants to know who we are can find out without hassles)
• Users are made aware that sending unsolicited mail (even well-intentioned) is not acceptable
• Policy violations are handled swiftly, easy because users can be contacted if needed
• After a few incidents with hacked passwords, we mandate strong passwords (but we don't require regular password changes as I consider that not helpful)
• We monitor the outgoing mail queue for stuck/blocked mails and analyze if the problem is with us or the recipient mail server, and try to remedy problems quickly
• We have a working abuse address and respond personally

On the incoming side, we have number of heuristics to identify likely spam sources:

• Mails from IPs without reverse DNS or with dynamic/generic names based on IP address will be rejected
• We keep a list of spammer-friendly hosting services. For example, if you host with DigitalOcean, OVH or some other service which has a non-responsive or non-existing abuse desk, your chances of getting mail through to us are slim
• There are a number of mechanisms to find out if a new sending domain is trying to hide its identity. Mail from such domains is rejected with a 450 SMTP response until we have checked the domain manually

The gist is that you as a mail-sending organization should be non-anonymous so you can be contacted in case of issues.

At the end of the day, all of this doesn't guarantee that mail can be sent. Mail providers will try all kinds of heuristics to block spam, and some of those will block non-spam, too. If the provider is unresponsive to your request for remedy you're just out of luck.