4

I don't have much knowledge of the subject but as I know, when servers receive an email they have two sources of data to classify it as SPAM or not. The contents of the message (subject + body) and the metadata that goes with it (from who / to who / server that sent the message, etc).

I know you can do basic filtering by blacklisting emails list and other sorts of metadata. But is it possible to do decent spam filtering without checking the contents of a message? Defining 'check the contents' as using any function that relies on data found on the body/subject of the message to classify it as spam or not spam.

Edit: The reason I ask for this because of the "scroogled" campaign where Microsoft accuses Google of scanning the contents of emails for advertising purposes, and I think that they do the same for spam prevention purposes. So I asked them and they said:

We have tools and system set up to filter spam emails to and from Outlook.com email accounts. However, we are not allowed to disclose how these spam filters work for security purposes.

ME: So can you assure me that no content from my personal email is being analysed and cross referenced with other data in any way?

...In response to your question, its a yes, we can assure you that.

  • analysed and cross referenced with other data <--- That doesn't mean it's not being compared to static heuristic rules. – Bobson Oct 11 '13 at 17:39
  • What does "decent" mean in actual numbers? Is 90% decent? 99% decent? Are you most worried about unknown domains? You'll need to define this a bit more to get a good answer otherwise answers are likely to border more on opinion based. – Michael Durrant Oct 11 '13 at 17:43
  • To be specific, you didnt ask them if they scanned content. You asked them if they scanned content and then cross-referenced the content with other data. They certainly scan the contents to catch spam, but they may not then be mining the contents for other purposes, or tracking whats in the emails. – GrandmasterB Oct 11 '13 at 17:45
  • 4
    @GrandmasterB - You pretty much hit the nail square on. Google reads your emails, and based on the content, might display ads about the topic. Microsoft claims they don't do this, and specifically calls out Google on it, they have an entire ad campaign on that very fact. Microsoft does indeed scan your email in order to catch email spam. – Ramhound Oct 11 '13 at 17:47
  • From these answers most mail validation is done from the headers: webmasters.stackexchange.com/questions/3728/… – dcaswell Oct 12 '13 at 3:45
12

is it possible to do decent spam filtering without checking the contents of a message?

Considering that 100% of the metadata that is useful in spam filtering can be spoofed,the answer to your question, is a big fat "NO". The actual contents is vital in order to determine if the contents matches previous spam email.

Lets put it another way, there is an infinite amount of email addresses, there isn't enough storage to filter every single email address that sends spam. While some of the metadata cannot be spoofed, the contents are not universal, email servers don't have the same behavior. Some email providers verify the sender was actually the sender, others will trust the information, and do no other verification process.

Microsoft's claims are connection to the fact they will not display ads based on the content of your email. Microsoft does indeed scan your email in order to block email spam from reaching you or at the very least flagging it as spam so you can delete it yourself.

Additional your question was "... is being analysed and cross referenced" which means their answer is accurate your email is NOt being analysed AND cross referenced its likely just being analysed. Furthermore the person replied isn't likely to be qualified to answer your question

  • Doesn't that leave out all possible machine learning algorithms that rely on learning for example, which words are common on spam messages? – Juan Enrique Muñoz Zolotoochin Oct 11 '13 at 17:41
  • The question was not about 100% coverage, it was "can a decent job" be done. That reveals that the answer will really depend on the OP's definition of decent. – Michael Durrant Oct 11 '13 at 17:42
  • 1
    @JuanEnriqueMuñozZolotoochin - If the contents are not looked out then machine learning algorithms would only be able to look at data that can be spoofed. – Ramhound Oct 11 '13 at 17:45
  • @MichaelDurrant - Without some heuristic data based on the data, helping out the filtering process, you would only have left data that could be spoofed. Even if you could filter without looking at the contents, it wouldn't be true for long, the spam companies would just say the subject was about "cute puppies playing" but actually be about human women allowing "their puppies to breath" for instance. – Ramhound Oct 11 '13 at 17:49
  • some do, some don't, my point was just that "they" are not monolithic entities. – Michael Durrant Oct 11 '13 at 19:34
0

A large percentage of spam is sent from botnets using barely-compliant SMTP implementations. If you implement a mail acceptance policy like https://www.nearlyfreespeech.net/about/email, what is left is still annoying, but requires far less effort to scan. (If Microsoft did this, however, the spammers would adapt, so I hope they don't.)

If you are concerned about the confidentiality of your message content, make sure it is encrypted before it leaves your (trusted) computer. And delete it from the server as soon as possible, even if it is encrypted. See https://ssd.eff.org/3rdparties to learn how little protection your messages have on a machine you don't own.

OT, but funny: most of the spam that hits my server uses SomeRandomAddress@outlook.com as the sender.

0

I looked at the link you supplied - what you got was a couple of 'canned' answers, neither of which actually address your question. It's one of the demons of outsourcing that agents get rewarded for giving something that looks plausible to a non-techy even if it's completely unrelated to the question that was asked.

Without a direct quote or link to an advert it's hard to know exactly which claim you're trying to verify with the support team.

Microsoft absolutely do algorithmically check incoming and outgoing message content on outlook/hotmail to check for spam and malware, but they don't use your email to build a profile of you for targeting advertisements (some of which now take the form of emails directly in your mailbox) which gmail does and always has (and has always been open about.) The point of the campaign was to point it out to people who use gmail are not have been aware of that practice and that many people disapprove of it in the hope they'd switch to Outlook/Hotmail.

There are also rumours that google employees regularly read mail to help tune their algorithms (as they do with web pages) but google deny this.


http://www.scroogled.com/mail/OurPosition says..

Outlook.com only scans the contents of your email to help protect you and display, categorize, and sort your mail appropriately. Just like the postal service sorts and scans mail and packages for dangerous explosives and biohazards, Outlook.com scans your mail to help prevent spam, gray mail, phishing scams, viruses, malware, and other dangers and annoyances.

  • Sorry, I am not an english speaker. What do you mean by "canned answers"? Also, my question is really simple, does outlook scan the contents of my email or not? I don't care if it's for targeted ads, criminal profiling, spam checking, or whatever, why would one be "worst" than the others? no humans are involved in the process in any way. Are you seriously talking about "rumors" on stackexchange? – Juan Enrique Muñoz Zolotoochin Oct 14 '13 at 19:12
  • Ok lets address that rambling stream of consciousnees one bit at a time... 1) A canned answer is a copy/paste response. Often the people giving them have not properly read your question and the answer they gave shows they didn't read it. – James Snell Oct 14 '13 at 20:31
  • 2) Your question is not simple, you've not included a link to what Microsoft claim - only your interpretation of it. Which given English isn't your first language may not be correct. – James Snell Oct 14 '13 at 20:32
  • 3) Yes, your email is scanned. – James Snell Oct 14 '13 at 20:32
  • 4) Why are they different? One is for your protection and there is no lasting effect or record, the other involves each communication you make being added to a file about you - which many people do not like. – James Snell Oct 14 '13 at 20:35

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.