The Problem with Bayesian Spam Filtering: Why Pattern Matching Is Not Enough

If you have ever wondered how your email provider decides what is spam and what is not, the answer is almost certainly a Bayesian classifier. Named after 18th-century mathematician Thomas Bayes, this approach has been the backbone of spam filtering since Paul Graham's influential 2002 essay "A Plan for Spam."

It works remarkably well. Bayesian filters catch 95-98% of spam. But the emails they miss — the sophisticated phishing, the brand impersonation, the social engineering — are precisely the ones that cause the most damage.

This article explains how Bayesian filtering works, why it is brilliant, and where it fundamentally falls short.

First: what do spam and ham mean?

In email filtering, every message is classified as one of two things:

• Spam — Unwanted, unsolicited, or malicious email. Phishing, scams, bulk marketing you never signed up for, malware delivery.
• Ham — Legitimate email you actually want. Personal messages, work communication, order confirmations, newsletters you subscribed to.

The terms come from the Monty Python sketch (spam, spam, spam) and its opposite. Every spam filter's job is to sort incoming email into these two buckets as accurately as possible.

How Bayesian classification works

The core idea is probability. The filter looks at the words in an email and asks: "Given the words in this message, what is the probability that it is spam?"

The training phase

The classifier starts by learning from a corpus of known spam and known ham:

1. Take 10,000 emails that humans have labeled as spam
2. Take 10,000 emails that humans have labeled as ham
3. For each word, calculate how often it appears in spam vs. ham

After training, the classifier knows things like:

Word	P(spam)	P(ham)	Meaning
"viagra"	0.99	0.01	Almost always spam
"invoice"	0.60	0.40	Slightly more likely spam (but ambiguous)
"meeting"	0.10	0.90	Almost always ham
"unsubscribe"	0.70	0.30	Leans spam (marketing email)
"regards"	0.25	0.75	Leans ham (personal/work email)

The classification phase

When a new email arrives, the classifier:

1. Extracts all words from the email
2. Looks up each word's spam/ham probability from training
3. Combines the probabilities using Bayes' theorem
4. Produces a single score: the overall probability that the email is spam

An email containing "viagra", "free", "click here", and "limited time" will score very high (almost certainly spam). An email containing "meeting", "tomorrow", "agenda", and "regards" will score very low (almost certainly ham).

Why Bayesian filtering is brilliant

Despite its simplicity, Bayesian classification has several remarkable properties:

• It learns automatically. Feed it examples and it builds its own model. No hand-written rules needed.
• It adapts over time. As spam evolves, new training data updates the probabilities. The filter evolves with the threat.
• It is fast. Classification is a series of lookups and multiplications. It can process thousands of emails per second.
• It handles unseen words gracefully. A word not in the training data gets a neutral probability. The classifier does not crash on new vocabulary.
• It is language-agnostic. It works on English, Dutch, Chinese, Russian — any language. It does not need to "understand" the language, just count patterns.

In Rspamd (the spam engine Cleanbox uses), the Bayesian classifier is one of the most impactful scoring symbols. BAYES_SPAM adds up to +5.1 to the score. BAYES_HAM subtracts up to -3.0. These are among the highest-weight symbols in the entire system.

Where it falls short

Bayesian classification is statistical, not contextual. It recognizes patterns it has seen before. It does not understand meaning. This fundamental limitation creates blind spots that modern attackers exploit.

1. Brand impersonation

A fake PayPal email from paypal-notifications-center.com contains the same words as a real PayPal email: "transaction", "unauthorized", "account", "verify". The Bayesian classifier sees PayPal-like word patterns and thinks it is legitimate.

A human immediately sees that paypal-notifications-center.com is not paypal.com. The classifier does not check domains against brands — it just counts words.

Real example: a fake PayPal phishing email scored 0.1 on Rspamd's Bayesian classifier. Delivered to the inbox. The content was nearly identical to a real PayPal notification.

2. Novel scam patterns

Sextortion emails ("I recorded you through your webcam, send Bitcoin") use language patterns that are genuinely unique. The first time this scam appeared, no Bayesian classifier had training data for it. The words "webcam", "recording", "Bitcoin" individually are not spam indicators — they appear in legitimate tech discussions.

The classifier needs to see a pattern before it can recognize it. Novel attack types fly under the radar until enough examples accumulate in the training corpus.

3. Context blindness

The word "password" appears in both:

• "Your password has been reset successfully" (ham — legitimate notification from a service you use)
• "I know your password is hunter2, send Bitcoin" (spam — extortion scam)

A Bayesian classifier sees the same word in both. It cannot distinguish the context in which "password" is used. A human understands instantly that one is a routine notification and the other is a threat.

4. Clean infrastructure spam

Modern spammers use legitimate email infrastructure: Mailchimp, SendGrid, Amazon SES. Their emails have valid SPF, valid DKIM, clean HTML, and professional formatting. The Bayesian classifier's word patterns look like legitimate marketing email — because they are sent through the same infrastructure.

Cold outreach ("I noticed your website could benefit from our SEO services") scores low because the language is professional and the infrastructure is clean. A human recognizes it as unsolicited commercial email. The classifier sees normal business vocabulary.

5. Low-content emails

A phishing email that contains only "You have a new voicemail. Click here to listen." and a link has very few words for the classifier to analyze. With so little text, the probability calculation is essentially a coin flip. These short, clean phishing emails routinely score low.

What Rspamd does beyond Bayes

To be fair, no modern spam filter relies solely on Bayesian classification. Rspamd combines Bayes with dozens of other checks:

• Authentication: SPF, DKIM, DMARC verification
• URL reputation: Known phishing/malware URLs
• Content rules: Zero-font detection, hidden text, phishing layouts
• IP reputation: DNSBL blacklists
• Header analysis: Missing headers, forged routing
• Fuzzy hashing: Matching against known spam fingerprints

These additional checks catch many threats that Bayes alone would miss. But they are all rule-based — they match specific patterns. They do not understand what the email is saying.

The accuracy gap

Combined, all of Rspamd's checks (Bayes + rules + authentication + reputation) catch roughly 95-98% of spam. That sounds excellent — and it is. But at 200 emails per day, 2-5% means 4-10 spam emails reaching your inbox every single day. And those 4-10 emails are the most sophisticated, most dangerous ones — because they are the ones that survived every check.

This is the fundamental challenge of statistical spam filtering: the emails it misses are precisely the ones designed to evade statistical detection. The remaining gap requires something that understands context, not just patterns.

Where the industry is heading

The next generation of spam filtering needs contextual understanding:

• Is the sender domain impersonating a known brand?
• Does the email structure match a known scam pattern (threat → demand → deadline)?
• Is the urgency genuine or manufactured?
• Does the "from" identity match the content's claimed identity?

These are questions a human answers instantly but a statistical classifier cannot answer at all. The gap between "counting words" and "understanding meaning" is where the next breakthrough in spam detection will come from.

Bayesian filtering is not going away — it is too effective and too efficient to replace. But it will increasingly be one layer in a multi-layer system where contextual analysis fills the gaps that statistics cannot.

For a deeper look at how Rspamd's scoring system works in practice, see Rspamd Spam Symbols Explained. For how Cleanbox combines multiple detection layers, see How does Cleanbox detect spam?