Commtouch's Recurrent Pattern Detection (RPD™)

Commtouch's Recurrent Pattern Detection (RPD) technology protects against spam outbreaks in real-time as messages are mass-distributed over the Internet. The unique content-agnostic technology detects and blocks spam in any language. Rather than evaluating the content of messages, the Commtouch Detection Center analyzes billions of emails in real-time, recognizing and protecting against new spam outbreaks the moment they emerge.

Benefits of RPD™

• High detection rate - block more than 97% of spam threatening email users
• Low false positives - less than 1 in 1.5 million reported false positives
• Complete global solution - protects against all types of spam with any content, in any location, format or
• language including image-based and double-byte languages
• Improved customer satisfaction due to real-time protection against spam
• Increased revenue by launching premium anti-spam solution in addition to current offerings
• Decreased engineering costs of keeping up with new spam attack tactics and sources

Commtouch GlobalView™ Mail Reputation Service

With spam and malware comprising the majority of all email, enterprises and service providers face bloated IT costs, and deterioration in the quality of service for valid traffic. Commtouch's GlobalView™ Mail Reputation Service fights unwanted mail at the perimeter, reducing more than 85% of incoming messages at the entry-point, before these messages enter the network. Additionally, GlobalView can optimize traffic flow so that legitimate sources gain optimal access, while unauthorized sources attempting to abuse the network are blocked.

GlobalView differs from other similar-sounding services in terms of its unique breadth of coverage, and its analysis and delivery of information in real-time. Built on Commtouch's comprehensive view of global Internet traffic, GlobalView distinguishes in real-time between legitimate corporate senders, valid publishers, zombies, and spammers/malware distributors.

Commtouch global detection centers analyze more than two billion Internet transactions per day, providing visibility into network traffic in every location around the world. This critical mass of data is analyzed using Commtouch's patented RPD technology, enabling the real-time delivery of IP address classification. GlobalView determines if a particular address is sending spam and/or legitimate email, and if it has been compromised into a zombie. These capabilities enable the GlobalView service to block distributed attacks the moment they start.

Benefits of GlobalView™ Mail Reputation 

• Statistical analysis of averages over time and recent changes of mail volume, spam ratio and valid bulk ratio
• Real-time zombie/botnet detection
• Continuously refined composite RBL score
• Use of IP DNS and whois attributes such as: domain age, geography, known dynamic IP, and numerous other factors.

Block e-mails on SMTP level

Typically, a server that utilizes Greylisting will record the following three pieces of information (referred to as triplet) for all incoming e-mail.

• The IP address of the connecting host
• The envelope sender address
• The envelope recipient address

The client is checked against the mail server's internal whitelists (if any) first. Then, if the triplet has never been seen before, it is greylisted for a period of time (how much time is dependent on the server configuration). The e-mail is rejected with a temporary error. The assumption is that since temporary failures are built into the RFC specifications for e-mail delivery, a legitimate server will attempt to connect again later on to deliver the e-mail.

Greylisting is effective because many mass e-mail tools utilized by spammers are not set up to handle temporary failures (or any failures for that matter) so the Spam is never received.

This feature can reduce e-mail traffic up to 50%. Greylisted e-mails never reach your mail server and your mail server will stop sending useless "Non Delivery Reports" to spammers.

The Proxmox Solution

Nowadays, e-mail domains are receiving a lot of e-mails to non-existing users. This could be up to 95 % of junk messages.

Proxmox can detect these e-mails to non-existing users on SMTP level, which means BEFORE the e-mails are transferred to your networks.

• reduced traffic, up to 90 %
• Your internal e-mail server is now working for you again
• Reduced load on your scanners
• 90 % less e-mails to analyze for spam and viruses

Open standard to prevent sender address forgery

Domains use public records (DNS) to direct requests for different services (web, e-mail, etc.) to the machines that perform those services.

All domains already publish e-mail (MX) records to tell the world what machines receive e-mail for the domain.

SPF works by domains publishing "reverse MX" records to tell the world what machines send e-mail for the domain.

When receiving a message from a domain, the recipient can check those records to make sure e-mail is coming from where it should be coming from.

DNS-based Block List

A DNS-based Blackhole List, or DNSBL, is a means by which an Internet site may publish a list of IP addresses, in a format which can be easily queried by computer programs on the Internet.

As the name suggests, the technology is built on top of the Internet DNS or Domain Name System. DNSBLs are chiefly used to publish lists of addresses linked to spamming.

Exclude senders from SMTP blocking

To prevent all SMTP checks (Greylisting, Receiver Verification, SPF and RBL) and accept all e-mails for the analysis in the filter rule system, you can add the following to this list:

• Domains (Sender)
• Domain (Receiver)
• Mail address (Sender)
• Mail address (Receiver)
• Regular Expression (Sender)
• Regular Expression (Receiver)
• IP address (Sender)
• IP network (Sender)

Statistical filters

Particular words have particular probabilities of occurring in spam e-mail and in legitimate e-mail. For instance, most e-mail users will frequently encounter the word "Viagra" in spam e-mail, but will seldom see it in other e-mail.

The filter doesn't know these probabilities in advance, and must first be trained so it can build them up. This is done automatically. For all words in each training e-mail, the filter will adjust the probabilities that each word will appear in spam or legitimate e-mail in its database.

For instance, Bayesian spam filters will typically have learned a very high spam probability for the words "Viagra" and "refinance", but a very low spam probability for words seen only in legitimate email, such as the names of friends and family members.

Tune the rule-system

Black- and Whitelists are an access control mechanism to accept or block (or quarantine) e-mail to recipients.

The following  objects are available:

• Domains
• E-mail Adress
• Regular Expression
• IP Address
• IP Network
• LDAP Group
• LDAP User

Text searching and analysis

E-mail headers are usually hidden, but they are required to deliver an e-mail to its destination. The system can detect inconsistent headers that are a feature of many spam e-mails. 

Around 600 text searching rules are used to detect phrases and other features common in spam e-mails.

SURBL

SURBLs differ from most other RBLs in that they're used to detect spam based on message body URIs (usually web sites).

Unlike most other RBLs, SURBLs are not used to block spam senders.

Instead they allow you to block messages that have spam hosts which are mentioned in message bodies.

Autolearning algorithm

The system gathers statistical information about spam e-mails. This information is used by an autolearning algorithm, so the system becomes smarter over time.

There are six states available

• ham: the message was learned as ham (non-spam)
• spam: the message was learned as spam
• no: the specific message didn't achieve the proper threshold values and requirements to be learned
• disabled: the configuration specifies bayes_auto_learn 0 or use_bayes 0 and so no autolearning is attempted
• failed: autolearning was attempted, but couldn't complete. This happens if SpamAssassin can't gain a lock on the Bayes database files, etc.
• unavailable: autolearning not completed for any reason not covered above. It could be the message was already learned.

Denial-of-service counter

Hashcash is a denial-of-service counter measure tool. Its main current use is to help hashcash users avoid losing email due to content based and blacklist based anti-spam systems.

A hashcash stamp constitutes a proof-of-work which takes a parameterizable amount of work to compute for the sender. The recipient can verify received hashcash stamps efficiently.