how to access the console…
SpamDupe Anti Spam is an antispam solution used with our enterprise class e-mail for our on server hosted mail users and for our clients who have their own mail servers at their sites.
SpamDupe Anti Spam provides inbound and outbound messaging hygiene/filtering protection with antimalware, antispam, antivirus and unwanted content blocking. We offer:
• Highly redundant, virtualized infrastructure with in service availability rates that exceed 99.99%.
• Accurate, effective spam filtering with better than 99.5% filter accuracy and less than 0.1% false positives (with SpamDupe acting as both inbound and outbound ESMTP gateways.)
• Bulk e-mail protection through our sophisticated Spamvertiser detection/blocking network.
• Virus defense by blocking e-mails with dangerous/unwanted attachments.
• Phishing and fraud defenses with sender reputation and web link validation checks.
• Denial of Service (DoS) and Distributed Denial of Service (DDoS) protection against e-mail attacks. Even under extreme pressure, SpamDupe continues to service e-mail traffic from known peers.
SpamDupe Anti Spam also provides enhanced e-mail privacy and security with;
• Ability to force e-mail encryption between peers to ensure that no e-mail is ever sent in clear text.
• Ability to force strong encryption options to ensure your message stays private.
SpamDupe Anti Spam is safe, secure and easy to use:
• Daily reports sent to user email for easy review and control (sample.)
• Mail queuing ensures that no inbound e-mail is lost when your mail servers are off-line or your Internet connection is unavailable.
• E-mail recovery provides you with ability to recover individual user, domain or site traffic going back for weeks.
• E-mail review and redirect gives management the power to review past e-mail exchanges between any employee and peer.
• Off-site disaster preparedness services allows you to continue to process e-mail if your mail server infrastructure should experience a failure (contact us for details.)
Key Benefits
• No infrastructure to buy, house, administer, back up or recover.
• Maintain user productivity by eliminating distractions and simplifying e-mail management.
• Reduce threats by filtering out viruses, spam, malware, advertising and phishing messages.
• Enforce acceptable use policies, improve management oversight and control.
• Switch over with no e-mail service interruptions.
• Integrate SpamDupe into your environment in 30 minutes or less.
How It Works
Basic Setup
SpamDupe is an Edge Transport Server based e-mail filter and relay solution that resides behind the firewall.
Internet <=> Firewall <=> SpamDupe <=> Mail Server
SpamDupe is a live filtering solution. It filters e-mail in real-time during the actual e-mail exchange.
When an e-mail message arrives, it is subject to our suite of validation and verification tests. The result is one of three decisions: Accepted, Tagged (uncertain), or Rejected.
Depending on your configuration the messages may be rejected or simply scored and filtered later.
Scoring Spam
Each e-mail is assigned a numerical score, generated by our anti-spam engine. The initial score of a message is “0”. We use many techniques to scan each message to see how “spammy” it is. The cumulative value of each test becomes the spam score of the message.
We have two thresholds, defined for each domain, that determine what happens to each message. The more spammy a message is, the higher the score. If the score reaches the tag threshold the e-mail will be tagged. If the score reaches the reject threshold the e-mail will be rejected.
Similarly, we look for evidence that the message is legitimate, reducing the spam score. Thus, the spam score can be a positive or negative number. The higher the number (positive) the more spammy it is; the lower the number (negative) the less spammy.
Tests that result in a high impact are examined first: virus scanning, black/white listing, sender history, etc. These tests take precedence; they can set the message result by themselves and may cause other tests to be skipped.
Some very expensive tests can get very good information about the sender; but they are done last and only if the test can change the disposition of the message.
We examine the traffic patterns between the sender and recipient. For legitimate senders, as their traffic history accumulates, their spam scores drop until the sender becomes implicitly white listed. This ensures their messages will never be blocked in error.
If the message is not accepted or rejected by the high impact tests, it is then classified based on its spam score and the Tag and Reject thresholds defined for the recipient.
SpamDupe uses three categories when scoring messages:
Accept
After being thoroughly scrutinized, the message was deemed wanted and is immediately forwarded to the intended recipient(s).
Reject
Messages that are rejected typically contain any of: unwanted content, obfuscated text, misleading or inaccurate e-mail header and/or envelope information, references to spam-friendly networks or other criteria that strongly indicates spam. As a result, SpamDupe refuses the message with an appropriate explanation to the sender. Reject messages are customizable so that in the unlikely chance the message was rejected in error, the sender can contact you by other means (phone).
Tag
SpamDupe tags messages that score above the Accept threshold but below the Reject threshold. We “Tag the subject line of the message [SPAM?] and deliver it to the recipient. The user does not need to check a separate quarantine. Typically less than 1% of all messages are tagged.
Tagged Messages
Tagged messages are message that are of indeterminate disposition; they have a score that puts them on the borderline between legitimate e-mail and spam. They are tagged with a text based note (by default it is [SPAM?]) on the subject line, but otherwise is delivered normally. (You can turn off the actual subject line tagging in Filter Settings, if needed.) Messages that are tagged and are also from unknown servers, seen for the first time, may be grey listed.
Grey listing is a process where the server reports that it is temporarily unable to service the e-mail. The sending server receives this notification while attempting the mail exchange. The normal behavior of mail servers is to try sending the message again after a short delay (usually 5 or 10 minutes). After 3 minutes or 3 attempts to deliver a “grey listed” message, that message will be accepted.
Note: Messages containing viruses, unwanted file attachments, or known Phishing (fraudulent) messages are always rejected.
Anti-Spam Tests
SpamDupe uses a variety of anti-spam tests:
♦ Sender Reputation
•Real-time Block Lists
•Incoming Sender Lists (Black/White lists, etc.)
•Real-time dynamic sender behavior analysis
♦ Historical Information
•Past server and sender behavior
•Analysis of e-mail traffic patterns
♦ Server Analysis
•Sending server analysis
•Sending address verification
•DNS configuration validation
•Server profiling and identification
♦ Sender Intention Checks
•Test for sender/origin obfuscation
•Phishing attempt identification
•Recipient validation
•Spam Traps
♦Content Scanning
•Anti-virus scanning
•Dangerous attachment filtering
•E-mail structure analysis
•Content black listing and watch words
•Anti-obfuscation engine
•OCR analysis
•Adaptive content filtering
Rejecting Spam
On any type of reject, a message delivery failure is immediately returned to the sending mail server. This occurs during the actual e-mail transaction which ensures a guaranteed delivery to the sending server.
Because SpamDupe™ never accepted the e-mail, the responsibility for dealing with that e-mail lies with the sending server. This behavior is markedly different from many delivery failure messages which are generated after a message has been accepted, scanned, then deemed to be spam.
This is a subtle difference but an important one. This ensures the responsibility for the e-mail lies with the sending server. We avoid the potential responsibility for such messages and avoid any legal requirements for storage, archiving, etc. that may otherwise be implied.
Further, many delivery failure messages are sent to spammers who do not accept them. This can literally choke your e-mail infrastructure with garbage messages that will never be sent.
ACTIVITY REPORT SAMPLE
WEB CONSOLE – FULL CONTROL FOR EVERY USER
Filtering Configuration
The Filtering configuration requires no changes to your PC’s. In this method the SpamDupe server scores each e-mail and filters out spam. Only the filtered e-mail is forwarded the mail server and client PC’s.
In this implementation:
– SpamDupe analyzes message
– SpamDupe discards spam and viruses
– SpamDupe forwards good e-mail
– All forwarded e-mail goes into your in-box
– Administrators release messages as needed
– Administrators have better understanding of e-mail issues
– Reduced e-mail traffic as spam is filtered
Scoring Spam
Each e-mail is assigned a numerical score, generated by our anti-spam engine. The initial score of a message is “0”. We use many techniques to scan each message to see how “spammy” it is. The cumulative value of each test becomes the spam score of the message.
We have two thresholds, defined for each domain, that determine what happens to each message. The more spammy a message is, the higher the score. If the score reaches the tag threshold the e-mail will be tagged. If the score reaches the reject threshold the e-mail will be rejected.
Similarly, we look for evidence that the message is legitimate, reducing the spam score. Thus, the spam score can be a positive or negative number. The higher the number (positive) the more spammy it is; the lower the number (negative) the less spammy.
Tests that result in a high impact are examined first: virus scanning, black/white listing, sender history, etc. These tests take precedence; they can set the message result by themselves and may cause other tests to be skipped.
Some very expensive tests can get very good information about the sender; but they are done last and only if the test can change the disposition of the message.
We examine the traffic patterns between the sender and recipient. For legitimate senders, as their traffic history accumulates, their spam scores drop until the sender becomes implicitly white listed. This ensures their messages will never be blocked in error.
If the message is not accepted or rejected by the high impact tests, it is then classified based on it’s spam score and the Tag and Reject thresholds defined for the recipient.
SpamDupe uses three categories when scoring messages:
After being thoroughly scrutinized, the message was deemed wanted and is immediately forwarded to the intended recipient(s).RejectMessages that are rejected typically contain any of: unwanted content, obfuscated text, misleading or inaccurate e-mail header and/or envelope information, references to spam-friendly networks or other criteria that strongly indicates spam. As a result, SpamDupe refuses the message with an appropriate explanation to the sender. Reject messages are customizable so that in the unlikely chance the message was rejected in error, the sender can contact you by other means (phone).TagSpamDupe tags messages that score above the Accept threshold but below the Reject threshold. We “Tag” the subject line of the message [SPAM?] and deliver it to the recipient. The user does not need to check a separate quarantine. Typically less than 1% of all messages are tagged.
Note: Messages containing viruses, unwanted file attachments, or known Phishing (fraudulent) messages are always rejected.
Tagging Messages – Uncertain Message Disposition
SpamDupe prepends a tag phrase ([SPAM?] by default) to the subject line of any Tagged messages. SpamDupe records the details of each message in its reputation system so that, as the sender’s reputation is established, SpamDupe will be less likely to Tag that senders messages.
Concerns can occasionally arise in your user community when a low frequency (or first-time) legitimate sender has receives a Tag score (and the [SPAM?] marker) on the subject line.
After a week administrators should take time to fine-tune SpamDupe so that the number of Tagged messages is safely and accurately reduced. A few moments spent fine-tuning SpamDupe will result in a more pleasant experience for users (fewer Tags) and fewer support calls for administrators.
Why Tag Messages?
We know that correctly handling legitimate e-mail is much more important than blocking spam. We designed SpamDupe to quickly and accurately discriminate between unwanted e-mail from unknown senders and valuable e-mail from your established e-mail peers.
SpamDupe accomplishes this task through adaptive learning. SpamDupe watches all e-mail traffic and quickly learns who e-mails whom. Once e-mail relationships are established, SpamDupe auto white lists the e-mail peer.
SpamDupe can discover e-mail peer relationships between active e-mail peers in as little as a few hours to a few days; and this happens automatically
to quickly and accurately discriminate between unwanted e-mail from unknown senders and valuable e-mail from your established e-mail peers.
SpamDupe accomplishes this task through adaptive learning. SpamDupe watches all e-mail traffic and quickly learns who e-mails whom. Once e-mail relationships are established, SpamDupe auto white lists the e-mail peer.
SpamDupe can discover e-mail peer relationships between active e-mail peers in as little as a few hours to a few days; and this happens automatically
False Positives
False Positive Legitimate e-mail that is falsely rejected by an anti-spam product.
False Negative Spam e-mail that is falsely accepted by an anti-spam product.
No e-mail scanning solution can be 100% effective. There is always a trade off between accepting spam and rejected legitimate e-mails.
Our Goal: Zero false positives
The key to achieving zero false positives is in SpamDupe history and reputation engine. By watching and recording e-mail activity, SpamDupe learns the identity of your e-mail addresses and their e-mail peers; as well as building and learning the reputations of both known and unknown mail servers.
SpamDupe learns and remembers who you e-mail, so we can let those messages through.
Quarantines
A quarantine is a holding area for e-mail. Anti-spam filters use quarantines when they cannot decide what to do with an e-mail. This creates a problem of E-mail Delivery Uncertainty. The anti-spam solution is uncertain about the disposition of the e-mail (spam or legitimate?) and the sender and recipient are uncertain about the delivery of the e-mail.
“Where is my e-mail from … ???”
“Why did … not receive my e-mail???”
Because SpamDupe is a live filtering solution it eliminates the problem of e-mail uncertainty. If SpamDupe accepts the e-mail it is delivered. If SpamDupe rejects the e-mail, this is done during message transmission – guaranteeing the sending server receives the reject status, which is then passed to the sender.
(Of course we can only guarantee delivery of the delivery status to your mail server; we have no control over what your mail server does with this information. In most cases a delivery message is sent to the sender, though sometimes these messages get discarded. This is a local e-mail setting, so your local e-mail administrator can help you with this, if this is a problem you are experiencing.)
Requiring users to check a quarantine for messages is a false economy. The user still needs to review their spam messages and they may have to do it using a separate application or website! All the quarantine has done is added a layer of complexity to checking e-mail. Many users will not check their personal quarantine – EVER. Messages held there are forever lost.
SpamDupe attempts to de-emphasize the quarantine! Our uncertainty rate is small enough that simply forwarding messages with uncertain dispositions removes the need of a quarantine. The trade-off is having a couple of spam messages in your in-box. This compromise ensures no e-mail gets lost. That is e-mail certainty!
Instead of a quarantine SpamDupe offers two simple End-User Empowerment Tools: E-mail Activity Reports and our Self-Service Console. These tools give you more than your quarantine; they give you the ability to manage all of your e-mail.
E-mail Recovery
Rather than having a quarantine, SpamDupe has a short term message storage facility where it keeps a copy of all e-mail that has passed through it, including a copy of most of the e-mail that
Anti-Virus
Anti-Virus software examines your e-mail for known rogue software, including computer viruses, worms and dangerous files. Additionally our anti-virus engine has been leveraged to identify known phishing scams and social engineering scams. All such content has the potential to harm your users and your company.
Additionally, SpamDupe lets you to filter out e-mail attachments which may be dangerous to your users.
SpamDupe offers professional grade anti-virus filtering with ClamAV. To stay up-to-date your SpamDupe product checks for virus signature updates automatically every ten minutes.
Anti-Virus software examines your e-mail for known computer viruses and other rogue software, including worms and dangerous files. In addition, our anti-virus engine has been leveraged to identify known phishing scams and social engineering scams. All such content has the potential to harm your users and your company.
Additionally, SpamDupe lets you to filter out e-mail attachments which may be dangerous to your users.
SpamDupe offers professional grade anti-virus filtering with ClamAV. To stay up-to-date your SpamDupe product checks for virus signature updates automatically every ten minutes.
Important: SpamDupe anti-virus filtering cannot replace your desktop anti-virus software. Sometimes e-mail can contain web links to viruses and executable files that SpamDupe simply can’t block. We check all web links against a list of known malware sites, but this kind of filtering is far from perfect. You still need to rely on your desktop anti-virus software to analyze and block viruses and malware at the desktop.
rejected.
Not only can you release messages that may have been inadvertently rejected, but you can also resend messages that may have been lost for some other reason.
In fact, if you lose your entire mail server you can recover your e-mail with our Message Replay Wizard. This wizard will let you select and simply re-transmit any lost e-mail activity.
If your mail server does stop functioning, SpamDupe will spool up your e-mail and automatically forward it to your mail server when it comes back up. Then if necessary, you just replay the lost time period. Easy!
Training the Filters
As SpamDupe is mostly self-tuning, there are few tuning chores to distract administrators from their other duties. In fact, most SpamDupe products are run lights out (without administrator involvement) after their first week of service.
SpamDupe’s inherent accuracy is enhanced by its embedded reputation system. SpamDupe’s reputation system helps ensure the highest overall accuracy (typically better than 99.9+% and zero false positives). SpamDupe auto-discovers protected e-mail users and peers as well as legitimate and malicious mail servers. It watches live activity to make the best overall decision.
Let SpamDupe watch your e-mail traffic, both inbound and outbound, for about a week. After that use the web interface to review how your messages have scored. Look at the messages that score around your reject and tag thresholds. After about a week you can start lowering these thresholds. Lower the scores by about 2 points. Depending on your traffic, this will have a huge impact on the amount of spam that comes through your system. Then for the following couple of weeks perform this exercise again, dropping the scores 1 or 2 points, watching for any false rejects, until you find an acceptable setting.
Threshold Values
Administrators must assign values for the Tag and Reject thresholds for each domain protected by SpamDupe. It is common practice to start with higher values, to ensure no false positives (legitimate mail rejected as unwanted) and then adjust values down over time. Higher initial values will allow some amount of unwanted e-mail (spam) to sneak in under the Tag and Reject thresholds. Over time, reduce these to safe long-term values for Tag and Reject thresholds.
SpamDupe’s reputation system will learn your users and their peers with a few days to a few weeks of service. Because SpamDupe strongly favors users and peers with an established reputation, it is safe to reduce Tag and Reject thresholds without the risk of introducing false-positive scores.
Optimal settings need to be determined empirically because each SpamDupe interacts with a unique set of users, mail peers and mail servers. To assist you, we suggest the following settings based on our own experience with the product:
| Tag | Reject | |
|---|---|---|
| Initial Deployment or for each new Domain | 16 | 26 |
| Retail ISP and non-business settings | 14 | 24 |
| Safe long term settings | 12 | 22 |
| More aggressive long term settings | 11 | 18 |
Note that scores have no meaning other than to indicate the magnitude of suspicious or undesirable activity discovered within a message. The overall range of scores that you might encounter is –50 or less for messages between peers with well established history, to 50+ for messages from one-time senders of strongly objectionable content.
Tracking E-mail with Message IDs
Each e-mail message is assigned a unique Message ID (e.g. o7IBY66I013204) by each mail server the message is relayed through. Often these Message ID’s are recorded by the mail server in a Received: e-mail header. The SpamDupe message view page shows the Message ID for each message at the top of the page.
You can use the Message ID to locate e-mail exchange details in the Raw Log for a specific message; or inversely use the Message ID from the Raw Log to locate and display the contents of an actual e-mail.
When tracking the progress of an e-mail, take a close look at the “stat=” portion of log entries in the Raw Log. The “stat=” will say what happened to the e-mail and will often give the Message ID that was assigned to the message on the mail server it was sent to.
E-mail Activity Reports
E-mail Activity Reports show a digest view of your e-mail. The report is typically sent once a day, though this can be configured by your SpamDupe Administrator.
Each e-mail is assigned a numerical score, generated by our anti-spam engine. The initial score of a message is “0”. We use many techniques to scan each message to see how spammy it is. The total value of the tests is the spam score of the message. The larger the spam score, the more spammy the message is; the lower the spam score, the less spammy the message is.
The E-mail Activity Report allows you to manage your e-mail directly from your e-mail client or by clicking through to use our web-based Self Service Console. The action icons in this report will allow you to view, release and resend messages, as well as allowing you to report e-mail as spam to both your server and our spam clearing house for further analysis.
E-mail Message Groups
All e-mail messages are analyzed by our anti-spam/anti-virus engine and are classified into the following general groups.
Each e-mail is assigned a numerical score, generated by our anti-spam engine. The initial score of a message is “0”. We use many techniques to scan each message to see how spammy it is. The total value of the tests is the spam score of the message. The larger the spam score, the more spammy the message is; the lower the spam score, the less spammy the message is. We have two thresholds, that determine what happens to each message.
The tag threshold (normally set to 12) is the spam score where we are uncertain if the message is spam. Messages are tagged on their subject line with a short message to say it may be spam. (This can be configured by your SpamDupe Administrator.
The reject threshold (normally set to 20) is the spam score we are we are certain a message is spam. The message will be quarantined on the server for a period of time set by your SpamDupe Administrator. To release a quarantined message use the release action link on your E-mail Activity Report or release the message using the Self Service Console.
Quarantined Mail – Messages that score as spam via their content score or for other reasons that are not definite are quarantined by the server. These messages are stored on the server and can be released if necessary. In all other aspects the behavior is the same as for Rejected Mail; the sending mail server receives a failure notice, during transmission, stating that the message was blocked for spam. Quarantined messages are stored on the SpamDupe server for a limited period of time, typically one week by default.
Refused Mail – Messages that were absolutely refused. These messages contain dangerous content such as viruses or executable content. This content is not stored by SpamDupe and cannot be inadvertently released.
Rejected Mail – Messages that received a spam score high enough to be rejected by SpamDupe. These messages can be viewed and released using the Administrative Interface, the Self Service Console and the E-mail Activity Report.
Delivered Mail – Messages that were accepted and delivered. These messages can be viewed, resent or reported as spam using the Administrative Interface, the Self Service Console and the E-mail Activity Report.
Outbound Mail – Messages that were sent out to the internet. These messages can be viewed and resent using the Administrative Interface, the Self Service Console and the E-mail Activity Report.
Internal Mail – This is a special group to track internal messages in SpamDupe. Normally, SpamDupe only tracks e-mail going to and from the Internet. An optional setting allows your mail server to forward all internal e-mail to SpamDupe for informational purposes only. These messages can be viewed using the Administrative Interface.
E-mail Filtering Results
Within each Message Group e-mail can be further classified by its Filtering Result.
Quarantined Mail
Content-Block – Message was blocked due to content analysis.
Word-Block – Message was blocked for content, with a strong score for words or phrases contained in the e-mail.
SPF-BLock – Message was blocked due to a Sender Policy Framework validation failure. SPF can generate false positives occasionally, so these messages are quarantined rather than rejected.
Rejected Mail
RBL-Block – Message was rejected because it originated from a known spam source.
Spam-Trap – Message was blocked because it targeted a spam trap account.
Host-Block– Message was blocked because the sending e-mail server was listed on a local black list on the SpamDupe server.
Sender-Block – Message was blocked because the sending e-mail address was listed on a local black list on the SpamDupe server.
Outbound-Block – The outbound message was blocked because it either contained content the server is configured to explicitly reject; or if the “from” address is from other than the hosted (or additionally specified) domains. These settings are configurable in the Administrator’s Web Interface.
Refused Mail
Virus – Message was refused outright because it contained a virus.
Attachment-Block – Message was refused outright because it contained a dangerous executable attachment.
Delivered Mail
Accept – Message was accepted and delivered.
WhiteList – Message was accepted and delivered and exists on a local white list on the SpamDupe server.
Tag – Message content was questionable. The message may have been tagged as possible spam on the subject line; however it was still accepted and delivered.
Reported-Spam – Message was previously accepted but has since been reported as spam. This message was used for training purposes on the local server and sent to a central clearing house for further analysis.
Outbound Mail
Outbound – Messages that were sent out to the internet.
Internal Mail
Internal – This is a special group to track internal messages in SpamDupe. Normally, SpamDupe only tracks e-mail going to and from the Internet. An optional setting allows your mail server to forward all internal e-mail to SpamDupe for informational purposes only.
Action Links and Notifications
The E-mail Activity Report allows you to manage your e-mail directly from your e-mail client or by clicking through to use our web-based Self Service Console. The action icons in this report will allow you to view, release and resend messages, as well as allowing you to report e-mail as spam to both your server and our spam clearing house for further analysis.
Self Service Console – The Self Service Console is a web-based interface that lets you manage your e-mail. The Self Service Console extends the information and functionality of the E-mail Activity Report, giving you access to all of your e-mail activity and letting you change the format of your E-mail Activity Report. The E-mail Activity Report provides a link for accessing this interface.
View Message – Open the e-mail in a new web browser window or tab. From this window you can view, release/resend, view the scoring information and report the message as spam.
Resend/Release Message – The e-mail will be resent or released from the message quarantine. The status for this e-mail will not change in the current E-mail Activity Report (which is a static report.) Further reports will show the message as being sent as will the Self Service Console.
Report Message as Spam – Report an e-mail as spam. This will mark the message as spam and help to train the content filters on your SpamDupe server. Further, a spam report will be sent to our spam clearing house for further analysis. The status for this e-mail will not change in the current E-mail Activity Report (which is a static report.)
Message Transfer was Encrypted – This is not an action link, it is a notification that the message was transferred using an encrypted transport mechanism. This is an important indicator for organizations where secure e-mail transport is necessary.
Reporting Spam
You can help in the fight against spam! How? Report it to your anti-spam provider. We rely on your feedback. At SpamDupe we offer several ways to report spam so that reporting can have an immediate impact on your local spam server and assist in our development activities.
A steady stream of spam feedback helps us to develop new tests and gives us a feel of what is actually getting past our filters in the real world.
The simplest way to report spam is to forward your spam messages to spam@spamdupe.com Our preference is to forward the messages rather than including them as attachments, but both ways will work.
The best way to report spam is using the web-based user interface. The Mail Log lets you select messages using check boxes to report them as spam. Some administrators review their messages on a daily basis and report spam messages as they find them. For end-users an E-mail Activity Report and Self Service Console are also available so they can select and report spam without having to rely on their e-mail administrator.