Tag Archives: spam

Just recently the results of Virus Bulletin’s VBSpam testing were released in which our new Kaspersky Linux Mail Security (KLMS) – unexpectedly for our competitors but quite expectedly for us – was among the winners – actually second – with an outstanding result of a 93.93% spam catch rate and 0.01% false positives. “Who wants to come second?” might come the refrain from those used to nothing but first place for KL. But in answer I’d say, “I do!” Here’s why…

So here we are once again on a subject that it seems will never go away – spam, this time about a particular kind thereof – “image spam” – and the protective technologies that fight it …

Hi all,

Once again, the subject is spam.

Depending on the “stars” and the time of year, the proportion of spam can range from anywhere between 70 and 90% of all email traffic.

Sounds like a lot, eh? But when you take all Internet traffic into consideration, it’s not actually that much – email traffic accounts for around just 1%. On the other hand, you can’t just forget about spam. Here is a bit more about spam’s role in the cybercrime ecosystem. Combating this particular evil is part of the massive war we are waging on cybercriminals. It’s no exaggeration to say that if we fail on this front, the rest of our efforts will amount to nothing.

In other words, we love anti-spam technologies and promote them as much as possible. There is, however, a subtle difference from anti-malware technologies. More precisely, there are different criteria for evaluating the quality of protection for anti-spam and anti-malware technologies. For malware it’s fairly easy: the higher the detection level, the better. For spam it’s more important to have no false positives. This is quite reasonable: it’s much better for the user to take a couple of seconds to delete a spam message that sneaks through the filter than miss important business correspondence. So, protection against spam is, in a way, a more complicated task, literally trying to kill two birds with one stone. In this difficult task, cloud technologies are a great help.

As I wrote earlier, we’ve been using cloud technologies for a while, and with considerable success. But one interesting detail has amazingly been overlooked, and unfairly so. In the cloud-based Kaspersky Security Network (KSN), (video, details) there’s a rather impressive anti-spam cloud. It started from the Urgent Detection System (UDS). The link to similar anti-malware technology is no coincidence: both are based on similar principles.

This is how the traditional anti-spam technology works.

Let’s say an email arrives at a computer. It is immediately assailed by various anti-spam technologies, both local and cloud-based, which test the message and give verdicts. Based on these, the system decides whether this message lives or dies.

And this is what happens in the UDS.

The system takes a micro-signature from the email message and sends it to the cloud to check it against a dedicated spam database. Earlier we used 16-byte hashes; in 2011 we started the UDS2 (UDS 2^nd generation) procedure involving 4-byte fuzzy hashes, which are more effective against obfuscated texts and are therefore better at filtering out spam. Importantly, these hashes do not create extra work for the analyst, since the system creates them automatically based on collected spam samples.

Read more: Serious ambitions for the elite 100/0 club …

Filtering out spam may not seem such a big deal – after all, even a kid knows the difference between a Viagra advert and a normal message! In the security world things are much more complicated as we have to create something akin to artificial intelligence that is capable of doing the job automatically, on the fly.

That’s no easy task and entails all sorts of demands in terms of efficiency, reliability, compatibility and so on. And you no doubt know where things stand with AI – there are plenty who claim to have got it figured, but there’s nothing really to show for it (or if there is, they’re doing a good job of keeping it a secret).

Anti-spam security is no easier a task than anti-malware protection. And may even be more difficult (or maybe I just understand more about viruses…). The spam industry is a multi-billion dollar business and tens of thousands of skilled bloodsuckers are behind the huge variety of junk that is sent out. And these parasites show great ingenuity when it comes to linguistics and other stuff to make spam reach your inbox.

On the face of it, a spammer’s work looks fairly easy – write a spam message, test it against several of the most popular anti-spam filters and spawn via a botnet.  But few customers realize that a spam message’s lifecycle is just half an hour to an hour long. 90% of a mass mailing will never reach its intended recipients – spam filters, activated with an update or triggered by statistics, will intercept it.

And it’s that black box – the thing that withstands the worst things that email traffic throws at it and keeps your inbox clean – that I want to discuss here.

First of all, a bit of background. Since 2002 our anti-spam solution (KAS) has got through four generations of engine and we’re now developing a fifth. A single blog post would hardly suffice to recount everything. Basically, KAS has acquired lots of bits and bobs over the last 10 years. It boasts over 10 methods of spam analysis alone. That’s why I’ll start with our new ‘Möbius’ technology – just in time for its debut in the latest version of KAS for Exchange Server.

Read more: Anti-spam bottleneck and how we solved it …

Try to define what you think of as spam. Most people assume it’s about Viagra, Nigerian letters and other pathetic, lame scams which jam up your inbox and slow down your daily business. But here’s the thing: spam is far more than just unsolicited ads. That Viagra offer is just the tip of the iceberg, while spam as a phenomenon is a crucial part of a huge cybercrime ecosystem. And the apparent “innocence” of spam is the illusion that I will be debunking here …