Number of the Month: 70K per Day.

Anti-malware: it’s a dirty job, but someone’s got to do it. Or at least it used to be… but I’ll get to that later…

For your average Joe it can be hard to understand all the finer details of the work of an anti-malware company. But oh how we want to tell everyone about them! So we’re trying as best we can to translate them all into understandable, non-gobbledygook language – not to mention also in the English language!

The tip of the malware-fight iceberg one gets a peek at from collections of facts and figures, which illustrate the basic ins and outs of anti-malware. For example, here we have the kinds of infographics we issue on a regular basis:

[click on the image to see the details]

One of the most frequently asked questions we get is: “How many viruses do you find every day?“.

At first you may laugh – but you shouldn’t. No. Here I should remind you (forgotten your better schoolteachers’ words?) that there’s no such thing as a stupid question! So what do I do with this question – simple though it seems? Answer with a simple answer. Right? But I thought that the answer was the much bandied about figure that we’ve had for a while – 35,000 viruses a day. Or somewhere in that vicinity, on average, and without going into mind-numbing detail about malware families, polymorphism, vector patterns, peculiarities of counting database records, etc.

But since for several months now the answer to this question has normally been followed up by further enquiries seeking clarifications, as it is thought that this figure is too small, we decided to get to the bottom of this once and for all. We got down to some sums and, well, were rather astonished at what we discovered. The result necessitated an update: 70,000 daily! Just a 100% increase, but there you go.

Again, I won’t get into all the gritty details (if anyone is interested – please ask in the comments section below). I’d do better telling you how generally we get to shovel up these 70,000 samples in a day.

Many of you will know that our “family’s” talisman is a woodpecker. And this is no accident.

Back in the day our anti-virus lab worked like a woodpecker! Like on a conveyor – we sat there and pecked away at viruses. Speaking of which, it was a very difficult, fatiguing, albeit respected profession – the dirty job that someone had to do. And guess who was the first woodpecker in the company? You guessed right: for too for many years I was pecking away daily!

However, those days are long gone. With the current high-volume streams of malware we discover every day, to peck is just unrealistic and ineffective, or rather, simply daft.

Already for years, as if on red-alert combat status… we’ve been auto-woodpecking! For human input is left just the most intellectual work – examining the trickiest samples, investigating botnets and targeted attacks, and making sure that the auto-woodpeckers don’t give any false positives, and of course, to train and develop these little helpers of ours.

There are several sources from which we receive samples of malware for analysis: self-propelling ones (which get caught in special traps), “submits” (sent by users), collection exchanges with other anti-virus companies, and our cloud-based KSN service (video, details). And in terms of the contributions towards protection of users, the latter is in the lead. So here let’s have a quick look at KSN and see how it drives our automation of malware processing.

The participating computers in KSN (currently more than a cool 50 million) send to the cloud statistics (anonymously, of course – no Big Brother participation here!) on how our products operate. Specifically, the cloud collects information about detected malicious programs and infected sites and much more besides to help reveal new outbreaks – for example, suspicious activity of programs, downloaded file checksums, and more.

Let’s say a user launches a previously unknown file. The anti-virus checks it with all the tools it has – and determines the file to be clean. Then the AV asks the cloud, and the response – no data. Ok, so we give the all-clear for the launch. But then it turns out that the file somehow strangely positions itself in the system registry, tries to access system services, sets up suspicious connections, has a double extension (e.g., jpg.exe), or does something else fishy.

A signal arrives at KSN, where the system automatically evaluates the reputation of the file (based on all the signs of trouble), and then takes a decision as to detection as “bad” or “good” or “not sure – best be careful”. If “bad”, to the protected computer is sent an “attack” command, the file is blocked, and its actions are rolled back. Of course, the more messages about a file from different computers, the higher the priority of processing and the higher the accuracy and criticality of the verdict. And if a suspicious file turns up on other users’ computers plugged into KSN – they get told straight away that it’s suspect, and advised they should not take any chances.

Let’s take another example:

Several users simultaneously download a file from one and the same link. But each time the file has a different checksum. Smells not like Teen Spirit, smells like malicious polymorphism! KSN starts to untangle the nitty-gritty and sees that, for example, the site was registered just a few days earlier, there’s some kind of iframe on it, and maybe it had even sent out infected files earlier (well, that’s a lot of signals that something ain’t right). And again, the cloud gives marks out of ten for reputation and, if not so good, sends a command to block not only the file itself and everything else originating from it, but also access to the site.

Thanks to this approach, on average between detection and verdict there are just 40 seconds!

But our auto-woodpeckers don’t stop there. Another system pumps from the network the most suspicious files and hands them over for analysis to an automatic handler.

There’s an array of all sorts of patented and yet-to-be patented technologies related to all this, so I won’t dig deeper here – I’ll save that for another day. For now just let me say that this handler develops and tests the familiar-to-everyone updates and sends them to the server so that all of us can download them and stay optimally protected.

I remember eight years ago how competitors enviously admired how we had the knack of doing things right and getting results with the few resources we had to conduct such a huge amount of work. The answer: automation! But it’s only competent automation that’s able to cope with today’s crazy flow of malware! And btw, though the “battlefront” in anti-malware work is constantly expanding, we have for a year now not increased the number of employees in the anti-virus lab.

So how do small anti-virus companies survive, you may ask. For example how can an AV company with as few as just two virus analysts – there is such a company! – handle this avalanche of new malware?

It’s taken as read that to run a good anti-virus lab you need not only money but also brains. So how do they find the resources to keep afloat, while spending little on R&D?

This is a big topic.

For several years in the anti-virus industry “detection adoption” has been flourishing. Instead of analyzing malware, developing home-grown expertise and nourishing new technologies, some (that is, around ten) companies simply peep at the results of the hard work of others and blindly add to their databases detections based on infected files’ checksums.

And they are helped along by incompetent tests that don’t reflect protection levels in real-world conditions. As a result, coming out top of the class is not the very best software, though the quantity of its installations increases as a result of the bogus gold medals, and the overall global level of protection falls. Honest companies dedicated to genuine, painstaking anti-malware research may lose motivation to conduct research as the effectiveness of investment in R&D falls, but sales of the copy-cats’ products rise, and cybercriminals rejoice.

But this is a topic for another post…

Despite this gloomy picture we still continue our work and keep investing in anti-malware R&D. And we’re planning a lot of new technologies in next year’s releases. At the same time, I’ve no idea at the moment how to protect our investments and avoid detection adoption, which naturally thwarts the fight against cybercrime.

To finish on a merrier note, here are some pics of what some of our woodpeckers look like (some of them are still available in our merchandizing store!):

And even a #kozmo woodpecker:

BTW, if anyone has more KL branded woodpeckers I missed here, please send me links below in the comments or use the contact form. I will be regularly updating my collection, available here.

Do svidaniya, comrades! Have a nice weekend!