To think of all the yummy stuff the Internet has brought us, though interesting, would probably be a waste of time: by the time you’d have finished totting up all the scrumptiousness you remember, just as much new scrummyness would have appeared. But there is one particular Internet-delicacy concept that, due to its importance and value, should really never be overlooked, even in just a “Best Hits” of the Internet. This concept deserves closer consideration. And this concept is crowdsourcing.
I won’t go into loads of detail – you can get that at the other end of the Wikipedia link above (incidentally, Wikipedia is also a crowdsourcing project :) or via a search engine. Here, let me briefly go through the idea:
The WWW permits large numbers of folks from all over the world to very quickly all get together and combine efforts to solve some kind of difficult task or other. The result is collective intelligence, backed up by gigahertz, gigabytes and terabytes of computers and communication channels. Technically, its all about the sharing and allocation of computing power. For example, I remember well how at the end of the nineties many at night connected their comps to SETI@Home – a non-commercial project that searched for radio signals of extraterrestrial civilizations. The project is still going, with 1.2 million participants and a total processing power running up to 1.6 petaflops.
Perhaps surprisingly, you’ll generally find network crowdsourcing being applied in practically every sphere of life. And security is no exception. Recent examples: the international brainstorming that went into solving the Duqu Framework, and into trying to crack the mystery of the encrypted Gauss payload. (For the former, by the way, we received a rather flattering write-up on darkreading.com.) Still, these cases aren’t really the best examples of crowdsourcing at work…
The best example is probably to be found in the way we (KL) successfully process 125,000 samples of malware every day (up from 70,000 late last year). Of course, robots and other technologies of automation and data-flow analysis help, but the most important ingredient to make it all work – the statistical food – is furnished by you! Yes, you! The system’s a big you-scratch-my-back, I’ll-scratch-yours gig in which our users help both us and one another in the business of preventing cyber break-ins around the world, and in particular of tackling unknown threats. And everyone helps anonymously and voluntarily after having clearly expressed a willingness to take part; and none of it affecting computer performance!
More: Let me tell you how it works …