The Age of Dinosaurs
There is a view of the current security market that
is often recycled by the media these days. It assumes a split between
‘first-gen(eration)’ or ‘traditional’ (or even ‘fossil’ or ‘dinosaur’) malware
detection technology – which is invariably claimed to rely on reactive
signature detection – and (allegedly) superior technologies using
‘next-gen(eration)’ signature-less detection. This picture is much favored by
some ‘next-gen’ companies in their marketing, but it doesn’t reflect reality.
The Theory of Evolution
First of all, I’d take issue with that term
‘first-generation’. A modern mainstream security suite can no more to be lumped
in with early ‘single layer’ technologies – such as static signature scanners,
change detection and vaccines – than Microsoft Word can be with ed or edlin. They may have the same fundamental
purpose as those long-gone applications – be it detection and/or blocking of
malicious software, or the creation and processing of text – but they have a
much wider range of functionality. A modern word processor incorporates
elements that decades ago would have been considered purely the domains of
desktop publishing, spreadsheets and databases.
The Origin of Species
A modern anti-malware-focused security suite isn’t
quite so wide-ranging in the programmatic elements it incorporates.
Nevertheless, it includes layers of generic protection that go far beyond
signatures (even generic signatures). They have evolved into very different
generations of product, incorporating technologies that didn’t exist when the
first security products were launched. To talk about newcomers to the market as
if they alone are ‘the next generation’ that goes beyond primitive
signature-specific technology is misconceived and utterly misleading.
Signatures? What signatures?
Nowadays, even modern, commercial single-layer
anti-malware scanners go far beyond looking for specific samples and simple
static signatures. They augment detection of known, hash-specific families of
malware with the inclusion of elements of whitelisting, behavior analysis,
behavior blocking, and change-detection (for instance) that were once
considered to be pure ‘generic’ technologies. Not that I recommend in general
that people should rely totally on a single-layer scanner such as those often
offered for free by mainstream companies: they should be using other ‘layers’
of protection as well, either by using a commercial-grade security suite, or by
replicating the multi-layered functionality of such a suite, while using
components drawn from a variety of sources, including a single-layer
anti-malware scanner. However, the latter approach requires a level of
understanding of threat and security technologies that most individuals don’t
have. Come to that, not all organizations have access to such a knowledgeable
resource in-house, which leaves them potentially at the mercy of marketing masquerading
as technical advice.
Back to basics
“It’s clear that the
distinctions between ‘fossilized’ and ‘next-gen’ products are often
terminological rather than technological.”
Although some next-gen products are so secretive
about how their technology actually works that they make mainstream
anti-malware products look like open source, it’s clear that the distinctions
between ‘fossilized’ and ‘next-gen’ products are often terminological rather
than technological. I don’t consider that ‘next-gen’ products have gone further
beyond these basic approaches to defeating malware, defined long ago by Fred Cohen (whose introduction and definition of the term
‘computer virus’ in 1984 to all intents and purposes jump-started the
anti-malware industry), than have ‘traditional’ solutions:
·
Identifying
and blocking malicious behavior
·
Detecting
unexpected and inappropriate changes
·
Detecting
patterns that indicate the presence of known or unknown malware
The ways of implementing those approaches have, of
course, become immeasurably more advanced, but that progression is not the
exclusive property of recently-launched products. For example, what we
generally see described as ‘Indicators of Compromise’ could also be described
as (rather weak) signatures. More than one vendor has failed to differentiate
convincingly between mainstream anti-malware use of behavior analysis and blocking,
between its own use of (for instance) behavioral analysis/monitoring/blocking,
traffic analysis (and so on) and the use of the same technologies by mainstream
anti-malware. Instead, they’ve chosen to promote a deceptive view of ‘fossil
technology’ and peppered their marketing with a hailstorm of technological
buzzwords.
Welcome to the machine
Consider, for instance, the frequent lauding of
‘behavior analysis’ and ‘pure’ Machine Learning (ML) as technologies that set
next-gen apart from first-gen. In the real world, Machine Learning isn’t unique
to one market sector. Progress in areas like neural networking and parallel
processing are as useful in mainstream security as in other areas of computing:
for example, without some degree of automation in the sample classification
process, we couldn’t begin to cope with the daily avalanche of hundreds of
thousands of threat samples that must be examined in order to generate accurate
detection.
“The use of terms like
‘pure ML’ in next-gen marketing is oratorical, not technological.”
However, the use of terms like ‘pure ML’ in
next-gen marketing is oratorical, not technological. It implies not only that
ML alone somehow provides better detection than any other technology, but also
that it is so effective that there is no need for human oversight. In fact,
while ML approaches have long been well-known and well-used in the mainstream
anti-malware industry, they have their pros and cons like any other approach.
Not least, in that the creators of malware are often as aware of ML as the
security vendors who detect malware, and devote much effort to finding ways of
evading it, as is the case with other anti-malware technologies.
On your best behavior
Similarly, when next-gen vendors talk about
behavioral analysis as their exclusive discovery, they’re at best misinformed:
the term behavioral analysis and the technologies taking that approach have
both been used in mainstream anti-malware for decades. In fact, almost any
detection method that goes beyond static signatures can be defined as behavior
analysis.
Natural and unnatural selection
Is there any way that the industry can help the
user compare and choose between 1st […] and 2nd generation […] for the
detection of malware?
Leaving aside the totally misleading 1st versus
2nd-generation terminology, yes, of course there is. In fact, some of the
companies self-promoted as ‘2nd-generation’ and claiming that their technology
is too advanced to test have nevertheless pushed an already open door even
wider by their own attempts to compare the effectiveness of their own products
and those of ‘first-gen’ vendors. For example, at least one next-gen vendor has
taken to using malware samples in its own public demonstrations: if different
generations of product can’t be compared in an independent test environment,
how can such demonstrations be claimed to be accurate in a public relations
exercise? Other misleading marketing from next-gen vendors includes claims that
“1st-gen products don’t detect ‘file-less’ malware in memory” (which we’ve done
for decades). One particularly inept example used a poorly constructed survey
based on Freedom of Information requests to ‘prove’ ‘traditional’
anti-malware’s ‘abject failure’ without attempting to distinguish between
attacks and successful attacks.
Testing and Pseudo-testing
More commonly, VirusTotal (VT) is misused by
misrepresenting its reports as if VT and similar services are suitable for use
as ‘multi-engine AV testing services’, which is not the case. As VT puts it:
VirusTotal should not be used to generate
comparative metrics between different antivirus products. Antivirus engines can
be sophisticated tools that have additional detection features that may not
function within the VirusTotal scanning environment. Because of this,
VirusTotal scan results aren’t intended to be used for the comparison of the
effectiveness of antivirus products.
VT can be said to ‘test’ a file by exposing it to a
batch of malware detection engines. But it doesn’t use the full range of
detection technologies incorporated into those products, so it doesn’t
accurately test or represent product effectiveness. One nextgen vendor talked
up its own detection of a specific ransomware sample a month before the same
sample was submitted to VirusTotal. However, at least one
mainstream/traditional vendor was detecting that hash a month before that
next-gen detection was announced. You simply can’t measure a product’s
effectiveness from VirusTotal reports, because VT is not a tester and its reports
only reflect part of the functionality of the products it makes use of.
Otherwise, there’d be no need for reputable mainstream testers like Virus Bulletin, SE Labs, AV-Comparatives and AV-Test, who go to
enormous lengths to make their tests as accurate and representative as possible.
Towards cooperation
One of the more dramatic turnarounds in 2016 took
place when VirusTotal changed its terms of engagement
in order to make it harder for next-gen companies to benefit from access to
samples submitted by “1stgen” companies to VirusTotal without contributing to
VT themselves. To quote VirusTotal’s blog:
…all scanning companies will now be required to
integrate their detection scanner in the public VT interface, in order to be
eligible to receive antivirus results as part of their VirusTotal API services.
Additionally, new scanners joining the community will need to prove a
certification and/ or independent reviews from security testers according to
best practices of Anti-Malware Testing Standards Organization (AMTSO).
While many vendors in the next-gen space initially
responded along the lines of “It’s not fair”, “The dinosaurs are ganging up on
us”, and “We don’t use signatures so we don’t need VT and we don’t care”, it
seems that several big names were subsequently prepared to meet those
requirements by joining AMTSO and thus opening themselves up to
independent testing. (By that I mean real testing, not pseudo-testing with
VirusTotal.) Since next-gen vendors have tended in the past to protest that
their own products cannot be tested, especially by the ‘biased’ testers
represented in AMTSO, perhaps this suggests the possibility of an encouraging
realization that not all customers rely purely on marketing when they make
purchasing decisions.
Share and share alike
“Vendors (of any
generation) benefit from access to VirusTotal’s resources and that huge sample
pool.”
Why have (some) next-gen vendors now decided that
they do need to work with VirusTotal? Well, VT shares the samples it receives
with vendors and provides an API that can be used to check files automatically
against all the engines VT uses. This allows vendors not only to access a
common pool of samples shared by mainstream vendors, but to check them against
indeterminate samples and their own detections, thereby training their machine
learning algorithms (where applicable).
And why not? That’s not dissimilar to the way in
which longer-established vendors use VirusTotal. The difference lies in the
fact that under the updated terms of engagement the benefit is three-way.
Vendors (of any generation) benefit from access to VirusTotal’s resources and
that huge sample pool. VirusTotal benefits as an aggregator of information as
well as in its role as a provider of premium services. And the rest of the
world benefits from the existence of a free service that allows them to check
individual suspect files with a wide range of products. Widening that range of
products to include less-traditional technologies should improve the accuracy
of that service, while the newer participants will, perhaps, be more scrupulous
about not misusing VT reports for pseudo-testing and marketing when they
themselves are exposed to that kind of manipulation.
Whole-product testing
The way that AMTSO-aligned testers have moved
towards ‘whole-product testing’ in recent years is exactly the direction in
which testers need to go in order to evaluate those less ‘traditional’ products
fairly. (Or, at any rate, as fairly as they do mainstream products.) It can be
argued, though, that testers can be conservative in their methodology. It’s not
so long ago that static testing was the order of the day (and to some extent
still is among testers not aligned to AMTSO, which has discouraged it since the
organization’s inception). AMTSO, despite all its faults, is greater (and more
disinterested) than the sum of its parts because it includes a range of
researchers both from vendors and from testing organizations, and marketing
people aren’t strongly represented. Thus, individual companies on either side
of the divide are less able to exert undue influence on the organization as a
whole in pursuit of their own self-interest. If the next-gen companies can grit
their teeth and engage with that culture, we’ll all benefit. AMTSO has suffered
in the past from the presence of organizations whose agenda seemed to have been
overly-focused on manipulation or worse, but a better balance of ‘old and new’
vendors and testers within the organization stands a good chance of surviving
any such shenanigans.
But can we imagine a world without AV, since
apparently the last rites are being read already? … Would the same companies
currently dissing AV while piggybacking its research be able to match the
expertise of the people currently working in anti-malware labs?
I think perhaps we have an answer to that. But if
the self-styled next generation can come to terms with its own limitations,
moderate its aggressive marketing, and learn the benefits of cooperation between
companies with differing strengths and capabilities, we may yet all benefit
from the détente.
