Re: Multiple AV Vendors ignoring tar.gz archives

[James Eaton-Lee <james.mailing () gmail com>]

First off, thanks for the e-mail! It was well argued, and you obviously
took a lot of time on it; this is much appreciated. With that, let the
reply begin..

On Mon, 2005-02-07 at 15:32 -0500, bkfsec wrote: 
> James Eaton-Lee wrote:
>
> > For many SMEs, the distinction is irrelevant, as a significant number of
> > e-mail servers do *NOT* incorporate antivirus software designed with
> > gateway scanning in mind - they run desktop scanning tools on e-mail;
> > thus, for many companies, the distinction between 'gateway' and
> > 'desktop' antivirus software is both, since one scanning engine and set
> > of definitions play the same role. 
> >  
> I think that the distinction that Nick was making was that any AV that 
> is intended to do gateway scanning should implement this, which is 
> implied by his whole "gateway scanners may have a problem with this..." 
> point. 
>
> If corporations are using desktop scanners as gateway scanners, then 
> they're misusing the product. 
>
> I could try to tow 3 tons of bricks with my little Honda Civic, but 
> would it be Honda's fault if my engine gave out?  I'd be misusing the 
> product. 'nuff said.

A good point - and one which is worth addressing. Obviously, arguing 'ad
absurdum', this point becomes irrelevant (as, in fact, do most points) -
but the devil is in the detail, especially in this particular case; I've
read through the documentation (which I happen to have in my office)
which comes with three extremely common corporate antivirus products
this afternoon (Norton, CA InnoculateIT, and Sophos SBE), and although
admittedly I've been busy and I *haven't* read through *every* piece of
paper and manual which comes with any of them (and I haven't read any of
the documentation on the vendors websites), I haven't found a single
paragraph concerning use in this particular manner. 

If a particular piece of antivirus software isn't designed for a
specific role, then vendors need to make this obvious in order that they
can avoid, as you put it, "misusing the product". Failure to do so is
the vendor's problem, not the user's. (No, it's not common sense.)

This swings both ways; IT staff shouldn't use products in a way which is
likely to backfire - but at the same time, especially in departments on
tight budgets, make-do-and-mend is practically a life philosophy. Some
vendors are more open than others about allowing their customers to
'hack' their product; sophos in particular have been extremely
supportive over the phone when I've 'hacked' their product using
vbs/perl/python (note: this may be, and is almost certainly a first-line
policy rather than a corporate one), as have several other major
software vendors. 

Many vendors *expect* their customers to write their own solutions,
scripts, and hacks based around their software packages - Microsoft have
one scripting language (vbs) which is really only used to hack
windows(!). If companies don't extend the same flexibility to their
product usage as others, they need to make thus abundantly clear.

> > Antivirus technology is something which even non-technical office staff are very
> > much aware of, and they base many aspects of their work on assumptions
> > such as the fact that if an antivirus scanner has not detected 'a virus'
> > in a file they have sent/downloaded/copied, that it is safe - although
> > they may not be at risk from a virus in an archive file that their
> > antivirus software does not detect, other people may. 
> >  
> Well, this is largely a perception problem.  People think that a clean 
> scan means that something is safe and that's wrong.  It's not just wrong 
> in AV, it's wrong in all  security analysis issues.  It's wrong in IDS.  
> It's wrong in forensics.  It's wrong in pen-testing.

Like I said, peoples use of software is based on assumptions; this
assumption in particular (that a clean scan == safe) has had very little
attempt made to dispel it. You know what exactly a 'clean scan' means, I
know what it means, and half of the users on full disclosure probably do
- but this doesn't mean that users do. Users don't have technical staff
staring over their shoulder 24/7, and they cannot (and should not) phone
up a helpdesk each time they have a niggling query about what exactly
something like this means; this is another issue where, I believe, if an
antivirus vendor doesn't mean X by Y (where Y is a product
feature/message and X is an assumption about what the product has done
or is telling the user), then they need to say so! (They don't!)

For a user who knows that "the antivirus software detects viruses and
quarantines them", the statement "no viruses have been found on your
computer" can be extrapolated to mean "There are no viruses on your
computer", since a package designed to detect viruses not having
detected viruses must mean that there aren't any viruses.. right?
(Again, I know this isn't true, so do you. They don't. This is the
point.)

In fact, such is the hysteria surrounding viruses at the moment that we
as an industry are very bipolar about how we portray security. Again, I
know what these concepts and terms mean and so do you - but (as an
industry), we don't preach these concepts to users. (disclaimer: you
might, the industry as a whole doesn't). Home users in particular have
had antivirus software and firewalling shoved down their throats in the
last 18 months - with little explanation of what these things involve
and what protection they afford. The lacklustre effort to actually give
people a holistic understanding of how things work has resulted in a set
of common misconceptions (such as firewall == no hackers, clean virus
scan == no viruses, etc). This *IS* a lack of understanding on the part
of the user, but they are not the expert on viruses, antivirus vendors
are. 

> What the outcome really means is, literally, that nothing that the 
> product was designed to detect was detected.  It means nothing more and 
> nothing less.
> However, people turn that into "the coast is clear" because people don't 
> want to live in a constant state of paranoia and fear.  By their nature, 
> security and usefulness have to be balanced, at least in this way.

Agreed with you on the first point; I've covered the second already; as
far as I'm concerned, people are at fault here, but they have no way of
educating themselves on these issues; the security industry has to do it
for them, and sitting on our hands and agreeing with each other that
people are not using software properly, and that they need to educate
themselves or buy another software package simply isn't going to
accomplish anything. You can pick your philosophy here; be right and let
people burn or marginally compromise your philosophical principles and
do some good.

> However, this all comes down to one point: If the AV can detect the 
> malware uncompressed, but can't detect it compressed, then there's no 
> problem.  The malware has to be decompressed to be dangerous.  That was 
> Nick's point and it's 100% correct. 

Yes and no - if the AV can detect the malware uncompressed, there's less
of an issue. But the malware really shouldn't make it onto the network
in the first place - as I pointed out, clients are fallable, servers are
less so, and therefore security measures should be kept as
server-orientated as possible if businesses are to have maximum
security. I think I'm taking the braces and belt line here. 

> IF your AV software is functioning normally.
> IF your AV software has proper real-time detection capabilities.
> IF your AV is properly setup and scans the programs you run at the time 
> they're read from the HD.
> IF your AV will detect the malware uncompressed.
>
> Then, as should be true for the vast majority of situations out there, 
> the malware will be caught as it's being extracted from the archive.  
> Or, barring detection on writes, when it's being executed in the first 
> place.

If, If, If. This is a chain argument, and as soon as you break one link
in a chain argument, the conclusion fails to be valid! I'd far rather
have a situation where If X then Z, or if Y then Z. Failure of client
antivirus software should not leave clients open to viruses - without
compression support (and lots of other things), I believe that under
certain circumstances, it does.

> If the problem you're pointing out is that SMEs are carrying out 
> cost-cutting by not putting AV on their workstations and blindly relying 
> on gateway scanning,  then that SME has a much bigger set of problems 
> than not having compressed tarball support on their gateway scanner, and 
> their cost-cutting is ultimately going to cost them.
>
> That SME has made a grave mistake and hopefully they'll learn their lesson.

This is the same argument as before; I know that SMEs shouldn't do this,
and you know it too - but the fact is that they do. Security
professionals and vendors need to consider *all* eventualities, and not
rely on *one* method of doing everything based on the idea that if
anyone is doing any of this differently, they're wrong and they'll pay
the price. The job of a security professional isn't to consign companies
to security hell for political reasons - it's to make that company more
secure, whether or not they like the way in which they're doing it.

> > Harking back to SMEs, who seem to be at the focus of most of the points
> > that I've made, it's quite possible that the inability to scan an
> > archive file could be extremely damaging to a business's reputation when
> > forwarded to a partner or customer
> In what situation can you imagine where a person blindly forwards 
> compressed (unscanned) content to a business partner?

Virtually every situation. People are stressed, lazy, and ignorant, and
they forward e-mails to other people *all the time*. But this isn't the
point - stupidity shouldn't be what causes network security to fall down
- network security should be *STUPIDITY INDEPENDENT*. This axiom
capitalised in order to promote my new line of 'Stupidity Independent'
network appliances and software. (Ok, that was less than funny, but this
is a long and serious e-mail and it needs punctuating in what has
otherwise been a rather pompous and arrogant thread in the mailing
list).

As an example of this point, consider this: Would you make every user on
your network a Domain Administrator based on the logic that if they
break anything it's due to stupidity and therefore their problem and not
yours? Like it or not, this is basically the same logic at work.

> Again, this can only be because of cost-cutting issues at the SME or 
> laziness on the part of the SME's employee.  Again, the problem is not 
> the issue of the AV, but rather the fault of the SME for not being more 
> careful.
>
> > - since you're obviously sure of your
> > positions on these issues, I shouldn't have to remind you that antivirus
> > software isn't about being theoretically perfect, it's about preventing
> > business loss.
> >  
> This is the wrong way to think about it.
>
> The goal of antivirus is, plainly said, to detect and block malware from 
> running.
>
> Preventing business loss is a side-effect of this.  There are many 
> reasons for keeping malware off of systems, business benefit is only one 
> of them.

Companies don't buy software because it's what the IT industry likes,
beacuse it's commonly deployed, or because it looks nice (usually). The
fundamental reason for IT upgrades in businesses is because it causes
the business to run more efficiently. I buy cheap dell workstations for
my businesses in full knowledge that an apple mac running OSX is running
a superior OS on a superior platform - because for *the business*,
another workstation running windows xp is of more value than the OSX
workstation, even thought the OSX workstation is technologically
superior.

Businesses do not (should not) buy things because they're
technologically superior, they buy them because they can use them to
further the business and make more money. In the case of purchases which
*are* made on the grounds that they're technologically superior
(executive toys?), I suppose the argument would be that the
technological superiority is necessary in order to impress clients or
accomplish a goal. 

My point is, technology is *means*, not an *end*. Following this logic,
antivirus software is deployed because it provides a palpable benefit,
and for this reason, the primary goal of antivirus software *is* to
provide business continuity in the same way that a cleaner mops the
floors in order to provide business continuity. They simply go about
their tasks (the means to the end) in a different way in order to do so.

> A hammer is a hammer.  Its sole intent is to bash things (and, possibly, 
> pry them out).  It can be used to build houses, but it is not a 
> house-builder.

That's a bad example; you've taken one thing which accomplishes one
thing (antivirus software) and compared it to something which can be
used for a wide variety of tasks.

> > Antivirus software is deployed based on many sets of assumptions.
> > Failure to live up to these assumptions is generally what causes the
> > most damage to businesses as protection they thought they had in place
> > fails - this issue is something which falls into this category;
> > antivirus software is, in the majority of SMEs, implemented by staff
> > without extensive experience in antivirus software, and they are highly
> > unlikely to be aware of issues such as this one (especially since in
> > most antivirus software, the option is given to 'scan archive files',
> > not 'scan archive files apart from the ones we don't understand') - not
> > a serious issue, but definitely a significant one, and one which should
> > be fixed upstream by antivirus vendors.
> >
> >  
> It is expressly impossible to determine what the uneducated, untrained, 
> and willfully incapable of reading documentation will do when left to 
> their own devices.
>
> User-friendly software tries to cater to these users, by making things 
> as simple as possible, but that does not mean that all of these 
> conditions can be predicted.  I'm very much in agreement that AV 
> programs should support compressed tarballs and other archival formats.  
> However, any organization that is bitten by this relatively small flaw 
> will be bitten because they lack common sense.

Possibly, possibly not. How about we just avoid the problem in the first
place?

> The OEMs out there, along with the AV companies for obviously 
> self-serving reasons, have gone a long way towards trying to spread the 
> word that virus protection should be on all clients out there.  This is 
> not an arcane planning issue like, say, properly implementing an IDS.  
> It's a common sense, best practices, no BS doctrine.
>
> And there are no excuses for an organization that purposefully puts 
> themselves into a position where a minor defect like this can harm their 
> business.
>
>                 -Barry


Thankyou for your intelligent, considered e-mail! Fundamentally, I think we have the same ideas at work, but with some 
different experiences; as a side note, I'd like to mention how much I love the evolution 2.2 for catching an 
application failure and keeping 99% of the text I wrote here after it locked up 10 minutes into writing this e-mail.

regards,

 -  James.

_______________________________________________
Full-Disclosure - We believe in it.
Charter: http://lists.netsys.com/full-disclosure-charter.html