Re: Firewalls - ITSEC Rating?

[John Alsop <jalsop () borderware com>]

At 08:30 AM 02/03/2000 -0500, Marcus J. Ranum wrote:

> I'm sure that many on this list will be shocked to hear me say
> this, but the ICSA firewall product certification is orders of
> magnitude more valuable to real customers than ITSEC evaluation.

My company (BorderWare Technologies Inc.) develops a firewall that is
both ICSA and EAL4 certified, and based on our experience with the
process, I must disagree with the above statement.

ICSA certification consists primarily of "black box" testing. i.e. a set
of tests is performed against the target firewall, and the results are
used to determine whether it meets the criteria defined by ICSA.  There
are 42 firewalls listed as being certified by ICSA as of Jan 31, ranging
in functionality from Cisco's IOS firewall feature set all the way to
high-end firewalls. ICSA certification does not include evaluation of
the vendor's internal processes or the vendor specific feature and
function claims.

Common Criteria certification, which is the latest incarnation of ITSEC,
involves a much more rigorous and in-depth analysis of the target
product.  This includes design and architecture, development processes
and security, software QA processes, and obviously, penetration testing.

Unlike the older ITSEC certification process, the Common Criteria
process involves evaluating the target product against objective
security parameters for the type of product, in addition to vendor
specific claims.

Paul Emerson wrote:
> ITSEC is really quite pitiful. For example FW-1 was evaluated and 
> passed E-3, but the GUI was not included with the target.  So I guess 
> in order to use FW-1 as evaluated the GUI should not be used.

This is a valid comment, and illustrates the point that customers should
not blindly accept any certification without checking what is actually
covered.

In the case of the BorderWare Firewall Server, we have published the
scope of our EAL4 certification for public review on our web site
(http://www.borderware.com/certifications.html).  Both the GUI and
underlying secure operating system are included in our certification.
i.e. the product in its normally used mode of operation on generic Intel
hardware is fully certified.

To go back to Marcus' observation, I would certainly agree that EAL4
certification is orders of magnitude harder and more expensive to get
than ICSA; however it is possible to certify a fully functional
commercial firewall, and the result does provide a significantly higher
level of assurance to customers.

--
John Alsop
President & CEO
Borderware Technologies Inc.
jalsop () borderware com
Tel: 905-804-1855 x223 Fax: 905-804-1865

Attachment: smime.p7s
Description: S/MIME Cryptographic Signature