Re: [PEN-TEST] Resend: Fwd: Re: IDS identification and a personal cry for help :)

[Peter Van Epp <vanepp () SFU CA>]

<snip>
> > and either no management connection or a properly isolated management
> > connection (i.e. no connection to the Internet) it really doesn't matter
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
<snip>

        I think by and large we agree. In the case of argus, a complete record
of your network activity is extremely sensitive data. That means a shomiti
isolating the sniffing port(s) from responding to any external stimulus at
all (at least back to the attacker, although blind attacks may be possible,
although first you have to find it). Obviously having an open management port
on the inside is almost as bad as having the sniffing port accessable. I'd
advocate a physically private internal network (or an ssh or ipvsec tunnel)
between your sensor and an as heavily secured (i.e nothing except sshd on the
management station and sshd and ntpd on the sensor, no inetd, portmapper, or
any other daemon running on either machine) as the mimimum acceptable security
posture to run something like this. There is no reason for Internet access
on the management machine, it is for the analyst's convienience of not having
to physically go to the sensor (I have several campuses a long way apart
physically for instance). If you aren't willing to do that you aren't serious
enough about security and probably shouldn't be running something as sensitive
as an argus system (an IDS may be less sensitive since it doesn't record
everything on your net, although it is still extremely sensitive in terms of
what you have no chance of detecting ...). It is inconvienient but not
particularly expensive to properly secure an IDS installation. You should have
enough other equally secure machines around to make it non trivial to figure
out which (other than all) of the secure machines are of interest. The security
of those machines should be such that they are all hard to compromise and
compromising one doesn't help with compromising another (atlhough thats hard
to make true in all cases). As I said, ideally it is on a physically seperate
network with no access from the Internet to make the job even harder. You are
correct the management ports are as much a threat and need as much attention
to security as the sniffing port, I guess I had assumed this group probably
would imply that from the comment about isolating the management port, but
perhaps it does need to be pointed out explicitly. As always security is only
as strong as the weakest link, and in this case that may well be the management
port.


(Since I clipped atributions above: Dragos Ruiu wrote:)
> Even with passive taps on the input spigot of the IDS....  They're all
> vulnerable in the back end on the output/log spigot... because, I
> have yet to see any net admin have the balls and masochism(:-) to
> _not_ inter-network their IDS back end net (if they even have one)
> so they can't access it remotely in _any_ way - only from the local kb.
> The only way it's really safe, is if it is not networked at _all_ and
> physically access controlled (and even then there are some half duplex
> exploits possible - "blind" buffer overflow and clean up scripts. It's
> possible... remember the  tcpdump decode overflow? Or the Sniffit one....
> or....). Oh, and burn the logs to non-volatile media like CD just for good
> measure. :-)

        Good point. Dump your logs to writable CD or worm (NOT rewritable CD!)
and sign and date the resulting CD and physically secure it in a facility
approved by your lawyer and preferably a police officer or better yet a DA
(because the cop and DA probably have more experience with evidence being
tossed in your particular juristiction because of inadequate handling than
even a lawyer :-)) as being probably accepable for chain of evidence purposes
in case you need to use them in a court case. This is so you can testify in
court that after you wrote and signed the CD, it was secured under your sole
control in such a way that you can certify that the CD submitted as evidence
is the same one you signed and are certifying represents a true copy of the
data in the log files that are being submitted as evidence.
        While I'm not aware of a precident for this yet (at least in Canada),
there is precident with other types of paper log files and mag tapes that the
judge may accept this as properly secured evidence in a court case if a proper
chain of evidence can be presented.
        As with argus logs of what went on in your network in the past, this is
cheap insurance against a hopefully unlikely disaster of being taken to court
over something alleged to have come from your network (such as a DDOS attack).
While you might make out with only argus logs (I'd bet you will be hosed, not
being able to prove anything one way or the other without an argus or argus
like log of everything that came and went from your network though!) that
aren't signed and secured, why not take that little extra effort to properly
secure the logs as if they were for use in a criminal trial? Civil may be less
picky, although I think by not much. I dumped some tapes as a favor for
someone external in a civil law suit and there was a lawyer physically present
with the tapes the entire time, maintaining chain of evidence oversight of the
tapes (which had been seized by police in a raid) while I mounted and dumped
them for them, so I guess it is the same as criminal (or at least is treated
as such by the smart ...).
        I also remember many years ago instructing a senior manager how to
mount a tape on a mainframe so that he could testify that he had mounted and
dumped the data off of the tape and signed the resulting dump (I think on
paper) of the data on the tape in a criminal matter. I assume he would also
testify about personallly doing all the steps in determining the correct tape
with the data on it all the way through the orginization so that there was a
single person that could maintian the entire chain of evidence (instead of
losing half the company to court dates as witnesses, which is another thing
to keep in mind).


> Perhaps draconian... but if you want to be secure, why mess around
> potentially wasting all that work securing with one little slip up, if you
> _do_ have some sort of wired access into the IDS.  Less chances for
> this and you will likely have a more robust secure net overall.
> Now, that's a paranoid install that I like... :-) Sneakernet all the way,
> and, again, even air-gap sneakernet can be compromised by targeted
> log cleaner viruses. And then the solution for that is.... :-) and so on.

        And the solution for that is to have a proper airgap that doesn't
transfer data in the reverse direction. There should be no reason to get data
back to the sensor, and the sniffing interface shouldn't be interpreting the
data at all, just recording it (even to the extent of doing DNS lookups, do
that from the managment station if you must) although I'll agree that there
will usually be some way to do something bad with the input data. Ideally
that causes an IDS hang which a separate watch dog process detects and yammers
for human intervention as in the "fill the syslog" case, where an abnormal
increase in the size and/or rate of syslogging information should trip an alarm
that attracts human attention to the unusual (which may or may not be an
attack). The machines aren't smart enough to do everything, they can very
usefully filter out enough noise to make on call human response to a possible
attack situation feasable to deal with the cases (hopefully few) that the
machine can't handle.


> But by the time you progress to worrying about stuff like _that_, you
> either have a lot of money to protect or a three letter acronym and/or
> some serious firepower at the other end of those computers, methinks.

        As Marcus Ranum pointed out in the current issue of Usenix login:, DDOS
type attacks and ensuing litigation by E-business for losses may well make
this level of paranoia the required minimum for any institution with deep
pockets and an Internet connection. I'll be prepared if and when it hits, how
about you? Because I have a record of what went out from my network (a permenant
record because argus logs are small enough to keep forever ...), if accused
I can confirm or better yet refute that the attack orginated here because while
I see the replies from the attacked machine being routed back to my network,
there are no attack packets orginating from my net in the log indicating the
attack was forged from another site without antispoof filters on their border
router. I'll bet I can explain this in terms that will convince a judge that it
wasn't our site (and it is someone elses problem to figure out who it really
was, good luck!).
        That will still be expensive in lawyers fees, but much better than
having to say "gee I can't prove one way or the other whether the attack
originated here" to our risk manager and lawyers. The converse is also true of
course, I could end up confirming that indeed we were responsible, in which
case I expect an out of court settlement would be forth coming (followed by
draconian internal policy changes ...). This is likely to be migated by the
fact a DDOS attack outbound from here will get quickly detected and killed (it
has happened on a small scale a couple of times) which shows 1) due diligence
in attempting to prevent such attacks, and 2) that we were responsible for a
limited amount of the damage which may help as well. Knowledge is power ...
        To get this at least a little close the the purpose of this list,
obviously the IDS/ tcp audit stations should be a high priority crack, since
gaining access will lay out all sorts of other profitable places to probe for
weak points by telling the attacker what types of attacks will get detected
or in the argus case showing some traffic getting through that suggests an
entry point to try.
        This may or may not get detected depending on what kind of post
processing of the argus logs looking for attack signatures is going on,
or what other IDS systems are also active looking for trouble on the net.
In my view you should have an argus server recording data on the outside of
your Internet connection and probably another inside your firewall (the
hardware for even a 100baseT network is cheap) and automatically going to
writable CD and being kept along side you IDS system or systems that are
defending your network. That way when a new attack comes out you can (and I
have) look back in the argus logs to see if you have been probed successfully
or unsuccessfully in the past by the new attack. With a real time IDS which
can't keep a copy of the input data you don't have that option. It is
amazingly valuable. When the various DDOS zombies that use icmp echo reply
packets became public I could and did look back and seeing unsuccessful probes
starting months before for such machines on my net. I had a complaint 3 months
later about a scan of anther site. Again I looked back, saw that I had detected
the scan outbound from my network and dealt with our user. I could then suggest
that we had dealt with the problem when it occurred and perhaps the remote site
should read their firewall logs more often than once every three months ...



> hard-core security, pull the plug, :-)

        And grind the machine back up in to sand and iron filings again, those
management types can always put the plug back in again and cause trouble
by wanting to use it ... :-)

Peter Van Epp / Operations and Technical Support
Simon Fraser University, Burnaby, B.C. Canada