Re: Linux hacked

[Miles Stevenson <miles () mstevenson org>]

Dale,

First of all, I'd like to point out that you are asking all the right 
questions, and that I'm impressed by how far you've come without having any 
sysadmin experience. 

Contrary to the advice that you have been given thus far, I'm hoping that you 
have not interacted with the system at all so far, aside from unplugging it 
from the network and/or shutting it down. If this is the case, then don't. 
The first thing you want to do is take a forensically sound "image" of your 
system, from which you can work. This way, you can work from the image, and 
not the real system in trying to determine what happened and how you were 
attacked. I think the best approach, is to boot your system with a separate 
bootable CD, such as Knoppix STD, Phlak, or another forensics-focused 
bootable linux OS. After you boot up into the OS running from CD, you can 
connect the system back to your internal network. You can then use the dd and 
netcat utilities, to take a perfect forensic snapshot of your system, and 
send that snapshot to another system on the network.

Instead of explaining how to do this, I will point you to another resource in 
order to save space: 
http://www.rajeevnet.com/hacks_hints/os_clone/os_cloning.html

Once you have a forensic copy of your system, you can now safely continue your 
investigation of what went wrong and why. You can also choose to completely 
wipe and rebuild your system if that is the most appropriate course of action 
for you, and you decide to investigate later. But, the longer you wait to 
perform an investigation, the more difficult that investigation is going to 
be. Choose carefully.

The most important thing for you to keep in mind here, is that once your 
system has been compromised, you can *no longer trust ANY of the data on your 
system*. Netstat might lie to you. Your kernel might lie to you. In essence, 
the attacker could have made any alterations to your systems to change the 
way it behaves or what it reports to you. You can't trust the logs, you can't 
even trust the output of the commands. This is why you have to run these 
tools from a separate, TRUSTED source, such as from a read-only forensic CD 
like Phlak.  Don't trust the "ls" command on the hacked system, but DO trust 
the "ls" command on your forensics disk. This is VERY important.

This process is going to get more and more complicated as you continue, and is 
best handled by someone with experience. If you can get to this point, and 
then hand things over to someone else, I recommend it. If you are unable to 
do that, then I am willing to help you as much as I can. But I think you 
should first get to this point of taking a forensic snapshot of your system, 
and obtaining a bootable forensic cd (I personally like Phlak, but there are 
many others) that you can use as a tool. Once you get to this point, let me 
know your situation, and we can continue. If I cover too much right now, not 
only will I run the risk of "information overload", but I also have to start 
making assumptions about your system in order to recommend how to proceed, 
and these assumptions can be disasterous, even when made by those of us that 
know what we are doing. You can contact me off-list if you prefer.

Good luck. 

On Wednesday 20 October 2004 12:52 pm, Nicholson, Dale wrote:
> First let me say I'm a security novice.  Please bear with me.
>
> My home linux (gentoo) machine was hacked last Thursday.  Installed active
> on the box was ssh, apache, php 5, and a squirl mail.  Iptables was set up
> for a firewall.  The box was set up as a web server with a number of
> websites and about 35 email accounts (separate passwords for the mail than
> the user accounts on the box).
>
> I'm guessing it was some sort of script kiddie if the names taking credit
> for the hack in the hidden folders I found are any indication.  I did some
> research on the person taking credit and found all kinds of information on
> him, he's an 18yr old kid in Germany.  I doubt he is very knowledgeable or
> he would not have alerted me to the intrusion by somehow locking out all
> accounts from the machine.
>
> To get in I have to boot from cd and chroot in.  Everything I've tried has
> been unsuccessful in getting root back.
>
> I found a hidden directory /var/tmp/.tmp that has a bunch of directories
> under it with names like +_01_+++++++HaXorEd by ... and
> +_05_++++++++++Movies++++++....
>
> I unplugged the machine from the internet shortly after the hack and can
> find no evidence of any uploads.  I do see that the person somehow was able
> to break root.  I was only able to find the hidden directories because the
> person forgot to clean up root's history file where I found the command
> used to create the them.  The box was set up to not allow remote login of
> root via ssh but you could su in once logged in as one of three users.
>
> I'm a novice at security and had been depending on my system admin to keep
> the box up to date.  He tells me he's been doing an emerge world every week
> but I don't know how to tell.
>
> Can someone help me with where to get a listing of everything I have
> installed and the versions?  I can't remember if the kernel is a 2.4 or 2.6
> but I think it's 2.6.  Plus I know there have been problems with ssh in the
> past but I don't know which versions have problems and I'm not sure how to
> find out what version I'm running.  I'm kind of stuck as my sys-admin
> normally handles these things but he cannot ssh in to the box without me
> first fixing the problem since he lives 13 hours from me (the box is in my
> basement).
>
> Also, I need something that can detect root kits etc. on linux.  I've heard
> knoppix mentioned as having good tools on this list for an example, but I
> wouldn't know what tools to use for this particular case.
>
> This is what I tried so far:
> I logged in using a boot CD, mounted the hard disks, chrooted in, blanked
> out the root password in the /etc/shadow file, changed the root password,
> rebooted and tried to log in normally.  This did not work.  I also checked
> that the correct users were in both /etc/passwd and /etc/shadow.
>
> Note that both the email and websites were still working despite not being
> able to log in, although not now of course since I unplugged the ethernet
> cable.
>
> Any comments/assistance will be greatly appreciated.

-- 
Miles Stevenson
miles () mstevenson org
PGP FP: 035F 7D40 44A9 28FA 7453 BDF4 329F 889D 767D 2F63

Attachment: _bin
Description: