IP: Process-table attack

[Dave Farber <farber () cis upenn edu>]

Date: Fri, 19 Feb 1999 16:08:06 -0500 
From: "Simson L. Garfinkel" <simsong () vineyard net> 
Subject: Process-table attack
Wide-ranging attack works against almost any UNIX systems on the Internet
ABSTRACT:
The Process Table Attack is a [relatively] new kind of denial-of-service 
attack that can be waged against numerous network services on a variety of 
different UNIX systems. The attack is launched against network services 
which fork() or otherwise allocate a new process for each incoming TCP/IP 
connection. Although the standard UNIX operating system places limits on 
the number of processes that any one user may launch, there are no limits on 
the number of processes that the superuser can create other than the hard 
limits imposed by the operating system. Since incoming TCP/IP connections 
are usually handled by servers that run as root, it is possible to 
completely fill a target machine's process table with multiple 
instantiations of network servers. Properly executed, this attack prevents 
any other command from being executed on the target machine.
DETAILS
In the book Practical UNIX and Internet Security, Gene Spafford and I 
observed that the UNIX operating system originally contained few defenses to 
protect it from a denial-of-service attack. This is changing. With the 
growth of the Internet, there has been a concerted effort in recent years to 
strengthen the operating system and its network services to these attacks.
Each time a network client makes a connection to a network server, a number 
of resources on the server are consumed. The most important resources 
consumed are memory, disk space, and CPU time. Some network services, such 
as sendmail, now monitor system resources and will not accept incoming 
network connections if accepting them would place the system in jeopardy.
One system resource that has escaped monitoring is the number of processes 
that are currently running on a computer. Most versions of UNIX will only 
allow a certain number of processes to be running at one time. Each process 
takes up a slot in the system's process table. By filling this table, it is 
possible to prevent the operating system from creating new processes, even 
when other resources (such as memory, disk space, and CPU time) are widely 
available.
The implementation of many network services leaves them open to a process 
table attack ? that is, an attack in which the attacker fills up the target 
computer's process table so that no new programs can be executed. The 
design of some network protocols actually leads the programmer into making 
these mistakes.
An example of such a protocol is the finger protocol (TCP port 79). The 
protocol follows this sequence:
1. The client makes a connection to the server. 
2. The server accepts the connection, and creates a process to service 
the request. 
3. The client sends a single line to the server consisting of the name 
of the entity that the client wishes to finger. 
4. The server performs the necessary database lookup and sends the 
information back to the client. 
5. The server closes the connection.
To launch a process table attack, the client need only open a connection to 
the server and not send any information. As long as the client holds the 
connection open, the server's process will occupy a slot in the server's 
process table.
On most computers, finger is launched by inetd. The authors of inetd placed 
several checks into the program's source code which must be bypassed in 
order to initiate a successful process attack. If the inetd receives more 
than 40 connections to a particular service within 1 minute, that service is 
disabled for 10 minutes. The purpose of these checks was not to protect the 
server against a process table attack, but to protect the server against 
buggy code that might create many connections in rapid-fire sequence.
To launch a successful process table attack against a computer running inetd 
and finger, the following sequence may be followed:
1. Open a connection to the target's finger port. 
2. Wait for 4 seconds. 
3. Repeat steps 1-2.
The attack program is not without technical difficulty. Many systems limit 
the number TCP connections that may be initiated by a single process. Thus, 
it may be necessary to launch the attack from multiple processes, perhaps 
running on multiple computers.
We have tested a variety of network services on a variety of operating 
systems. We believe that the UW imap and sendmail servers are also 
vulnerable. The UW imap contains no checks for rapid-fire connections. 
Thus, it is possible to shut down a computer by opening multiple connections 
to the imap server in rapid succession. With sendmail the situation is 
reversed. Normally, sendmail will not accept connections after the system 
load has jumped above a predefined level. Thus, to initiate a successful 
sendmail attack it is necessary to open the connections very slowly, so that 
the process table keeps growing in size while the system load remains more 
or less constant.
We have also seen a variety of problems on BSD-based machines being used as 
the attacker. When the target machine freezes or crashes, the attacker 
machine sometimes crashes as well. Apparently the TCP stack does not 
gracefully handle hundreds of connections to the same port on the same 
machine simultaneously going into the FIN_WAIT_2 state.
There are variants of this attack:
1. Use IP spoofing so that the incoming connections appear to come from 
many different locations on the Internet. This makes tracking 
considerably harder to do. 
2. Begin the attack by sending 50 requests in rapid fire to the telnet, 
rlogin and rsh ports on the target machine. This will cause inetd to 
shut down those services on the target machine, which will deny 
administrative access during the attack. 
3. Instead of initiating a new connection every 4 seconds, initiate one 
every minute or so. The attack slowly builds, making it more difficult 
to detect on packet traces.
There are several ways to defend against the attack:
1. inetd and other programs should check to see the number of free slots 
in the process table before accepting new connections. If there is less 
than a predefined number of free slots, new connections should be 
accepted. 
2. Alternatively, if there are more than a preset number of network daemons 
for the service running, incoming requests should be queued rather than 
serviced. 
3. Network services (such as finger) should implement timeouts. For 
example, the statement alarm(30) could be inserted into the finger 
daemon source code so that the program would stop running after 30 
seconds of execution.
Simson L. Garfinkel, Sandstorm Enterprises, Inc. <www.sandstorm.net>
[Simson informed me over a year ago that he had discovered this attack 
and had notified many relevant operating system vendors. To the best of 
my knowledge, no one has addressed the problem in the intervening year. 
We thus include this item in the hopes of spurring some action, or at 
least awareness and public discussion. On the other hand, we of course 
do not recommend conducting experiments to demonstrate this flaw on 
other people's systems. PGN]