The Upgrade Process: Restarting vs Rebooting.

[InfoSec News <isn () c4i org>]

+------------------------------------------------------------------+
|  Linux Security: Tips, Tricks, and Hackery                       |
|  Published by Onsight, Inc.                                      |
|                                                                  |
|  09-April-2003                                                   |
|  http://www.hackinglinuxexposed.com/articles/20030409.html       |
+------------------------------------------------------------------+

This issue sponsored by Linuxfest Northwest 2003, Bellingham, WA,
April 26

LFNW is a showcase for what Northwest Linux users are doing with
Linux and open source software. It's a place for Linux enthusiasts to
get together to share their passion for what good software can do.
This year's fest will include boatloads of demonstrations, tutorials,
and lectures by gurus such as John "Mad Dog" Hall, Randal Schwartz,
and your very own Linux Security geek, Brian Hatch.

Admission is free. For more information, see http://
www.linuxnorthwest.com. Come join the fun!

--------------------------------------------------------------------

The Upgrade Process: Restarting vs Rebooting.
By Brian Hatch

Summary: Upgrading your software is a constant task. But when does it
require a reboot, and when can you get by without?

Every operating system in the world requires software updates from
time to time. If you're used to the world of Windows, you have
probably become accustomed to a simple fact of windows life: you need
to reboot your machine after any system change, such as upgrading
software, installing patches, installing new software, moving your
mouse, and so on. In the Linux world, this is not the case. You can
keep your machine running for months or years without rebooting as
long as you properly handle software upgrades.

There are four main types of software upgrades that you could
require. I'll point out the actions you should take after performing
an upgrade to be sure that your machine is running smoothly and
you're not accidentally running old versions.

User-level software

User level software are your standard programs in places like /bin
and /usr/bin that are executed from the command line interactively,
from system scripts, cron jobs, you name it. Things like awk, emacs,
grep, mutt, passwd, perl, ping, vi, and so on. These programs are run
for short periods of time, rather than hanging around forever like a
webserver.

These programs, since they are not running continually, do not
require any reboot of the machine after an upgrade. The next time
they're called, the new version will automatically be run. On the oft
chance you have extremely long lived processes (say you've had mutt
running on a screen session since the beginning of time) then you
should stop and restart them, but this is rare.

Daemon software

Daemon software is anything that runs continually unattended, such as
a webserver, mail server, database server, network time server, log
analyser, etc. Often you can identify these by looking at their
parent process ID in ps output - it will usually be '1', the init
process. For example:

  $ ps -fC syslog-ng
  UID        PID  PPID  C STIME TTY          TIME CMD
  root       221     1  0 Jun23 ?        00:04:53 /sbin/syslog-ng

These processes are usually controlled by a script in /etc/init.d.
After you upgrade this sort of software, you should stop and restart
the daemon. For example

  # /etc/init.d/syslog-ng restart
  # /etc/init.d/apache restart

  or

  # /etc/init.d/ntpd stop
  # /etc/init.d/ntpd start

Doing this after an upgrade assures your daemon is the newly upgraded
version. The service will be down for a brief moment while it's
restarting, but that's small change compared to the time required for
a reboot of the computer.

Inetd-style Daemon software

Some daemon software, particularly network daemons, is started by
super servers like inetd, xinetd, or Bernstein's tcpserver. Each time
a client connects to one of these services, the super server will
accept the inbound connection and launch a program to handle it, for
example an IMAP or POP server. These programs are only running for
the time that the connection is active, after which they stop. For
example, here's a snippet of ps output which shows a POP server
that's controlled by xinetd:

  UID        PID  PPID  C STIME TTY          TIME CMD
  root      1128     1  0 Jun23 ?        00:00:25 xinetd -stayalive -reuse -pidfil
  ryan     10169  1128  1 12:30 ?        00:00:01 [ipop3d]
  reegen   32015  1128  1 12:14 ?        00:00:10 [ipop3d]
  maddie   32015  1128  1 12:31 ?        00:00:05 [ipop3d]
  chris    32015  1128  1 12:28 ?        00:00:03 [ipop3d]

All the [ipop3d] processes were started by xinetd and when they are
done servicing the request, they will terminate.

If you were to upgrade the ipop3d software, the processes that are
still running would be the vulnerable version, but all new
connections will be the bug free version. To stop the buggy versions
rudely, you could simply

  # killall ipop3d

This will stop the old version, and the clients would need to
reconnect. Again, no reboot is needed.

Libraries

A library is a file that contains compiled code that can be used by
more than one program. Most Linux programs are compiled to use shared
libraries, as opposed to being compiled 'statically', which would
mean that all the code is contained in the binary itself. Using
existing libraries is good - it means that should a bug be found in
the implementation of printf, the library needs to be upgraded, but
the rest of the software doesn't need to be recompiled.

Different Linux distributions may handle library upgrades slightly
differently, however they all do share one common feature: the
existing library is deleted, and the new one is installed. The
problem is that any program that is running has already mapped the
old buggy library, so it keeps using the old one, even though the new
one is available.

Let's take a look at this. Say on my Debian system I have slapd (the
OpenLDAP server) running. Turns out, there's an update for the ldap
shared libraries -- the libldap2[1] package -- that I need to apply.

  $ dpkg -L libldap2 
  /usr/lib/libldap.so.2.0.15
  /usr/lib/libldap_r.so.2.0.15
  /usr/lib/libldap.so.2
  /usr/lib/libldap_r.so.2

So, it looks like the current (buggy) version has files named /usr/
lib/libldap*. Let's see what programs are using these:

  $ lsof /usr/lib/libldap*
  COMMAND   PID  USER  FD TYPE DEVICE   SIZE    NODE NAME
  slapd   29401  root mem  REG    3,5 159648  350748 /usr/lib/libldap_r.so.2.0.15
  slapd   29402  root mem  REG    3,5 159648  350748 /usr/lib/libldap_r.so.2.0.15
  slapd   29403  root mem  REG    3,5 159648  350748 /usr/lib/libldap_r.so.2.0.15
  slapd   29445  root mem  REG    3,5 159648  350748 /usr/lib/libldap_r.so.2.0.15

Ok, slapd is using those libs, unsurprisingly. Let's upgrade and see
what happens afterwards:

  
  # apt-get install libldap2
  ... upgraded package installs ...

  $ lsof /usr/lib/libldap*
  COMMAND   PID  USER  FD TYPE DEVICE SIZE   NODE NAME
  slapd   29401  root mem  DEL    3,5      350748 /usr/lib/libldap_r.so.2.0.15.dpkg-new
  slapd   29402  root mem  DEL    3,5      350748 /usr/lib/libldap_r.so.2.0.15.dpkg-new
  slapd   29403  root mem  DEL    3,5      350748 /usr/lib/libldap_r.so.2.0.15.dpkg-new
  slapd   29445  root mem  DEL    3,5      350748 /usr/lib/libldap_r.so.2.0.15.dpkg-new

What happened here? The way the Debian upgrade worked was to rename
the old libraries to oldname.dpkg-new, then install the new versions
and delete the old ones. Slapd was already using the old ones, and
thus those files are still available to it, even though you can't see
them in the directory itself -- notice "DEL" in the "TYPE" column,
and the empty "SIZE" column.[2].

In order to have slapd start using the new libraries you should --
you guessed it -- restart slapd

  # /etc/init.d/slapd restart
  

Again, no reboot necessary, but there will be momentary downtime as
the program restarts.

The Kernel

So you want to upgrade the kernel to patch the most recent ptrace
bug, for example? Great idea. Does this require a reboot? You betcha.
No two ways about it, to run a new kernel you need to reboot. Sorry
for the bad news.

Summary

So, when do you need to reboot? Technically, the only time you need
to reboot is if you're installing a new kernel. Any other process can
be stopped and restarted without much worry.[3] However you may still
want to schedule a reboot for a convenient time anyway, perhaps at
3am on a weekend. This allows you to test any new changes and make
sure that the computer comes up again properly, and have time to fix
it when it's not much needed. You'd much rather test then, than have
a power outage take down your machine while you're on vacation and
find that it won't start up your software properly.

NOTES:

[1] I've taken the liberty of trimming out some of the extraneous
files to make this explanation cleaner. There are actually more files
in the libldap2 package.

[2] The Linux kernel won't actually remove a file from disk until all
files using it close the file, and all hard links to the file on disk
are removed from the filesystem. You might want to check out a
previous article about finding deleted files at http://
www.hackinglinuxexposed.com/articles/20020507.html

[3] Ok, /sbin/init can't really be restarted, but I don't expect
we'll see an upgrade needed for it, it's pretty rock solid.

                            -------------                            
Brian Hatch is Chief Hacker at Onsight, Inc and author of Hacking
Linux Exposed and Building Linux VPNs. He changes his bio in these
newsletters every week, and wonders if anyone notices. Brian can be
reached at brian () hackinglinuxexposed com.

--------------------------------------------------------------------
This newsletter is distributed by Onsight, Inc.

The list is managed with MailMan (http://www.list.org). You can
subscribe, unsubscribe, or change your password by visiting
http://lists.onsight.com/ or by sending email to
linux_security-request () lists onsight com.

Archives of this and previous newsletters are available at
http://www.hackinglinuxexposed.com/articles/

--------------------------------------------------------------------

Copyright 2003, Brian Hatch.



-
ISN is currently hosted by Attrition.org

To unsubscribe email majordomo () attrition org with 'unsubscribe isn'
in the BODY of the mail.