RE: Getting passwords from the heap?

[Michael Wojcik <Michael.Wojcik () merant com>]

> From: H D Moore [mailto:hdm () secureaustin com]
> Sent: Wednesday, June 27, 2001 3:24 AM

> [Jason Spence:]
> > Also, what is the difference between malloc(3) and calloc(3)?  calloc
> > says it's supposed to clear the memory, but malloc(3) does that too...

They have different syntax and semantics.  That should be clear enough from
the man page, or (better) from the C standard.

calloc is typically a wrapper around malloc that computes the size of the
area (a trival multiplication, obviously), mallocs it, and then if malloc
succeeds does a memset(area, 0, size) to clear the returned memory to
all-bits-zero.

Since all-bits-zero isn't necessarily what you want (it may not be the NULL
pointer representation, for example) and may even be invalid for fields in
the structure you're placing in the calloc'd area (it could be a trap
representation, for example), calloc is a Bad Thing.  Don't use it.  It's a
vestigal function included for backward compatibility.

(The correct way in C to allocate dynamic memory for a structure and then
reset it to the "zero state" it would have been in at program startup had it
been a static variable instead is to use an initializer and structure copy.
For example:

        {
        struct MyStruct MyStruct0 = {0};
        struct MyStruct *MyStructPtr;
        if (MyStructPtr = malloc(sizeof *MyStructPtr))
           *MyStructPtr = MyStruct0;
        }

See the comp.lang.c FAQ for more information.)

Note that malloc *need not* "clear the memory".  malloc, calloc, free, etc.
are standard C functions.  What they *must* do is determined only by the C
standard.  (That's ISO 9899-1999, aka "C99".)  It does not require that the
area returned by malloc be initialized in any way.

> malloc() doesn't clear the memory, it requests a page from the OS,

Sometimes.  In some implementations.

Typical Unix implementations of malloc maintain a pool of memory, usually
called a "heap" (not to be confused with the heap tree-in-an-array data
structure).  They use either sbrk(2) or mmap(2) to request additional pages
for the heap *when necessary*.  If there's already memory available in the
heap to satisfy the request, malloc doesn't need to request any new pages,
and the memory it returns will contain anything put there earlier *by the
same process*.

> and the OS 
> may clear that page that the memory is allocated from if it hasnt already 
> been allocated for that user.  So the first malloc of 2k will return clean

> memory, but the next malloc of 1k could return memory that has been mucked

> with by something else in that process ...

Mostly right, except that the usual Unix malloc doesn't make a system call
if it has memory available.

Of course, an implementation *could* give every process a maximum-size heap
at startup (perhaps using lazy allocation to reduce actual consumption), and
then malloc would never have to make any system calls at all.  Or it could
do any of a number of other things.

Note also that the C standard says nothing about the OS clearing pages given
to malloc.  I suspect this is guaranteed by some Unix standard (POSIX, SUS,
etc.), but I'm too lazy to look it up.  It's required by Orange Book level
C2.  I can't think of a modern OS I use that doesn't do it.

You're right, though, that the crucial points are that 1) memory returned by
malloc may have garbage in it, but 2) on any reasonable multiuser OS it'll
be your garbage, not some other process'.

> I have this fuzzy understanding that pages are normally 4k

That's a typical value - offhand I can't think of a Unix system I've used
that had pages of another size - but it's not carved in stone anywhere.

> once you have malloc'd more memory then is left in your "current" page,
the 
> OS gives you another (which it then clears)

Most malloc implementations probably request more than one page at a time.

> if anyone knows more about the exact workings of memory and page
allocation 
> under linux, windows, and other OS's I would be great if they shared...

The source to glibc's malloc is readily available, as are some other
implementations.  There's probably a decent amount of information about the
Win32 malloc in the MS Knowledgebase.

Michael Wojcik             michael.wojcik () merant com
MERANT
Department of English, Miami University