Re: Getting passwords from the heap?

["Aigars Grins" <aigars.grins () defcom com>]

> > Also, what is the difference between malloc(3) and calloc(3)?  calloc
> > says it's supposed to clear the memory, but malloc(3) does that too...
>
> malloc() doesn't clear the memory, it requests a page from the OS, and the
OS
> may clear that page that the memory is allocated from if it hasnt already
> been allocated for that user.  So the first malloc of 2k will return clean
> memory, but the next malloc of 1k could return memory that has been mucked
> with by something else in that process (variables in called functions,
libc,
> etc).  That could be completely wrong technically but thats what I have
seen
> in practice.  I have this fuzzy understanding that pages are normally 4k
and
> once you have malloc'd more memory then is left in your "current" page,
the
> OS gives you another (which it then clears). if anyone knows more about
the
> exact workings of memory and page allocation under linux, windows, and
other
> OS's I would be great if they shared...

Sorry for my bad english.. The following applies for OpenBSD 2.7 (and most
likely later versions as well). A more OpenBSD savvy programmer can propably
correct me on a lot of details (please do).

Malloc() uses sbrk() to access memory from the OS. This gives malloc() a
'static' area to work with. Static in the sense that it's always a single
continuous space (the end point is what changes when it gets/gives resources
from/to the OS). This space is continous from userland perspective, ie. the
space is not a continous physical RAM space.

Malloc() organizes this space on a per page (pages are 4096 bytes) basis
into parts (ie. some pages for 65-128 byte alloc's, some other for 129-256
byte alloc's etc.). When an alloc is made it marks the space as used and
returns a pointer to it. When the memory is free()'d it's marked as
non-used. When a page is wholly un-used it's marked for re-use. If a space
large enough (about half a page) is asked for (from the user) whole pages
are marked as used.

If there is free space at 'the end' of the area some of this is returned to
the OS. When requesting more space from the OS more then necessery is
requested. Both these things are done to increase performance.

There is of course a problem with the approach (from a resource utilization
point of view). If a process malloc()'s and free()'s a lot, a situation with
a large amount of half used and half non-used pages can occur (which wastes
memory resources). This rarely happens and the approach gives an overall
better performance then trying to utilize all space 'perfectly'.

Calloc() simply calls malloc() and then zeros out the space. Malloc does no
zero'ing at all (hence it's more efficient).

A call to malloc() doesn't cost that much as long as there are pages
available to it (about the same as two small function calls). If not it has
to call sbrk() to get more from the OS (ie. extend it's available area).
This is costly. The OS should of course (from a security point of view) wipe
these areas before giving access to them to the process. I don't know if
this is done by a call to sbrk().

There are other ways a process gets access to memory. There is of course the
space that is allocated for the process code and static variables. Also
mmap() can be called to get memory.

I have worked on a project where we implemented something of a copy of
malloc() that used mmap() as underlying memory access method instead of
sbrk(). This gave us a possibility to split alloc's over two different areas
giving us an easier way of handing out-of-memory errors. The things I know
of how malloc() works is from that project, although I was not the one that
implemented the routines.

A programmer should of course never count on anything wiping anything except
if he/she says so explicitly.

--
Aigars