CVE-2009-1388 kernel: do_coredump() vs ptrace_start() deadlock

[Eugene Teo <eugene () redhat com>]

The OpenVZ Linux kernel team has found deadlock between ptrace and 
coredump code. It affects 2.6.18 but does not affect the upstream kernel.

"ptrace_start() spins waiting for child->state == 
TASK_TRACED/TASK_STOPPED. If we race with the coredumping, we have to 
wait until it completes.

If the tracer participates in coredumping too, we deadlock. 
do_coredump() waits for tracer to exit and report 
complete(mm->core_startup_done), the tracer spins in an endless loop.

Change ptrace_start() to abort if child->mm->core_waiters != 0."

Patch:
https://bugzilla.redhat.com/attachment.cgi?id=346742

Reference:
https://bugzilla.redhat.com/show_bug.cgi?id=CVE-2009-1388

Thanks, Eugene
--
Eugene Teo / Red Hat Security Response Team