Re: LMS-2014-06-16-6: LZ4 Core

["Don A. Bailey" <donb () securitymouse com>]

Ahhh, so that's who this is. I only read Yann's blog post which was largely
an emotional response, and so I ignored most (read: all) of it. Now that I
understand who he is, this makes a *lot* more sense. Thanks for sending the
email.

I also never saw his responses on the lz4 code site. I posted a note there,
but never received updates or saw that he responded to my messages. I
checked several times throughout this process. Unfortunately, this just
breaks down to a communication error.

I think the larger issue is that this was noted as a security problem some
time ago on the lz4 site. I've never tried to hide that. What was
interesting is the issue was dismissed by the lz4 developer, who I now know
is Yann. It was never fixed and it was given a low priority. So, from this
perspective, I think it is unfortunate that he is so angry. If he thought
it was a serious issue back then he should have raised the priority. The
original researcher never even pursued the issue after it was deemed a
non-issue, nor did they attempt to seek out alternative implementations.
The maintainers of LZ4 variants used in ZFS, for example, were never
notified. I contacted those individuals myself.

Yann is technically wrong about a lot here, however.
 - 64bit systems are still vulnerable, but impractical to exploit due to
memory constraints
 - I mentioned that the security flaw is unlikely to be exploited due to
memory constraints
 - I mentioned the ZFS 128k limit in my blog post as an example of why
things *aren't* vulnerable
 - There is no constraint or ceiling in the LZ4 decompression routine on
the size of the input/output buffer

First issue - 64bit systems. It is still possible to generate an integer
overflow on 64bit systems. The amount of memory (terabytes upon terabytes)
would be necessary to succeed. This is indeed infeasible. But, this is also
potentially the same logic that kept this bug hidden for 20 years. I am not
here to speculate on whether something might be vulnerable or not. It is
vulnerable code. Period.

The security flaw is indeed unlikely to be exploited in most environments.
I have never disputed this. I think the LZ4 bug is interesting because it
is more easily instrumented than the LZO exploit. You can actually get a
more precise overwrite with LZ4 than you can with LZO. As a result, it is
extremely practical to write exploits for it, including RCE. Is it
practical outside of the core library? Probably not, but that doesn't mean
it shouldn't be secured.

I noted in my blog that ZFS is constrained to 128k. For some reason Yann
didn't read this far. I think he misunderstands context. The LZ4 code as is
in the library is vulnerable. Period. It can be instrumented for precise
overwrites. Period. But, as I address in the blog, context - and thus the
threat model - changes drastically with use of the library. This is why -
and again, I called for this in my blog post - auditing of each
implementation is imperative. Since these algorithms are widely used, and
there is no enforced constraint on a call to LZ4's algorithm, there is no
way to determine who is using this "correctly" or not. As an example, I
have RCE examples for MPlayer2 on 8 or so different target platforms from
x86, x86_64, and ARM on BSDs and Linux. This is because libav's
implementation is slightly different enough to be easily instrumented by an
attacker. Is LZ4 used similarly in another product? I don't know. That's
why I'm calling for audits to find out. Let's find out!

Finally, Yann is right that there are block sizes, etc. But the
decompression routine itself does not care or enforce a size constraint.
Just like LZO's decompression routine, this means that it can be passed any
amount of data the caller wants, and like LZO, users will implement this
incorrectly.

It's unfortunate that Yann's feelings were hurt, and I feel bad that he was
upset enough to react so caustically. I had no intention to make him look
bad, or hurt his project. But, when I saw the bug reports during the Linux
kernel audit from years ago with no reaction or patch, I suppose I presumed
the worst. That's my fault, and I apologize for that.

Hope this helps illuminate my perspective.

Best,
Don A. Bailey
Founder / CEO
Lab Mouse Security
@InfoSecMouse
https://www.securitymouse.com/



On Thu, Jun 26, 2014 at 8:37 PM, Solar Designer <solar () openwall com> wrote:

> On Thu, Jun 26, 2014 at 12:58:37PM -0600, Don A. Bailey wrote:
> > A vulnerability has been identified in the LZ4 core implementation.
> Please
> > review the bug report attached inline.
> [...]
> > Report ID: LMS-2014-06-16-6
> >
> > CVE ID: CVE-2014-4611
> [...]
> > Vulnerability Status: Reported / No response
>
> Yann Collet, the author of LZ4 and maintainer of the LZ4 reference
> implementation, has now posted a different point of view:
>
>
> http://fastcompression.blogspot.fr/2014/06/debunking-lz4-20-years-old-bug-myth.html
>
> Aside from the bitterness (which I think is excessive, albeit
> understandable), there's technical detail on why the vulnerability is
> less severe, and a mention of it having been reported via "a brief note
> on the LZ4 issue board".  I've just found this note here:
>
> https://code.google.com/p/lz4/issues/detail?id=52&can=1
>
> I guess there was some miscommunication, because there _was_ response
> via comments on this issue.  Don's comment was posted on June 19, and
> Yann replied via multiple comments on June 20, 22, 26.  The latest one
> of these says "Fixed into r118", which is:
>
> https://code.google.com/p/lz4/source/detail?r=118
>
> and the commit message includes:
>
> "fix :  Issue 52  (malicious address space overflow in 32-bits mode when
> using custom format)"
>
> Per Yann's blog post, and per comments on issue 52, we should credit
> Ludvig Strigeus for earlier discovery of this issue specifically in LZ4,
> although it was not treated as a security issue until Don's rediscovery
> (per Yann's good reasons, it shouldn't have been, but that's arguable).
>
> Given the above, I think all of Ludvig, Don, and indeed Yann deserve
> credit for getting this issue fixed, and I find it unfortunate that
> feelings were hurt.
>
> Alexander