A Familiar World of Chaos

[Dave Aitel via Dailydave <dailydave () lists aitelfoundation org>]

After spending some time looking at "Secure by Design/Default" I have no
doubt many of you feel like something is missing - something that's hard to
put your finger on. So you go back to the treadmill of reading about bugs
in Palo Alto devices, or the latest Project Zero blogpost, or something the
Microsoft Threat Team is naming RidonculousBreeze, or whatever.

For those of you who chose to read the latest Project Zero post, one way to
look at Mateusz Jurczyk's vast destruction
<https://googleprojectzero.blogspot.com/2024/04/the-windows-registry-adventure-1.html>
of the Windows Registry API, resulting in what can only be described as a
"boatload" of Local Privilege Escalations, is that securing legacy code is
hard, there's a talent shortage in how many people want to do the reverse
engineering work necessary to understand and fix complicated and critical
old code, and our investments in automated security engineering toolkits
and better software development practices, while valuable, have not paid
off in the kind of hardened Rust-only systems we dreamed about.

Another way to look at this kind of wholesale destruction, a true tour de
force, is that you cannot both put advertisements in your Start menu, and
develop a secure operating system, for reasons that are more philosophical
than technical.

It's ironic that it is often Google that demonstrates this about other
vendors, when of course, the lack of any ad blocking in Chrome or Android
presents the exact same dilemma. You can't both make your systems secure,
and sit beside the great river of Advertising Revenue with a ladle, dipping
it in every quarter to fill up a cauldron of greater and greater value for
the shareholders. It's hard to draw a straight line from an internal
PowerPoint slide saying "Ads in the Start Menu are a good idea, actually"
to the inevitable conclusion of 0days, ransomware, and US Government emails
are being read by some old Russian who understands cryptography and Azure
keys better than you were hoping.

But in some respect this cause and effect is as fundamental and simple as
how that tattoo on your arm is actually there because one night you decided
to start off with shots of Limoncello.

When Project Zero started, and even when it got to the towering behemoth of
talent that it is now, I knew people in the offensive industry who were
quite scared of it - of the possibility that a large and funded team of top
researchers, with access to one of the only five real computers on the
planet, could drain the lake of software vulnerabilities we all fished in.

But I had no such fears. An organization so dependent on advertising
revenue to survive can no more fix systemic security issues than a Sperm
Whale can medal in Olympic Skiing. It is contrary to their very nature,
although they will probably smash a bunch of trees on the way down.

Like many of you, I spent my Saturday porting code to use LLAMA3:70b,
largely by annoying my 18yo with questions about ollama and Docker, since I
find modern Linux system administration as foreign as an octopus finds
calculus.

But search engines, like surface warships, are clearly on their last legs.
They went from something you used every day, multiple times a day, to
something your LLM uses for you, as just one tool among many. It is, for
reasons that must be obvious even to executives drunk on the heady fumes of
their stock options maturing, hard to make money selling advertisements
that are only read by LLMs.

But having spent the better part of a couple years doing LLM work now, I
feel like I understand why these behemoths are investing so much money in
them, despite the obvious cannibalization of their cash umbilical. It's
because they can!

There's just not that many businesses that generate ten billion dollars of
revenue year on year to get into. You've got some elements of
manufacturing, tech, education, health care, video games. It's not a big
list. Apple gave up on manufacturing cars because the niche they wanted
(impractably weird and expensive) was already filled by Tesla.

But by investing in LLMs and AI in general you kinda get to put your thumbs
in every other billion dollars business all at once. It's a straight line
shot from something you already know, to the next place. So of course, they
are throwing dollars at it like it was the only thing they knew how to do.
And what we get is the pretentious superiority of ChatGPT, or the
sanctimonious holiness of Claude, or the ever-sadness of Gemini, the
impertinence of Mistral or the trollishness that is LLAMA.  A world of
chaos, yet something so familiar.

-dave

_______________________________________________
Dailydave mailing list -- dailydave () lists aitelfoundation org
To unsubscribe send an email to dailydave-leave () lists aitelfoundation org