Re: The Real AI Threat?

[Mel Beckman <mel () beckman org>]

Miles,

You realize that “AI” as general artificial intelligence is science fiction, right? There is no general AI, and even ML 
is not actually learning in the sense that humans or animals learn. “Neural networks”, likewise, have nothing to do at 
all with the way biological neurons work in cognition (which science doesn’t understand). That’s all mythology, 
amplified by science fiction and TV fantasies like Star Trek’s character “Data”. It’s just anthropomorphizing 
technology. 

We create unnecessary risk when we anthropomorphize technology. The truth is, any kind of automation incurs risk. There 
is nothing related to intelligence, AI or otherwise. It’s all just automation to varying degrees. ML, for example, 
simply builds data structures based on prior input, and uses those structures to guide future actions. But that’s not 
general behavior — it all has to be purpose-designed for specific tasks.

The Musk-stoked fear that if we build automated systems and then “put them together” in the same network, or whatever, 
that they will somehow gain new capabilities not originally designed and go on a rampage is just plain silly. Mongering 
that fear, however, is quite lucrative. It’s up to us, the real technologists, to smack down the fear mongers and tell 
truth, not hype. 

Since the academics’  promised general intelligence of AI never materialized, they had to dumb-down their terminology, 
and came up with “narrow AI”. Or “not AI”, as I prefer to say. But narrow AI is mathematically indistinguishable from 
any other kind of automation, and it has nothing whatsoever to do with intelligence, which science doesn’t remotely yet 
understand. It’s all automation, all the time.

All automated systems require safeguards. If you don’t put safeguards in, things blow up: rockets on launchpads, guns 
on ships, Ansible on steroids. When things blow up, it’s never because systems unilaterally exploited general 
intelligence to “hook up” and become self-smarted. It’s because you were stupid.

For a nice, rational look at why general AI is fiction, and what “narrow AI”, such as ML, can actually do, get Meredith 
Broussard’s excellent book "Artificial Unintelligence - How computers misunderstand the world". 

https://www.amazon.com/Artificial-Unintelligence-Computers-Misunderstand-World/dp/026253701X

Or if you prefer a video summary, she has a quick talk on YouTube, "ERROR – The Art of Imperfection Conference: The 
Fragile”:

https://www.youtube.com/watch?v=OuDFhSUwOAQ

At 2:20 into the video, she puts the kibosh on the mythology of general AI.

 -mel


> On Dec 9, 2020, at 11:07 AM, Miles Fidelman <mfidelman () meetinghouse net> wrote:
>
> Hi Folks,
> It occurs to me that network & systems admins are the the folks who really have to worry about AI threats.
>
> After watching yet another AI takes over the world show - you know, the 
> same general theme, AI wipes out humans to preserve its existence - it 
> occurred to me:
>
> Perhaps the real AI threat is "self-healing systems" gone wild. Consider:
>
> - automated system management
> - automated load management
> - automated resource management - spin up more instances of <whatever> 
> as necessary
> - automated threat detection & response
> - automated vulnerability analysis & response
>
> Put them together, and the nightmare scenario is:
> - machine learning algorithm detects need for more resources
> - machine learning algorithm makes use of vulnerability analysis library 
> to find other systems with resources to spare, and starts attaching 
> those resources
> - unbounded demand for more resources
>
> Kind of what spambots have done to the global email system.
>
> "For Homo Sapiens, the telephone bell had tolled."
> (Dial F for Frankenstein, Arthur C. Clarke)
>
> I think I need to start putting whisky in my morning coffee.  And maybe not thinking 
> about NOT replacing third shift with AI tools.
>
> Miles Fidelman
> -- 
> In theory, there is no difference between theory and practice.
> In practice, there is.  .... Yogi Berra
>
> Theory is when you know everything but nothing works. 
> Practice is when everything works but no one knows why. 
> In our lab, theory and practice are combined: 
> nothing works and no one knows why.  ... unknown