Re: Sorry

[Dean Webb <zzzptm () gmail com>]

Hello all,

Michal's comment made me recall this recent article from The Bulletin
of Atomic Scientists:
http://thebulletin.org/how-next-us-nuclear-accident-could-happen8441

We can become numb to automation, and then it becomes a blind spot for
us. We assume that things will work because they're in production, yet
they may not have had sufficient testing at boundary conditions to see
how they perform under stress.

Consider also the recent story of a driverless car that charged a
group of reporters in Brazil, injuring two. The Volvo in question did
not have the optional package of pedestrian detection. Even with that
package, it would have acknowledged the pedestrians and then charged
them, because it did not have the additional optional package that
would have it brake for pedestrians.

The robotic arm in question must have been in a boundary condition it
hadn't been tested for and while its reaction stuns, shocks, and
horrifies us, we have to keep in mind that, logically, the machine
made all the right moves, given its programming.

Which then goes back to another idea I read at The Bulletin. Namely,
what happens with autonomous, armed military drones in boundary
conditions? How much autonomy do we permit them to have, and when? The
answers to those conditions could mean either a failure to operate in
a combat zone and/or full activation inside a friendly base, and a
catastrophic case of friendly fire, or, as the brass during Vietnam
once said, "accidental misdeliverance of combat ordnance."

Kind regards,

Dean

On 7/3/15, Michal Zalewski <lcamtuf () coredump cx> wrote:
> > In all seriousness it is terrifying how little environmental awareness is
> > needed (and thus, is present) with modern robotics.
>
> Industrial processes are terrifying. Many people die or lose limbs
> every year when operating all sorts of factory-floor machinery;
> everything from CNC machining centers to metal forming presses can and
> does kill. I suspect we find this tragic story captivating is because
> robotic arms are menacingly anthropomorphic, whereas machining centers
> are not? But I bet that by fully automating production lines, they on
> balance save lives.
>
> /mz
> _______________________________________________
> Fun and Misc security discussion for OT posts.
> http://lists.linuxbox.org/cgi-bin/mailman/listinfo/funsec
> Note: funsec is a public and open mailing list.
_______________________________________________
Fun and Misc security discussion for OT posts.
http://lists.linuxbox.org/cgi-bin/mailman/listinfo/funsec
Note: funsec is a public and open mailing list.