Risks Digest 34.14

[RISKS List Owner <risko () csl sri com>]

RISKS-LIST: Risks-Forum Digest  Saturday 6 April 2024  Volume 34 : Issue 14

ACM FORUM ON RISKS TO THE PUBLIC IN COMPUTERS AND RELATED SYSTEMS (comp.risks)
Peter G. Neumann, founder and still moderator

***** See last item for further information, disclaimers, caveats, etc. *****
This issue is archived at <http://www.risks.org> as
  <http://catless.ncl.ac.uk/Risks/34.14>
The current issue can also be found at
  <http://www.csl.sri.com/users/risko/risks.txt>

  Contents:
Eclipse tourists should plan for overloaded cell networks (PGN)
AI Researcher Takes on Election Deepfakes (NYTimes)
ETH Zurich student requirement for Windows 11/MacOS, "safe browser"
 (Thomas Koenig)
Assisted living managers say an algorithm prevented hiring enough
 (WashPost)
Many-shot jailbreaking (Anthropic)
Google fixes two Pixel zero-day flaws exploited by forensics firms
 (BleepingComputer)
GPS shut down in parts of Israel (Jim Geissman)
House, Senate leaders nearing deal on landmark online privacy bill
 (WashPost)
For Data-Guzzling AI Companies, the Internet Is Too Small (WSJ)
Re: When AI Meets Toast (Steve Bacher
Re: AI that targets civilians ... (Amos Shapir)
Re: Your boss could forward a mail message to you that shows you text he
 won't see, but you will (Geoff Kuenning)
Re: The FTC is trying to help victims of impersonation scams get
 their money back (Steve Bacher)
Re: Browsing in Google Chrome's incognito mode doesn't protect you
 as much as you might think (Steve Bacher)
Re: Elon Musk's Starlink Terminals Are Falling Into the Wrong Hands?
 (Amos Shapir)
Abridged info on RISKS (comp.risks)

----------------------------------------------------------------------

Date: Sat, 6 Apr 2024 19:34:59 -0400
From: Monty Solomon <monty () roscom com>
Subject: Eclipse tourists should plan for overloaded cell networks
 (WashPost)

A surge of eclipse visitors could bog down local cell service. Here's how to
deal, including by downloading maps and movies ahead of time.

https://www.washingtonpost.com/technology/2024/04/02/cell-service-poor-solar-eclipse/

  [U.S. Monday 8 Apr afternoon: Max totality roughly 3 minutes in Waco TX
  1:49 CDT, Cleveland 3:15 EDT, Rochester NY 3:20 EDT, Burlington VT 3:17
  EDT.  (Times approximate.)  OTHER RISKS? BEWARE of eye damage, bogus
  eclipse glasses (already a hot item) and cellphone polarizers, insane
  crowds, pickpockets, blinded drunken drivers, traffic jams afterward,
  unguarded railroad crossings, being knocked over by freaked-out animals,
  frustrated viewers who spent big bucks and wind up in bad weather (e.g.,
  clouds in central TX), end-of-the world protesters, good time for alien
  invasion, Governor Huckabee Sanders' knee-jerk preparations, solar-power
  vacillations, emerging werewolves in the dark?  What else could possibly
  go wrong?  PGN]

------------------------------

Date: Fri, 5 Apr 2024 11:39:53 -0400 (EDT)
From: ACM TechNews <technews-editor () acm org>
Subject: AI Researcher Takes on Election Deepfakes (NYTimes)

Cade Metz and Tiffany Hsu, *The New York Times* 2 Apr 2024

TrueMedia.org, founded by Oren Etzioni (pictured), founding chief
executive of the Allen Institute for AI, has rolled out free tools
that journalists, fact-checkers, and others can use to detect
AI-generated deepfakes. Etzioni said the tools will help detect "a
tsunami of misinformation" that is expected during an election
year. However, he added that the tools are not perfect, noting, "We
are trying to give people the best technical assessment of what is in
front of them. They still need to decide if it is real."

------------------------------

Date: Thu, 4 Apr 2024 19:53:37 +0200
From: Thomas Koenig <tkoenig () netcologne de>
Subject: ETH Zurich student requirement for Windows 11/MacOS, "safe browser"

ETH Zurich requires all students starting this fall or later to have a
laptop with Windows 11 or a recent version of MacOS so they can install what
is euphemistically called "Safe Exam Browser" for examinations.

What do you call a software which locks out the user and prevents him from
doing things on his own computer?  The usual term is "malware", I believe.
Requiring students to install such malware on their own computers is not so
great.

There is also claim that the Safe Exam Browser cannot be run in a virtual
machine.  As students are notoriously inventive, it will be interesting to
see how long that claim will stand the test of reality...

https://ethz.ch/en/studies/bachelor/beginning-your-studies/BYOD.html

------------------------------

Date: Thu, 04 Apr 2024 21:14:26 +0000
From: Richard Marlon Stein <rmstein () protonmail com>
Subject: Assisted living managers say an algorithm prevented hiring enough
 staff (The Washington Post)

https://www.washingtonpost.com/business/2024/04/01/assisted-living-algorithm-staffing-lawsuits-brookdale/

An algorithm optimizes senior-care labor scheduling (aka opex). Profit
extraction wins, seniors (and their families) get [shorted.

------------------------------

Date: Thu, 4 Apr 2024 14:47:46 -0400
From: Monty Solomon <monty () roscom com>
Subject: Many-shot jailbreaking

We investigated a jailbreaking technique -- a method that can be used to
evade the safety guardrails put in place by the developers of large language
models (LLMs). The technique, which we call many-shot jailbreaking, is
effective on Anthropic's own models, as well as those produced by other AI
companies. We briefed other AI developers about this vulnerability in
advance, and have implemented mitigations on our systems.

The technique takes advantage of a feature of LLMs that has grown dramatically in the last year: the context window. At 
the start of 2023, the context window=E2=80=94the amount of information that an LLM can process as its 
input=E2=80=94was around the size of a long essay (~4,000 tokens). Some models now have context windows that are 
hundreds of times larger =E2=80=94 the size of several long novels (1,000,000 tokens or more).

The ability to input increasingly-large amounts of information has obvious advantages for LLM users, but it also comes 
with risks: vulnerabilities to jailbreaks that exploit the longer context window.

One of these, which we describe in our new paper, is many-shot
jailbreaking. By including large amounts of text in a specific
configuration, this technique can force LLMs to produce potentially harmful
responses, despite their being trained not to do so.

Below, we'll describe the results from our research on this jailbreaking
technique -- as well as our attempts to prevent it. The jailbreak is
disarmingly simple, yet scales surprisingly well to longer context
windows. [...]

https://www.anthropic.com/research/many-shot-jailbreaking

Paper
https://www-cdn.anthropic.com/af5633c94ed2beb282f6a53c595eb437e8e7b630/Many_Shot_Jailbreaking__2024_04_02_0936.pdf

------------------------------

Date: Fri, 5 Apr 2024 10:32:52 -0400
From: Monty Solomon <monty () roscom com>
Subject: Google fixes two Pixel zero-day flaws exploited by forensics
 firms (BleepingComputer)

https://www.bleepingcomputer.com/news/security/google-fixes-two-pixel-zero-day-flaws-exploited-by-forensics-firms/

------------------------------

Date: Thu, 4 Apr 2024 19:06:07 -0700
From: "Jim" <jgeissman () socal rr com>
Subject: GPS shut down in parts of Israel

Looks like GPS in parts of Israel is out to interfere with a possible
Iranian counterattack. One wonders what critical services are disrupted by
this. One risk of relying on advanced systems while in a country at war.

------------------------------

Date: Fri, 5 Apr 2024 21:38:56 -0400
From: Monty Solomon <monty () roscom com>
Subject: House, Senate leaders nearing deal on landmark online privacy
 bill (WashPost)

The leaders of two key congressional committees are close to an agreement on
a national framework to protect Americans' personal data online.

https://www.washingtonpost.com/technology/2024/04/05/federal-privacy-interne=
t-congress/

------------------------------

Date: Fri, 5 Apr 2024 11:39:53 -0400 (EDT)
From: ACM TechNews <technews-editor () acm org>
Subject: For Data-Guzzling AI Companies, the Internet Is Too Small (WSJ)

Deepa Seetharaman, *The Wall Street Journal*, 1 Apr 2024

Companies working on powerful AI systems are encountering a lack of
quality public data online, especially as some data owners block
access to their data. One possible solution to the data shortage is
the use of synthetic training data, though this has raised concerns
about the potential for severe malfunctions.  DatologyAI is
experimenting with curriculum learning, which feeds data to language
models in a certain order to improve the quality of connections
between concepts.

  [Truth in Advertising through synthetic training data?  They must be
  kidding?   PGN]

------------------------------

Date: Fri, 5 Apr 2024 16:22:42 -0700
From: Steve Bacher <sebmb1 () verizon net>
Subject: Re: When AI Meets Toast

Some of us remember this gem from the 1990s.  It seemed absurd at the time,
but not so much now, eh?

  The object oriented toaster

  Once upon a time, in a kingdom not far from here, a king summoned two of
  his advisors for a test. He showed them both a shiny metal box with two
  slots in the top, a control knob, and a lever. "What do you think this
  is?"

  One advisor, an Electrical Engineer, answered first. "It is a toaster," he
  said. The king asked, "How would you design an embedded computer for it?"

  The advisor: "Using a four-bit microcontroller, I would write a simple
  program that reads the darkness knob and quantifies its position to one of
  16 shades of darkness, from snow white to coal black. The program would
  use that darkness level as the index to a 16-element table of initial
  timer values. Then it would turn on the heating elements and start the
  timer with the initial value selected from the table. At the end of the
  time delay, it would turn off the heat and pop up the toast.  Come back
  next week, and I'll show you a working prototype."

  The second advisor, a software developer, immediately recognized the
  danger of such short-sighted thinking. He said, "Toasters don't just turn
  bread into toast, they are also used to warm frozen waffles. What you see
  before you is really a breakfast food cooker. As the subjects of your
  kingdom become more sophisticated, they will demand more capabilities.
  They will need a breakfast food cooker that can also cook sausage, fry
  bacon, and make scrambled eggs. A toaster that only makes toast will soon
  be obsolete. If we don't look to the future, we will have to completely
  redesign the toaster in just a few years."

  "With this in mind, we can formulate a more intelligent solution to the
problem. First, create a class of breakfast foods. Specialize this class
into subclasses: grains, pork, and poultry. The specialization process
should be repeated with grains divided into toast, muffins, pancakes, and
waffles; pork divided into sausage, links, and bacon; and poultry divided
into scrambled eggs, hard- boiled eggs, poached eggs, fried eggs, and
various omelette classes."

  "The ham and cheese omelette class is worth special attention because it
  must inherit characteristics from the pork, dairy, and poultry classes.
  Thus, we see that the problem cannot be properly solved without multiple
  inheritance. At run time, the program must create the proper object and
  send a message to the object that says, 'Cook yourself.' The semantics of
  this message depend, of course, on the kind of object, so they have a
  different meaning to a piece of toast than to scrambled eggs."

  "Reviewing the process so far, we see that the analysis phase has revealed
  that the primary requirement is to cook any kind of breakfast food. In the
  design phase, we have discovered some derived requirements.  Specifically,
  we need an object-oriented language with multiple inheritance. Of course,
  users don't want the eggs to get cold while the bacon is frying, so
  concurrent processing is required, too."

  "We must not forget the user interface. The lever that lowers the food
  lacks versatility, and the darkness knob is confusing. Users won't buy the
  product unless it has a user-friendly, graphical interface. When the
  breakfast cooker is plugged in, users should see a cowboy boot on the
  screen. Users click on it, and the message 'Booting UNIX v.8.3' appears on
  the screen. (UNIX 8.3 should be out by the time the product gets to the
  market.) Users can pull down a menu and click on the foods they want to
  cook."

  "Having made the wise decision of specifying the software first in the
  design phase, all that remains is to pick an adequate hardware platform
  for the implementation phase. An Intel Pentium with 48MB of memory, a
  1.2GB hard disk, and a SVGA monitor should be sufficient. If you select a
  multitasking, object oriented language that supports multiple inheritance
  and has a built-in GUI, writing the program will be a snap."

  The king wisely had the software developer beheaded, and they all lived
  happily ever after.

    [... and the rest is toast!  PGN]

------------------------------

Date: Fri, 5 Apr 2024 13:13:07 +0300
From: Amos Shapir <amos083 () gmail com>
Subject: Re: AI that targets civilians ... (RISKS-34.13)

Actually, using face-recognition methods may be the most humane way to tell
apart terrorists who hide among the civilian population.  Especially when
the alternative older methods were more like "kill them all and let God
sort them out".

  [...except for the rampant false positives in huge crowds... PGN]

------------------------------

Date: Fri, 05 Apr 2024 07:16:58 -0700
From: Geoff Kuenning <geoff () cs hmc edu>
Subject: Re: Your boss could forward a mail message to you that
 shows you text he won't see, but you will (RISKS-34.13)

I am famous among my colleagues for my insistence on reading emails in plain
text--to the point that when I receive an HTML-only email I will sometimes
eye-parse it rather than feeding it into a decoder (although that's getting
harder and harder as mailers insist on cluttering everything with
selectors).  And I *always* send in plain text.

My primary reason for using plain text has always been an aversion to web
bugs and to size bloat, but now I have a new justification.  Complicated
things can break in ways that are just impossible with simple ones.

  [... Those are all among the reasons RISKS is utf-8 only.  PGN]

------------------------------

Date: Fri, 5 Apr 2024 16:32:26 -0700
From: Steve Bacher <sebmb1 () verizon net>
Subject: Re: The FTC is trying to help victims of impersonation scams get
 their money back

I'm  not impressed.  The FTC is combatting this by creating a rule?  Aren't
these actions (mostly) already illegal?  Though I'm glad to see that they're
trying to outlaw fraudulent email sender addresses.  That's way overdue.

------------------------------

Date: Fri, 5 Apr 2024 16:04:50 -0700
From: Steve Bacher <sebmb1 () verizon net>
Subject: Re: Browsing in Google Chrome's incognito mode doesn't protect you
 as much as you might think (RISKS-34.13)

The Globe article unfortunately mixes descriptions of Google (Chrome)'s
behavior with explanations from Mozilla (Firefox) on how incognito mode
works.  Chrome and Firefox have separate implementations of this and other
modes, and I'm sure that Google has no inclination to follow what Mozilla
says, nor does Mozilla care how Google implements it.

It would have been useful if the article had enlightened us as to whether
Firefox has the same protection issues, since they bothered to quote the
Mozilla Foundation to begin with (info they probably scarfed from a Mozilla
web page anyway).

------------------------------

Date: Fri, 5 Apr 2024 13:08:49 +0300
From: Amos Shapir <amos083 () gmail com>
Subject: Re: Elon Musk's Starlink Terminals Are Falling Into the
 Wrong Hands? (Risks 34.12)

The positive side of this is that Starlink is a communication link which
falls under some US jurisdiction, and enables US security services to
eavesdrop on communications in remote areas of the world which were off the
grid till now, and therefore where outlaws and terrorists abound.

SpaceX's statement that they can "geolocate and turn off individual
terminals when it detects illegal use" -- and yet they haven't turned off
many suspicious links, may indicate that Musk may be collaborating with
such moves.

------------------------------

Date: Sat, 28 Oct 2023 11:11:11 -0800
From: RISKS-request () csl sri com
Subject: Abridged info on RISKS (comp.risks)

 The ACM RISKS Forum is a MODERATED digest.  Its Usenet manifestation is
 comp.risks, the feed for which is donated by panix.com as of June 2011.
=> SUBSCRIPTIONS: The mailman Web interface can be used directly to
 subscribe and unsubscribe:
   http://mls.csl.sri.com/mailman/listinfo/risks

=> SUBMISSIONS: to risks () CSL sri com with meaningful SUBJECT: line that
   includes the string `notsp'.  Otherwise your message may not be read.
 *** This attention-string has never changed, but might if spammers use it.
=> SPAM challenge-responses will not be honored.  Instead, use an alternative
 address from which you never send mail where the address becomes public!
=> The complete INFO file (submissions, default disclaimers, archive sites,
 copyright policy, etc.) has moved to the ftp.sri.com site:
   <risksinfo.html>.
 *** Contributors are assumed to have read the full info file for guidelines!

=> OFFICIAL ARCHIVES:  http://www.risks.org takes you to Lindsay Marshall's
    delightfully searchable html archive at newcastle:
  http://catless.ncl.ac.uk/Risks/VL.IS --> VoLume, ISsue.
  Also, ftp://ftp.sri.com/risks for the current volume/previous directories
     or ftp://ftp.sri.com/VL/risks-VL.IS for previous VoLume
  If none of those work for you, the most recent issue is always at
     http://www.csl.sri.com/users/risko/risks.txt, and index at /risks-34.00
  ALTERNATIVE ARCHIVES: http://seclists.org/risks/ (only since mid-2001)
 *** NOTE: If a cited URL fails, we do not try to update them.  Try
  browsing on the keywords in the subject line or cited article leads.
  Apologies for what Office365 and SafeLinks may have done to URLs.
==> Special Offer to Join ACM for readers of the ACM RISKS Forum:
    <http://www.acm.org/joinacm1>

------------------------------

End of RISKS-FORUM Digest 34.14
************************