Power Utility Substations At Risk

[Audrey McNeil <audrey () riskbasedsecurity com>]

http://www.darkreading.com/vulnerability/power-utility-substations-at-risk/240165567

Nearly 30 security vulnerabilities so far have been found in products using
a popular ICS/SCADA communications protocol, prompting about half of the
affected vendors to patch their products, and at least one vendor to pull
its affected software from the market and urge its customers to instead
install another one of its products.

The findings by researchers Adam Crain and Chris Sistrunk of potentially
dangerous bugs in ICS/SCADA products running the so-called DNP3 protocol --
used for "master" host systems to communicate with equipment at power plant
substations -- could be easily exploited by an attacker to disrupt parts of
the power grid by crashing the master system so it can no longer monitor
and control the SCADA network at a substation or substations. The attacks
would entail sending malformed DNP3 response packets back to the master
host system by exploiting flaws in the way software using DNP3 is written
and deployed.

Cooper Power Systems, which was notified by the researchers of an improper
input validation flaw in its Cybectec DNP3 Master OPC Server software,
discontinued the server product rather than patch it, and is urging its
customers to use its SMP Gateway product -- which doesn't carry the flaw --
as a replacement. The bug could allow an attacker to crash the system and,
ultimately, disrupt the process it was running.

Last week at the S4x14 conference in Miami, Crain and Sistrunk disclosed
new details on the so-called Project Robus research that they quietly began
in April 2013. The researchers have been using Crain's homegrown fuzzing
tool for DNP3 implementations, and so far have reported some 28 flaws,
resulting in 16 security advisories from the ICS-CERT and related vendor
patches. Only two products that the researchers tested have not had DNP3
flaws, and the researchers are awaiting word on nearly a dozen additional
bugs that they have reported.

Some 75 percent of North American power facilities run DNP3, which was
developed in 1993. The protocol is used for "master" servers to communicate
with remote terminal units in electric substations, gas pumping plants for
gas pipelines, and water utilities, for instance. That includes monitoring
voltage or water levels, for instance.

Sistrunk, an engineer with an electric utility, a few months ago decided to
try out Crain's open-source DNP3 fuzzer in his lab. "I tested it on a few
things I have access to that had DNP3, and they broke. So I said, 'Time out
-- we need to have a pow-wow and talk about what we're going to do because
this is pretty big,'" says Sistrunk, who conducted the DNP3 research
independently of his utility company, which he ask not be named.

An attacker could exploit these bugs and take down a remote site such that
the utility would have no visibility or control over it anymore, says Dale
Peterson, founder and CEO of Digital Bond, an ICS/SCADA consultancy that
hosts the S4 Conference. "What it really means is that someone can go to an
unmanned facility and take out the visibility of the entire SCADA system
... There's no need to go to the control center. They can pick [a power
substation] in the middle of nowhere, go and break in, hook something up,
and the whole thing goes down," Peterson says.

Sistrunk and Crain said that they also have found 90 or so DNP3 devices
exposed on the public Internet. "The majority are misconfigured ... this is
the [tip] of the iceberg. How many are on the Net that don't say anything?"
said Crain, who is CEO of Automatak and the principal author of the Open
DNP3 stack.

The exposed equipment is yet another example of the millions of public
Internet-facing equipment found vulnerable and wide open to attack. Project
SHINE, which has been gathering data on SCADA/ICS devices from SHODAN for a
year-and-a-half, has identified more than 1 million unique IP addresses to
date, and 2,000 to 8,000 new devices each day. According to Bob
Radvanovsky, one of the Project SHINE researchers, the devices contain
buffer overflows, misconfigurations, and cross-site scripting flaws, among
other vulnerabilities.

The good news is that patching DNP3-based systems doesn't come with the
baggage and risk of patching a PLC or other plant-floor system, where
patching comes with risk of shutting down critical systems if a newly
patched system goes awry. "It wouldn't be that much of a headache. I think
that's an important point: We're not talking about the systems in the
substations. We're talking about the master servers," says Ralph Langner,
founder of Langner Communications, an ICS/SCADA consultancy. "It's like
average IT equipment running a Microsoft OS."

And it's a relatively small number of "master" systems that are set up with
redundant systems so that taking one down doesn't take down an entire
plant, notes Digital Bond's Peterson. "I would expect to see something like
this being patched. There's no excuse not to ... I expect over the next
year or two a large percentage will apply the patches."

But that doesn't mean the sites with Cooper's now-defunct and vulnerable
system, for example, will swap it out for the vendor's more secure product
right away. "It'll be out there until it fails. Owner-operators, especially
with the recent economic climate, will run things until they break,"
Sistrunk says. "Jiggling the red wire is cheap. It's cheaper than replacing
[a system]," he says.

One of the vendors whose code was found vulnerable was Triangle Microworks,
a.k.a. TMW, which sells source code for deploying DNP3 to several SCADA
vendors. The company has since patched its DNP3 Master and Outstation
Source Code Library packages. The improper input validation bug could allow
an attacker to remotely send a malicious TCP packet from the master station
on an IP network, sending the substation device into an "infinite loop." An
attacker with physical access to the master station could do the same thing
to a serially connected device.

"It would be missing data and run until someone notices," Crain said. "If
an operator isn't directly on that server, depending on how the software is
architected internally, he might never know, and it might remain deadlocked
until someone notices the data has not refreshed."

Not all of the vendor patches have gone smoothly. One vendor initially
released details of the bug in its release notes, and two others issued
so-called "silent fixes," meaning that they didn't coordinate their patches
with ICS-CERT or notify their customers of the problem. And another vendor
attempted to fix the flaw a couple of times, but ultimately "gave up,"
Sistrunk says.

There are some ways to mitigate DNP3-borne exploits, including not allowing
DNP3 networks to touch the corporate firewall, ensuring strong physical
security at substations, and ensuring that third-party software is tested
for security before you buy it, Sistrunk says. "Ask for secure
authentication and encryption," he says. "You can do SIEM on the enterprise
side and just watch and know if you see something is wrong."

Meanwhile, the DNP Users Group board of directors emphasized in a document
about the vulnerability reports that the problems were not with the DNP3
protocol itself, but rather with software implementations of it. DNP3
includes secure authentication in the application layer for both serial and
IP communications.

The users group worked with Sistrunk and Crain on guidance for application
developers on DNP3, which is available here (PDF) for download.

"The DNP3 protocol is sound. It is indeed possible to write a robust DNP3
implementation. This is evidenced by the few devices that Crain and
Sistrunk were not able to crash with the Aegis fuzzer," says Jacob Brodsky,
Chair of the DNP Users Group. "However, there is no denying that the DNP3
protocol is subtle and complex, mostly because SCADA systems themselves are
a subtle and complex endeavor."

Sistrunk says deploying secure authentication doesn't preclude all of the
bugs he and Crain discovered.

_______________________________________________
Dataloss Mailing List (dataloss () datalossdb org)
Archived at http://seclists.org/dataloss/
Unsubscribe at http://lists.osvdb.org/mailman/listinfo/dataloss
For inquiries regarding use or licensing of data, e-mail
        sales () riskbasedsecurity com 

Supporters:

Risk Based Security (http://www.riskbasedsecurity.com/)
YourCISO is an affordable SaaS solution that provides a comprehensive information security program that ensures focus 
on the right security.  If you need security help or want to provide real risk reduction for your clients contact us!