Strange traffic (fwd)

[Michal Zalewski <lcamtuf () DIONE IDS PL>]

After several months of investigation, I decide to make this information
public, as I (and most of the people involved in this problem in some way)
can't see the way to solve this puzzle. I hope INCIDENTS readers could
support us. Let's start at the top of our iceberg:

Belsow you will find almost extact mail I've sent to exodus.net regarding
the activity we have noticed some time ago. I fixed some typos, cut some
parts and added few short explainations to make it more clean, but the
main sense is still the same. Firstly, I wanted to send it to
abuse () exodus net, but it seems to be spam-reporting address only (?), so I
tried hostmaster@, root@ and support@ as well. I haven't received any
response (well, I expected that) and I'm doubt if any response will ever
arrive, so I'm sending it here. The only thing is that activity
disappeared almost immediately after my e-mail and never came back. I am
somewhat disappointed with Exodus, but I am not going to re-send this mail
again and again hoping for a response - feel free to comment it.

During the investigation mentioned in this mail, we have noticed really
interesting activities from several other systems as well - for example,
some not really ugly examples of client invigilation done by the biggest
web companies. But for now, we are not going to start the hype, and would
like what readers of this list think about the activity we have seen. One
day, we might report other observations as well.

For polish-speaking people (you are lucky!), full documentation can be
found at http://lcamtuf.hack.pl/wtf/ - it currently almost 240 kB of logs,
hypotestis and analysis. It couldn't be done without extensive support
from numerous people - http://lcamtuf.hack.pl/wtf/wtf-1.html - which were
known as the RST+ACK team. Praise their work. I would like to thank jam,
smarkacz, szur, MadKarrde, neq, poncki, and LinuxNews.pl team at the
first. No english translation is available for now.

Michal Zalewski,
founder of the "RST+ACK project"

---------- Forwarded message ----------
Date: Tue, 10 Oct 2000 08:05:44 +0200 (CEST)
From: Michal Zalewski <lcamtuf () tpi pl>
Subject: Strange traffic

I'm writing this mail because I am really curious about the thing we - as
a group of network administrators - are observing for quite long time,
having no way to find any satisfying answer. You might think: "what the
hell it has to do with network abuses, this is not a violation of our
rules"... But I believe it's worth reading to the end. Ok, lets go:

About six months ago, in my test network, I've noticed strange RST+ACK
packets, coming from one of hosts belonging to your (Exodus, not assigned
to customers) network in Santa Clara - 216.32.132.250 - FreeBSD "irc
server" named 'irc.idle.net'. This machine seems to be firewalled by
statefull firewalling subsystem, and has rDNS entry
vip-testing.sntc04.exodus.net.

Such RST+ACK packets are certainly not the thing I could expect in
completely unused network - usually, such packet might appear only if some
"off-topic" (not related to open connection) packet with ACK bit set has
been sent to destination machine - but that was impossible, as my machines
were not able to send anything. I had complete TCP traffic logs, and for
sure there was no _any_ traffic coming from our network anywhere (as I
said, it was freshly launched test segment)...

Packet... delay... packet... packets were addressed to different nodes,
most of them - not existing (thus, my firewall replied with ICMP message).
Delays were quite often really close to 4 minutes, but _never_ below. Four
minutes - the default timeout for IDS scan detection routines. Funny:

Sun Jul 16 19:53:10 2000 : + TCP 0x14 216.32.132.250:7859 ->
213.25.176.126:10493 ttl=51 off=0x4000 id=0xd783 tos=0x0 len=40 phys=46

Sun Jul 16 19:53:10 2000 : + Packet dump: 45 00 00 28 D7 83 40 00 33 06 8D
99 D8 20 84 FA D5 19 B0 7E 1E B3 28 FD 00 00 00 00 BF 76 E8 4B 50 14 00 01
DD A8 00 01 B6 80 00 00 0D 91

Sun Jul 16 19:54:53 2000 : + TCP 0x14 216.32.132.250:12411 ->
212.160.116.95:60721 ttl=51 off=0x4000 id=0x3e11 tos=0x0 len=40 phys=46

Sun Jul 16 19:54:53 2000 : + Packet dump: 45 00 00 28 3E 11 40 00 33 06 63
A4 D8 20 84 FA D4 A0 74 5F 30 7B ED 31 00 00 00 00 84 AB 6C 4C 50 14 00 01
FB 0E 00 01 FB AB 00 00 7D 78

None of those destination nodes were present, as I said, just unused
IPs... Also, I am in real doubt if there is any way and sense for attacker
to cause this host to RST+ACK packets from these ports. More! This host,
as I said, seems to be firewalled in stateful manner, returning ICMP
messages when trying to send TCP packets to these ports.

Something told me not to ignore this thing. The situation become more and
more interesting. I and other administrators in polish networks, grepped
their logs looking for this specific IP. Not suprisingly, we found such
activity appearing for at least two months (some of us found even '99
entries). For sure, none (or almost none) of our networks ever established
connection to this C class, not saying about this specific host.

Sometimes this traffic disappeared for a few days to come back after some
time. And all other observations confirmed the nature of this activity -
slow, regular, addressed to most of the nodes in every C class we had.

We thought it's an effect of DoS attack or so (think about spoofed SYN
flood). But next six months were surprising. We started monitoring for
such "alone" RST+ACK packets in approx. 15 networks in Poland and 4
outside this coutry. Additionally, we monitored all traffic coming from
this 216.32.132.250 host, and several other targets. Results were sent
back to me everyday for futher analysis. The conclusion is somewhat
shocking - such traffic is really uncommon in normal conditions, while
this vip-testing host (and *only* this host - we haven't noticed such
constant traffic to networks that haven't never ever tried to connect to
the target - from any other BSD systems, nor any other irc servers - only
a few lost packets somewhere) is generating such traffic constantly. Just
like the sonar.

Our logs are now keeping the record of thousands packets from vip-testing
- and all of them falls into the rules described above. Perfect time
intervals, no "double node hits" in short periods of time... And, almost
always, short ICMP response, telling so much about the distance, firewall
/ router software and rules and so on.

The possibilities we can imagine:

1) Someone is performing spoofed SYN-flood DoS attacks on this machine.
   That was our first theory, but unfortunately, it's flawed. It could
   explain one, two or three incidents of this kind, but is very mere
   explaination of constant, periodic, specific traffic observed for
   over 8 months in numerous networks. Also, some attempts performed
   while receiving such traffic proven service seems to be not under
   heavy attack. Also, we were unable to find any evidence of tools
   that can be used for such attack - common tools are generating
   quite specific sequence numbers and source addresses, completely
   different from those observed by us.

2) Software / hardware bug; this theory isn't really good, as well, as
   this host is a regular Unix box. There are thousands, if not millions,
   similar machines in the Internet, but only this box is generating such
   traffic. This elliminates "software implementation bug" possibility.
   "Hardware bug" is also something unbelivable - as low-level hardware
   couldn't corrupt IP packet preserving control checksums and so on.

2) Someone is spoofing this traffic; this theory is absolutely senseless
   - provides (technically) correct answer, but does not explain sense
   of spoofing such packets for so long (or at all ;).

3) The most believable theory assumed it's very wide so-called
   reverse-mapped network scanning. In fact, ICMP messages returned
   by routers and firewalls could provide really interesting and
   valuable information about the Internet and particular subnets.
   It has been proven such data HAS a value - take a look at Caida project
   [1], sponsored by government agencies and huge companies. In this case,
   we're fascinated who - and why - is doing such tests, and why he is
   trying to stay undetected, using such sophisticated - well, practically
   undetectable - technique.

[1] Caida project map:
http://www.caida.org/analysis/topology/as_core_network/AS_Network.xml

This traffic has been last noticed one day ago.

Please tell me if you know, what we're observing. If it's known to you,
but couldn't be published, please tell me at least I shouldn't try
bothering you with such problems :) And, if it sounds so fascinating for
you as sounds for me, please make me know what was the solution :)

Thanks in advance,
_______________________________________________________
Michal Zalewski [lcamtuf () tpi pl] [tp.internet/security]
[http://lcamtuf.na.export.pl] <=--=> bash$ :(){ :|:&};:
=-----=> God is real, unless declared integer. <=-----=