Re: From int $13 to distributed object clouds

["Brian Azzopardi" <brian () gfi com>]

If you are starting with zero knowledge and doing the grouping as more
data is becomes known is problematic:
- communicating the result of the tests. If the scanning software is
client-server (a peer network would be more complex to sync) then all
the agents need to send the data back to the server after each test
which does the grouping. Maybe not a big problem, but you need to sync
with the clients, slowing you down.
- you still have to do the all tests anyway. Sure, it might allow you to
early-out the rest of the tests, but you can only do this if you accept
only one group. Once you want to do regrouping later on as more data
comes in you need to do all the tests for all the ips.

You could first do a parallelized scanning run to get only the data
which you need for grouping, do the grouping on the master, and then
farm out the groups for parallel vuln checking.

As another poster mentioned, the interfaces a machine may be exposing
could make a difference. IMO, its better to just scan the IP range and
then do clustering as a post-process. The data that comes back might
hint at groupings you did not foresee. Post-processing is usually
cheaper/faster then the scanning itself - which is why Dave wanted to do
the scanning in parallel in the first place.

Considered the case where a machine might belong to multiple groups?

Languages with inbuilt parallelization (Occam, Erlang, etc) usually have
a steeper learning curve then Python/C with their easily understood
(hacked-on) threading models. Implicit parallelization is cool - but
doing it in practice for general-purpose tasks is nigh on impossible.


Brian


-----Original Message-----
From: Jon Passki [mailto:jon.passki () hursk com] 
Sent: Friday, December 22, 2006 8:15 PM
To: Brian Azzopardi
Cc: Dave Aitel; dailydave () lists immunitysec com
Subject: Re: [Dailydave] From int $13 to distributed object clouds

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1


On Dec 22, 2006, at 03:06 , Brian Azzopardi wrote:

> > They need to be grouped intelligently
>
> Can't you group IPs intelligently and then farm out the groups to be 
> handled in parallel?

If I'm not hitting on the hookah too hard and understand what Dave's
talking about , partitioning the universe of our hosts will occur more
than once.  This is different than parallelism with one-time
partitioning, ran until completion on a host.  The parallelism,
methinks, would have to be dynamic and flexible (perhaps intelligent
:-).  For example, assuming a zero-knowledge beginning, there's no way
to start grouping IP's/targets/assets until something is learned,
besides the asset identifier.  If you group as soon as something is
known, such as subnet location of externally-facing IP addresses, and
then want to regroup later on, such as subnet location of
internally-facing IP addresses, all the hosts performing the parallelism
potentially need to reshuffle their groups.


> > Some IP addresses are the same machine, and we need to know that
> 10.0.1.1 and 10.0.2.1 are the same machine
>
> You can do that as a post-process (assuming you don't do the 
> intelligent grouping first).

What's the point of messing with 1.1 if you already have it under the
identity of 2.1?  If the goal is to perform as little action as possible
(e.g. to be covert, to quickly gather data, and/or to reduce data
analysis and post-grouping), then this is a wasted action. It just
becomes a balance between the cost of reaching this goal and the cost of
violating this goal.  Going w/ the concept above of regrouping and
reshuffling the parallelism, one would want to perform the least amount
of overlapping tests across all assets or groups of assets.  Since, if
you reshuffle later, it would be ideal to combine disjointed sets versus
identical sets.  Again, there's a cost in doing this...

> >  intelligent parallelism handled by a language
>
> What do you understand by intelligent parallelism? Is Occam 
> intelligent enough? Do you prefer implicit parallelism?
>
> Just for the record, I am working (slowly) on an new language that has

> parallelism as fundamental part of the language, rather than tacked on

> to it via threads like Python/C++/etc.
>
>
> Brian
>
>
>
> -----Original Message-----
> From: dailydave-bounces () lists immunitysec com
> [mailto:dailydave-bounces () lists immunitysec com] On Behalf Of Dave 
> Aitel
> Sent: Friday, December 22, 2006 4:43 AM
> To: dailydave () lists immunitysec com
> Subject: [Dailydave] From int $13 to distributed object clouds
>
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> The question you have to ask yourself when dealing with, as Sinan 
> would call it, "NP Complete Stuff" (aka, anything academic and wanky) 
> is "How is this going to help me hack something". Lately I've been, in

> the back of my head, obsessed with distributed object languages. But 
> how can I explain that having your language abstract not just memory 
> management, but also parallelism, is going to help you break into more

> computers faster and better?
>
> The problem set is easy to understand: scanning a range of IP 
> addresses for exploitable vulnerabilities and then exploiting them.
> People look at that and say "Easy to parallelize. Just split it up 
> based on IP range.". They'd be wrong - IP addresses are connected to 
> each other in many ways. They need to be grouped intelligently, and 
> deep down, we're breaking into machines, not IP addresses. Some IP 
> addresses are the same machine, and we need to know that 10.0.1.1 and
> 10.0.2.1 are the same machine even if they've been split up across 
> scanning processes which reside on different computing clouds. We also

> need to use information gained from hacking 10.0.1.1 against 10.0.2.1.
>
> Something in my right brain is telling me parallelism is the next big 
> step for something like CANVAS. Not simple "split it up into bite size

> pieces", but intelligent parallelism handled by a language that is as 
> much like Python as possible, but time abstract. Possibly the easier 
> next step is built-in data-mining and CRM. When we do open source data

> collection on a target, I need somewhere to enter that in that can 
> reuse that information automatically. And when I own 10,000 machines, 
> I need to be able to mine that cloud for the information I'm 
> interested in, covertly.
>
> Of course, in the meantime it's shellcode shellcode shellcode. No 
> hacker ever truly gets away from that. Even here in Aotearoa there's 
> an int $13 waiting...
>
> - -dave
>
> P.S. Congrats to NFR :>

[snip]

Jon

Obvia conspicimus, nubem pellente Mathesi.

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.1 (Darwin)

iD8DBQFFjC7QZpJsLIS+QSIRAo4cAJ9OWSwExWSsZZoQSu/9Du9ylEuZBwCfbTDS
lPOWj/3zxuetRm8AAbNX5eo=
=OiTY
-----END PGP SIGNATURE-----

DISCLAIMER
The information contained in this electronic mail may be confidential or
legally privileged. It is for the intended recipient(s) only. Should you
receive this message in error, please notify the sender by replying to
this mail. Unless expressly stated, opinions in this message are those
of the individual sender and not of GFI. Unauthorized use of the
contents is strictly prohibited. While all care has been taken, GFI is
not responsible for the integrity of the contents of this electronic
mail and any attachments included within.  

This mail was checked for viruses by GFI MailSecurity. 
GFI also develops anti-spam software (GFI MailEssentials), a fax server (GFI FAXmaker), and network security and 
management software (GFI LANguard) - www.gfi.com 


_______________________________________________
Dailydave mailing list
Dailydave () lists immunitysec com
http://lists.immunitysec.com/mailman/listinfo/dailydave