Re: Asynchronous

[Thomas Ptacek <tqbf () matasano com>]

Generally I find threaded code harder to bug than async, because there
are more dimensions to the problem; you have to get synchronization
right *and* you have to be catching accidental serialization.

The code is ostensibly easier to read, but at any time it's operating
on any of three levels (the code, the interactions with other threads,
and the implied scheduling of access to other resources), which isn't
the case in async (except when you have timers attached to temporary
objects).

People that need to scale up large async programs often use half-sync
half-async patterns. It's often the case that the things in your
program that benefit from straight-line code don't need so much
performance or sharing, and the things that need high performance can
be scoped down tightly enough to make callback state machines
palatable.

It helps very much to be in a language with good anonymous functions.
Javascript is one of them; Lisp is the canonical example; Ruby is
serviceable as well. Callback state machines split among lots of
little named functions are a bitch.

Message middleware is a huge win here; you can pick and choose what's
going to be async and what's going to be straight-line and communicate
between them on the message bus.

There's a huge literature of this stuff that the Node crowd has not
yet discovered. You have family members that know a lot about this
stuff, Dave. Another great place to go is Schmidt's ACE patterns page
(but DON'T USE ACE), or his giant ugly Pattern Oriented Software
Volume 2 --- which incidentally if you're inclined to scoff at
"pattern wanking" may change your mind about the fundamental concept.


On Wed, Oct 5, 2011 at 11:38 AM, Dave Aitel <dave () immunityinc com> wrote:
> So while I'm in the process of running a large asynchronous event-driven
> product in a VM in another window, it's a good time to read all sorts of
> things about asynchronous programming.
>
> Frankly, I'm not a huge fan of it, but Chris is, and he's a better
> programmer than me, so we'll leave it at that. Largely, I think people are
> fans of Async because most languages and kernels are terrible at threads.
> Python, for example, does not have threads. "No worky worky", as we say
> around here.
>
> The downside of async is that it is basically impossible to debug, and you
> need a programmer as good as Chris to even begin to use it well. Likewise,
> the locking/blocking problems don't go away, they just get mapped into a
> more inscrutable form.
>
> So for systems that have working threads, you use them for anything IO-heavy
> (aka, web servers/crawlers/other useful hacking tools). But since almost no
> system HAS working, scalable, threads, people get excited about async.  And
> then you spend your whole life saying "Hey, this thing I do sometimes has to
> do a lot of work on the CPU, so let's put it in another thread please". Or
> "Hey, this whole giant DB library we have to use isn't built from the ground
> up to use Async, so we need it to be in its own thread and manage a feeder
> Queue to it".
>
> And then eventually you're like "Why on earth am I spending so much time
> worrying about how efficient an algorithm that runs on one machine is?" and
> you go off and build something that scales horizontally onto multiple
> machines. (Where "multiple" is > 100).
>
> But in the meantime you have things like asyncore and Node.js and stuff. I
> can't do them justice, but these posts below are the funniest thing you'll
> read since Steve Yegge.
>
> http://teddziuba.com/2011/10/node-js-is-cancer.html
> http://teddziuba.com/2011/10/straight-talk-on-event-loops.html
> http://www.unlimitednovelty.com/2011/10/nodejs-has-jumped-shark.html
>
> -dave
>
> --
> INFILTRATE 2012 January 12th-13th in Miami - the world's best offensive
> information security conference.
> www.infiltratecon.com
>
> _______________________________________________
> Dailydave mailing list
> Dailydave () lists immunityinc com
> https://lists.immunityinc.com/mailman/listinfo/dailydave



-- 
---
Thomas H. Ptacek // matasano security
read us on the web: http://www.matasano.com/log
_______________________________________________
Dailydave mailing list
Dailydave () lists immunityinc com
https://lists.immunityinc.com/mailman/listinfo/dailydave