Re: RFC: Forking Snort

[Martin Roesch <roesch () sourcefire com>]

Wow.

Mea culpa, I haven't been as good a communicator with the Snort community as
I should have been, my fault.  Having a baby and a startup place many
demands on my time and it's hard to be as publicly active as I've been in
the past, but rest assured I'm still leading the development of Snort behind
the scenes as much as ever.

Yes, Sourcefire is heavily involved with the development of Snort because in
order for Sourcefire to be successful, Snort has to be the best technology
available for performing intrusion detection, period.  It is the opinion of
Sourcefire's CEO[1] (as well as my own) that Snort must remain open source
and be the best that we can make it for us to be successful.  That means
making it faster, more capable, more configurable and more undeniably the
most excellent detection technology that we can muster.  *All* of this code
that goes into Snort is released open source, you get it for free.  The
Snort you used yesterday will still be available for free tomorrow and be
better, faster, more capable and more robust.  How this could possibly be
contrary to the users of the free product is a mystery to me.

Code scrubbing is another matter.  Things that detract from Snort's
performance or stability don't help anyone, including Sourcefire.  The fact
of the matter is that all the complex output plugins should be moving over
to Barnyard, leaving them in the core Snort code is an invitation to dropped
packets and crashes, we've all seen them over the years.  (How many times
have we heard 'hey, my CPU is using 99% when I'm logging to MySQL!' or 'I
just ran out of inodes!'?)  Barnyard is *not* that hard to use, and is also
open source.  By using it, people get better use out of Snort because it
performs better, and they get Snort's data produced in a nice archival
format at the same time (unified format).  Criticisms leveled at existing
plugins for speed and stability are valid and I stand behind them, my code
is far from perfect as well but I try not to repeat the same mistakes over
and over again, I fix my broken code, and I've been at it and doing it for
free for almost 4 years.

Let me see if I can make this easy for everyone to understand.  I laid these
basic points out for our investors when I was talking to them over the past
few months, everyone was aware of them and ok with them before I'd move
forward:

1) Snort is open source code and will remain open source code, I've said it
in public for years and have always maintained the exact same stance.

2) Sourcefire can only succeed if Snort is the best we can make it from a
performance, feature and stability standpoint.

3) Having an open source community that uses the open source product is
vital to Sourcefire's success because it's our best mechanism to developing
the sensor technology in ways that people find useful.

In short, Sourcefire succeeds only if Snort is successful and free.  The
board of directors of Sourcefire is in complete agreement with that
proposition, including the venture capitalists (in fact, they see it as the
great equalizer between us and giant competitors like Cisco).

Towards that end, some serious technology has been developed to keep Snort
competitive over the past 1.5 years since I started Sourcefire:

[*] stream4 - stateful inspection, tcp stream reassembly, stats collection
for post-processed statistical analysis, tcp anomaly detection, tcp flow
keyword, survivable under attack

[*] frag2 - IP defragmentation, IP fragmentation anomaly (DoS) detection,
survivable under attack

[*] Spo_unified - unified logging system for integration with barnyard,
output manipulation is performed in a separate process for performance and
scalability

[*] Barnyard - unified output processor, let's Snort do its job while
complex and relatively slow tasks like DB writes happen OOB

[*] Ruleset - major cleanups, lots of additions

[*] Spp_bo - back orifice detection (known cyphertext attack against the
crypto protocol instead of the old brute force method)

[*] Spp_telnet_negotiation - telnet protocol normalization

[*] Spp_rpc_decode - rpc protocol normalization

[*] Stability and bugfixing - guys like Chris Green and Andrew Baker can
work on Snort full time now, I get to occasionally too :)

There's a lot more cool stuff in development or on the drawing board as
well.

Running a major open source project is a lot of work, it seems that the more
you try to do to make it more fully a part of your life, the less people
like it.  I try to be understanding, I have a lot of people that want my
time (Snort, company, wife & daughter, SANS, etc) and it's hard to balance
it all.  That said, I have no desire to see Snort forked or brought into a
committee form or management, there are tons of inefficiencies there and I
don't see many benefits to anyone except from the very same corporate
interests that this is supposed to save us all from.  I've worked very hard
to make sure that Snort is the best IDS available and that you (the open
source users) can always continue to use it.  If that's not good enough then
I'm at a loss as to how to help people, but I will say that I would
personally feel very hurt if a branch was to happen because all of you had
lost faith in my ability to develop Snort in an honest and open manner and
enforce that with the rest of the developers as well.

I hope that this clears the air to some degree, there is no "corporate
conspiracy" to wrest Snort from the open source community, nor is there a
systematic attack on Jed Pickel's legacy code contributions.

I won't address the rest of Jed's comments at this time, but I feel the
basis for his arguments are unfounded and that Sourcefire's existence is
complementary to Snort's continued development, as it was designed to be
from the beginning.  I hope that this email will help people understand
where Snort's development is going and what the developers who work at
Sourcefire are doing with regards to it.  Jed, please feel free to mail me
directly or call me at Sourcefire if this doesn't assuage your fears,
(410)-290-1616.


      -Marty


[1]http://www.sunspot.net/business/bal-bz.sourcefire26jun26.story?coll=bal-p
e-business

On 7/2/02 2:50 AM, "Jed Pickel" <jed () pickel net> wrote:

> This document is intended to gauge the interest of the Snort community
> in creating a fork of Snort that is governed by a consortium (similar to
> Apache's "Apache Software Foundation") rather than a single profit
> driven corporate entity. Below I will provide some background as to why
> I am bringing this up. There are advantages and disadvantages to this
> from nearly every perspective; thus, I encourage comments and discussion
> of all opinions.
>
> Snort has come to a critical point in its evolution. Due to the hard
> work from numerous developers and thousands of users, Snort is now
> monitoring many of the worlds most sensitive networks. Also, a growing
> number of companies are offering commercial solutions based on Snort and
> standardization efforts have leveraged Snort as a conduit toward
> furthering security standards. As a result, the number of Snort users
> continues to grow as it becomes more commercially accepted.
>
> Few would disagree that Snort has successfully become a "killer app".
> The challenge Snort now faces is how to avoid becoming a victim of its
> own success. Apache is an example of open source code that has
> successfully bridged the gap from killer app to significant piece of
> Internet Infrastructure. This success can be attributed to governing and
> regulating Apaches growth through a consortium. I believe Snort could
> benefit from the same type of arrangement.
>
> Unfortunately, the forces that have brought Snort this level of success
> are falling out of balance. With Marty at the helm of both a wildly
> successfully open source project and Sourcefire (a growing, soon to be
> 800 pound gorilla in the intrusion detection market) he is faced with
> answering to a board of directors on one hand and the security community
> on the other. These are opposing forces with dramatically different
> goals. It is simply not possible for a single person to serve both of
> these roles and act in the best interest of each.
>
> While the number of users of Snort is growing, the percentage of
> community contributed code is decreasing. The reasons for this are not
> immediately obvious. Although there is plenty of community interest in
> contributing code, these interests are aparently in conflict with the
> goals of Sourcefire. Thus, some contributions have had been subjected to
> stealth deletions, others have never been incorporated in the codebase
> or have been re-written by Sourcefire to be more accommodating toward
> their goals.
>
> The most successful of the contributed code has been subjected to
> consistent negative and inflammatory PR campaigns. Marty carries this
> out this by proclaiming to the community false and misleading statements
> such as --- "Many of the contributed plugins, Marty says, 'were
> bug-filled, crashy, and slowed things down.'"[1] This tactic began to
> manifest in an unhealthy way a little over a year ago, shortly after
> Sourcefire was getting started.
>
> One can only speculate the strategy of Sourcefire in the long run;
> however, it would be foolish to think the goals of Sourcefire do not
> include maximizing profits. I have plenty of respect for Marty and I
> believe he has the best of intentions; however, he is no longer the man
> with the final say at Sourcefire. The investors of Sourcefire now
> control the critical strategies and goals of the company. There will
> undoubtedly and understandably be pressure from Sourcefire investors to
> gain more control of Snort while creating barriers to entry and stifling
> the competition.
>
> There are a vast number of Snort add ons and wrappers (both open source
> and proprietary) that lead me to believe Snort is on the track toward
> becoming something of an operating system of intrusion detection that
> forms a base for numerous applications and business to grow and
> flourish. I would like to see an environment of healthy competition in
> this market to benefit the consumer, security community, and provide the
> opportunity for independent developers and business to find some niche
> and profit from their work.
>
> These are the reasons why I believe now is the time for the community to
> begin discussing forming a branch of Snort that is governed by a
> consortium that is not profit driven, but rather exists to support the
> best interests of the community and support healthy competition among
> all of the companies that are providing Snort based security solutions.
>
> This is a sensitive topic, but I believe the time has come to surface
> it. I'd like to hear your opinion... Is now the right time to begin
> considering a fork or branch or Snort? What benefits or advantages would
> this create for end users, business that use Snort, business that
> provide products or services based on Snort, or the security community
> as a whole? If a consortium were formed for governing a new fork of
> Snort who or what businesses, organizations, or individuals should that
> involve?
>
> All comments, flames, and opinions are welcome. The sole intention of
> this message is to initiate discussion.
>
> Regards,
>
> * Jed
>
> References
> -----------------------------------------
> [1] http://newsforge.com/newsforge/02/06/29/2127239.shtml?tid=3
>
>
> -------------------------------------------------------
> This sf.net email is sponsored by:ThinkGeek
> Welcome to geek heaven.
> http://thinkgeek.com/sf
> _______________________________________________
> Snort-users mailing list
> Snort-users () lists sourceforge net
> Go to this URL to change user options or unsubscribe:
> https://lists.sourceforge.net/lists/listinfo/snort-users
> Snort-users list archive:
> http://www.geocrawler.com/redir-sf.php3?list=snort-users

-- 
Martin Roesch - Founder/CTO Sourcefire Inc. - (410) 290-1616
Sourcefire: Professional Snort Sensor and Management Console appliances
roesch () sourcefire com - http://www.sourcefire.com
Snort: Open Source Network IDS - http://www.snort.org



-------------------------------------------------------
This sf.net email is sponsored by:ThinkGeek
No, I will not fix your computer.
http://thinkgeek.com/sf
_______________________________________________
Snort-users mailing list
Snort-users () lists sourceforge net
Go to this URL to change user options or unsubscribe:
https://lists.sourceforge.net/lists/listinfo/snort-users
Snort-users list archive:
http://www.geocrawler.com/redir-sf.php3?list=snort-users