Snort mailing list archives
RE: [Snort-devel] Re: RFC: Forking Snort
From: "Bob Walder" <bwalder () spamcop net>
Date: Fri, 5 Jul 2002 09:49:18 +0100
As an outsider with no vested interests here I will chip in with my 0.02 (whatever that is worth at current USD/GBP exchange rates...) Version 1.8.1 was the first version of Snort we included in our IDS Group test - it was not the first version we had "looked at". However, even from 1.8.1 to 1.8.6 (the version we most recently tested - see www.nss.co.uk/ids) we noted huge improvements in many aspects of the product. Improved resistance to most IDS evasion techniques, some pretty useful stateful reassembly with a session resync feature that is missing from a lot of commercial products, some useful-looking protocol analysis preprocessors, improved stability. And, of course, a huge improvement in the signature set.
From what I have seen on the mailing lists, there has been some considerable
effort put into tidying up the code "under the hood". All of these improvements seem to have come about since the incorporation of Sourcefire. Taking a purely cynical view: prior to that, Snort was shambolic in places (witness different methods of passing parameters to preprocessors in snort.conf - this is the sort of thing that needs to be made consistent in a commercial-grade product) - having a major commercial product with some serious VC funding behind it being based on Snort, and with the happy coincidence that the creator of Snort heads up that commercial operation, it is hardly surprising that some serious effort with some full-time developers has been put it. It HAD to. Prior to Sourcefire, Snort was not really suitable to use as the basis for a major commercial product - now it is very close. And - in contrast to some of the claims in recent posts - this has all happened pretty quickly as far as I can see (i.e. 1.8.1 to 1.8.6 took a little over 6 months) In other words, from the outside looking it I can only see GOOD things having come out of the Sourcefire link so far. Can it improve further? Of course it can - performance needs looking at now, but we are promised good things in the next major release. With commercial pressure driving the development, I am sure we will not be waiting as long for that release as we would if it were being developed by committee! Regards, Bob Walder Director The NSS Group
-----Original Message----- From: snort-devel-admin () lists sourceforge net [mailto:snort-devel-admin () lists sourceforge net]On Behalf Of Martin Roesch Sent: 05 July 2002 05:00 To: Jed Pickel; snort-devel () lists sourceforge net Cc: snort-users () lists sourceforge net; focus-ids () securityfocus com Subject: Re: [Snort-devel] Re: RFC: Forking Snort On 7/4/02 3:50 AM, "Jed Pickel" <jed () pickel net> wrote:First off: I sent the last mail based on what I've witnessed as a growing dis-satisfaction and concern over the managementof Snort overthe past year. My leaving the Snort project 6 months agowas part ofthat and I recently have received an increase in privatecommunicationabout the topic. Being in a decent position to stick my neck out I crafted the last message in an attempt to get a gauge onthe community. Interesting how there can be "growing dissatisfaction" in the user base without any of the core developers hearing of it. When you left the project back in January the only reason that you gave was "I don't have time for the project any more", not that it was somehow being mismanaged by myself. Who are you in "private communication" with about the management of the project? Do they have any commercial interest in Snort? Seeing as your neck is out instead of theirs is pretty interesting, why can't they speak for themselves?I will state for the record at this point that I believe afork in Snortis not the best route for the community at this time;however, I believesome changes in management philosophy and process for Snort are necessary to prevent a fork. The changes I am proposingare in the bestinterest of users, developers, third party businesses,Sourcefire, andthe projects current leadership in my opinion.Have you ever managed a large open source project before? What is the basis of your opinion?Mixing the "benevolent dictator" model with business ------------------------------------------------------------------ I believe Snort would benefit from a change in managementphilosophy andthat the current leadership is capable of implementing andfollowingthrough on these changes. To explain the reasons lets consider what would happen ifLinus Torvaldswere the CTO of RedHat.Alan Cox is the lead developer on the production kernels and works for RedHat, so you can probably say that the situation exists. Has Linux's development suffered as a result of it's developers being able to work on it full time and for pay?* Would Linus accept contributions to the Linux kernelfrom the Debianor Suse folks?Yes, just as I continue to take contributions from a number of people from the community (Mike Fisk, Jed Haile, Glenn Mansfield Keeni, Roman Danyliw, Phil Wood, etc.). The reason that things get into or out of Snort is based on what they do (functionality) and how well they do it and not a whole lot else. That's always been the case, performance, portability and stability are the criteria against which contributions are judged. If code doesn't make it into the distro, people still have the option of maintaining it as a patch against the core distro, such as the "ac" patches for Linux or the SnortSAM module for Snort. Because it doesn't make it in doesn't mean you can't maintain it separately. This brings up a whole new discussion: who maintains the code. For example, if you have walked off 6 months ago and hadn't had Roman Danyliw there to pick up maintenance of the code, who would have maintained it? What if nobody but me was left to do it? Should I be required to? Would I be an "evil corporate guy out for his own interests" if I were to remove the code from the distro because I didn't want to maintain it? If we're going to go down the road of hypotheticals there are all sorts of bizarre scenarios we can come up with I'm sure. Why are we bothering to talk about things that haven't come to pass as debating points for whether we should take drastic action?* Would existing third party code contributions be "scrubbed" if it meant solidifying RedHats market share and increasing sales?Scrubbing means nothing more than bug fixing and sanity checking. For example, does it make more sense to use automatic variables (like a char array) in functions or is it better to malloc to a pointer, use the allocated memory space, then free that pointer at the end of the function? Who's responsibility is it to fix the code if it's doing the latter instead of the former due to the inexperience of the coder? What if the advice is given and not heeded? When can I as maintainer take the initiative to make a contributor's work better? Since it's "my" codebase, shouldn't I be able to do what I want with it? Linus certainly seems to feel free to run the Linux project and manage patches/contributions anyway he feels fit.* (Or -- following on a previous post) What if a contribution would be a huge leap in Linux kernel technology but it wouldmean rewritingthe proprietary pieces of RedHat Linux all while dealingwith customersdemanding this feature. Would that contribution to the"open-source"Linux kernel be delayed or ignored?Like the pre-emptive kernel patches that Linus hasn't allowed into the kernel because they violate his submission guidelines? There's a good example. He's not working for a company with a commercial interest in the development of Linux, yet something along those same lines is happening because the change is as yet unproven and the patch is so big that Linus rejected it for violating his submission guidelines. You don't have to have commercial interest in the project to reject things. I'm not sure that allowing the community to play Monday morning quarterback for the inclusion of code is a good thing. I've found over and over again that many contributors to Snort have little concept of writing high performance code or effective testing and validation techniques. Is it better to have a committee of people with no focus on the real needs of the project or a bunch of like minded individuals who have a similar level of experience and perspective of what's important?* What would happen to the Linux kernel if the RedHatinvestors soldthe company to Microsoft for some ridiculous sum of money and Linus had signed a non-compete? The "benevolent dictator" would be out of commission and the Linux community as a whole would suffer, particularly if the intention of Microsoft was to disrupttheir maincompetitor. Would the fallout halt the progress of Linuxwhile a newmanagement structure forms out of the chaos?That's fantasy land stuff, I won't even address it. I'm not going to discuss Sourcefire's corporate strategy where all my competitors can see it, Snort will always be free and I'm going to be the guy heading it until and unless I hand it off to another highly qualified individual with the skills and the personality to handle the project.* Or would the Linux kernel either pro-actively fork or beforced intoa different style of management because of community pressure to avoid the above problems?If people want to fork Snort they can do as they please, but without the support of the core developers all you've got is 64,000+ lines of legacy code to start maintaining. I'm only seeing "pressure" from an exceedingly small group of the overall Snort community, so I don't see any great cry to address the specters that you raise here. Quite frankly, the fact that this was done "Pearl Harbor" style continues to have me scratching my head as to the identity of the silent 3rd parties that are advocating this course of action.Sourcefire can continue as is for a while but conflicts of interest are going to come up regardless of the promises from Marty. Walking the fine line of preventing a fork and maximizing profits can be avoided all together by pro-actively changing themanagement style andphilosophy of open-source Snort.You state as fact things that you can't possibly know. Why are there suddenly going to be conflicts of interest where there have been none before? Have there been any conflicts of interest? Why am I going to go back on the one key promise that is essential to the success of my corporate endeavor? You state as an inevitability something that hasn't happened in 18 months of Sourcefire's history, indeed Snort's development has accelerated rapidly since the founding of Sourcefire and I think few people disagree that Snort is vastly improved over the last pre-Sourcefire version (version 1.7).I'm a big fan of the "benevolent dictator" model and Ithink it worksbeautifully when profit is not one of the primary motivators of the dictator. When profit motivations are thrown into the mixmeasures haveto be put in place to retain the public confidence thatthese conflictsof interest can not manifest. No doubt there are timeswhen business andopen source goals run in parallel. The above hypotheticalanalogy ofLinus as the CTO of RedHat are a few examples where they do not."Profit" means different things to different people. Is profit the notoriety associated with running a successful open source project? It sure as hell is! Up until Sourcefire was started, I made *nothing* directly on Snort but quite a bit on being the author of Snort. I didn't get paid for developing Snort, but I gained some amount of fame which in turn led to better jobs and opportunities to get paid for speaking. You seem to ignore my stated position that Sourcefire can only succeed by making Snort as good as possible, do you doubt that that is the case?One of certainly many possible ways to address this issuecould be forSourcefire to create a non-profit corporation to manage open-source snort. A possible way to structure such an organization is how the Apache folks have. Take a look at this url (http://www.apache.org/foundation/roles.html) for anexample. The idealwould be to ensure that a diverse group is represented onthe Board OfDirectors for this non-profit corp including users, developers, and businesses that provide Snort based solutions. Take significant steps now to prohibit conflict ofinterest issues and Ibelieve that Snort will unify and get stronger than any of us ever imagined. I believe this would ensure the goals for Snortmentioned inMarty's message would not only be met, but significantly exceeded.I doubt that. There are people out there who have their commercial aspirations placed *way* ahead of the good of Snort and the community, some of whom I used to trust implicitly. If such an organization were established they'd be first in line to get a leading position in such an organization. I would refute that I have mismanaged the project, and I doubt that signing up a predatory committee to run the development process is going to help anything. I've been screwed way too many times over the past 3 years by supposed friends and partners to turn Snort over to you when you're acting as a proxy for a as-yet-unknown 3rd party with an interest in controlling Snort.Well defined processes for code contributions and "scrubbing" code ------------------------------------------------------------------ One way to increase the number and quality ofcontributions is to have awell defined process for contributing code. Why do I mention this? Because shortly after I sent the last message half of the private responses I received came from people or organizationsthat tried tocontribute code or bugfixes. They never received anyresponse whatsoeverfrom the Snort leadership.How many total people is that? One? One thousand? What significant patches/contributions ended up not being used?Obviously, certain requirements have to be met for codecontributions.When a contribution does not meet those requirements tell the contributer the decision and if possible point out thedeficiencies inthe contribution. Contributers improve the qualityflexibility of Snort.Snort would not be what it is today without its contributers. As for scrubbing --- scrub away, but tell the maintainerof that codewhy their code is being scrubbed or what they can do to address the problems to prevent it from being scrubbed. Manycontributers have oftenmade a large investment in Snorts success and they deserveat least thatmuch.How much do you think they'd complain about me being a big meanie when I told them that their code was slow, crashed constantly, had terrible memory management and introduced DoS conditions into Snort? Do we need to introduce political correctness into the process as well? That'd certainly help people keep from getting their feelings hurt, we wouldn't want that either...As a disclaimer --- My past code contributions are not mymotivation forbringing up these issues. I personally don't care what isdone with anycode I've contributed in the past. Scrub it, delete it,throw darts atit. I won't loose any sleep over it. I am aware of a number of organizations that use some of that code in their production snort installations and couple companies and organizations thathave builtproducts and services around some of that code. They mightloose sleep! You stated in your original message that the treatment that you had received at my hands was part of your motivation.... People who build products on top of Snort should probably try to make sure that they stay in close communications with the core development team to make sure that they're always apprised of the development trends. As a group we're pretty easy to get a hold of. BTW, the sense of entitlement that people have for free code is pretty interesting, I'm constantly amazed at what people feel are my obligations towards them because they use something that I develop for free...Now for a few responses.... ------------------------------------------------------------------ Alfred Huger wrote...You don't happen to have a business plan in hand thatrequires Snortby chance do you? Or better yet are you working somewherethat does?No! My motivations are solely to protect open-source snort, see its continued growth and improvement, and continued increase in market share. To fully disclose, my only business in the info-secindustry isoccasional independent consulting (Incident Response and Code Development). My past, present, or future role in theSnort community isnot relevant to that effort. The majority of my current business ventures are in other industries.But you also claim to be acting on behalf of unknown third parties as well, what is their interest in Snort?Martin Roesh wrote (these 2 comments are from differentmessages)...Having code accepted for inclusion in the base Snort distro is a privilege, not a right, you aren't entitled to seeing your code go into Snort just because you ship it over...I get a lot of patches and contributions otherwise and only review what immediately piques my interest or what I can get to.You have every right to make that decision. I believe thatSnort wouldfind more success if contributions are encouraged and ifthose that havemade the investment to contribute have a well definedprocess, includingguidelines and requirements. Contributors also deserve aresponse and areason if their code is not accepted. Or not... But theconsequence isthat pressure will continue to build for a fork.If you want to fork the code, be my guest, have fun maintaining it. Contributions are and always have been encouraged, what leads you to believe otherwise? I guess I'm confused. If people don't like the current cycle of major bug fixing and feature additions from myself and the rest of the core developer group, I'd like to hear about what directions they think we should be going in. More instability? Slower code? Their own pet features? The fact of the matter is that Snort's development has never been more vibrant and active than since Sourcefire was started and, more recently, brought on Andrew Baker and Chris Green. You seem to imply that this insurmountable pressure from this as-of-yet unheard from group of fellow dissenters is going to force us to do something or risk a fork. That sounds a lot like an ultimatum, I really hate those...Andrew Baker wrote...It was assumed the developers of the existing output plugins would port over their existing code to Barnyard. However, thishas not evercome to pass and thus to bridge the gapHere are two examples of how this statement is false. * Database plugin: After discussing porting my Snort contributed db plugin to Barnyard on the snort-admin list and completing90% of theport you released an incomplete plugin that only included MySql support. Had I known that your intentions were to replace the database plugin, I would have never bothered investing mytime porting. This completely flies in the face of the Cathedral and the Bazaar, if your code was better it would have quickly edged out Andrews code. As it was, we never saw anything from you and you quit the project 3 months later. Andrew's intentions (as I recall) was to solve a problem that we were facing (not having DB support in BY with no port in sight from you) and he did that. Why couldn't you keep going? We had spp_unidecode and spp_http_decode for a long time before we merged the two functionalities, why does your plugin need to have an exclusive place in the hierarchy of contributions for you to continue with it? This doesn't make any sense, it seems like you're ignoring the entire ideology of open source development when you make statements like this.* A Barnyard port of the XML plugin was submitted 3 monthsago yet Istill have not seen it in the distribution.Due to the license changes (GPL->QPL) the lawyers at CMU/CERT needed to leap into action, we're (well, Andrew and Roman) waiting to see what they say. Funny you haven't discussed this with Roman at all...So exactly what is required here to "bridge the gap"? Idon't understandyour underlying motivations or the logic behind criticizing for not contributing code while at the same time not acknowledgingor acceptingcontributions. Another mystery about Barnyard is the reason for the license changed from GPL to QPL.A commercial entity was using it with modifications and selling it as part of a product without open sourcing those mods and refusing to contribute them back. The QPL allows Andrew and myself to force compliance with the open source license (everyone using the license has a perpetual nonexclusive license to code developed on top of it) and it's OSI approved. Go to http://www.opensource.com for more info.Ryan Russel wrote...And there's nothing to stop them. Even if your worst case comes true, why would it stop the independents? The worstmonopoly in ourindustry, Microsoft, still leaves tons of room for others to make money. And you're a whole lot better of with Snort, because it's open source and you CAN fork if SourceFire goes nuts and releases the next version only to paying customers.Very true. It is a nice insurance policy. Although, Iwould not like tosee a fragmented community. Hopefully Sourcefire will dothe right thingto prevent that from happening.The "right thing" being what you recommend? I would refute that that is the right thing at all. Just because you say it is doesn't make it so.Jed Haile -- the other Jed :) wrote his one...Another thing I should point out, as one of thedevelopers of hogwash, isthat if you want to do your own thing with Snort, have atit. Marty and therest of the Snort team have been fully supportive of thehogwash team'sefforts. Hogwash is in many aspects a fork of Snort, itis also a projectthat I hope to one day see merged with Snort. Marty hasexpressed that hewould like to see that also.Obviously anyone can fork at any time they want. There is also the snort-adapter fork. The fewer forks the better IMHO. Thereason this hasnot happened more often is that this requires significantinvestment ofboth time and money, not because people have not wantedto. I'd like to Additionally, forking sounds a lot better on paper than it works in reality. How many people are going to sign up to get a few hundred emails to their inbox a day with no expectation of getting compensated in any way for their time and effort and relentless criticism when they don't reply in a timely and courteous fashion? Who has the patience to answer the same FAQ for the 25347th time from people who can't be bothered to read the docs or don't speak english or that have a grudge against them? Who is going to work a full 8-16 hour day at their job to come home and handle this stuff? Who is going to coordinate the efforts of developers from all over the world and make the determination what gets in and what doesn't for the code base as well as arbitrate the inevitable disputes? Who has the personality to do this successfully without becoming a raving lunatic or non-benevolent dictator? How long will this go on before the forkers get bored and go onto things where they can get paid to be smart and personable and manage a herd of cats? Who is going to dedicate time away from friends and family for the sake of a bunch of people who are going to turn around someday and suggest that because they'd like to get some compensation by devising a nice complementary way of solving the problems that large users face that they no longer have the best interests of the users at heart any more? Really Jed, your line of reasoning is *really* starting to make me scratch my head, you don't have the *any* idea why Sourcefire exists, what it's principles are, what my intentions are for Snort, or anything even remotely approaching an idea of what reality looks like with regards to Snort these days. If you would like to find out, here's what I'll do. If you want to drive out to the Sourcefire office in Columbia MD (it's about 3 hours drive from you in Pittsburg) I'll personally take you on a guided tour of what we're up to and how it relates to Snort. If you walk away from that meeting feeling like we're about to make Snort proprietary or that we have anything less than the interests of the community in mind, then you'll have a basis for your complaint. Otherwise this is all just idle speculation and hardly anything to take drastic action on. Once again, my office phone number is 410-290-1616, feel free to call and schedule a meeting, I'll give you a full day of my time if necessary.see people unify around one version of Snort and I believesome changesin management style would facilitate this.I believe that ultimately you have no idea if this would solve anything and you're looking for change because you felt slighted by me in public and you'd like me to not have anywhere near the voice that I do, or at least that seemed to be the theme of your first message. This one seems to be different, however, so I'm not sure what you're really looking for. The fact that you still haven't talked to any of the core developers of Snort or myself but yet you continue to speak to and coordinate your message with this cabal of disgruntled 3rd parties really has me asking a lot of questions about your motivations and real intentions. Even the thought of somehow developing a committee with this group of people who are sitting in the shadows is a non-starter, I don't work with people I can't trust and there's way too much subterfuge going on here.My Concluding Thoughts ------------------------------------------------------------------ I want to thank everyone for their opinions. This hashelped clarifymany issues to me and hopefully the users and developerson these lists.This discussion has led me to believe that a fork is notthe best routeat this time. Nevertheless, I believe we need to encouragethe currentleadership to put measures in place to avoid conflicts ofinterest andhave a well defined and open process for code contributions.I encourage you to take the time to understand everything that Sourcefire is doing, you have been offered unprecedented access to our technical and management staff and if you're truly interested in what's best for the Snort project you'll get active in it again instead of presenting wild conjecture and hypotheticals as truth and recommending courses of action without full knowledge of why you're doing it. I would like to suggest that your involvement with these "unknown 3rd parties" has only given you half of the actual story and that if you're really interested in Snort's future you'll take me up on this. -Marty -- Martin Roesch - Founder/CTO Sourcefire Inc. - (410) 290-1616 Sourcefire: Professional Snort Sensor and Management Console appliances roesch () sourcefire com - http://www.sourcefire.com Snort: Open Source Network IDS - http://www.snort.org ------------------------------------------------------- This sf.net email is sponsored by:ThinkGeek Caffeinated soap. No kidding. http://thinkgeek.com/sf _______________________________________________ Snort-devel mailing list Snort-devel () lists sourceforge net https://lists.sourceforge.net/lists/listinfo/snort-devel
------------------------------------------------------- This sf.net email is sponsored by:ThinkGeek Bringing you mounds of caffeinated joy. http://thinkgeek.com/sf _______________________________________________ Snort-users mailing list Snort-users () lists sourceforge net Go to this URL to change user options or unsubscribe: https://lists.sourceforge.net/lists/listinfo/snort-users Snort-users list archive: http://www.geocrawler.com/redir-sf.php3?list=snort-users
Current thread:
- Re: [Snort-devel] Re: RFC: Forking Snort Martin Roesch (Jul 02)
- <Possible follow-ups>
- Re: [Snort-devel] Re: RFC: Forking Snort Martin Roesch (Jul 03)
- Re: [Snort-devel] Re: RFC: Forking Snort Matt Jonkman (Jul 03)
- Re: [Snort-devel] Re: RFC: Forking Snort Jeff Nathan (Jul 04)
- Re: [Snort-devel] Re: RFC: Forking Snort Matt Jonkman (Jul 03)
- RE: [Snort-devel] Re: RFC: Forking Snort Bob Walder (Jul 05)
- RE: [Snort-devel] Re: RFC: Forking Snort Bob Walder (Jul 05)