Re: "stuck at RHEL5"?

[JP Vossen <jp () jpsdomain org>]

I'll comment in-line (with trimming) on both Joel and Nigel's replies.

Thanks to you both for the useful and informative answers.


On 01/08/2011 01:57 PM, Joel Esler wrote:
> On Sat, Jan 8, 2011 at 5:53 AM, JP Vossen<jp () jpsdomain org>  wrote:

[...]
> > Main point up front:
> > Who else votes for better RHEL5/CentOS-5 support and longer life-cycles?!?
> >
> > And who else votes for actual support of RHEL6 (and CentOS-6 whenever it
> > finally gets here) that conforms the the RHEL life-cycle not the SF
> > whatever-the-hell-the-devs-feel-like-this-week Snort life-cycle?
>
> We don't do things "just for the hell of it".  There are reasons for the
> decisions we make, and maybe I need to do a better job of explaining those
> reasons so our community understands them.

That was probably unfair of me, but that's sometimes what it feels like 
from this end.


[...]
> In the case of Snort 2.9, that meant FC13 and would have meant RHEL/CentOS
> 5.5.  We do not update our platform support for patch releases (i.e.
> 2.9.0.3) since there is a larger QA effort in terms of reimaging systems
> that we simply cannot swallow.  That is one of the many reasons we release
> source code in tarball format, so users can build on any platform they
> desire.

I get the point about patch releases, though I might argue it gets 
nebulous fast.  RHEL 5.0 to 5.3 are officially "unsupported" [1] so we 
can't be using those, so you do need to support patch releases.  On the 
other hand, by the very nature of "RHEL" and LTS, stuff should Just 
Work.  That's the whole point, and why stuff doesn't change much.  So 
you're damned if you do and damned if you don't.  But some of this has 
more to do with CYA than technical issues.  More on that in a bit.

[1] https://access.redhat.com/support/policy/updates/errata/

Perhaps a slight terminology change might help (for new stuff going 
forward).  Instead of "/centos-5-4/" it could be "/centos-5/" or maybe 
"/centos-5-x/".


> With Snort 2.9 and the release of DAQ, we now require libpcap 1.0 or higher.
> This is simply not a package that is included in the standard distro build
> of CentOS 5.5 (FC13), and the Snort development team are not going to
> repackage libpcap within the Snort RPMs, so we're stuck there as well.

That one I have a problem with.  "not a package that is included in the 
standard distro" and "not going to repackage libpcap within the Snort 
RPMs" should be mutually exclusive IMHO.  For a *supported* OS, either 
you use what is provided in the distro, or you provide the missing 
pieces.  Period, end of story.

Somewhere in _Absolute FreeBSD_ Lucas talks about one of the main goals 
of good technical writing:  "you do the work so your readers don't have 
to."  The same goes for packaging software: you do the work so your 
users don't have to.

If you want folks to use Snort, you have to make the barrier to entry as 
low as possible.  That's why I was involved in the Snort RPMs for a 
while circa 2004 (grep the spec file), and that's a large part of why 
I'm writing this.  (I already solved *my* problems, with help from 
Vincent to avoid reinventing some wheels.)

That means *really* supporting the things you say you support.  Which 
brings us back to the need to find out what needs to be supported.  As 
I've mentioned, maybe there are fewer RHEL/CentOS users than I think 
there are.  (Maybe that question should go on the Users list and blog, 
as already discussed.)

If only a few folks use RHEL/CentOS then you should not devote a lot of 
resources to them and I'll shut up.  My gut feeling is that lots use 
them, but hey, I've been wrong before.


> There is a plan to build on CentOS 6 for Snort 2.9.1 (our next planned
> version) assuming it's available, so there will (read: should) be RPMS for
> the RHEL6 platform.

Excellent.


> We will not be able to support RHEL 5 because of the
> above points.

That's disappointing to hear, especially since Vincent has already 
solved the problems for you.

Maybe that's a way to work-around some of these build and packaging 
issues.  Instead of having the community host the builds, have us solve 
the problems and then you implement and host the packages.  Maybe you 
could give such folks a break on subscriptions or something.


[...]
> > Due to NDAs if we want certain rules we *have* to use the pre-compiled
> > ones.  OK, I get it.  I don't like it, but I get it.  Also not SF's fault.
>
> Thank you for recognizing that.  We'll make some changes around this soon I
> hope.

Looking forward to that.  Please blog & ML the details as much in 
advance as you can.


[...]
> > Huh?!?  FC9, 11, 12, but not 10, and all of which are obsolete and
> > unsupported.  But not F13 (that Snort is actually compiled for) or F14
> > (current), not CentOS-5.5 (current).  RHEL-5.0, also unsupported but not
> > RHEL-5.5 (or just use the CentOS).  And why "8.04" (correct) but "10-4"?
> >   WTH is "10-4?"  (80's flashback: 10-4 good buddy! :)
> Okay, we can correct this, thanks for bringing it to our attention.

Carefully...  Even though it's inconsistent and arguably wrong, people 
have probably scripted against it, or are expecting it to stay the way 
it is.  You can't just go change it one day...  (I know, I'm 
impossible... :)  I'd say you could sym or hard link it, but that'll 
fail on Windows.  You could just dup it, but that makes the tarball 
bigger.  Honestly, you probably should leave it alone, and just put some 
SoP in place to be consistent in the future.

Sorry if it sounds like I'm being wishy-washy; I wanted to point out the 
issue but I'm not sure there is an ideal solution.  (Damned if you do or 
don't...)


> The VRT
> maintains a separate build environment that is much larger than the Snort
> team's build env, simply for the Shared Object rules.  (adding OpenBSD to
> that above list very soon as well.)  Maybe we can get to a point in the near

 From and outside and naive perspective I want to ask why you can't all 
just get along...but I won't; I get it.  I'm usually not a fan of The 
Cloud, but perhaps in this case EC2 instances used for a few hours a 
month might be worthwhile and in budget?  Though there's that setup cost 
again.  Community help?


> future where we can align the builds for VRT and Snort Dev to make it easier
> for the community, but then we'll run into the reverse effect, and we'll
> catch scorn for that as well.  So we are between a rock and hard place. But
> we'll sit down internally and figure this stuff out.

Not sure what reverse effect you're talking about, but I do agree you 
can easily get into damned if you do and damned if you don't situations. 
  Heck, I've listed a bunch of them in this message.


> Personally, I have a box here at the house that is Fedora Core 10.  It's
> running the FC-9 Shared Object rules.  They work fine.  Undocumented, but
> they work.  That's my own personal work around.   I have to maintain my own
> compiles for libpcap, libdnet, and such as well.  Unfortunately that's the
> price I pay for not wanting to move my personal box to a higher version.
>   Not a realistic expectation in the enterprise world.  But that's the price
> of free software for me.

And now we get to one of the other main points that I didn't articulate 
originally.  Some of this is just plain big enterprise CYA.  If 
something breaks I don't want to have to explain to management why I am 
using something that is "officially unsupported" regardless of the fact 
that we all know it'll almost certainly work.  And that puts me between 
a rock and a hard place.

I want to use CentOS, but in order for that to be "current" and 
"supported" I need to use 5.4 and soon 5.5.  There are precompiled rules 
for Centos-5-4, but not an "official" Snort.org working Snort/DAQ/pcap. 
  You get the idea...


[...]
> Right, I understand that.  Personally (and not speaking for the company on
> this point), I'd rather we don't build RPMs at all. I'd rather we just put
> out the tarball and let the package maintainers keep the distros up to date.
>   Unfortunately, they don't.  We have some packages way back in 2.6 land
> laying around because the package maintainer either stopped doing it, got
> lazy, whatever.  Doesn't matter, the point is they are out of date.  So we
> put out a few.

Yeah, I get that.  My knee-jerk reaction was going to be to suggest you 
dedicate some time from an internal resource to this.  Once you get it 
up and running (non-trivial), maintaining it wouldn't be that big a 
deal, especially with ready access to the devs.  But when I thought 
about it some more, you will run into policy problems, certainly with 
Debian, and probably with most/all the others, regarding version 
"freezes" and such. :-(  IOW they won't let you move nearly as fast as 
you want to.  So then the package maintainer has to try to back-port 
fixes and features, which is impossible without other changes, as we've 
beaten to death above, and which will be disallowed by distro policy anyway.

Some faster-moving projects (e.g., Wine, OpenOffice, and now 
LibreOffice) maintain "third-party" repos and/or Ubuntu PPAs, but now 
we're really taking about some startup costs (in terms of time/effort). 
  Maybe get a summer intern to do that or something?

So I have to come back to: find out what your users want, publish that 
and prioritize.  I didn't look all that hard, but I didn't find any 
handy "requirements" or "supported" lists on the Snort.org site.  I'd 
stick it on the downloads page if it were me.  Maybe also point to the 
community repos, but that gets dicey for "security" tools.  Signed 
packages with sigs posted on the official site will be a help there.


[...]
> The big thing with RHEL 5* is the lack of libpcap 5.0.

Yup, and Vincent showed us how to work around that (more below).


> The version of Snort that will run on RHEL 5 without libpcap modification is
> 2.8.6.1.  VRT still maintains rules for this version, and will until the
> next point release of Snort comes out (2.9.1) (+90 days) in accordance with
> our EOL policy.  http://www.snort.org/vrt/rules/eol_policy

But you are going to EoL that *long* before EoL on RHEL5...


> OK, rant over.  (If anyone actually read this far... :)
> > <rant off>
> JP, it's perfectly fine!  Just understand what I read above.  We don't make
> this stuff up on a whim, there are technical reasons for it.  When we
> provide new and better functionality, like with DAQ, sometimes that requires
> something newer.  We have to release new functionality, so we do, but we get
> in trouble there, but if we don't release new functionality, we get accused
> of being dead and get in trouble there too and can't detect newer problems.

Yup, it's certainly a tightrope.


[...]
> Thank you for the kudos.  You are more than welcome to tell me what we are
> doing wrong, and I will see what I can do to make it better. If we can't
> make stuff better (like release a 2.8.6.2, or support RHEL 5), I'll try and
> lay out the reasons for that on the blog/here.  I am trying to make the blog
> worth reading, and hopefully I have so far.

Definitely.


> Yes, Kudos to Vincent.  He is also going to start signing his RHEL supported
> packages for you guys, which is excellent as well.  As far as I know, he did
> exactly what I suggested above.  He downloaded the libpcap source code, and
> made an RPM out of it.  We can't do it, so the community stepped up. I
> appreciate this very much, and I think it's phenomenal that our community is
> getting more involved.

He also tweaked things in a way that allows *both* the stock RHEL5 and 
the newer libpcap to be installed at the same time.  That's a Big Deal, 
because you don't have to do evil things like '--no-deps' to get it to 
install, then have conflicts with things like the stock PPP.



On 01/08/2011 04:07 PM, Nigel Houghton wrote:
[...]
> I can shed light on the platform support for the pre-compiled rules
> since it is my group within the VRT who build and maintain those
> systems that the so rules are built on.
>
> Our intention is to keep pace with the major distributions as far as
> the platforms go. That is, we intend to keep those systems up to date
> with the latest supported version of each distro along with at least
> one supported version back. Right now for example, we have Ubuntu 10-4
> and 8-04. The latest version of Ubuntu is 10-10 yes, however in this
> case Ubuntu 10-4 LTS is the one we are sticking with since that is the
> one designated for long term support (hence the LTS).

I *strongly* support you supporting *only* LTS releases!  As I noted a 
few different ways, 6 month life-cycles (OK, they are a bit longer than 
that, but still) are way too short.  Debian is more-or-less inherently 
LTS anyway, as are RHEL and CentOS.  I don't follow the BSDs as closely, 
so I won't comment there.

Nothing in InfoSec is "set it and forget it" but I think it's fair to 
say that sensor OS upgrades are something we want to deal with as little 
and infrequently as possible.  (Engine upgrades too, but Joel's point 
about innovating or dieing is well taken.)


> As for RHEL, we are planning on adding support for RHEL 6 as soon as
> resources allow, at which point we will also address the 5-0 vs 5-5
> issue.

Cool.


> FC-10 was not added since 11 and 12 were already out, so we went with
> those. The support for FC-9 will more than likely end in the near
> future and we will re-purpose those resources so we can support other
> distros and versions.

Makes sense.  No doubt you'll catch flak for that too, but it won't be 
from me.


> On top of all this, we are adding more support for 64 bit platforms
> (another reason for FC-9 still lingering around at the moment since we
> don't have the 64 bit platforms for 11 and 12 yet). It is our intention
> to have i386 and x64 support for each distribution.

Yes, 64-bit support is more important every day.  (I just ran into a 
2038 bug over the weekend, totally unrelated to Snort.)


> We should be able to start shipping so rules for OpenBSD in the coming
> week, we still have some testing to do but that should be completed
> pretty soon. If I was going to stick my neck out and give a date, it
> would probably be Thursday.

Excellent!  Please blog it.


> All this effort does of course take careful planning and resource
> allocation to achieve these goals. We cannot reach them overnight, it
> takes time. A tremendous amount of work goes on behind the scenes to
> deliver this support, we have made progress already, we have a plan,
> we'll get to a consistent state sooner rather than later.

It's nice to know that you see the issues and are working towards fixing 
them.

Thanks again for the feedback.

Later,
JP
----------------------------|:::======|-------------------------------
JP Vossen, CISSP            |:::======|      http://bashcookbook.com/
My Account, My Opinions     |=========|      http://www.jpsdomain.org/
----------------------------|=========|-------------------------------
"Microsoft Tax" = the additional hardware & yearly fees for the add-on
software required to protect Windows from its own poorly designed and
implemented self, while the overhead incidentally flattens Moore's Law.

------------------------------------------------------------------------------
Protect Your Site and Customers from Malware Attacks
Learn about various malware tactics and how to avoid them. Understand 
malware threats, the impact they can have on your business, and how you 
can protect your company and customers by using code signing.
http://p.sf.net/sfu/oracle-sfdevnl
_______________________________________________
Snort-devel mailing list
Snort-devel () lists sourceforge net
https://lists.sourceforge.net/lists/listinfo/snort-devel