Secure Coding mailing list archives
Building Security In vs Auditing
From: James.McGovern at thehartford.com (McGovern, James F (HTSC, IT))
Date: Fri, 5 Jan 2007 17:16:52 -0500
Thanks for your response but I am not sure that it got at the essence of my thinking, so let me ask some additional questions. 1. I haven't gotten a sense that a bakeoff matters. For example, if I wanted to write a simple JSP application, it really doesn't matter if I use Tomcat, Jetty, Resin or BEA from a functionality perspective while they may each have stuff that others don't, at the end of the day they are all good enough. So is there really that much difference in comparing say Fortify to OunceLabs or whatever other tools in this space exist vs simply choosing which ever one wants to cut me the best deal (e.g. site license for $99 a year :-) ? 2. Continuing with your plumber analogy, usually you either are referred by someone who had a particular experience with a vendor or you simply choose whoever is available from the Yellow Pages that will show up when you want them to and will charge what you think the best value is. My circle doesn't include the first and I would like to become smarter about the second in that should I choose someone knowledgable from Accenture, TCS, Cognizant or other firms I am familiar with or would I be doing myself a huge disservice and should instead focus on a boutique. -----Original Message----- From: Paco Hope [mailto:paco at cigital.com] Sent: Thursday, January 04, 2007 9:33 AM To: McGovern, James F (HTSC, IT); sc-l at securecoding.org Subject: RE: [SC-L] Building Security In vs Auditing
Gary, I would love a little refinement of the benefits to badnessometers. Let's say I get a tool to tell me something I already suspect is wrong, what percentage of the population are better than they expected?
I won't speak for Gary, but working a few doors down I have seen a few of the same things he has. Occasionally developers internally run free tools and surrepetitiously fix problems that the tools find (this happens in some cultures where management is particularly antagonistic towards security as a first class concern). In those unusual instances, I could see the results of a badnessometer coming out better than expected. Management would perceive that such things had never been run, and would be pleasantly surprised to learn that the sky might not be falling. Other than that, few people run a tool for the first time and see results better than they expected. Tools codify all manner of stuff that your developers almost certainly do not know how to check for (and if they do, they probably don't have time).
Is it better to do such a badness test by doing a POC with one of the tool vendors in this space or do I get additional lift by going with a consulting firm in this regard?
I'm a consultant, take that as implied bias. But, I think you do get lift, and here's my analogy. Consider yourself a handy guy around the house who is going to do something moderately complicated, like redo a whole bathroom. You can buy all the tools and read all the books on how to do it for a lot less money than hiring a contractor to do the whole thing. There's some pretty specialized tools in plumbing, though, and they're tools you probably haven't used more than once or twice. Do you gain some extra insight into the use of those tools by hiring someone experienced to assist on the complicated parts? I think so. That someone experienced will come in, help you wield the unfamiliar tool, show you some things that he has experienced, and get you through the difficult parts. Then you, being the handy guy you are, are left to finish the bathroom, doing things you know how to do well. I think this analogy holds with a lot of the tools in security. You learn a lot by getting the experience someone brings, assuming you get a good someone. We, for example, have run a bunch of tools on a lot of different code bases. We know which rules tend to be alarmist and which ones are really important if they fire. Tool vendors won't give you that objectivity on their own tool, and some of the sales engineers don't have the insight into their own tool to know which warnings are just noise and which warnings are a big deal. A consultant can help you have a bake-off between tools, whereas a tool vendor typically lacks that objectivity. Paco ---------------------------------------------------------------------------- This electronic message transmission contains information that may be confidential or privileged. The information contained herein is intended solely for the recipient and use by any other party is not authorized. If you are not the intended recipient (or otherwise authorized to receive this message by the intended recipient), any disclosure, copying, distribution or use of the contents of the information is prohibited. If you have received this electronic message transmission in error, please contact the sender by reply email and delete all copies of this message. Cigital, Inc. accepts no responsibility for any loss or damage resulting directly or indirectly from the use of this email or its contents. Thank You. ---------------------------------------------------------------------------- ************************************************************************* This communication, including attachments, is for the exclusive use of addressee and may contain proprietary, confidential and/or privileged information. If you are not the intended recipient, any use, copying, disclosure, dissemination or distribution is strictly prohibited. If you are not the intended recipient, please notify the sender immediately by return e-mail, delete this communication and destroy all copies. *************************************************************************
Current thread:
- Building Security In vs Auditing McGovern, James F (HTSC, IT) (Jan 02)
- Building Security In vs Auditing ljknews (Jan 02)
- <Possible follow-ups>
- Building Security In vs Auditing Gary McGraw (Jan 02)
- Building Security In vs Auditing McGovern, James F (HTSC, IT) (Jan 03)
- Building Security In vs Auditing Paco Hope (Jan 04)
- Building Security In vs Auditing McGovern, James F (HTSC, IT) (Jan 05)
- Building Security In vs Auditing Gunnar Peterson (Jan 06)
- Code Analysis Tool Bakeoff John Steven (Jan 08)
- Building Security In vs Auditing McGovern, James F (HTSC, IT) (Jan 05)