nanog mailing list archives
Re: Is Cisco equpiment de facto for you?
From: Joel M Snyder <Joel.Snyder () Opus1 COM>
Date: Mon, 10 Jan 2011 10:26:06 -0700
>I try to follow the Tolly Group who compares products, and they >continually show that Cisco equipment >is a poor performer in almost any equipment compared to others, I find >that so hard to believe.....
Just a rough comment here. Tolly's business model is a sponsored test one, and Cisco is the dominant vendor in the marketplace. So when anyone has a new product they want to promote, they can hire Tolly's lab to show how a particular feature set is implemented and can often show how one model exceeds Cisco's performance on some or all benchmarks.
While Tolly never cheats, remember that the guys who "win" the test are the ones that are helping to design the test methodology, and so they are focusing on features that they know that they can beat Cisco on. Thus, the Tolly reports are true, but may not be telling the whole story or may be focusing on a particular feature more than anything else. No one is lying to you, but you may not be given the whole picture.
There's nothing wrong with having Tolly show that your product is in the same league as Cisco's, but I would not necessarily read those reports to say that Cisco is always the "loser" in any new product comparison. The real value in Tolly's reports is to give credibility to the new and up-coming products by having a mostly-independent source show that the new products are comparable with the market leaders' in some ways.
Tolly is also very very important at doing spec verification: showing that a product actually DOES what the specification sheet says it does. Remember that those numbers are often made up by product managers based on poorly-done in-house tests, so having a third-party, even a paid one, say "yes, it does do that in our test lab" is WAY more information than you get from most vendors.
Tolly provides a valuable service to our community but you need to look at a combination of benchmarks from Tolly as well as other very-unbiased sources (IDG and CMP magazines are pretty thorough) AND YOUR OWN TESTING to know what is best for you.
On the other hand, compared to the analyst firms like Gartner who don't even have test labs and generate their reports based on phone calls, Google searches, and unverified specification sheets, Tolly is a source of infinite knowledge about products.
Finally, also remember that performance is only one part of system design and deployment. Management and feature completeness are, for the operator community, often more important than whether the box can do 80 Gbps or 85 Gbps. Product selection is best done by YOU setting YOUR requirements and doing YOUR OWN testing, not by looking at point tests to see if one product version/model/device beats another in a particular small set of performance tests.
Obligatory disclaimer: we don't compete directly with Tolly, but we do get paid by the magazines to do testing and there is some overlap, so we do compete indirectly.
jms -- Joel M Snyder, 1404 East Lind Road, Tucson, AZ, 85719 Senior Partner, Opus One Phone: +1 520 324 0494 jms () Opus1 COM http://www.opus1.com/jms
Current thread:
- Re: Is Cisco equpiment de facto for you?, (continued)
- Re: Is Cisco equpiment de facto for you? b nickell (Jan 10)
- Re: Is Cisco equpiment de facto for you? Miquel van Smoorenburg (Jan 10)
- Re: Is Cisco equpiment de facto for you? Jethro R Binks (Jan 11)
- RE: Is Cisco equpiment de facto for you? Brandon Kim (Jan 11)
- Re: Is Cisco equpiment de facto for you? Jack Bates (Jan 11)
- Re: Is Cisco equpiment de facto for you? Seth Mattinen (Jan 11)
- RE: Is Cisco equpiment de facto for you? Justin M. Streiner (Jan 11)
- Re: Is Cisco equpiment de facto for you? Ron Broersma (Jan 11)
- Re: Is Cisco equpiment de facto for you? Charles N Wyble (Jan 10)
- Re: Is Cisco equpiment de facto for you? Chuck Anderson (Jan 13)
- Re: Is Cisco equpiment de facto for you? Herro91 (Jan 13)
- RE: Is Cisco equpiment de facto for you? Brandon Kim (Jan 13)
- Re: Is Cisco equpiment de facto for you? Tony Varriale (Jan 13)
- Re: Is Cisco equpiment de facto for you? Jack Bates (Jan 13)
- Re: Is Cisco equpiment de facto for you? Tony Varriale (Jan 13)
- Re: Is Cisco equpiment de facto for you? Chris Adams (Jan 13)
- Re: Is Cisco equpiment de facto for you? Greg Whynott (Jan 13)