Re: Muni fiber: L1 or L2?

[Scott Helms <khelms () zcorum com>]

> > OK, think about it like this. The most efficient topology to provide both
> > coverage and resiliency is a ring with nodes (shelves) from which end
> users
> > are connected. That ring (usually Gig or 10Gig Ethernet today) needs to
> be
> > connected to a central location so you can interconnect to other
> providers
> > (your ISP customers) and/or to connect to the Internet if the city is
> > also going to provide direct L3 services. If you instead push down a L1
> > path then the most expensive pieces of gear in the access network (the
> FTTx
> > shelves) have to be replicated by everyone who wants to offer
> > services.
>
> In short, you're saying I *must* have a ring with active equipment
> scattered around it, and I *cannot* home run each property.
>
> No one else is saying that, and you don't appear to justify it later
> in this email:

I'm not saying that you have to, but that's the most efficient and
resilient (both of those are important right?) way of arranging the gear.
 The exact loop length from the shelves to the end users is up to you and
in certain circumstances (generally really compact areas) you can simply
home run everyone.  Most muni networks don't look that way though because
while town centers are generally compact where people (especially the
better subdivisions) live is away from the center of town in the US.  I
can't give you a lot insight on your specific area since I don't know it,
but those are the general rules.


> > This bad not just from the initial cost perspective but because people
> > and companies that identify themselves as ISPs seldom know anything
> beyond
> > Ethernet and IP and then only in a few manufacturers (mainly Cisco and
> > Juniper). They are most certainly not comfortable working with Calix,
> > Adtran, and the rest of the carrier (formerly telco) equipment
> > manufacturers.
>
> Well, ok, but those people who are not comfortable handling access gear
> like the Calix will be L2 clients, anyway, taking a groomed 802.1q handoff
> from my Calix/whatever core, so they won't *have* to care.

That works, so long as its an Ethernet hand off you're (usually there is
some goofy gear out there) in good shape.

> L1 access will be there a) cause it has to be anyway, to keep active
> equipment out of the outside plant, b) for people who really want PtP,
> and 3) for ISPs large enough to want to do it themselves, if any show
> up (they admittedly might not; we're only 6k households).

Keep in place, but I've worked with virtually all of the nationwide guys
and most of the regional ones and they don't as a rule want anything to do
with your fiber plant.  Even in major metro areas selling dark fiber
doesn't have a huge uptake because if you the network owner didn't light it
you have no idea how good or bad the splices and runs are.

> >                 To make matters more complicated in cases of problems
> > you don't have a good demarcation of responsibility. What do you do as
> the
> > L1 provider when one of your ISP partners tells you one of his customers
> > can't connect or stay connected to that ISP's gear? Whose responsible in
> > that case?
>
> Well that's an interesting question, but I don't see that it's not
> orthogonal to the issue you raised earlier.
>
> > What happens when your tech goes out with an OTDR (
> > http://en.wikipedia.org/wiki/Optical_time-domain_reflectometer) meter
> > and says the connection is fine but your ISP insists its your problem?
>
> On an L1 connection, you mean?  I'll do what people always do; I'll work
> the ticket; at that level, this stuff's relatively digital, no?

No, its not and I've seen several of networks fail because demarc issues.
 US Carrier (a statewide network here in GA) was recently sold for pennies
on the dollar largely because of blurry demarcs. You can and will get
sucked into scenarios you don't want to be in and will lose money on.

> > > You're talking about what I'm calling L2 clients. If layer 2 falls
> > > over it's my fault, and believe me, I'll know about it.
> >
> > What I'm telling you is that you can't reliably have L1 clients in
> > shared model.
>
> You're telling me that, but you're not giving me good reasons *why* you
> think so.

Because:
1) There won't be much interest in doing it from experienced operators so
you're only going to get customers for it that are also new to the
business.  So your combined troubleshooting and install time will be bad
for a long time until everyone in the chain kind of understand what they're
doing.

2)  Unless you can home run every single connection you're going to run
into a lot of access related issues.  You will be working for the city so
they won't have a problem with you getting into their building at the water
tower/sewage treatment plant/power sub station or other  city owned
property.  Your L1 customer isn't going to have that access (not with the
city manager/mayor/council's knowledge anyway) because of regulatory and
liability reasons.  If you do home run everything you still have an access
challenge (where are you going to be able to give access to these customers
economically?) but its probably more solvable since its one spot.  You also
increase your costs by home running each connection, but that may or may
not be a deal breaker in your situation.

3)  Your going to have to do a lot more work since at L1 all of the rough
edges and sharp corners are there to be dealt (like someone grabbing the
wrong cable from the patch panel) there is simply no inexpensive method of
safeguarding L1 adds, changes, and modifies so either you do them or you
let your L1 customer do them and run a risk.  This is also something that
the city management is going to have concerned over.

4)  Physical layer troubleshooting is much harder than layer 2 and up.
 Having several organizations and potentially several different equipment
vendors and L2 technologies will be very tough to deal with over time.

5)  I've seen at least 5 muni networks try this and not succeed.  If you
include non-muni networks with similar characteristics (like EMCs) then
I've seen it tried more than a dozen times with no success (or interest)
beyond a few dark fiber set ups for point to point connections across town.



> >        You can of course lease someone a dark fiber from point A to point
> > B, but that's not a traditional way of partnering with ISPs and in any
> > case will only be feasible for a small number of connections since you
> > (probably) can't afford to home run each location in your network.
>
> Well, I'll have to see on that, won't I?  That's my next practicality
> checkpoint; fiber passing costs.
>
> > > > The long and short of it is lots of people have tried to L1
> > > > sharing and its
> > > > not economical and nothing I've seen here or elsewhere changes
> > > > that.
> > >
> > > You just changed gears again, no?
> > >
> > > I'm not trying to share L1 *drops*. I'm trying to make it possible
> > > to share *the entire L1 deployment between providers*, a drop at a
> > > time.
>
> > That's what I'm trying to tell you can't do. Its more expensive in
> > both the initial and long term costs.
>
> I can see 'initial', maybe, but if I reduce the utility of the field
> network by putting active equipment in it, then I've already raised the
> OPEX, substantially, as well as reducing the intrinsic value of that
> network.

I think you're vastly overestimating the desire that customers have for a
bare fiber.  Having said that, your community may be different from what
I've experienced.


> Cheers,
> -- jra
> --
> Jay R. Ashworth                  Baylink
> jra () baylink com
> Designer                     The Things I Think                       RFC
> 2100
> Ashworth & Associates     http://baylink.pitas.com         2000 Land
> Rover DII
> St Petersburg FL USA               #natog                      +1 727 647
> 1274


-- 
Scott Helms
Vice President of Technology
ZCorum
(678) 507-5000
--------------------------------
http://twitter.com/kscotthelms
--------------------------------