RE: Failover how much complexity will it add?

[Blake Pfankuch <bpfankuch () cpgreeley com>]

> > -----Original Message-----
> > From: adel () baklawasecrets com [mailto:adel () baklawasecrets com]
> > Sent: Sunday, November 08, 2009 4:52 AM
> > To: nanog () nanog org
> > Subject: Failover how much complexity will it add?
> >
> > HI,
> >
> > I was recently brought onto a project where some failover is desired, but I think that the number of connections 
> > provisioned
> > is excessive.  Also hoping to get some guidance with regards to how well I can get the failover to actually work.  
> > So currently
> > 4 X 100Mb/s Internet connections have been provisioned.  One is to be used for general Internet, out of the 
> > organisation, it
> > also terminates VPNs from remote sites belonging to the organisation and some publicly accessible servers -routed 
> > DMZ and
> > translated IPs.  Second Internet connection to be used for a separate system which has a site-to-site VPN to a third 
> > party
> > support vendor.  Internet connections 3 and 4 are currently thought of as providing backups for one and two.  Both 
> > connections
> > firewalled by a Juniper SSG of some description.
> >
> > Now I couldn't get any good answers as to why Internet connections 1 and 2 need to be separate.  I think the idea 
> > was to make
> > sure that there was enough bandwidth for the third party support VPN.  I feel that I can consolidate this into one 
> > connection
> > and just use rate limiting to reserve some portion of the bandwidth on the connection and this should be fine.  Now 
> > if I was to
> > do this then I can make a case for just having one backup Internet connection.  However I'm still concerned about 
> > failover and
> > reliability issues.  So my questions regarding this are:
> >
> > - Should I make sure that the backup Internet connection is from a separate provider?

Yes yes yes yes a thousand times yes.  Depending on the criticality of internet connectivity you should also aim to 
have your redundant
connections coming from a complete separate direction.  Example, fiber from Level 3 come from the north in a dedicated 
conduit and
fiber from Verizon coming in a dedicated conduit from the south of the building.  Why?  Put simply we had construction 
ignore the
painted lines and dig up our conduit a few years back.  At that point we have 4 bonded T1's from a single carrier.  
That was a long
couple of days...  Carrier diversity is not a bad thing, spend some time shopping an additional provider.  Make sure 
they operate their
own network for last mile, and also make sure they don’t piggyback off the same network your main carrier does anywhere 
locally.
Comcast Ethernet, Verizon and Cogent make great secondary connections when you need high availability.  You don’t need 
your
secondary to have 99.999% uptime.  97% is usually good enough if it's on a separate network.  I wouldn't sway from the 
big names
for your primary connections either.

> > - How can I acheive a failover which doesn't require me to change all the remote VPN endpoints in case of a 
> > failover?  Its
> > possible to configure failover VPNs on the Junipers, which should take care of this, but how do I take care of the 
> > DMZ hosts and
> > external translation?

With recent experience with the Juniper SSG VPN functions put nicely they suck.  VPN failover is in there, but we had 
issues with the
tunnel staying active for extended periods of time.  Also depending on if you do a route based or a policy based VPN, 
it becomes so
much of a headache.  We used 2 SSG550 devices as a proof of concept and the one thing which annoyed me to no end was 
the complete and
total crap options within then VPN configuration.  When I typically set up a VPN, I use a SonicWall NSA or E-class 
device (yes I know
hiss boo) or an ASA.  Saying that the Juniper was lacking was a complete understatement.  I personally would completely 
avoid even
attempting VPN failover within a Juniper device.  I will say they are rock solid though for generic firewall 
functionality, just try
to keep the config simple or they turn into giant slow dogs.

> > - In fact I think I'm asking what are my options with regard to failover between one Internet connection and the 
> > other?

Considering you have 4x 100mbit lines, have you looked at BGP?  Even if you drop line 2 and its associated backup, you 
have 2x 100mbit
lines.  Or even if you have 3 unique carriers with a 100mbit from each of them it makes BGP very appealing.  I think 
this would be an
ideal situation for a BGP setup using a couple of small routers.  You could probably get away with something as small 
as a Cisco 3825
for each connection (purely redundancy).  If the Cisco name scares you Juniper routers are great as well.  Don’t forget 
Vyatta!

If you do BGP, you have 1 VPN to configure, you have 1 tunnel to configure, there is no VPN failover configuration and 
hopefully you
are not pushing more than 1 subnet across the VPN otherwise you end up doing a route based VPN instead of a policy 
based VPN and you
will be significantly happier.  That’s a Juniper headache for another day however.

> > I'm hoping to figure out whether adding an extra Internet connection actually gives us that much, in fact whether it 
> > justifies the
> > complexity and spend.

What you really need to know to ask this is how much is Mr. Customer going to yell and scream if someone cuts only 
internet connection
and it's going to be down for about 3 days while they patch the conduit and fiber.

> > Many Thanks for your comments.
> >
> > Adel