Re: Changing ISP?

["Winders, Timothy A" <twinders () SOUTHPLAINSCOLLEGE EDU>]

I am working on setting up a redundant internet connection now.  For
full redundancy, I am installing it at a different campus with a
different ISP.  I am also setting up a VPN tunnel between the two end
points so that if our internal network fails, we will have redundancy to
the other campus over the Internet.

Here are a few "gotchas" I am wrestling with now.

VPN termination - we have our own IP addresses.  I was terminating VPN
connections on the outside interface of our existing firewall.  When
advertising our routes with the new ISP, VPN broke for some users.
Their connections were coming in through the second ISP.  I will be
purchasing a dedicate VPN termination device, but, in the meantime, I've
changed the IP address of the VPN termination and am not announcing that
to ISP2 so all VPN connections come in the correct path.

A 2nd internet connection at a 2nd location requires a 2nd firewall and
management of that firewall.

I was previously doing Null routed annoucements into BGP to prevent any
bgp flapping and aggregate our network addresses.  This is fine as long
as our internal network is 100% accessible from either ISP.  However, in
the case of an internal network break, this breaks Internet
connectivity.  Suddenly, I was announcing routes to an ISP which were
not accessible, creating a black hole.  I have had to go to injecting
OSPF routes into BGP, aggregating where necessary and removing the Null
routes.

I am running BGP beween my two external routers for optimal route path
selection.  From ISP1 I was taking only that ISP customer routes because
the router didn't have enough memory for a full table.  I am taking full
routes from ISP2.  When I brought up the BGP session between router 1
and router 2, router 1 was not happy, requiring a reload to recover.
Router 1 will be replaced as soon as the replacement comes in.

Probably more, but this is a good start.  :-)

---
Tim Winders | Associate Dean of Information Technology | South Plains
College
 

> -----Original Message-----
> From: John Kaftan [mailto:jkaftan () UTICA EDU] 
> Sent: Wednesday, October 04, 2006 9:16 AM
> To: SECURITY () LISTSERV EDUCAUSE EDU
> Subject: [SECURITY] Changing ISP?
>
> We recently had an extended outage on our MCI 
> (VerizonBusiness) connection.
> We were down for 2 days and customer support was very poor 
> during this time.
>
> Now we are faced with a decision as well as building 
> redundancy into our
> connection.  Here are my questions:
>
> 1.    Should we leave MCI and suffer the pain of changing ISPs and
> receiving a new set of Public IPs?
>
> 2.    Can anyone else relate their recent experiences with 
> MCI positive or
> negative?
>
> 3.    Also, is it sufficient to use a single ISP for 
> redundancy if they
> give us separate local loops, via separate ILECs, into 
> opposite ends of the
> campus, to separate COs?
>
> 4.    Can anyone speak to setting up redundancy with separate ISPs and
> BGP?
>
> We have not talked about cost with any ISPs but I imagine it 
> would be much
> more affordable using a single ISP.   
>
> We are planning owning our next set of IP addresses.  I'm 
> told there might
> be a chance that we could keep our current set.  We'll see.
>
> John Kaftan
> Utica College