Re: New Office, New Network. Questions.

[Nikolai Petrov <prnpetrov () yandex com>]

Please find my replies inline. Thanks for your time!

> hi nikolai
>
> - oops.. this got long based on my experiences/opinions :-)
>
> On 07/10/16 at 09:53pm, Nikolai Petrov wrote:
>
> > We are moving to our new offices in two months and I have access to the building already.
> > My task is to set up the entire network for the company.
> > The previous administrator has left the company and I thought of taking the chance to remove some "technical debt" 
> > and make everything from scratch again.
>
> all good ...
>
> > I was told to move the networks this week
>
> do you have the routers, switches, cables, few servers for testing ?
> has your ISP installed their internet uplink connectivity to the bldg ?
>
> if so, than the above management is on their toes
> otherwise, you'd need to rattle some $$$ loose to buying missing hw :-)

Since we have received funding for this scale up of the company the management decided I should buy all the hardware 
new. I have a large budget and I could even make the network 10 Gb/s with that money. I guess I am a lucky person. 
Large budget to do a network without using old stuff! The uplink is installed already from our ISP and the entire 
building is available to me to install the network.

> > and I have spent a lot of time thinking about how I should do it.
>
> good ... now's the chance to fix the problems if any ..

This is what I thought!

> > 1. Currently we do not have IPv6 in our network
>
> implies a learning IPv6 curve ( red flag for possible time-wasting hogs )
>
> if the task is to mvoe the entire "mid-sized" from current bldg to new bdlg,
> i'd suggest use "known/good/working/best-practices" methodology to move
> the company. first get the new bldg with new test servers working
> with IPv4 ( the way you want it done ) and "working" the current bldg
> which should take a few hours :-)
>
> than work with IPv6 issues

This is my plan. I have designed the IPv4 network already but I also want to install the IPv6 network right after it 
and not "some time in the future".

> > but I have seen the ISP is giving us a "/56 Block"
>
> good
>
> > which from what I understand is a couple hundred "/64 Subnets".
> > I think you can only have /64 subnets in IPv6.
>
> nah ... you can subnet your /56 into whatever you want
>
> > In our IPv4 setup we have 32 addresses,
> > four of which I will use for NAT
> > and the remaining needed for online services and servers.
>
> good ... use that to test everything
>
> since you want or going to use NAT, you have the standard
> internal LAN for the bldg can use the standard 10/8 or
> 192.168/16 or 172.16/12
>
> so far.. nothing new/special/problematic

Exactly. I intend to keep the same IP range as before since I do not have problems with it.. I may change 1-2 subnet 
sizes but that's it.. I want to avoid issues with DNS servers, static IP configurations since these are not 100% under 
my control and I do not know if the server team knows all places where an IP is located..

> > In IPv6 we have a lot of addresses but I am not sure whether
> > I should give an address of the ISP to every device.
>
> why would you want to complicate time-restricted ( 1month )
> to get the new bldg working with IPv6 w/out having prior
> IPv6 experience ?
>
> remember, "all eyes" will be looking to you to move the
> whole company from current bldg to new bldg without delay

I think I can do the IPv4 part sooner than needed so I should use the remaining time for IPv6 + testing..

> > I found that there is an organization that can help avoid collisions
> > in private IPs: https://www.sixxs.net/tools/grh/ula/ .
>
> there should never be any collision in IP#, ipv4 or ipv6

Of course. I meant global collisions in case of a network merge or something.. 

> > From what I can tell it is just a registry, but I am thinking of
> > registering the ranges there and then use these subnets and
> > NAT them to the IPv6 address of the router.
>
> the ISP provides you the range of IPv6 assigned to you
>
> if your current bldg does NOT have IPv6, you might not be
> able to easily test the new IPv6 stuff in the new bldg

The new building has full connectivity and when I add our router it gets the IPv4 and IPv6 addresses. The optical fiber 
of the ISP comes to the building directly.

> you might be able to test your new IPv6 connections
> at the local coffee shop or other public places but
> that's a major security violation since your new IPv6
> has no security pre-cautions installed yet

I am not sure I understand this part. How can I test our IPv6 company set up in a coffee shop?

> you should be paranoid about trojans/worms/mailware piggie
> backing into your new un-secured new bldg IPv6 infrastructure
> or IPv4 infrastructure
>
> > However, I noticed something strange. The WAN port of our
> > router gets a /64 IPv6 address which is not in our IPv6.
>
> why strange ??
>
> routers get its IP# from dhcpv6 or statically assigned

Because it is a /64 range and not a /126. In my previous company we used an IPv4 /30 for our BGP. Isn't a /64 a waste?

> > Should I use this for NAT or one of "our" addresses?
>
> you need to fix this problem before continuing ..
> ( explain why the IPv6/64 is not what you're expecting )
>
> NAT is NOT the solution ...
>
> > 2. The previous administrator did some bad job in some parts of the network.
>
> :-) that will always be true 90% of the time :-)
>
> some things are always gonna be "bad"
>
> > We have an internal router protocol to move traffic between routers,
> > but in some cases he used NAT instead of adding these subnets to the
> > router protocol. Everything works and all things that have to be
> > reached are reachable,
>
> if it works .. why is is "bad" ??
>
> there might be dozens of different ways to make things work
> ( "things that have to be reachable are reachable" )
>
> > however I think this is bad
>
> not necessarily a bad thing
>
> > and we should use the router protocol for all parts of the network.
>
> why ?

Well, currently we have issues with this: NAT makes all computers communicate with all computers but we need to give 
users access to the routers ( ! ) so they can port forward so employees from other departments can reach their 
computers and we also do UPnP. Currently there are employees with access to three routers so they can port forward the 
port forward of the port forward to reach other workstations.. If we used the router protocols it would be direct host 
to host communication.

> > I have found two protocols in our router that are good and support
> > IPv6 and they are OSPF and BGP.
>
> there might be more :-)

I have found some more like RIP but it is not good enough I think. I also found ISIS but I don't think it provides much 
benefit over OSPF and it is not supported by all routers..

> > I did not manage to have BGP work
>
> what part is not working ?

I can successfully peer with the other router using IPv6 but unfortunately no IPv6 routes are exchanged between the 
routers or propagated to others if I set the IP ranges manually.

> google/yahoo the error messages :-)
>
> > and it is slow so I am thinking of OSPF.
>
> sometimes, which works first/better/easiest might be
> a good option, thus trying other things is good, but
> that can also create more headaches too .. more problems
> to (fun) solve
>
> > Do uou think it is a good choice for IPv6 and IPv4?
>
> i'd work with IPv4 first ...
> and more importantly... there is NO excuse why IPv4 doesn't
> or cannot work in the new bldg
>
> after IPv4 works in the new bldg as good as it does in the current
> bldg, you have time for "( IPv6 ) learning experiements"
>
> > If I have two separate paths of 1 Gb/s, will I transfer files at 2 Gb/s?
>
> no ... you will be able to transfer 1Gb/s each ..

Let me clarify this is not transfer to the Internet but between two hosts in the internal network.. 

> if you "channel bond" the two 1Gb/s into "one link",
> than you might be able to see 1.9Gb/s uplinks .. never 2G/s
>
> if you have 2 1G/s uplinks ... you should have the 2 routers
> crosslinked for failover unless uplink speed is more
> important than failover
>
> > 3. In our old network we use "VRRP" which from what I know
> > is a system for routers to shae IPs and load balance or "failover" the traffic.
>
> good
>
> > I have seen that IPv6 has a built-in system which is similar
> > and has something like priorities, etc. What happens if I have
> > two routers with same priority?
>
> same rules/issues apply to IPv4
>
> one router/path should always have priority over the other
> depending on destinations .... lots of testing to see which
> packets goes thru which uplinks
>
> > Whic is used as default gateway?
>
> depends..
>
> engineering/manufacturing uses router1
> hr/accting uses router2
>
> or
>
> public DMZ uses router1
> corp LAN uses router2
>
> but in either case, router1 and rotuer2 should be crosslinked
> if failover is important
>
> > Is it load balancing? Also, can I use "VRRP" to load balance traffic
> > to our DNS look-up "recursor"?
>
> dozen ways to do load balancing ... more problems to resolvea
> and prioritize based on your company visibility online
>
> load balancing should be worried about:
> - dns, www traffic, email traffic, DVD/video/music downloading,
>
> also always have 3 hot-swap complete infrastrucure and backups
> fw1 + dns1 + www1 + mail1 + NAT1
> fw2 + dns2 + www2 + mail2 + NAT2
> fw3 + dns3 + www3 + mail3 + NAT3
>
> fw only runs iptables for inline fw for entire dmz/localLan
> dns only runs bind and iptables and nothing else
> www only runs apache and iptables and nothing else
> mail only runs sendmail and iptables and nothing else
> nat only runs NAT + iptables
>
> each backup its bind/sendmail/apache data to the other 2 boxes, but
> bind/sendmail/apache itself is turned off on the other hot backups
>
> magic pixie dust
> alvin
> #
> # DDoS-Mitigator.net
> #