Re: PTP/Syncronized Ethernet maturity

[geir egeland via NANOG <nanog () nanog org>]

Hi Taichi,
It depends. GNSS at the cell site has its own operational challenges, for example making sure that the antenna has a 
clear enough view of the sky. A challenge in Asia is that very little of the fiber is in the ground, hence multiple 
fiber cuts happen on a daily basis which changes the path length when restoration mechanisms kicks in. A change in the 
path length is not a problem for PTP, but it require that we know the path asymmetry on all possible fiber paths 
between the master and the slave (we need to use protection on layer 1 in Asia due to the frequent fiber cuts).
Another challenge with GNSS is that we experience that the GNSS is either jammed or even worse, that the GNSS is 
spoofed.
 
When we did the PTP design, we also believed that the length of the fiber path length would be equal in both direction. 
However, in some of our old Metro networks, the line amplifier have embedded Dispersion Compensating Fiber (DCF) to 
compensate for the chromatic dispersion of different wavelengths. The length of fiber within DCF modules to compensate 
for the same length of fiber may vary significantly. Other parts in the optical domain can also cause asymmetry, 
e.g.transponders, software FEC, or FEC in general, and  digital signal processors in coherent optical systems. 
Asymmetry increases with link speed, so we could consider running PTP over 1GE interfaces, but this is a challenge in 
our Core networks.
 
We can overcome the DCF (DCF is cheap) issue by either measure the asymmetry of every fiber hop (not practical 
possible),  change the DCP modules to Bragg filters (expensive), or deploy grand masters in the access/aggregation 
network in order to have less asymmetry impact from the fiber network. 
 
When it comes to the instabilities with PTP implementation, we try to work with the vendors so that they fix failing 
line cards, port flapping etc. However, its not always easy to get the vendor’s attention on PTP issues.
 
OAM for PTP is another challenge, i.e. how can we make sure that the clock is healthy? We plan to solve this by 
deploying GNSS at certain locations in the network and use equipment that can compare the difference between GNSS-input 
and the received PTP clock.
 
So key message is that PTP does not work out of the box, it requires significant engineering effort. GNSS has many 
issues as well, and in certain parts of Europe we cannot rely on GNSS only. So it is not an either-or, -  we need both.

best regards,
Geir

> On 8 Sep 2020, at 18:11, m.Taichi <marc101.maxmaok () gmail com> wrote:
>
>
> Hi Geir,
>
> Can we say, from your production network experiences across Asia and Europe, that getting synchronization clock 
> signal via GNSS receiver directly on each cell site is a much more reliable, stable, and simpler way than getting it 
> by network-based PTP? Especially when there is WDM link used in between the BC and Slave Clock?
>
> Why does WDM link cause path asymmetry? I thought the optical fibers carry forward link and reverse link are almost 
> equal in length (distance). Aren't they?
>
> What are your solutions to overcome the PTP synchronization instability problems in your TDD 4G/5G networks?
>
> Thanks and best regards,
> Taichi
>
>
> On Tue, Sep 8, 2020 at 11:09 PM geir egeland via NANOG <nanog () nanog org <mailto:nanog () nanog org>> wrote:
> We have mobile NWs in both Asia and Europe and also experience a lot of issues with PTP, - almost with every vendor.
> The instabilities, SW-bugs etc. related to PTP seems to indicate that very little testing of this code has been done 
> in production networks. In some deployments we have been able to produce a clock service by installing GNSS on the 
> cell-site. However, in other countries there are regulatory directives that the phase sync must be PTP/Network based.
>  
> Currently, the optical domain is causing us huge problems when we try to engineer a T-BC/PTP solution. This is due to 
> the path asymmetry that exist in the WDM/fiber domain. In some networks we have a lot of DCF in the fiber path and 
> the only way we can get visibility in the asymmetry on these fiber hops is to measure in both direction:(
> Also, running T-BC over WDM/OTN will simply not work as the phase error introduced more or less eats up the phase 
> error budget for 4G/5G TDD-service. 
>
> best regards,
> Geir
>
> > On 5 Sep 2020, at 00:17, Macho Pellegrini via NANOG <nanog () nanog org <mailto:nanog () nanog org>> wrote:
> >
> > Hello everybody,
> >
> > We have deployed PTP in our mobile NW since late 2019 as a part of the 4G/5G, however we are seeing a lots of 
> > instabilities and interop issues, a lot of the issues have ended up with SW bugs in the OS, I have no specific 
> > question, however I got the impression that the technology/protocol is not yet mature, anybody here got his hands 
> > dirty with PTP?
> >
> > Thanks,
> > MP