Re: Strange behavior on the Juniper MX240

[Nehul Patel <nehul.patel () gmail com>]

ok got it thank you Nick

On Thu, May 5, 2022 at 3:43 PM Nick Olsen <Nick () 141networks com> wrote:

> Nehul,
>
> He was running the 15 code train. I think 15.1R6.7. But don't take that as
> fact. I just know it was 15 for sure.
> ------------------------------
> *From:* Nehul Patel <nehul.patel () gmail com>
> *Sent:* Thursday, May 5, 2022 6:40 PM
> *To:* Nick Olsen <Nick () 141networks com>
> *Cc:* nanog () nanog org <nanog () nanog org>
> *Subject:* Re: Strange behavior on the Juniper MX240
>
> Hi Nick,
>
> Thank you for the feedback on it. Would you please let me know which Juno
> OS version he had installed on the MX Chassis that works with the extended
> memory command of it?
>
> On Thu, May 5, 2022 at 12:50 PM Nick Olsen <Nick () 141networks com> wrote:
>
> Friend of mine had this issue recently on an MX chassis running DPC's and
> RE-2000's.
>
> The extend memory command others have mentioned worked for him.
>
> His instance drove us crazy for a bit. The device would learn a route,
> show that it was installed (show routes) but traffic to said prefix would
> bounce net unreachable. We even pushed a static just for S&G's and that
> still didn't resolve it. It was a single prefix that a customer had
> reported.
>
> Some things to consider, as others have mentioned.
>
>
>    1. IPv6 routes share the same space. And use more per-route. You can
>    extend the life of this box (probably considerably) by dropping full tables
>    for IPv6. Perhaps taking just a default (Same goes for v4).
>    2. It seems from your previous output that you're taking ~1 full v4
>    table. And 2x v6 tables. Do you really need a full table if you're only
>    taking 1 v4 table? Consider switching to a default only? In my Colleagues
>    case, he was taking 2 full tables of v4 and v6 until he hit the same issue.
>    3. While you're RE's could use a nice upgrade too. Your linecards are
>    actually the problem here. If you move to anything > DPC you get the trio
>    chipset with much more FIB space (2 Million routes I believe?). I'd
>    consider new RE's and new line cards for this box. Which might also mean
>    new switch fabric controllers.... Basically, we'd be talking a full
>    overhaul sans the power supplies and chassis.
>    4. Consider taking a default + full routes. Then filtering > /24 (if
>    you even have anything < /24 learned now) (/48 on IPv6).
>
> Start with the memory command first and see where that gets you. But keep
> a watchful eye out for this to happen again (as the DFZ grows). Eventually
> your only option will be to filter routes and rely on a default or upgrade.
> ------------------------------
> *From:* NANOG <nanog-bounces+nick=141networks.com () nanog org> on behalf of
> Nehul Patel <nehul.patel () gmail com>
> *Sent:* Wednesday, May 4, 2022 3:56 PM
> *To:* nanog () nanog org <nanog () nanog org>
> *Subject:* Strange behavior on the Juniper MX240
>
>
> Hi NANOG,
>
> We are seeing some strange behavior on our Juniper MX240 Chassis it is
> randomly dropping the routes to the certain destination IP address getting
> the following errors on the MX240 Chassis
>
> If Someone has seen these errors before please suggest how to resolve it
>
>
> May  4 12:42:00 cr01 newsyslog[44735]: logfile turned over due to
> size>1024K
> May  4 12:42:01   /kernel: RT_PFE: RT msg op 1 (PREFIX ADD) failed, err 5
> (Invalid)
> May  4 12:42:01   /kernel: RT_PFE: RT msg op 3 (PREFIX CHANGE) failed, err
> 5 (Invalid)
> May  4 12:42:01   last message repeated 4 times
> May  4 12:42:01   fpc0 RT: IPv6:0 - 2600:40fc:1011::/48 (add rt entry into
> jtree failed)
> May  4 12:42:01   fpc0 RT-HAL,rt_entry_add_msg_proc,2028:
> rt_halp_vectors->rt_create failed
> May  4 12:42:01   fpc0 RT-HAL,rt_entry_add_msg_proc,2092: proto ipv6,len
> 48 prefix 2600:40fc:1011::/48 nh 1048576
> May  4 12:42:01   fpc0 RT-HAL,rt_msg_handler,540: route process failed
> May  4 12:42:01   fpc0 RT: Failed prefix add IPv6 - 2001:67c:20fc::/48 (No
> memory) on FE 0
> May  4 12:42:01   fpc0 RT: IPv6:0 - 2001:67c:20fc::/48 (add rt entry into
> jtree failed)
> May  4 12:42:01   fpc0 RT-HAL,rt_entry_add_msg_proc,2028:
> rt_halp_vectors->rt_create failed
> May  4 12:42:01   fpc0 RT-HAL,rt_entry_add_msg_proc,2092: proto ipv6,len
> 48 prefix 2001:67c:20fc::/48 nh 1048576
> May  4 12:42:01   fpc0 RT-HAL,rt_msg_handler,540: route process failed
> May  4 12:42:01   fpc0 RT: Failed prefix add IPv6 - 2606:2800:e004::/48
> (No memory) on FE 0
> May  4 12:42:01   fpc0 RT: Failed prefix add IPv6 - 2a05:3181:ffff::/48
> (No memory) on FE 0
> May  4 12:42:01   /kernel: RT_PFE: RT msg op 3 (PREFIX CHANGE) failed, err
> 5 (Invalid)
> May  4 12:42:01   /kernel: RT_PFE: RT msg op 1 (PREFIX ADD) failed, err 5
> (Invalid)
> May  4 12:42:01   /kernel: RT_PFE: RT msg op 2 (PREFIX DELETE) failed, err
> 5 (Invalid)
> May  4 12:42:02   fpc0 RT: Failed prefix add IPv4 - 79.120.22/24 (No
> memory) on FE 0
> May  4 12:42:02   fpc0 RT: IPv4:0 - 79.120.22/24 (add rt entry into jtree
> failed)
> May  4 12:42:02   fpc0 RT-HAL,rt_entry_add_msg_proc,2028:
> rt_halp_vectors->rt_create failed
> May  4 12:42:02   fpc0 RT-HAL,rt_entry_add_msg_proc,2092: proto ipv4,len
> 24 prefix 79.120.22/24 nh 1048583
> May  4 12:42:02   /kernel: RT_PFE: RT msg op 1 (PREFIX ADD) failed, err 5
> (Invalid)
> May  4 12:42:02   fpc0 RT-HAL,rt_msg_handler,540: route process failed
>
> May  4 09:33:17   fpc0 RSMON: Resource Category:jtree
>  Instance:jtree2-seg0 Type:free-pages Available:20 is less than LWM
> limit:1638, rsmon_syslog_limit()
> May  4 09:33:17   fpc0 RSMON: Resource Category:jtree
>  Instance:jtree2-seg0 Type:free-dwords Available:1280 is less than LWM
> limit:104857, rsmon_syslog_limit()
> May  4 09:33:18   fpc0 RSMON: Resource Category:jtree
>  Instance:jtree3-seg0 Type:free-pages Available:19 is less than LWM
> limit:1638, rsmon_syslog_limit()
> May  4 09:33:18   fpc0 RSMON: Resource Category:jtree
>  Instance:jtree3-seg0 Type:free-dwords Available:1216 is less than LWM
> limit:104857, rsmon_syslog_limit()
> May  4 09:33:18   fpc1 RSMON: Resource Category:jtree
>  Instance:jtree0-seg0 Type:free-pages Available:16 is less than LWM
> limit:1638, rsmon_syslog_limit()
> May  4 09:33:18   fpc1 RSMON: Resource Category:jtree
>  Instance:jtree0-seg0 Type:free-dwords Available:1024 is less than LWM
> limit:104857, rsmon_syslog_limit()
> May  4 09:33:18   fpc1 RSMON: Resource Category:jtree
>  Instance:jtree1-seg0 Type:free-pages Available:15 is less than LWM
> limit:1638, rsmon_syslog_limit()
> May  4 09:33:18   fpc1 RSMON: Resource Category:jtree
>  Instance:jtree1-seg0 Type:free-dwords Available:960 is less than LWM
> limit:104857, rsmon_syslog_limit()
> May  4 09:33:18   fpc1 RSMON: Resource Category:jtree
>  Instance:jtree2-seg0 Type:free-pages Available:19 is less than LWM
> limit:1638, rsmon_syslog_limit()
> May  4 09:33:19   fpc1 RSMON: Resource Category:jtree
>  Instance:jtree2-seg0 Type:free-dwords Available:1216 is less than LWM
> limit:104857, rsmon_syslog_limit()
> May  4 09:33:19   fpc1 RSMON: Resource Category:jtree
>  Instance:jtree3-seg0 Type:free-pages Available:17 is less than LWM
> limit:1638, rsmon_syslog_limit()
> May  4 09:33:19   fpc1 RSMON: Resource Category:jtree
>  Instance:jtree3-seg0 Type:free-dwords Available:1088 is less than LWM
> limit:104857, rsmon_syslog_limit()
> May  4 09:33:19   fpc2 RSMON: Resource Category:jtree
>  Instance:jtree0-seg0 Type:free-pages Available:15 is less than LWM
> limit:1638, rsmon_syslog_limit()
> May  4 09:33:19   fpc2 RSMON: Resource Category:jtree
>  Instance:jtree0-seg0 Type:free-dwords Available:960 is less than LWM
> limit:104857, rsmon_syslog_limit()
> May  4 09:33:19   fpc2 RSMON: Resource Category:jtree
>  Instance:jtree1-seg0 Type:free-pages Available:15 is less than LWM
> limit:1638, rsmon_syslog_limit()
> May  4 09:33:19   fpc2 RSMON: Resource Category:jtree
>  Instance:jtree1-seg0 Type:free-dwords Available:960 is less than LWM
> limit:104857, rsmon_syslog_limit()
>
> Any suggestions will be helpful
>
>
> Please do let me know if you have any questions.
>
>
>
>
> Regards and thanks,
>  Nehul