Re: Strange behavior on the Juniper MX240

[Mark Tinka <mark@tinka.africa>]

On 5/4/22 21:56, Nehul Patel wrote:
> Hi NANOG,
>
> We are seeing some strange behavior on our Juniper MX240 Chassis it is 
> randomly dropping the routes to the certain destination IP address 
> getting the following errors on the MX240 Chassis
>
> If Someone has seen these errors before please suggest how to resolve it
>
>
> May  4 12:42:00 cr01 newsyslog[44735]: logfile turned over due to 
> size>1024K
> May  4 12:42:01   /kernel: RT_PFE: RT msg op 1 (PREFIX ADD) failed, 
> err 5 (Invalid)
> May  4 12:42:01   /kernel: RT_PFE: RT msg op 3 (PREFIX CHANGE) failed, 
> err 5 (Invalid)
> May  4 12:42:01   last message repeated 4 times
> May  4 12:42:01   fpc0 RT: IPv6:0 - 2600:40fc:1011::/48 (add rt entry 
> into jtree failed)
> May  4 12:42:01   fpc0 RT-HAL,rt_entry_add_msg_proc,2028: 
> rt_halp_vectors->rt_create failed
> May  4 12:42:01   fpc0 RT-HAL,rt_entry_add_msg_proc,2092: proto 
> ipv6,len 48 prefix 2600:40fc:1011::/48 nh 1048576
> May  4 12:42:01   fpc0 RT-HAL,rt_msg_handler,540: route process failed
> May  4 12:42:01   fpc0 RT: Failed prefix add IPv6 - 2001:67c:20fc::/48 
> (No memory) on FE 0
> May  4 12:42:01   fpc0 RT: IPv6:0 - 2001:67c:20fc::/48 (add rt entry 
> into jtree failed)
> May  4 12:42:01   fpc0 RT-HAL,rt_entry_add_msg_proc,2028: 
> rt_halp_vectors->rt_create failed
> May  4 12:42:01   fpc0 RT-HAL,rt_entry_add_msg_proc,2092: proto 
> ipv6,len 48 prefix 2001:67c:20fc::/48 nh 1048576
> May  4 12:42:01   fpc0 RT-HAL,rt_msg_handler,540: route process failed
> May  4 12:42:01   fpc0 RT: Failed prefix add IPv6 - 
> 2606:2800:e004::/48 (No memory) on FE 0
> May  4 12:42:01   fpc0 RT: Failed prefix add IPv6 - 
> 2a05:3181:ffff::/48 (No memory) on FE 0
> May  4 12:42:01   /kernel: RT_PFE: RT msg op 3 (PREFIX CHANGE) failed, 
> err 5 (Invalid)
> May  4 12:42:01   /kernel: RT_PFE: RT msg op 1 (PREFIX ADD) failed, 
> err 5 (Invalid)
> May  4 12:42:01   /kernel: RT_PFE: RT msg op 2 (PREFIX DELETE) failed, 
> err 5 (Invalid)
> May  4 12:42:02   fpc0 RT: Failed prefix add IPv4 - 79.120.22/24 (No 
> memory) on FE 0
> May  4 12:42:02   fpc0 RT: IPv4:0 - 79.120.22/24 (add rt entry into 
> jtree failed)
> May  4 12:42:02   fpc0 RT-HAL,rt_entry_add_msg_proc,2028: 
> rt_halp_vectors->rt_create failed
> May  4 12:42:02   fpc0 RT-HAL,rt_entry_add_msg_proc,2092: proto 
> ipv4,len 24 prefix 79.120.22/24 nh 1048583
> May  4 12:42:02   /kernel: RT_PFE: RT msg op 1 (PREFIX ADD) failed, 
> err 5 (Invalid)
> May  4 12:42:02   fpc0 RT-HAL,rt_msg_handler,540: route process failed
>
> May  4 09:33:17   fpc0 RSMON: Resource Category:jtree 
>  Instance:jtree2-seg0 Type:free-pages Available:20 is less than LWM 
> limit:1638, rsmon_syslog_limit()
> May  4 09:33:17   fpc0 RSMON: Resource Category:jtree 
>  Instance:jtree2-seg0 Type:free-dwords Available:1280 is less than LWM 
> limit:104857, rsmon_syslog_limit()
> May  4 09:33:18   fpc0 RSMON: Resource Category:jtree 
>  Instance:jtree3-seg0 Type:free-pages Available:19 is less than LWM 
> limit:1638, rsmon_syslog_limit()
> May  4 09:33:18   fpc0 RSMON: Resource Category:jtree 
>  Instance:jtree3-seg0 Type:free-dwords Available:1216 is less than LWM 
> limit:104857, rsmon_syslog_limit()
> May  4 09:33:18   fpc1 RSMON: Resource Category:jtree 
>  Instance:jtree0-seg0 Type:free-pages Available:16 is less than LWM 
> limit:1638, rsmon_syslog_limit()
> May  4 09:33:18   fpc1 RSMON: Resource Category:jtree 
>  Instance:jtree0-seg0 Type:free-dwords Available:1024 is less than LWM 
> limit:104857, rsmon_syslog_limit()
> May  4 09:33:18   fpc1 RSMON: Resource Category:jtree 
>  Instance:jtree1-seg0 Type:free-pages Available:15 is less than LWM 
> limit:1638, rsmon_syslog_limit()
> May  4 09:33:18   fpc1 RSMON: Resource Category:jtree 
>  Instance:jtree1-seg0 Type:free-dwords Available:960 is less than LWM 
> limit:104857, rsmon_syslog_limit()
> May  4 09:33:18   fpc1 RSMON: Resource Category:jtree 
>  Instance:jtree2-seg0 Type:free-pages Available:19 is less than LWM 
> limit:1638, rsmon_syslog_limit()
> May  4 09:33:19   fpc1 RSMON: Resource Category:jtree 
>  Instance:jtree2-seg0 Type:free-dwords Available:1216 is less than LWM 
> limit:104857, rsmon_syslog_limit()
> May  4 09:33:19   fpc1 RSMON: Resource Category:jtree 
>  Instance:jtree3-seg0 Type:free-pages Available:17 is less than LWM 
> limit:1638, rsmon_syslog_limit()
> May  4 09:33:19   fpc1 RSMON: Resource Category:jtree 
>  Instance:jtree3-seg0 Type:free-dwords Available:1088 is less than LWM 
> limit:104857, rsmon_syslog_limit()
> May  4 09:33:19   fpc2 RSMON: Resource Category:jtree 
>  Instance:jtree0-seg0 Type:free-pages Available:15 is less than LWM 
> limit:1638, rsmon_syslog_limit()
> May  4 09:33:19   fpc2 RSMON: Resource Category:jtree 
>  Instance:jtree0-seg0 Type:free-dwords Available:960 is less than LWM 
> limit:104857, rsmon_syslog_limit()
> May  4 09:33:19   fpc2 RSMON: Resource Category:jtree 
>  Instance:jtree1-seg0 Type:free-pages Available:15 is less than LWM 
> limit:1638, rsmon_syslog_limit()
> May  4 09:33:19   fpc2 RSMON: Resource Category:jtree 
>  Instance:jtree1-seg0 Type:free-dwords Available:960 is less than LWM 
> limit:104857, rsmon_syslog_limit()
>
> Any suggestions will be helpful

Looks like you are out of FIB slots.

Would recommend reducing the number of routes you need to send into FIB, 
or upgrading to newer hardware that has more space.

Mark.