I have discovered a interesting default behaviour on a Nexus 7000 Router while troubleshooting. The Cisco Nexus 7000 switches (in this case a router), seem to drop ICMP echo replies.

Pinging from a directy connected IOS/IOS-XE device:

S1#ping 192.0.2.1 repeat 1000
Type escape sequence to abort.
Sending 1000, 100-byte ICMP Echos to 192.0.2.1, timeout is 2 seconds:
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!.!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!.!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!!!!!!!!!!!!!!!!!!!!
Success rate is 99 percent (998/1000), round-trip min/avg/max = 1/3/24 ms

You always want to have Success rate is 100 percent if devices are directly connected.

If pinging a target IOS/IOS-XE device over a routed network the result looks like this:

S1#ping 198.51.100.250 repeat 1000 size 1500 df-bit 
Type escape sequence to abort.
Sending 1000, 1500-byte ICMP Echos to 198.51.100.250, timeout is 2 seconds:
Packet sent with the DF bit set
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!!!!!!!!!!!!!!!!!!!!
Success rate is 100 percent (1000/1000), round-trip min/avg/max = 1/4/20 ms

Success rate is 100 percent with the df-bit and the maximum packet size. That is called a reliable link.

  • No packetloss if pinging over a routed network
  • Packetloss on directly connected devices

On the NX-OS router pinging the directly connected S1 switch. S1 has the IP 192.0.2.10.

R1# ping S1 count 1000
...
64 bytes from 192.0.2.10: icmp_seq=995 ttl=254 time=1.396 ms
64 bytes from 192.0.2.10: icmp_seq=996 ttl=254 time=2.188 ms
64 bytes from 192.0.2.10: icmp_seq=997 ttl=254 time=1.339 ms
64 bytes from 192.0.2.10: icmp_seq=998 ttl=254 time=1.352 ms
64 bytes from 192.0.2.10: icmp_seq=999 ttl=254 time=1.333 ms

--- S1 ping statistics ---
1000 packets transmitted, 996 packets received, 0.40% packet loss
round-trip min/avg/max = 1.014/1.786/17.317 ms

This link is not reliable enough. It looks like something is blocking the traffic.

And it is true. The documentation reveals a default security feature called CoPP (Control Plane Policing).

R1# sh copp status                                                                                          
Last Config Operation: None
Last Config Operation Timestamp: None
Last Config Operation Status: None
Policy-map attached to the control-plane: copp-system-p-policy-strict

It says the control plane has a policy-map attached to it. Strict or policy-stict. Looking deeper reveals:

R1# sh policy-map type ?
  control-plane  Control-Plane
  network-qos    Type network-qos
  qos            Type qos
  queuing        Type queuing

...4 different types of policy-maps are configured. The one we are interested in is the control-plane policy map:

R1# sh policy-map type control-plane 
...
class copp-system-p-class-monitoring
      set cos 1
  police cir 130 kbps bc 1000 ms 
    conform transmit violate drop 
...

And here a detailed output:

R1# sh policy-map type control-plane expand
...
class copp-system-p-class-monitoring
match access-group name copp-system-p-acl-icmp
match access-group name copp-system-p-acl-icmp6
match access-group name copp-system-p-acl-mpls-oam
match access-group name copp-system-p-acl-traceroute
match access-group name copp-system-p-acl-http-response
match access-group name copp-system-p-acl-smtp-response
match access-group name copp-system-p-acl-http6-response
match access-group name copp-system-p-acl-smtp6-response
set cos 1
police cir 130 kbps bc 1000 ms 
  conform transmit violate drop 
...

There is a class named copp-system-p-class-monitoring This particular class applies to following ACL's f.e:

  • copp-system-p-acl-icmp
  • copp-system-p-acl-icmp6

And some other well known protocols listed.

The policy map assignes the matched traffic (ICMP,HTTP,SMTP...) as class 1. This policy map has a Commited information rate (CIR) of 130 kbps. If the ICMP Traffic exceeds this threshhold the traffic will be dropped.

Hence the Quest is over.

We found a configuration snipplett for rate-limits incoming/outgoing traffic.

Limits are applied to modules 3-6:

R1# sh policy-map interface Control-plane | b monitoring
    class-map copp-system-p-class-monitoring (match-any)
      match access-group name copp-system-p-acl-icmp
      match access-group name copp-system-p-acl-icmp6
      match access-group name copp-system-p-acl-mpls-oam
      match access-group name copp-system-p-acl-traceroute
      match access-group name copp-system-p-acl-http-response
      match access-group name copp-system-p-acl-smtp-response
      match access-group name copp-system-p-acl-http6-response
      match access-group name copp-system-p-acl-smtp6-response
      set cos 1
      police cir 130 kbps bc 1000 ms 
        conform action: transmit 
        violate action: drop 
      module 3:
        conformed 194465448 bytes,
          5-min offered rate 16 bytes/sec
          peak rate 80 bytes/sec at Sun Dec 11 07:59:43 2016
        violated 0 bytes,
          5-min violate rate 0 bytes/sec
          peak rate 0 bytes/sec
      module 4:
        conformed 2019193 bytes,
          5-min offered rate 0 bytes/sec
          peak rate 550 bytes/sec at Thu Dec 29 14:29:46 2016
        violated 510 bytes,
          5-min violate rate 0 bytes/sec
          peak rate 1 bytes/sec at Thu Dec 29 14:29:46 2016
      module 5:
        conformed 0 bytes,
          5-min offered rate 0 bytes/sec
          peak rate 0 bytes/sec
        violated 0 bytes,
          5-min violate rate 0 bytes/sec
          peak rate 0 bytes/sec
      module 6:
        conformed 275859216 bytes,
          5-min offered rate 13 bytes/sec
          peak rate 505 bytes/sec at Sun Oct 02 09:14:32 2016
        violated 64 bytes,
          5-min violate rate 0 bytes/sec
          peak rate 0 bytes/sec

In the example above the drop policy has been hit on Thu Dec 29 14:29:46 2016 to module 4.

NX-OS cisco devices have default settings preconfigured to protect the control plane.

Further documentation about NX-OS CoPP feature is found here:

oneliner to see the troublemakers:

R1#show policy-map interface control-plane | i "class|conform|violated"