k3s.live

Based on the IT journey of Michael Rickert

Spanning tree loops, VRRP, and more with Dell N4000 switches

 

So you got yourself a bunch of shiny new Dell N4064F switches, and you want to use all of the advanced features like VRRP and MSTP. Sit down awhile and listen…. or just read these important notes for hard learned best practices when using advanced services on these higher end Dell switches. For reference I’ve built out campus wide networks on the back of the N4000 series with fairly positive outcomes _after_ learning these caveats.

 

Firmware, firmware, firmware!

There are some very, very very very VERY nasty bugs in firmware versions older than 6.3.1.13, if you havn’t already upgraded do so ASAP or expect reboots and crashes near constantly. Now, moving past firmware version 6.3.1.13 things become nice and stable.
copy usb://N4000v6.3.2.3.stk backup
boot system backup
copy run start
reload

Disable some still buggy CPU hogging features:

If you’re like me and don’t use hivemanager, go ahead and disable that now:

  eula-consent hiveagent reject
eula-consent support-assist reject

Also igmp snooping will use 30+% of cpu in most cases due to poor implementation in the DellOS code, so best to disable that as well:
no ip igmp snooping
no ipv6 mld snooping

I like to have my ports reset if they go errdisable, instead of sitting around dead forever…
errdisable recovery cause arp-inspection
errdisable recovery cause bpduguard
errdisable recovery cause dhcp-rate-limit
errdisable recovery cause sfp-mismatch
errdisable recovery cause sfpplus-mismatch
errdisable recovery cause udld
errdisable recovery cause ucast-storm
errdisable recovery cause bcast-storm
errdisable recovery cause mcast-storm
errdisable recovery cause bpdustorm
errdisable recovery cause loop-protect
errdisable recovery cause mac-locking
errdisable recovery cause denial-of-service
errdisable recovery interval 30

Also LLDP and ISDP come in handy quite frequently, lets turn those on too:
lldp med all
isdp enable

Important MSTP changes

First off, a warning: Sometimes things like VRRP like to bleed through blocked ports on the Dell N4000 switches, I was told this is because they operate as a part of the ‘management’ channel or some such, as long as you properly wrangle your packets it won’t be an issue.

I personally use MSTP for creating regions, and in doing so found that just adding all vlans to one instance served well:
  spanning-tree priority 24576
  spanning-tree mst 1 priority 24576
  spanning-tree mst configuration
  name “Region1”
  revision 2
  instance 1 add vlan 1-4094
  exit

Note instance 1 has all possible vlans added, this will ensure something doesn’t get tossed into instance 0 (default instance) later on if another vlan is added in. Remember that firmware warning earlier? If you’re running a version older than 6.3.1.13 you’ll get an immediate crash and restart on entering the ‘instance 1 add vlan 1-4094’ command, so be careful!

Also keep in mind that although you have regions, you still have a larger ‘root’ region (instance 0) that will still propagate out everywhere.

 

VRRP

On the Dell N4000 series you’ll notice that VRRP packets like to propagate. They like to make it very very very far down the stack. In fact I’ve see them make it all the way to end devices by sniffing around with wireshark. This also extends to OSPF hello packets and is definitely not best practice for network security or congestion. The good news is the DellOS firmware does honor mac address table access lists, and you can stop packets from making outside of your core network that way.

On your core switches that will connect out to your distribution layer, lets make a new access-list that blocks both vrrp and ospf packets at the MAC address layer:

  mac access-list extended block_vrrp_ospf
1000 deny any 0100.5E00.0012 0000.0000.0000
1001 deny any 0100.5E00.0005 0000.0000.0000
1010 permit any any
exit

Then just put in the range of ports you want vrrp/ospf packets to _not_ propagate down to:

  int range te1/0/1-44
  mac access-group block_vrrp_ospf out
  mac access-group block_vrrp_ospf in
  exit

Easy right? Well I can tell you first hand not blocking those VRRP packets is a fast and easy way to some crazy network traffic and routing loops, its worth the short amount of time to filter them out!

 

Finally, BPDU guard/protection/fast

Dell does BPDU guard a bit differently than other vendors… It’s not necessarily bad, and does work, its just different syntax:

First you’ll want to enable bpdu-protection at the global level, that way if a port designated as portfast/access/end client sends out a bpdu (managed switch packet) the port will shut off.

  spanning-tree bpdu-protection

By itself this command does nothing. If you have no ports set to both access and portfast you will never get bpdu-protection on any switchports. Lets set some ports to access and portfast, which will in turn enable bpdu-protection on those ports.

  interface range gigabitethernet 1/0/1-48
  spanning-tree portfast

  switchport access vlan 100

  switchport mode access

And your all set, any port on the switch in question that has an access port with portfast will now have bpdu-protection in place.

2 comments for “Spanning tree loops, VRRP, and more with Dell N4000 switches

Leave a Reply