Forwarding in vPC through FHRP backup
In this post, we will cover the benefits introduced by Virtual Port-Channel (vPC) for First Hop Redundancy Protocols (FHRP).
Hot Standby Router Protocol (HSRP) and Virtual Router Redundancy Protocol (VRRP) are two of these FHRPs.
HSRP is Cisco proprietary whereas VRRP is a standard-based protocol and allows interoperability between different vendors.
vPC enhances FHRPs by allowing them to operate in active-active mode at data plane while at control plane the FHRP still works in active-backup mode.
The above statement considers active as the ability to forward traffic and it is just a coincidence with the HSRP state. In VRRP, the master forwards the traffic, whereas the backup does not.
Coming back to the vPC use case, at the control plane, the active router is still responsible to reply to ARP Requests with the virtual MAC address. In case the ARP request is received by the backup router, the ARP request is forwarded to the active router through the VPC peer-link which then replies with the virtual MAC address.
The feature that allows FHRPs to work in active-active mode at data plane is vPC built-in and there is no additional configuration required to enable it.
The active-active operation mode is facilitated by imposing G bit (Gateway bit) for the FHRP virtual MAC in the MAC address table on both vPC peer devices. We are going to see what it means later.
Let’s see an example using VRRP as FHRP on two Nexus devices.
This is the network diagram:
R_ACCESS has a default gateway 172.16.123.254 which is the virtual IP address configured in VRRP.
R_ACCESS#show ip route Gateway of last resort is 172.16.123.254 to network 0.0.0.0 S* 0.0.0.0/0 [1/0] via 172.16.123.254 172.16.0.0/16 is variably subnetted, 2 subnets, 2 masks C 172.16.123.0/24 is directly connected, GigabitEthernet0/1 L 172.16.123.100/32 is directly connected, GigabitEthernet0/1 R_ACCESS#
The ARP table on R_ACCESS shows the virtual MAC address for the default gateway IP:
R_ACCESS#show ip arp Protocol Address Age (min) Hardware Addr Type Interface Internet 172.16.123.254 0 0000.5e00.0101 ARPA GigabitEthernet0/1 Internet 172.16.123.100 - fa16.3e8b.ce06 ARPA GigabitEthernet0/1 R_ACCESS#
A best practice recommendation is to have at control-plane the active router the same device that is the vPC primary peer device.
In this case, NX_OS-1 is the master router and in the same time it is the primary peer.
NX_OS-1# show vrrp detail Vlan1000 - Group 1 (IPV4) State is Master Virtual IP address is 172.16.123.254 Priority 110, Configured 110 Forwarding threshold(for VPC), lower: 1 upper: 110 Advertisement interval 1 Preemption enabled Authentication text "VRRP_AU" Virtual MAC address is 0000.5e00.0101 Master router is Local NX_OS-1# show vpc Legend: (*) - local vPC is down, forwarding via vPC peer-link vPC domain id : 21 Peer status : peer adjacency formed ok vPC keep-alive status : peer is alive Configuration consistency status : success Per-vlan consistency status : success Type-2 consistency status : success vPC role : primary Number of vPCs configured : 1 Peer Gateway : Disabled Dual-active excluded VLANs : - Graceful Consistency Check : Enabled Auto-recovery status : Disabled Delay-restore status : Timer is off.(timeout = 30s) Delay-restore SVI status : Timer is off.(timeout = 10s) Operational Layer3 Peer-router : Disabled vPC Peer-link status --------------------------------------------------------------------- id Port Status Active vlans -- ---- ------ ------------------------------------------------- 1 Po21 up 1000 vPC status ---------------------------------------------------------------------------- Id Port Status Consistency Reason Active vlans -- ------------ ------ ----------- ------ --------------- 1 Po1 up success success 1000
Please check “show vpc consistency-parameters vpc” for the consistency reason of down vpc and for type-2 consistency reasons for any vpc.
NX_OS-1#
The Port-Channel between the access switch and the two Nexus devices is up:
SW_ACCESS#show etherchannel summary Flags: D - down P - bundled in port-channel I - stand-alone s - suspended H - Hot-standby (LACP only) R - Layer3 S - Layer2 U - in use N - not in use, no aggregation f - failed to allocate aggregator M - not in use, minimum links not met m - not in use, port not aggregated due to minimum links not met u - unsuitable for bundling w - waiting to be aggregated d - default port A - formed by Auto LAG Number of channel-groups in use: 1 Number of aggregators: 1 Group Port-channel Protocol Ports ------+-------------+-----------+----------------------------------------------- 1 Po1(SU) LACP Gi0/1(P) Gi0/2(P) SW_ACCESS#
R_CORE is running OSPF with the two Nexus devices for end-to-end reachability:
R_CORE#show ip route 172.16.0.0/24 is subnetted, 1 subnets O 172.16.123.0 [110/41] via 10.10.13.2, 00:01:57, GigabitEthernet0/2 [110/41] via 10.10.12.2, 00:03:30, GigabitEthernet0/1 R_CORE#
This being said, when the traffic from R_ACCESS needs to go R_CORE (that is, pass through one of the two Nexus devices), it will reach the access switch. At this point, the switch will decide based on the hashing algorithm which of the two links from the Port-Channel will use. It can go directly to the master VRRP router or it can go to the backup VRRP router.
The below diagram shows the situation when the traffic hits NX_OS-1 which is the master VRRP:
On R_CORE device, there is an access list applied on each the two interfaces matching and logging the traffic from R_ACCESS to R_CORE’s loopback IP address to confirm that traffic is received for this pair (172.16.123.100,1.1.1.1) on both interfaces:
interface GigabitEthernet0/1 ip address 10.10.12.1 255.255.255.0 ip access-group VIA_NX_OS-1 in duplex auto speed auto media-type rj45 ! interface GigabitEthernet0/2 ip address 10.10.13.1 255.255.255.0 ip access-group VIA_NX_OS-2 in duplex auto speed auto media-type rj45 ! ip access-list extended VIA_NX_OS-1 permit icmp host 172.16.123.100 host 1.1.1.1 log-input permit ip any any ip access-list extended VIA_NX_OS-2 permit icmp host 172.16.123.100 host 1.1.1.1 log-input permit ip any any
When traffic takes the path through NX_OS-1, the packet matches the ACL from GigabitEthernet0/1:
%SEC-6-IPACCESSLOGDP: list VIA_NX_OS-1 permitted icmp 172.16.123.100 (GigabitEthernet0/1 5e00.0000.0007) -> 1.1.1.1 (0/0), 5 packets
And the ACL counter is increasing:
R_CORE#show ip access-lists Extended IP access list VIA_NX_OS-1 10 permit icmp host 172.16.123.100 host 1.1.1.1 log-input (5 matches) 20 permit ip any any (16 matches) Extended IP access list VIA_NX_OS-2 10 permit icmp host 172.16.123.100 host 1.1.1.1 log-input 20 permit ip any any (8 matches) R_CORE#
The diagram below shows when the traffic hits NX_OS-2:
When the traffic takes this path, the ACL matches the packet and logs it:
%SEC-6-IPACCESSLOGDP: list VIA_NX_OS-2 permitted icmp 172.16.123.100 (GigabitEthernet0/2 5e00.0001.0007) -> 1.1.1.1 (0/0), 5 packets
Again, the ACL counters are increasing:
R_CORE#show ip access-lists Extended IP access list VIA_NX_OS-1 10 permit icmp host 172.16.123.100 host 1.1.1.1 log-input (5 matches) 20 permit ip any any (25 matches) Extended IP access list VIA_NX_OS-2 10 permit icmp host 172.16.123.100 host 1.1.1.1 log-input (5 matches) 20 permit ip any any (8 matches) R_CORE#
As you can, see whenever the traffic goes to NX_OS-2, the traffic is routed directly to the destination according to the routing table.
This is how the ARP/MAC table looks on NX_OS-1, the master VRRP:
NX_OS-1# show ip arp Flags: * - Adjacencies learnt on non-active FHRP router + - Adjacencies synced via CFSoE # - Adjacencies Throttled for Glean CP - Added via L2RIB, Control plane Adjacencies PS - Added via L2RIB, Peer Sync RO - Dervied from L2RIB Peer Sync Entry D - Static Adjacencies attached to down interface IP ARP Table for context default Total number of entries: 3 Address Age MAC Address Interface Flags 10.10.12.1 00:01:22 fa16.3ecd.b31a Ethernet1/4 172.16.123.2 00:00:09 5e00.0001.0007 Vlan1000 172.16.123.254 - 0000.5e00.0101 Vlan1000 NX_OS-1# show mac address-table Legend: * - primary entry, G - Gateway MAC, (R) - Routed MAC, O - Overlay MAC age - seconds since last seen,+ - primary entry using vPC Peer-Link, (T) - True, (F) - False, C - ControlPlane MAC VLAN MAC Address Type age Secure NTFY Ports ---------+-----------------+--------+---------+------+----+------------------ G 1000 0000.5e00.0101 static - F F sup-eth1(R) G - 5e00.0000.0007 static - F F sup-eth1(R) G 1000 5e00.0000.0007 static - F F sup-eth1(R) * 1000 5e00.0001.0007 static - F F vPC Peer-Link(R) NX_OS-1#
And on NX_OS-2:
NX_OS-2# show ip arp Flags: * - Adjacencies learnt on non-active FHRP router + - Adjacencies synced via CFSoE # - Adjacencies Throttled for Glean CP - Added via L2RIB, Control plane Adjacencies PS - Added via L2RIB, Peer Sync RO - Dervied from L2RIB Peer Sync Entry D - Static Adjacencies attached to down interface IP ARP Table for context default Total number of entries: 3 Address Age MAC Address Interface Flags 10.10.13.1 00:15:50 fa16.3ee0.57bf Ethernet1/4 172.16.123.1 00:01:18 5e00.0000.0007 Vlan1000 172.16.123.254 00:00:05 0000.5e00.0101 Vlan1000 NX_OS-2# show mac address-table Legend: * - primary entry, G - Gateway MAC, (R) - Routed MAC, O - Overlay MAC age - seconds since last seen,+ - primary entry using vPC Peer-Link, (T) - True, (F) - False, C - ControlPlane MAC VLAN MAC Address Type age Secure NTFY Ports ---------+-----------------+--------+---------+------+----+------------------ G 1000 0000.5e00.0101 static - F F vPC Peer-Link(R) * 1000 5e00.0000.0007 static - F F vPC Peer-Link(R) G - 5e00.0001.0007 static - F F sup-eth1(R) G 1000 5e00.0001.0007 static - F F sup-eth1(R) NX_OS-2#
On the active router at control plane, the virtual MAC points to sup-eth1(R), while on the backup router at control plane, the virtual MAC points to the vPC peer-link and this is a good way to determine which router is the active and which is the backup on control plane level.
As mentioned at the beginning of the article, you do not need to do anything to benefit from this mode of operation of FHRP in combination with vPCs.
Reference:
1. Design and Configuration Guide: Best Practices for Virtual Port Channels (vPC) on Cisco Nexus 7000 Series Switches
Thank you to Paris Arau for his contributions to this article.