avatar elvin thoughts
  • Home
  • About
  • Now

Understanding EVPN-VPWS

Jun 03, 22

Overview

We will explore the theory behind L2VPN Point-to-Point (P2P) with EVPN-VPWS, we will also dive into its implementation in Junos, but the main purpose of this article is to provide the fundamental theory behind it.

Introduction

A plethora of technologies exist to interconnect layer 2 domains over multiprotocol label switching (MPLS) networks or IP-only transit networks, among them one of the latest additions to the L2VPN category is the Ethernet VPN (EVPN) family. In particular EVPN-VPWS, defined in RFC 8214, provides the set of functions to provide P2P reachability over an MPLS network. By leveraging BGP, we can distribute the EVPN-VPWS services across the service provider network to the intended egress Label Switch Routers (LSR).

Among the advantages that EVPN-VPWS provide are:

  1. BGP Signaling of Pseudowires - Only BGP is used to signal the PW labels, no other protocols (such as LDP) are triggered for the PW to establish.
  2. Single-Active and All-Active Multihoming - EVPN family provides the capabilities to perform single active ro all active multihoming while not causing data plane issues
  3. Entropy Signaling with Control Word
  4. Fast Protection upon Node or Link Failure

Unlike Multipoint L2VPNs, P2P L2VPN services do not require MAC lookups in order to forward frames between endpoints, therefore there is no concept of bridge-domains or MAC tables in this type of service as there is only a single endpoint where the frames are required to be forwarded. The customer-facing interface known as Attachment Circuit (AC) at the ingress LSR receives an Ethernet frame which subsequently is wrapped with a service label signal through BGP EVPN SAFI and sent across the backbone towards the egress LSR using the corresponding transport label.

EVPN-VPWS Auto-Discovery

RFC 7434 defines the architecture for the EVPN family and applications, and within it provides the definition of different route types required for these services to function, each route with a role. Specifically for EVPN-VPWS, we need to dive into the EVPN Type-1 route also known as Ethernet Auto-Discovery (A-D) route in its two variants: a) Per-ESI, b) Per-EVI.

In simple terms, Per-EVI is what will be generated when we have a single-home scenario, for multi-homing, a per-ESI route would be required. The Ethernet Auto Discovery (Ethernet A-D) route is defined as follows:

Image 1 - Type-1: Ethernet A-D route

            	+---------------------------------------------------+

            	|  Route Distinguisher (RD) (8 octets)     |

            	+---------------------------------------------------+

            	| Ethernet Segment Identifier (10 octets)|

            	+---------------------------------------------------+

            	|  Ethernet Tag ID (4 octets)       	     |

            	+---------------------------------------------------+

            	|  MPLS Label (3 octets)            	     |

            	+---------------------------------------------------+

  1. Route Distinguisher - Route distinguisher serves the purpose of uniquely identify the prefix in the AS
  2. Ethernet Segment Identifier - The ESI is 10 bytes, and is set to 0 for Single Home (SH) and to non-zero values when used in multi-homed (MH) environments
  3. Ethernet Tag ID - For VPWS services, the Ethernet Tag ID is set to the local service identifier of the Attachment Circuit
  4. MPLS Label - Service label announced by BGP for the L2VPN EVPN prefix

Also, extended communities are used in the BGP advertisement in order to signal additional attributes.

Image 2 - EVPN-VPWS Extended Communities

              +-------------------------------------------------------+

               |  Type (0x06) / Sub-type (0x04) (2 octets)  |

               +-------------------------------------------------------+

               |  Control Flags  (2 octets)                            |

               +--------------------------------------------------------+

               |  L2 MTU (2 octets)                                      |

               +--------------------------------------------------------+

               |  Reserved (2 octets)                                   |

               +--------------------------------------------------------+

Flags described below:

  1. C-flag to signal Control Word
  2. L2 MTU flag
  3. P-bit is set if device is Primary (only applicable for Multihoming topologies)
  4. B-bit is set if device is Backup (only applicable for Multihoming topologies)
  5. Reserved fields

Topology

Configuration

For our example, R2 and R4 are our PE routers and these are Juniper vMX devices, R1 (CE), R3 (P), and R5 (CE) are Cisco routers.

R1
!
hostname R1
!
interface Ethernet0/0.15
 encapsulation dot1Q 15
 ip address 15.0.0.1 255.255.255.0
!
end
R2
set system host-name R2

# core-facing
set interfaces ge-0/0/0 vlan-tagging
set interfaces ge-0/0/0 unit 23 vlan-id 23
set interfaces ge-0/0/0 unit 23 family inet address 10.2.3.2/24
set interfaces ge-0/0/0 unit 23 family mpls

# loopback0
set interfaces lo0 unit 0 family inet address 2.2.2.2/32
set interfaces lo0 unit 0 family mpls

# bgp
set routing-options autonomous-system 100
set protocols bgp group IBGP_AS100 type internal
set protocols bgp group IBGP_AS100 local-address 2.2.2.2
set protocols bgp group IBGP_AS100 family evpn signaling
set protocols bgp group IBGP_AS100 peer-as 100
set protocols bgp group IBGP_AS100 neighbor 4.4.4.4 description "IBGP AS100 Peer"

# igp
set protocols ospf area 0.0.0.0 interface lo0.0
set protocols ospf area 0.0.0.0 interface ge-0/0/0.23 interface-type p2p

# mpls
set protocols mpls interface lo0.0
set protocols mpls interface ge-0/0/0.23
set protocols ldp track-igp-metric
set protocols ldp mtu-discovery
set protocols ldp deaggregate
set protocols ldp interface ge-0/0/0.23
set protocols ldp interface lo0.0
R3

hostname R3
!
mpls ldp entropy-label
!
interface Loopback0
 ip address 3.3.3.3 255.255.255.255
 ip ospf 1 area 0
!
interface Ethernet0/0.23
 encapsulation dot1Q 23
 ip address 10.2.3.3 255.255.255.0
 ip ospf network point-to-point
 ip ospf 1 area 0
!
interface Ethernet0/0.34
 encapsulation dot1Q 34
 ip address 10.3.4.3 255.255.255.0
 ip ospf network point-to-point
 ip ospf 1 area 0
!
interface Ethernet0/0.36
 encapsulation dot1Q 36
 ip address 10.3.6.3 255.255.255.0
 ip ospf 1 area 0
!
router ospf 1
 router-id 3.3.3.3
 mpls ldp autoconfig
 !
end
R4
set system host-name R4

# core-facing
set interfaces ge-0/0/0 vlan-tagging
set interfaces ge-0/0/0 unit 34 vlan-id 34
set interfaces ge-0/0/0 unit 34 family inet address 10.3.4.4/24
set interfaces ge-0/0/0 unit 34 family mpls

# loopback0
set interfaces lo0 unit 0 family inet address 4.4.4.4/32
set interfaces lo0 unit 0 family mpls

# bgp
set routing-options autonomous-system 100
set protocols bgp group IBGP_AS100 type internal
set protocols bgp group IBGP_AS100 local-address 4.4.4.4
set protocols bgp group IBGP_AS100 family evpn signaling
set protocols bgp group IBGP_AS100 peer-as 100
set protocols bgp group IBGP_AS100 neighbor 2.2.2.2 description "IBGP AS100 Peer"

# igp
set protocols ospf area 0.0.0.0 interface lo0.0
set protocols ospf area 0.0.0.0 interface ge-0/0/0.34 interface-type p2p

# mpls
set protocols mpls interface lo0.0
set protocols mpls interface ge-0/0/0.34
set protocols ldp track-igp-metric
set protocols ldp mtu-discovery
set protocols ldp deaggregate
set protocols ldp interface ge-0/0/0.34
set protocols ldp interface lo0.0
R5

hostname R5
!
interface Ethernet0/0.15
 encapsulation dot1Q 15
 ip address 15.0.0.5 255.255.255.0
!
end

The infrastructure is in place, we can do a ping mpls in order to test MPLS infrastructure.

# R2
root@R2# run traceroute mpls ldp 4.4.4.4/32
  Probe options: ttl 64, retries 3, wait 10, paths 16, exp 7, fanout 16

  ttl	Label  Protocol	Address      	Previous Hop 	Probe Status
	1   	19  LDP     	10.2.3.3     	(null)       	Success
  FEC-Stack-Sent: LDP
  ttl	Label  Protocol	Address      	Previous Hop 	Probe Status
	2    	3  Unknown 	10.3.4.4     	10.2.3.3     	Egress
  FEC-Stack-Sent: LDP

  Path 1 via ge-0/0/0.23 destination 127.0.0.64

# R4
root@R4# run traceroute mpls ldp 2.2.2.2/32
  Probe options: ttl 64, retries 3, wait 10, paths 16, exp 7, fanout 16

  ttl	Label  Protocol	Address      	Previous Hop 	Probe Status
	1   	18  LDP     	10.3.4.3     	(null)       	Success
  FEC-Stack-Sent: LDP
  ttl	Label  Protocol	Address      	Previous Hop 	Probe Status
	2    	3  Unknown 	10.2.3.2     	10.3.4.3     	Egress
  FEC-Stack-Sent: LDP

  Path 1 via ge-0/0/0.34 destination 127.0.0.64

Having the infrastructure set, we can now proceed with the configuration of EVPN-VPWS, this includes the Attachment Circuit, which is the PE-to-CE facing interface, plus a routing-instance that will hold the attributes for the EVPN-VPWS Instance we are creating.

R2

# ac
set interfaces ge-0/0/1 flexible-vlan-tagging
set interfaces ge-0/0/1 encapsulation flexible-ethernet-services
set interfaces ge-0/0/1 unit 15 encapsulation vlan-ccc
set interfaces ge-0/0/1 unit 15 vlan-id 15

# routing-instance EVPN-VPWS
set routing-instances EVPN-VPWS-15 description "EVPN-VPWS-15"
set routing-instances EVPN-VPWS-15 instance-type evpn-vpws
set routing-instances EVPN-VPWS-15 interface ge-0/0/1.15
set routing-instances EVPN-VPWS-15 route-distinguisher 2.2.2.2:15
set routing-instances EVPN-VPWS-15 vrf-target target:100:15
set routing-instances EVPN-VPWS-15 protocols evpn interface ge-0/0/1.15 vpws-service-id local 15
set routing-instances EVPN-VPWS-15 protocols evpn interface ge-0/0/1.15 vpws-service-id remote 15

Note the vpws-service-id, which is a value advertised as part of the Ethernet A-D route, in this case it is signaling the local attachment circuit ID assigned, as well as associating the remote AC for the particular endpoint. This information will be encoded in the EVPN prefix in BGP, more on this later.

R4

# ac
set interfaces ge-0/0/1 flexible-vlan-tagging
set interfaces ge-0/0/1 encapsulation flexible-ethernet-services
set interfaces ge-0/0/1 unit 15 encapsulation vlan-ccc
set interfaces ge-0/0/1 unit 15 vlan-id 15

# routing-instance EVPN-VPWS
set routing-instances EVPN-VPWS-15 description "EVPN-VPWS-15"
set routing-instances EVPN-VPWS-15 instance-type evpn-vpws
set routing-instances EVPN-VPWS-15 interface ge-0/0/1.15
set routing-instances EVPN-VPWS-15 route-distinguisher 4.4.4.4:15
set routing-instances EVPN-VPWS-15 vrf-target target:100:15
set routing-instances EVPN-VPWS-15 protocols evpn interface ge-0/0/1.15 vpws-service-id local 15
set routing-instances EVPN-VPWS-15 protocols evpn interface ge-0/0/1.15 vpws-service-id remote 15

Verification

For the Junos implementation, as soon as we configure our routing-instance type evpn-vpws, the router will assign a local label for the attachment circuit for the EVPN process and install it as an xconnect (CCC) type in the mpls.0 table for the purposes of traffic forwarding towards it, this label will also be used for crafting the the Ethernet A-D route so that the peer installs it as the bottom label for the service.

The Ethernet TAG ID for the Ethernet A-D route will be set to 15, which is NOT the VLAN-ID, but the local service ID assigned to the circuit. For simplicity purposes, I did configure the vpws-service-id **to be the same on both PEs as well as the VLAN ID, but these don’t need to match. For exchanging the EVPN prefixes, Junos will create a new table called **bgp.evpn.0, where all EVPN prefixes will be received and held if a route-target matches with a local routing-instance.

All the characteristics of the EVPN-VPWS-15 routing instance can be observed below.

root@R2# run show evpn vpws-instance EVPN-VPWS-15
Instance: EVPN-VPWS-15, Instance type: EVPN VPWS
  Route Distinguisher: 2.2.2.2:15
  Number of local interfaces: 1 (1 up)

	Interface name  ESI                        	Mode        	Role   	Status	Control-Word
	ge-0/0/1.15 	00:00:00:00:00:00:00:00:00:00  single-homed	Primary	Up     	No
    	Local SID: 15 Advertised Label: 299888
    	Remote SID: 15
        	Local Interface: ge-0/0/1.15 (Up)
        	PE addr     	ESI                       	Label  Mode       	Role 	TS                  	Status
        	4.4.4.4     	00:00:00:00:00:00:00:00:00:00 299808 single-homed   Primary  2022-06-08 17:35:26.934 Resolved
  Number of protect interfaces: 0

For Single-Homed (SH) scenarios, the Ethernet Segment Identifier (ESI) is set to all zeros, local Segment Identifier is 15 with the associated label of 299888 (local), remote SID is set to 15 (note these don’t have to match, just be correctly associated within the EVPN instance). These attributes are being used to construct the BGP advertisement, as we can observe below.

root@R2# run show route table EVPN-VPWS-15.evpn.0 protocol evpn extensive

EVPN-VPWS-15.evpn.0: 2 destinations, 2 routes (2 active, 0 holddown, 0 hidden)
1:2.2.2.2:15::0::15/192 AD/EVI (1 entry, 1 announced)
TSI:
Page 0 idx 0, (group IBGP_AS100 type Internal) Type 1 val 0xb20c6a4 (adv_entry)
   Advertised metrics:
 	Nexthop: 2.2.2.2
 	Localpref: 100
 	AS path: [100] I
 	Communities: target:100:15 evpn-l2-info:0x0 (mtu 0)
Path 1:2.2.2.2:15::0::15 Vector len 4.  Val: 0
    	*EVPN   Preference: 170
            	Next hop type: Indirect, Next hop index: 0
            	Address: 0xc631270
            	Next-hop reference count: 3
            	Protocol next hop: 2.2.2.2
            	Indirect next hop: 0x0 - INH Session ID: 0x0
            	State: <Active Int Ext>
            	Age: 10:13
            	Validation State: unverified
            	Task: EVPN-VPWS-15-evpn
            	Announcement bits (1): 1-BGP_RT_Background
            	AS path: I
            	Communities: evpn-l2-info:0x0 (mtu 0)
            	Route Label: 299888

Note the label assigned in the route of 299888. We can see what operation this label is doing by checking the **mpls.0 **table, where we can see that the label is associated with the attachment circuit of the routing instance.

root@R2# run show route table mpls.0 label 299888 protocol evpn

mpls.0: 14 destinations, 14 routes (14 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both

299888         	*[EVPN/7] 00:11:34
                	> via ge-0/0/1.15, Pop <<< ! remove label

If we check on R4, we should see the route from R2 held in the bgp.evpn.0 **table. The key pieces here are the route-target, which is allowing the routing instance to import this EVPN prefix, the **protocol next-hop of R2 and the Ethernet TAG ID, which is telling us the service ID or AC ID R2 is advertising.

Also note that in addition to the route-target, there is another extended community set in the advertisement named evpn-l2-info:0x0 (mtu 0) which is use to signal additional capabilities such as Control-Word.

root@R4# run show route table bgp.evpn.0 extensive

bgp.evpn.0: 1 destinations, 1 routes (1 active, 0 holddown, 0 hidden)
1:2.2.2.2:15::0::15/192 AD/EVI (1 entry, 0 announced)
    	*BGP	Preference: 170/-101
            	Route Distinguisher: 2.2.2.2:15
            	Next hop type: Indirect, Next hop index: 0
            	Address: 0xc632b30
            	Next-hop reference count: 2
            	Source: 2.2.2.2
            	Protocol next hop: 2.2.2.2
            	Indirect next hop: 0x2 no-forward INH Session ID: 0x0
            	State: <Active Int Ext>
            	Local AS:   100 Peer AS:   100
            	Age: 17:22  	Metric2: 11
            	Validation State: unverified
            	Task: BGP_100.2.2.2.2+179
            	AS path: I
            	Communities: target:100:15 evpn-l2-info:0x0 (mtu 0)
            	Import Accepted
            	Route Label: 299888
            	Localpref: 100
            	Router ID: 2.2.2.2
            	Secondary Tables: EVPN-VPWS-15.evpn.0
            	Indirect next hops: 1
                    	Protocol next hop: 2.2.2.2 Metric: 11
                    	Indirect next hop: 0x2 no-forward INH Session ID: 0x0
                    	Indirect path forwarding next hops: 1
                            	Next hop type: Router
                            	Next hop: 10.3.4.3 via ge-0/0/0.34
                            	Session Id: 0x140
                            	2.2.2.2/32 Originating RIB: inet.3
                              	Metric: 11	Node path count: 1
                              	Forwarding nexthops: 1
                                    	Nexthop: 10.3.4.3 via ge-0/0/0.34
                                    	Session Id: 140

We can see successful communication between R1 and R5.

R5#ping 15.0.0.1
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 15.0.0.1, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 1/96/475 ms

Let’s trace it hop-by-hop and see the flow from R1 towards R5.

Route from our peer is installed, this confirms the control plane (BGP) is OK, and we are importing the prefix into the routing instance.

root@R4# run show route table EVPN-VPWS-15.evpn.0 protocol bgp

EVPN-VPWS-15.evpn.0: 2 destinations, 2 routes (2 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both

1:2.2.2.2:15::0::15/192 AD/EVI
               	*[BGP/170] 00:24:32, localpref 100, from 2.2.2.2
                  	AS path: I, validation-state: unverified
                	> to 10.3.4.3 via ge-0/0/0.34, Push 17

If we check the xconnect information for ge-0/0/1.15, we will observe the operations that R4 as the ingress LSR is doing.

root@R4# run show route ccc ge-0/0/1.15

mpls.0: 14 destinations, 14 routes (14 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both

ge-0/0/1.15    	*[EVPN/7] 00:32:04, route-type Egress
                	> to 10.3.4.3 via ge-0/0/0.34, Push 299888, Push 17(top)

First, if traffic is received in ge-0/0/1.15, it will be classified and tagged according to the AC configuration and label 299888 would be pushed as the bottom label or service label, after that, R4 will check the protocol next-hop advertised in the BGP route and impose label for this IP, which corresponds to lo0 address of R2, label 17. This can be seen in the inet.3 table.

root@R4# run show route table inet.3 2.2.2.2/32

inet.3: 3 destinations, 3 routes (3 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both

2.2.2.2/32     	*[LDP/9] 00:35:58, metric 11
                	> to 10.3.4.3 via ge-0/0/0.34, Push 17

Checking our P/core router R3, we can see that label 17 has a Pop operation.

R3#show mpls forwarding-table labels 17
Local  	Outgoing   Prefix       	Bytes Label   Outgoing   Next Hop
Label  	Label  	or Tunnel Id 	Switched  	interface
17     	Pop Label  2.2.2.2/32   	96164     	Et0/0.23   10.2.3.2

Finally, the frame reaches R2 as our egress LSR with the bottom label of 299888, let’s see what R2 will do with this label.

root@R2# run show route table mpls.0 label 299888

mpls.0: 14 destinations, 14 routes (14 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both

299888         	*[EVPN/7] 00:38:24
                	> via ge-0/0/1.15, Pop  <<< !

Traffic will be forwarded over ge-0/0/1.15 as a native frame. Checking the FPC0, we can see further information of how the programming is being held in the line card by grabbing the forwarding index.

root@R2# run show route table mpls.0 label 299888 extensive | match index
            	Next hop type: Router, Next hop index: 585 <<<

Once we have the index, we have a session into the FPC0.

root@R2# run start shell
root@R2:~ # vty fpc0

VMX- platform (2592Mhz Intel(R) Atom(TM) CPU processor, 1792MB memory, 8192KB flash)

VMX-0(R2 vty)# show nhdb id 585 extensive
   ID  	Type  	Interface	Next Hop Addr	Protocol   	Encap 	MTU           	Flags  PFE internal Flags
-----  --------  -------------  ---------------  ----------  ------------  ----  ------------------  ------------------
  585   Unicast  ge-0/0/1.15	-             	MPLS->CCC  	CCC/VLAN  1522  0x0000000000200005  0x0000000000000000 <<< !

BFD Session Id: 0

Flags:          	0x0000000000200005
PFE internal flags: 0x0000000000000000
Interface: 335

Topo-link:
[pfe-0]: 0x11c0000000024900
	ModifyNH: Subcode=SetNH-Token(7),Desc=0x0,Data=0x24900,NextNH=0
 (pfeDest:0, VC memberId:0, token:0x249/585)


Dram Bytes 	: 396
Flags      	: 0x0000000000200005
Parent NHID	: 0
Route Table ID : 0
PreComputed MTU: 0
Tunnel Self ID 	: 0x0
Tunnel Parent ID   : 0x0
Parent NHID	: 0

NH Egress/Fabric:
    	Feature List: NH
       	[pfe-0]: 0x23031f7400004817;
      	f_mask:0x8000000000; c_mask:0x8000000000000000; f_num:27; c_num:1,  inst:0x0
    	Idx#24  	ucast:
       	[pfe-0]: 0x23031f7400004817

PFE:0

Encap-ptr chain:
----------------
Encapsulation Pointer (0xe0bb7c28) data:
	Encap-ptr-type:RSVD-EID
	Ref Count:1
	RSVD-EID: 0

Dram Bytes: 532

EVPN-VPWS is the point-to-point technology within the EVPN family, simple to configure and deploy.

HTH,

Elvin