Understanding EVPN-VPWS
Jun 03, 22Overview
We will explore the theory behind L2VPN Point-to-Point (P2P) with EVPN-VPWS, we will also dive into its implementation in Junos, but the main purpose of this article is to provide the fundamental theory behind it.
Introduction
A plethora of technologies exist to interconnect layer 2 domains over multiprotocol label switching (MPLS) networks or IP-only transit networks, among them one of the latest additions to the L2VPN category is the Ethernet VPN (EVPN) family. In particular EVPN-VPWS, defined in RFC 8214, provides the set of functions to provide P2P reachability over an MPLS network. By leveraging BGP, we can distribute the EVPN-VPWS services across the service provider network to the intended egress Label Switch Routers (LSR).
Among the advantages that EVPN-VPWS provide are:
- BGP Signaling of Pseudowires - Only BGP is used to signal the PW labels, no other protocols (such as LDP) are triggered for the PW to establish.
- Single-Active and All-Active Multihoming - EVPN family provides the capabilities to perform single active ro all active multihoming while not causing data plane issues
- Entropy Signaling with Control Word
- Fast Protection upon Node or Link Failure
Unlike Multipoint L2VPNs, P2P L2VPN services do not require MAC lookups in order to forward frames between endpoints, therefore there is no concept of bridge-domains or MAC tables in this type of service as there is only a single endpoint where the frames are required to be forwarded. The customer-facing interface known as Attachment Circuit (AC) at the ingress LSR receives an Ethernet frame which subsequently is wrapped with a service label signal through BGP EVPN SAFI and sent across the backbone towards the egress LSR using the corresponding transport label.
EVPN-VPWS Auto-Discovery
RFC 7434 defines the architecture for the EVPN family and applications, and within it provides the definition of different route types required for these services to function, each route with a role. Specifically for EVPN-VPWS, we need to dive into the EVPN Type-1 route also known as Ethernet Auto-Discovery (A-D) route in its two variants: a) Per-ESI, b) Per-EVI.
In simple terms, Per-EVI is what will be generated when we have a single-home scenario, for multi-homing, a per-ESI route would be required. The Ethernet Auto Discovery (Ethernet A-D) route is defined as follows:
Image 1 - Type-1: Ethernet A-D route
+---------------------------------------------------+
| Route Distinguisher (RD) (8 octets) |
+---------------------------------------------------+
| Ethernet Segment Identifier (10 octets)|
+---------------------------------------------------+
| Ethernet Tag ID (4 octets) |
+---------------------------------------------------+
| MPLS Label (3 octets) |
+---------------------------------------------------+
- Route Distinguisher - Route distinguisher serves the purpose of uniquely identify the prefix in the AS
- Ethernet Segment Identifier - The ESI is 10 bytes, and is set to 0 for Single Home (SH) and to non-zero values when used in multi-homed (MH) environments
- Ethernet Tag ID - For VPWS services, the Ethernet Tag ID is set to the local service identifier of the Attachment Circuit
- MPLS Label - Service label announced by BGP for the L2VPN EVPN prefix
Also, extended communities are used in the BGP advertisement in order to signal additional attributes.
Image 2 - EVPN-VPWS Extended Communities
+-------------------------------------------------------+
| Type (0x06) / Sub-type (0x04) (2 octets) |
+-------------------------------------------------------+
| Control Flags (2 octets) |
+--------------------------------------------------------+
| L2 MTU (2 octets) |
+--------------------------------------------------------+
| Reserved (2 octets) |
+--------------------------------------------------------+
Flags described below:
- C-flag to signal Control Word
- L2 MTU flag
- P-bit is set if device is Primary (only applicable for Multihoming topologies)
- B-bit is set if device is Backup (only applicable for Multihoming topologies)
- Reserved fields
Topology
Configuration
For our example, R2 and R4 are our PE routers and these are Juniper vMX devices, R1 (CE), R3 (P), and R5 (CE) are Cisco routers.
R1
!
hostname R1
!
interface Ethernet0/0.15
encapsulation dot1Q 15
ip address 15.0.0.1 255.255.255.0
!
end
R2
set system host-name R2
# core-facing
set interfaces ge-0/0/0 vlan-tagging
set interfaces ge-0/0/0 unit 23 vlan-id 23
set interfaces ge-0/0/0 unit 23 family inet address 10.2.3.2/24
set interfaces ge-0/0/0 unit 23 family mpls
# loopback0
set interfaces lo0 unit 0 family inet address 2.2.2.2/32
set interfaces lo0 unit 0 family mpls
# bgp
set routing-options autonomous-system 100
set protocols bgp group IBGP_AS100 type internal
set protocols bgp group IBGP_AS100 local-address 2.2.2.2
set protocols bgp group IBGP_AS100 family evpn signaling
set protocols bgp group IBGP_AS100 peer-as 100
set protocols bgp group IBGP_AS100 neighbor 4.4.4.4 description "IBGP AS100 Peer"
# igp
set protocols ospf area 0.0.0.0 interface lo0.0
set protocols ospf area 0.0.0.0 interface ge-0/0/0.23 interface-type p2p
# mpls
set protocols mpls interface lo0.0
set protocols mpls interface ge-0/0/0.23
set protocols ldp track-igp-metric
set protocols ldp mtu-discovery
set protocols ldp deaggregate
set protocols ldp interface ge-0/0/0.23
set protocols ldp interface lo0.0
R3
hostname R3
!
mpls ldp entropy-label
!
interface Loopback0
ip address 3.3.3.3 255.255.255.255
ip ospf 1 area 0
!
interface Ethernet0/0.23
encapsulation dot1Q 23
ip address 10.2.3.3 255.255.255.0
ip ospf network point-to-point
ip ospf 1 area 0
!
interface Ethernet0/0.34
encapsulation dot1Q 34
ip address 10.3.4.3 255.255.255.0
ip ospf network point-to-point
ip ospf 1 area 0
!
interface Ethernet0/0.36
encapsulation dot1Q 36
ip address 10.3.6.3 255.255.255.0
ip ospf 1 area 0
!
router ospf 1
router-id 3.3.3.3
mpls ldp autoconfig
!
end
R4
set system host-name R4
# core-facing
set interfaces ge-0/0/0 vlan-tagging
set interfaces ge-0/0/0 unit 34 vlan-id 34
set interfaces ge-0/0/0 unit 34 family inet address 10.3.4.4/24
set interfaces ge-0/0/0 unit 34 family mpls
# loopback0
set interfaces lo0 unit 0 family inet address 4.4.4.4/32
set interfaces lo0 unit 0 family mpls
# bgp
set routing-options autonomous-system 100
set protocols bgp group IBGP_AS100 type internal
set protocols bgp group IBGP_AS100 local-address 4.4.4.4
set protocols bgp group IBGP_AS100 family evpn signaling
set protocols bgp group IBGP_AS100 peer-as 100
set protocols bgp group IBGP_AS100 neighbor 2.2.2.2 description "IBGP AS100 Peer"
# igp
set protocols ospf area 0.0.0.0 interface lo0.0
set protocols ospf area 0.0.0.0 interface ge-0/0/0.34 interface-type p2p
# mpls
set protocols mpls interface lo0.0
set protocols mpls interface ge-0/0/0.34
set protocols ldp track-igp-metric
set protocols ldp mtu-discovery
set protocols ldp deaggregate
set protocols ldp interface ge-0/0/0.34
set protocols ldp interface lo0.0
R5
hostname R5
!
interface Ethernet0/0.15
encapsulation dot1Q 15
ip address 15.0.0.5 255.255.255.0
!
end
The infrastructure is in place, we can do a ping mpls in order to test MPLS infrastructure.
# R2
root@R2# run traceroute mpls ldp 4.4.4.4/32
Probe options: ttl 64, retries 3, wait 10, paths 16, exp 7, fanout 16
ttl Label Protocol Address Previous Hop Probe Status
1 19 LDP 10.2.3.3 (null) Success
FEC-Stack-Sent: LDP
ttl Label Protocol Address Previous Hop Probe Status
2 3 Unknown 10.3.4.4 10.2.3.3 Egress
FEC-Stack-Sent: LDP
Path 1 via ge-0/0/0.23 destination 127.0.0.64
# R4
root@R4# run traceroute mpls ldp 2.2.2.2/32
Probe options: ttl 64, retries 3, wait 10, paths 16, exp 7, fanout 16
ttl Label Protocol Address Previous Hop Probe Status
1 18 LDP 10.3.4.3 (null) Success
FEC-Stack-Sent: LDP
ttl Label Protocol Address Previous Hop Probe Status
2 3 Unknown 10.2.3.2 10.3.4.3 Egress
FEC-Stack-Sent: LDP
Path 1 via ge-0/0/0.34 destination 127.0.0.64
Having the infrastructure set, we can now proceed with the configuration of EVPN-VPWS, this includes the Attachment Circuit, which is the PE-to-CE facing interface, plus a routing-instance that will hold the attributes for the EVPN-VPWS Instance we are creating.
R2
# ac
set interfaces ge-0/0/1 flexible-vlan-tagging
set interfaces ge-0/0/1 encapsulation flexible-ethernet-services
set interfaces ge-0/0/1 unit 15 encapsulation vlan-ccc
set interfaces ge-0/0/1 unit 15 vlan-id 15
# routing-instance EVPN-VPWS
set routing-instances EVPN-VPWS-15 description "EVPN-VPWS-15"
set routing-instances EVPN-VPWS-15 instance-type evpn-vpws
set routing-instances EVPN-VPWS-15 interface ge-0/0/1.15
set routing-instances EVPN-VPWS-15 route-distinguisher 2.2.2.2:15
set routing-instances EVPN-VPWS-15 vrf-target target:100:15
set routing-instances EVPN-VPWS-15 protocols evpn interface ge-0/0/1.15 vpws-service-id local 15
set routing-instances EVPN-VPWS-15 protocols evpn interface ge-0/0/1.15 vpws-service-id remote 15
Note the vpws-service-id, which is a value advertised as part of the Ethernet A-D route, in this case it is signaling the local attachment circuit ID assigned, as well as associating the remote AC for the particular endpoint. This information will be encoded in the EVPN prefix in BGP, more on this later.
R4
# ac
set interfaces ge-0/0/1 flexible-vlan-tagging
set interfaces ge-0/0/1 encapsulation flexible-ethernet-services
set interfaces ge-0/0/1 unit 15 encapsulation vlan-ccc
set interfaces ge-0/0/1 unit 15 vlan-id 15
# routing-instance EVPN-VPWS
set routing-instances EVPN-VPWS-15 description "EVPN-VPWS-15"
set routing-instances EVPN-VPWS-15 instance-type evpn-vpws
set routing-instances EVPN-VPWS-15 interface ge-0/0/1.15
set routing-instances EVPN-VPWS-15 route-distinguisher 4.4.4.4:15
set routing-instances EVPN-VPWS-15 vrf-target target:100:15
set routing-instances EVPN-VPWS-15 protocols evpn interface ge-0/0/1.15 vpws-service-id local 15
set routing-instances EVPN-VPWS-15 protocols evpn interface ge-0/0/1.15 vpws-service-id remote 15
Verification
For the Junos implementation, as soon as we configure our routing-instance type evpn-vpws, the router will assign a local label for the attachment circuit for the EVPN process and install it as an xconnect (CCC) type in the mpls.0 table for the purposes of traffic forwarding towards it, this label will also be used for crafting the the Ethernet A-D route so that the peer installs it as the bottom label for the service.
The Ethernet TAG ID for the Ethernet A-D route will be set to 15, which is NOT the VLAN-ID, but the local service ID assigned to the circuit. For simplicity purposes, I did configure the vpws-service-id **to be the same on both PEs as well as the VLAN ID, but these don’t need to match. For exchanging the EVPN prefixes, Junos will create a new table called **bgp.evpn.0, where all EVPN prefixes will be received and held if a route-target matches with a local routing-instance.
All the characteristics of the EVPN-VPWS-15 routing instance can be observed below.
root@R2# run show evpn vpws-instance EVPN-VPWS-15
Instance: EVPN-VPWS-15, Instance type: EVPN VPWS
Route Distinguisher: 2.2.2.2:15
Number of local interfaces: 1 (1 up)
Interface name ESI Mode Role Status Control-Word
ge-0/0/1.15 00:00:00:00:00:00:00:00:00:00 single-homed Primary Up No
Local SID: 15 Advertised Label: 299888
Remote SID: 15
Local Interface: ge-0/0/1.15 (Up)
PE addr ESI Label Mode Role TS Status
4.4.4.4 00:00:00:00:00:00:00:00:00:00 299808 single-homed Primary 2022-06-08 17:35:26.934 Resolved
Number of protect interfaces: 0
For Single-Homed (SH) scenarios, the Ethernet Segment Identifier (ESI) is set to all zeros, local Segment Identifier is 15 with the associated label of 299888 (local), remote SID is set to 15 (note these don’t have to match, just be correctly associated within the EVPN instance). These attributes are being used to construct the BGP advertisement, as we can observe below.
root@R2# run show route table EVPN-VPWS-15.evpn.0 protocol evpn extensive
EVPN-VPWS-15.evpn.0: 2 destinations, 2 routes (2 active, 0 holddown, 0 hidden)
1:2.2.2.2:15::0::15/192 AD/EVI (1 entry, 1 announced)
TSI:
Page 0 idx 0, (group IBGP_AS100 type Internal) Type 1 val 0xb20c6a4 (adv_entry)
Advertised metrics:
Nexthop: 2.2.2.2
Localpref: 100
AS path: [100] I
Communities: target:100:15 evpn-l2-info:0x0 (mtu 0)
Path 1:2.2.2.2:15::0::15 Vector len 4. Val: 0
*EVPN Preference: 170
Next hop type: Indirect, Next hop index: 0
Address: 0xc631270
Next-hop reference count: 3
Protocol next hop: 2.2.2.2
Indirect next hop: 0x0 - INH Session ID: 0x0
State: <Active Int Ext>
Age: 10:13
Validation State: unverified
Task: EVPN-VPWS-15-evpn
Announcement bits (1): 1-BGP_RT_Background
AS path: I
Communities: evpn-l2-info:0x0 (mtu 0)
Route Label: 299888
Note the label assigned in the route of 299888. We can see what operation this label is doing by checking the **mpls.0 **table, where we can see that the label is associated with the attachment circuit of the routing instance.
root@R2# run show route table mpls.0 label 299888 protocol evpn
mpls.0: 14 destinations, 14 routes (14 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both
299888 *[EVPN/7] 00:11:34
> via ge-0/0/1.15, Pop <<< ! remove label
If we check on R4, we should see the route from R2 held in the bgp.evpn.0 **table. The key pieces here are the route-target, which is allowing the routing instance to import this EVPN prefix, the **protocol next-hop of R2 and the Ethernet TAG ID, which is telling us the service ID or AC ID R2 is advertising.
Also note that in addition to the route-target, there is another extended community set in the advertisement named evpn-l2-info:0x0 (mtu 0) which is use to signal additional capabilities such as Control-Word.
root@R4# run show route table bgp.evpn.0 extensive
bgp.evpn.0: 1 destinations, 1 routes (1 active, 0 holddown, 0 hidden)
1:2.2.2.2:15::0::15/192 AD/EVI (1 entry, 0 announced)
*BGP Preference: 170/-101
Route Distinguisher: 2.2.2.2:15
Next hop type: Indirect, Next hop index: 0
Address: 0xc632b30
Next-hop reference count: 2
Source: 2.2.2.2
Protocol next hop: 2.2.2.2
Indirect next hop: 0x2 no-forward INH Session ID: 0x0
State: <Active Int Ext>
Local AS: 100 Peer AS: 100
Age: 17:22 Metric2: 11
Validation State: unverified
Task: BGP_100.2.2.2.2+179
AS path: I
Communities: target:100:15 evpn-l2-info:0x0 (mtu 0)
Import Accepted
Route Label: 299888
Localpref: 100
Router ID: 2.2.2.2
Secondary Tables: EVPN-VPWS-15.evpn.0
Indirect next hops: 1
Protocol next hop: 2.2.2.2 Metric: 11
Indirect next hop: 0x2 no-forward INH Session ID: 0x0
Indirect path forwarding next hops: 1
Next hop type: Router
Next hop: 10.3.4.3 via ge-0/0/0.34
Session Id: 0x140
2.2.2.2/32 Originating RIB: inet.3
Metric: 11 Node path count: 1
Forwarding nexthops: 1
Nexthop: 10.3.4.3 via ge-0/0/0.34
Session Id: 140
We can see successful communication between R1 and R5.
R5#ping 15.0.0.1
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 15.0.0.1, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 1/96/475 ms
Let’s trace it hop-by-hop and see the flow from R1 towards R5.
Route from our peer is installed, this confirms the control plane (BGP) is OK, and we are importing the prefix into the routing instance.
root@R4# run show route table EVPN-VPWS-15.evpn.0 protocol bgp
EVPN-VPWS-15.evpn.0: 2 destinations, 2 routes (2 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both
1:2.2.2.2:15::0::15/192 AD/EVI
*[BGP/170] 00:24:32, localpref 100, from 2.2.2.2
AS path: I, validation-state: unverified
> to 10.3.4.3 via ge-0/0/0.34, Push 17
If we check the xconnect information for ge-0/0/1.15, we will observe the operations that R4 as the ingress LSR is doing.
root@R4# run show route ccc ge-0/0/1.15
mpls.0: 14 destinations, 14 routes (14 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both
ge-0/0/1.15 *[EVPN/7] 00:32:04, route-type Egress
> to 10.3.4.3 via ge-0/0/0.34, Push 299888, Push 17(top)
First, if traffic is received in ge-0/0/1.15, it will be classified and tagged according to the AC configuration and label 299888 would be pushed as the bottom label or service label, after that, R4 will check the protocol next-hop advertised in the BGP route and impose label for this IP, which corresponds to lo0 address of R2, label 17. This can be seen in the inet.3 table.
root@R4# run show route table inet.3 2.2.2.2/32
inet.3: 3 destinations, 3 routes (3 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both
2.2.2.2/32 *[LDP/9] 00:35:58, metric 11
> to 10.3.4.3 via ge-0/0/0.34, Push 17
Checking our P/core router R3, we can see that label 17 has a Pop operation.
R3#show mpls forwarding-table labels 17
Local Outgoing Prefix Bytes Label Outgoing Next Hop
Label Label or Tunnel Id Switched interface
17 Pop Label 2.2.2.2/32 96164 Et0/0.23 10.2.3.2
Finally, the frame reaches R2 as our egress LSR with the bottom label of 299888, let’s see what R2 will do with this label.
root@R2# run show route table mpls.0 label 299888
mpls.0: 14 destinations, 14 routes (14 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both
299888 *[EVPN/7] 00:38:24
> via ge-0/0/1.15, Pop <<< !
Traffic will be forwarded over ge-0/0/1.15 as a native frame. Checking the FPC0, we can see further information of how the programming is being held in the line card by grabbing the forwarding index.
root@R2# run show route table mpls.0 label 299888 extensive | match index
Next hop type: Router, Next hop index: 585 <<<
Once we have the index, we have a session into the FPC0.
root@R2# run start shell
root@R2:~ # vty fpc0
VMX- platform (2592Mhz Intel(R) Atom(TM) CPU processor, 1792MB memory, 8192KB flash)
VMX-0(R2 vty)# show nhdb id 585 extensive
ID Type Interface Next Hop Addr Protocol Encap MTU Flags PFE internal Flags
----- -------- ------------- --------------- ---------- ------------ ---- ------------------ ------------------
585 Unicast ge-0/0/1.15 - MPLS->CCC CCC/VLAN 1522 0x0000000000200005 0x0000000000000000 <<< !
BFD Session Id: 0
Flags: 0x0000000000200005
PFE internal flags: 0x0000000000000000
Interface: 335
Topo-link:
[pfe-0]: 0x11c0000000024900
ModifyNH: Subcode=SetNH-Token(7),Desc=0x0,Data=0x24900,NextNH=0
(pfeDest:0, VC memberId:0, token:0x249/585)
Dram Bytes : 396
Flags : 0x0000000000200005
Parent NHID : 0
Route Table ID : 0
PreComputed MTU: 0
Tunnel Self ID : 0x0
Tunnel Parent ID : 0x0
Parent NHID : 0
NH Egress/Fabric:
Feature List: NH
[pfe-0]: 0x23031f7400004817;
f_mask:0x8000000000; c_mask:0x8000000000000000; f_num:27; c_num:1, inst:0x0
Idx#24 ucast:
[pfe-0]: 0x23031f7400004817
PFE:0
Encap-ptr chain:
----------------
Encapsulation Pointer (0xe0bb7c28) data:
Encap-ptr-type:RSVD-EID
Ref Count:1
RSVD-EID: 0
Dram Bytes: 532
EVPN-VPWS is the point-to-point technology within the EVPN family, simple to configure and deploy.
HTH,
Elvin