Take a look at example of connecting an office to a DMVPN cloud, where two Internet access channels from different Internet service providers are used: one of them provides access via a wired connection and the other via a stick modem.

Figure 9. Diagram showing how to connect a branch office router to Internet (with backup connection via modem) via the networks of two Internet service providers
Within this DMVPN scheme, two GRE tunnels are set up on the Spoke node: one to the first DMVPN Hub, and another one to the second. The Spoke node has IP connectivity via two independent Internet providers; the transport of the first Internet provider is used to connect to the first DMVPN Hub , using the uplink as a “wired” connection to the network, while the second ISP's transport is used to connect to the second DMVPN Hub using a stick modem. In this guide, a Huawei E8372 stick modem is used. |
The branch office router connects to two ISPs, each of which provides a channel with a white statically assigned IP address. For ease of configuration, assign names to both devices in this diagram.
hostname RT-OFFICE-5 |
hostname SW-OFFICE-5 |
As part of this guide, examine a simple access switch configuration for connecting the equipment of a branch office to a border router. The parameters of the office's local network are described in Table 24.
Table 24. Local network parameters for branch office No. 5.
| Purpose | VLAN | Subnet |
|---|---|---|
| Local area network of the branch office No. 5 | 100 | 192.168.15.0/24 |
Move on to configuring the switch. The client ports must be configured in general mode. The required VLAN ID is assigned to untagged traffic.
vlan database vlan 100 exit ! interface range GigabitEthernet1/0/1-24 switchport mode general switchport general allowed vlan add 100 untagged switchport general pvid 100 switchport protected-port exit |
To isolate clients in the broadcast domain, enable the port-protection feature. |
Configure the physical interface facing the router as a trunk port for the required VLAN ID
interface TenGigabitEthernet1/0/1 switchport mode general switchport general allowed vlan add 100 tagged switchport forbidden default-vlan exit |
Move on to configuring the border router. Add a subinterface for processing tagged traffic with the appropriate VLAN ID:
interface gigabitethernet 1/0/3.100 description "LAN | CUSTOMER" ip address 192.168.15.1/24 exit |
Add a security zone for the subinterface and ensure that incoming ICMP traffic from this security zone is accepted:
security zone CUSTOMER
description "LAN | CUSTOMER"
exit
interface gigabitethernet 1/0/3.100
security-zone CUSTOMER
exit
security zone-pair CUSTOMER self
rule 10
description "Permit | ICMP | From ANY | To ANY"
action permit
match protocol icmp
enable
exit
exit
|
Take a look at the process of connecting to the Internet using the data provided by ISP No. 1 for configuring static addressing on the interface. The parameters for connecting to the Internet provider are described in Table 25.
Table 25. Data from ISP No. 1 for configuring static IP addressing on the border router of branch office No. 5
| Parameter | Value |
|---|---|
| Router IP | 203.11.1.2 |
| Network mask | 255.255.255.0 |
| Gateway IP | 203.11.1.1 |
In this example, both ISP No. 1 and ISP No. 2 provide public (white) IP addresses for static addressing configuration. In cases where an Internet service provider provides a private (gray) IP address for configuration with subsequent NAT conversion to public IP addressing, the configuration will be similar. |
Move on to configuring the network interface and static routing. Separate the interface using VRF – the Front-Door VRF scheme will be configured later.
A connection scheme in which the transport network for a certain virtual network is moved to a separate network namespace is called Front-Door VRF. This scheme of organizing transport for a virtual network offers the following advantages:
|
ip vrf ISP-CORE exit interface gigabitethernet 1/0/1 ip vrf forwarding ISP-CORE description "ISP-CORE | Uplink" ip address 203.11.1.2/24 exit ip route vrf ISP-CORE 0.0.0.0/0 203.11.1.1 name ISP-CORE |
Configure a separate security zone for this network interface and allow ICMP traffic incoming to the router from this security zone:
security zone CORE
description "ISP-CORE | Uplink"
ip vrf forwarding ISP-CORE
exit
interface gigabitethernet 1/0/1
security-zone CORE
exit
security zone-pair CORE self
rule 10
description "Permit | ICMP | From ANY | To ANY"
action permit
match protocol icmp
enable
exit
exit |
Take a look at the process of connecting to the Internet using the data provided by ISP No. 2 for configuring static addressing on the interface. The parameters for connecting to the Internet provider are described in Table 26.
Table 26. Data from ISP No. 2 for configuring static IP addressing on the border router of branch office No. 5
| Parameter | Value |
|---|---|
| Router IP | 203.11.0.2 |
| Network mask | 255.255.255.0 |
| Gateway IP | 203.11.0.1 |
Move on to configuring the stick modem and static routing. Separate the interface using VRF – the Front-Door VRF scheme will be configured later.
The first step in configuring a modem connection is to connect the modem to the border router and wait for the modem manufacturer and slot number where the modem is connected to be detected. This information can be viewed using the show cellular status modem command. If there is no configuration for modems, the modem status will be disabled/Down.
esr# show cellular status modem Number USB port Manufacturer Model Current state Inteface Link device state ------ -------- ------------ ---------- ------------- --------------- ----- 1 3-1 huawei E8372 disabled -- Down |
The second step is to configure the stick modem interface itself. Proceed as follows:
ip vrf ISP-MODEM exit cellular profile 1 APN internet exit cellular modem 1 ip vrf forwarding ISP-MODEM description "ISP-MODEM | Uplink" device 3-1 profile 1 allowed-mode 3g allowed-mode 4g enable exit ip route vrf ISP-MODEM 0.0.0.0/0 interface modem 1 name ISP-MODEM |
When configuring the modem, a profile must be created specifying the APN of the operator to which the connection is made, and the USB port ID of the modem obtained in the first step using the show cellular status modem command must be specified. Additionally, the default route through the modem interface must be set to enable Internet access. |
Configure a separate security zone for the network interface connected to ISP No. 2 and allow ICMP traffic incoming to the border router from this security zone:
security zone MODEM
description "ISP-MODEM | Uplink"
ip vrf forwarding ISP-MODEM
exit
cellular modem 1
security-zone MODEM
exit
security zone-pair MODEM self
rule 10
description "Permit | ICMP | From ANY | To ANY"
action permit
match protocol icmp
enable
exit
exit |
Configuring IPsec for the future DMVPN cloud is an important part of this guide. Correct IPsec configuration ensures the privacy and security of traffic between offices. IKE and IPsec parameters shown in Table 27 will be used in further configuration.
Table 27. IKE and IPsec parameters used to configure IPsec tunneling on the DMVPN Spoke router of branch office No. 5.
| RT-OFFICE-5 | ||
|---|---|---|
| IKE parameters | Encryption algorithm | AES-256 |
| Hashing algorithm | SHA2-256 | |
| Diffie-Hellman group | 19 | |
| IKE session lifetime in seconds | 86400 | |
| IKE session identifier | spoke5.company.loc | |
| Interval for sending DPD messages | 40 | |
| Total timeout for DPD message response | 160 | |
| Action when DPD timeout occurs | Terminating the IKE session | |
| IPsec parameters | Encryption algorithm | AES-256 |
| Hashing algorithm | SHA2-256 | |
| Diffie-Hellman group for a PFS mechanism | 19 | |
| IPsec session lifetime in seconds | 28800 | |
| IPsec session lifetime in kilobytes | 4608000 | |
| Early reauthentication interval for IKE session/Early rekeying interval for IPsec session in seconds | 3600 | |
| Threshold value for early reauthentication interval of IKE session/Threshold value for early rekeying of IPsec session in kilobytes | 86400 | |
IPsec configuration begins with configuring encryption algorithm sets for the IKE protocol:
security ike proposal DMVPN_IKE_PROP_1 description "DMVPN | IKE proposal #1" authentication algorithm sha2-256 encryption algorithm aes256 dh-group 19 exit |
Create an IKE authentication keyring. Since domain names are going to be used as IPsec neighbor identifiers in further configuration, domain names will also be used in the keyring:
security ike keyring DMVPN_IKE_KEYRING description "DMVPN | IKE keyring" identity dns *.company.loc pre-shared-key ascii-text password exit |
Create an IKE policy. It includes encryption algorithm sets, authentication method selection and lifetime of IKE session (Phase 1):
security ike policy DMVPN_IKE_POL description "DMVPN | IKE policy" lifetime seconds 86400 keyring DMVPN_IKE_KEYRING authentication method keyring proposal DMVPN_IKE_PROP_1 exit |
Create a set of IKE gateways.
Due to the specifics of IPsec configuration in the DMVPN scheme on ESR service routers, multiple IPsec VPN policies must be created on routers acting as DMVPN Spokes: one for connecting to other DMVPN Spokes and one for each configured DMVPN Hub. |
The number of possible settings in IKE gateway is quite large, so focus on the most important configuration items:
|
security ike gateway DMVPN_IKE_GW_HUB_1 description "DMVPN | RT-HUB-1 IKE gateway" version v2-only ike-policy DMVPN_IKE_POL local interface gigabitethernet 1/0/1 local network dynamic protocol gre local id dns "spoke5.company.loc" remote id dns "hub1.company.loc" remote address 203.0.113.4 remote network 203.0.113.4/32 protocol gre mode policy-based mobike disable dead-peer-detection action clear dead-peer-detection interval 40 dead-peer-detection timeout 160 exit security ike gateway DMVPN_IKE_GW_HUB_2 description "DMVPN | RT-HUB-2 IKE gateway" version v2-only ike-policy DMVPN_IKE_POL local interface modem 1 local network dynamic protocol gre local id dns "spoke5.company.loc" remote id dns "hub2.company.loc" remote address 203.0.113.132 remote network 203.0.113.132/32 protocol gre mode policy-based mobike disable dead-peer-detection action clear dead-peer-detection interval 40 dead-peer-detection timeout 160 exit security ike gateway DMVPN_IKE_GW_SPOKES_1 description "DMVPN | Spokes IKE gateway via ISP-CORE" version v2-only ike-policy DMVPN_IKE_POL local interface gigabitethernet 1/0/1 local network dynamic protocol gre local id dns "spoke5.company.loc" remote address any remote network any protocol gre mode policy-based mobike disable dead-peer-detection action clear dead-peer-detection interval 40 dead-peer-detection timeout 160 exit security ike gateway DMVPN_IKE_GW_SPOKES_2 description "DMVPN | Spokes IKE gateway via ISP-MODEM" version v2-only ike-policy DMVPN_IKE_POL local interface modem 1 local network dynamic protocol gre local id dns "spoke5.company.loc" remote address any remote network any protocol gre mode policy-based mobike disable dead-peer-detection action clear dead-peer-detection interval 40 dead-peer-detection timeout 160 exit |
1) Specify the interface facing the ISP as the "local interface." 2) Using "local id" and "remote id" is a convenient way to identify IKE neighbors. Since domain names are used in "security ike keyring," the same type of ID should be used here. 3) Four gateways will be created — for connecting to RT-HUB-1, RT-HUB-2 and for both DMVPN Spokes to be separetely connected to each ISP. |
Configure the policy for duplicate IKE sessions to replace existing IKE sessions when duplicates occur:
security ike session uniqueids replace |
Create a set of encryption algorithms specifically for the IPsec tunnel:
security ipsec proposal DMVPN_IPSEC_PROP_1 description "DMVPN | IPsec proposal #1" authentication algorithm sha2-256 encryption algorithm aes256 pfs dh-group 19 exit |
Next create an IPsec policy. It includes sets of encryption algorithms and the lifetime of the IPsec session directly responsible for encrypting user traffic:
security ipsec policy DMVPN_IPSEC_POL description "DMVPN | IPsec policy" lifetime seconds 28800 lifetime kilobytes 4608000 proposal DMVPN_IPSEC_PROP_1 exit |
Configured IKE and IPsec settings can be combined into general VPN profiles. As with IKE gateways, four IPsec VPN profiles will be created. For IPsec VPN profiles used on GRE tunnels in the DMVPN scheme, it is necessary to enable transport mode:
security ipsec vpn DMVPN_IPSEC_VPN_HUB_1 description "DMVPN | RT-HUB-1 IPsec VPN profile" type transport ip vrf forwarding ISP-CORE ike establish-tunnel route ike gateway DMVPN_IKE_GW_HUB_1 ike ipsec-policy DMVPN_IPSEC_POL enable exit security ipsec vpn DMVPN_IPSEC_VPN_HUB_2 description "DMVPN | RT-HUB-2 IPsec VPN profile" type transport ip vrf forwarding ISP-MODEM ike establish-tunnel route ike gateway DMVPN_IKE_GW_HUB_2 ike ipsec-policy DMVPN_IPSEC_POL enable exit security ipsec vpn DMVPN_IPSEC_VPN_SPOKES_1 description "DMVPN | Spokes IPsec VPN profile via ISP-CORE" type transport ip vrf forwarding ISP-CORE ike establish-tunnel route ike gateway DMVPN_IKE_GW_SPOKES_1 ike ipsec-policy DMVPN_IPSEC_POL enable exit security ipsec vpn DMVPN_IPSEC_VPN_SPOKES_2 description "DMVPN | Spokes IPsec VPN profile via ISP-MODEM" type transport ip vrf forwarding ISP-MODEM ike establish-tunnel route ike gateway DMVPN_IKE_GW_SPOKES_2 ike ipsec-policy DMVPN_IPSEC_POL enable exit |
Allow traffic associated with IPsec tunnels to pass through. To do this, first describe the port profiles for the IKE protocol and the encrypted traffic of the IKE and ESP protocols packaged in UDP:
object-group service IKE_AND_IPSEC description "IKE, IKE encrypted, ESP encap UDP" port-range 500 port-range 4500 exit |
1) The UDP/500 port is used by the IKEv2 protocol during the initial packet exchange with a remote IKE neighbor. 2) The UDP/4500 port is used to send IKEv2 protocol messages when determining the presence of NAT between the peers. 3) The UDP/4500 port is also used to send ESP protocol messages encapsulated in UDP, which are encrypted user traffic in the tunnel. |
Allow incoming IPsec tunnel traffic for each ISP (Firewall):
# For ISP-CORE
security zone-pair CORE self
rule 20
description "Permit | IKE/IPsec | From ISP | To ANY"
action permit
match protocol udp
match destination-port object-group IKE_AND_IPSEC
enable
exit
rule 30
description "Permit | ESP | From ISP | To ANY"
action permit
match protocol esp
enable
exit
exit |
# For ISP-MODEM
security zone-pair MODEM self
rule 20
description "Permit | IKE/IPsec | From ISP | To ANY"
action permit
match protocol udp
match destination-port object-group IKE_AND_IPSEC
enable
exit
rule 30
description "Permit | ESP | From ISP | To ANY"
action permit
match protocol esp
enable
exit
exit |
User traffic is encapsulated in the ESP protocol and if there is NAT between IPsec neighbors, ESP protocol messages are in turn encapsulated in the UDP protocol, port 4500. |
Configure GRE tunnels in multipoint mode with NHRP protocol support on DMVPN Spoke. Since two DMVPN Hubs are deployed in the central office with addresses from two different ISPs, the connection to them will be made using two separate mGRE tunnels. The difference from the DMVPN Spoke configuration with a connection to a single Internet service provider is that the NBMA address for each GRE tunnel will differ depending on the IP address of the interface to that Internet service provider. The main parameters of the GRE tunnels for connecting to both DMVPN Hubs are shown in Table 28.
Table 28. Parameters of GRE tunnels on the DMVPN Spoke router of branch office No. 5
| DMVPN Hub | DMVPN Cloud | GRE tunnel number | DMVPN Spoke tunnel addressing | DMVPN Hub tunnel address | DMVPN Hub NBMA address | GRE tunnel key | Lifetime of NHRP entries in seconds |
|---|---|---|---|---|---|---|---|
| RT-HUB-1 | ISP-CORE Cloud | 11 | 172.16.1.15/24 | 172.16.1.1 | 203.0.113.4 | 1000 | 600 |
| RT-HUB-2 | ISP-MODEM Cloud | 12 | 172.16.2.15/24 | 172.16.2.1 | 203.0.113.132 | 2000 | 600 |
First, change the general settings for GRE tunnels on each DMVPN Hub. These settings include:
tunnel gre 11 description "DMVPN | Cloud 1" key 1000 ttl 64 mtu 1400 multipoint tunnel-source vrf ISP-CORE local interface gigabitethernet 1/0/1 ip address 172.16.1.15/24 ip tcp adjust-mss 1360 enable exit tunnel gre 12 description "DMVPN | Cloud 2" key 2000 ttl 64 mtu 1400 multipoint tunnel-source vrf ISP-MODEM local interface modem 1 ip address 172.16.2.15/24 ip tcp adjust-mss 1360 enable exit |
From the perspective of the NHRP protocol, DMVPN Spoke routers act as NHRP clients, which, after registering in the DMVPN cloud on one or more DMVPN Hubs, can request information about other members of the DMVPN cloud from them. In this regard, for DMVPN Spoke to work correctly, it is necessary to describe the NHRP protocol settings for connecting to both DMVPN Hubs.
To correctly establish Spoke-to-Spoke tunnels, where routing directs all traffic to the DMVPN Hub, enable the "ip nhrp shortcut" option, which activates the DMVPN Spoke's response to special NHRP "Traffic Indication" messages that the DMVPN Hub will send in response to packets destined for another DMVPN Spoke. Processing such a message will result in the construction of a Spoke-to-Spoke tunnel and traffic between DMVPN Spokes will flow directly, bypassing the DMVPN Hub. This scheme for organizing routing and establishing Spoke-to-Spoke tunnels in DMVPN clouds is commonly referred to as DMVPN Phase 3. |
It should be noted that the recreation of Spoke-to-Spoke tunnels is initiated only after the specified holding-time parameter has expired. Thus, in the event of an Internet provider failure and, accordingly, a loss of connection to the DMVPN Hub, the Spoke-to-Spoke tunnel will be reconfigured to the backup Internet provider upon expiration of the Expire time for the corresponding Spoke-to-Spoke tunnel. |
tunnel gre 11 ip nhrp authentication password ip nhrp holding-time 600 ip nhrp shortcut ip nhrp ipsec DMVPN_IPSEC_VPN_HUB_1 static ip nhrp ipsec DMVPN_IPSEC_VPN_SPOKES_1 dynamic ip nhrp map 172.16.1.1 203.0.113.4 ip nhrp nhs 172.16.1.1 ip nhrp multicast nhs ip nhrp enable exit tunnel gre 12 ip nhrp authentication password ip nhrp holding-time 600 ip nhrp shortcut ip nhrp ipsec DMVPN_IPSEC_VPN_HUB_2 static ip nhrp ipsec DMVPN_IPSEC_VPN_SPOKES_2 dynamic ip nhrp map 172.16.2.1 203.0.113.132 ip nhrp nhs 172.16.2.1 ip nhrp multicast nhs ip nhrp enable exit |
Due to the nature of traffic decapsulation from IPsec tunnels operating in transport encapsulation mode, decrypted traffic ends up on the same network interface that terminates the IPsec tunnel. Therefore, the firewall rules on the DMVPN Spoke must allow not only encrypted IPsec packets, but also GRE packets that arrive at the interface after decryption. |
security zone-pair CORE self
rule 40
description "Permit | ESP | From GRE | To ANY"
action permit
match protocol gre
enable
exit
exit
|
security zone-pair MODEM self
rule 40
description "Permit | ESP | From GRE | To ANY"
action permit
match protocol gre
enable
exit
exit
|
To filter traffic within the DMVPN cloud, a separate security zone must be created and assigned to GRE tunnels. Allow incoming ICMP traffic to pass through this zone:
security zone DMVPN_NET
description "DMVPN | Cloud"
exit
tunnel gre 11
security-zone DMVPN_NET
exit
tunnel gre 12
security-zone DMVPN_NET
exit
security zone-pair DMVPN_NET self
rule 10
description "Permit | ICMP | From ANY | To ANY"
action permit
match protocol icmp
enable
exit
exit
|
Configure BGP neighbors facing the DMVPN Hub:
router bgp log-neighbor-changes
router bgp 65000
neighbor 172.16.1.1
description "DMVPN | RT-HUB-1"
remote-as 65001
update-source gre 11
address-family ipv4 unicast
enable
exit
enable
exit
neighbor 172.16.2.1
description "DMVPN | RT-HUB-2"
remote-as 65002
update-source gre 12
address-family ipv4 unicast
enable
exit
enable
exit
enable
exit
|
Since DMVPN Spoke and DMVPN Hub are located in different autonomous systems, route information will not be advertised by default. Create a route map that allows sending routes to local network on DMVPN Hub. When default routes are distributed, they are announced with different metrics, which allows for the selection of a priority direction. If the second Internet provider, and therefore the second hub, is unavailable, routing is switched using the BFD protocol mechanism.
Specify the created route map for the IPv4 route family in the created BGP neighbors. Also, add the local network to the advertised routes:
route-map DMVPN_NET_OUT
rule 10
description "DMZ | LAN for DMVPN Hub"
match ip address 192.168.15.0/24
exit
exit
router bgp 65000
neighbor 172.16.1.1
description "DMVPN | RT-HUB-1"
remote-as 65001
update-source gre 11
address-family ipv4 unicast
route-map DMVPN_NET_OUT out
enable
exit
enable
exit
neighbor 172.16.2.1
description "DMVPN | RT-HUB-2"
remote-as 65002
update-source gre 12
address-family ipv4 unicast
route-map DMVPN_NET_OUT out
enable
exit
enable
exit
address-family ipv4 unicast
network 192.168.15.0/24
exit
enable
exit |
Enable BFD support for created BGP neighbors. Increase BFD timers, taking into account the convergence speed of IPsec and mGRE tunnels:
ip bfd log-adjacency-changes ip bfd min-rx-interval 1000 ip bfd min-tx-interval 1000 ip bfd multiplier 8 router bgp 65000 neighbor 172.16.1.1 fall-over bfd exit neighbor 172.16.2.1 fall-over bfd exit exit |
Allow incoming BGP and BFD traffic to pass through the security zone configured on GRE tunnels:
object-group service BGP
description "BGP"
port-range 179
exit
object-group service BFD
description "BFD"
port-range 3784
exit
security zone-pair DMVPN_NET self
rule 20
description "Permit | BGP | From ANY | To ANY"
action permit
match protocol tcp
match destination-port object-group BGP
enable
exit
rule 30
description "Permit | BFD | From ANY | To ANY"
action permit
match protocol udp
match destination-port object-group BFD
enable
exit
exit
|
Since the DMVPN cloud that has now been created allows traffic of local users at the branch office to exit via the central office's Internet gateway, additional firewall settings are required.
Allow transit traffic from the local network to pass to the DMVPN cloud:
security zone-pair DMVPN_NET CUSTOMER
rule 10
description "Permit | ANY | From DMVPN Cloud | To CUSTOMER"
action permit
enable
exit
exit
security zone-pair CUSTOMER DMVPN_NET
rule 10
description "Permit | ANY | From CUSTOMER | To DMVPN Cloud"
action permit
enable
exit
exit |
At this point, configuration of DMVPN with a backup channel in the branch office can be considered complete.