Scenario:
HQ(Site A) has a dedicated 10Gbps leased link to DR(Site B). As a network engineer, you have been asked to take advantage of the speed and capacity of the inter-site link to extend capabilities of storage systems, backups and replications to the DR site which is Site B. You have been asked to stay away from traditional vlans and ensure the interlink is purely Layer-3 with no possibility of VLANS spanning across to the other site.

OTV is an IP-based functionality that has been designed from the ground up to provide Layer 2 extension capabilities over any transport infrastructure: Layer 2 based, Layer 3 based, IP switched, label switched, and so on. The only requirement from the transport infrastructure is providing IP connectivity between remote data center sites. In addition, OTV provides an overlay that enables Layer 2 connectivity between separate Layer 2 domains while keeping these domains independent and preserving the fault-isolation, resiliency, and load-balancing benefits of an IP-based interconnection.

With OTV you can deploy virtual computing resources and clusters across geographically distributed data centers, delivering:

1. Transparent workload mobility
2. Business resiliency
3. Superior computing resource efficiencies

VXLAN (Virtual extensible LAN ) an open IETF specification designed to standardise an overlay encapsulation protocol, capable of relaying layer-2 traffic over IP networks.

Using VXLAN to Extend L2 Networks Across Layer 3 Links

Aruba has some recommended steps for configuring VXLAN Tunnels
The base requirement for the above topology and use case is a VXLAN: (VXLAN uses the UDP Port 4789 by default). The RFC7438 might be a good read in your spare time. Let’s get right into it.

VXLAN is an overlay technology that extends Layer 2 VLANS across a Layer 3 IP Network. It creates a Layer 2 Tunnel also called a VXLAN Segment between 2 Virtual Tunnel Endpoints also known as VTEP.

VXLAN (Virtual Extensible LAN) – The technology that provides the same Ethernet Layer 2 network services as VLAN does today, but with greater extensibility and flexibility.

VNID (Vxlan Network Identifier) – 24 bit segment ID that defines the broadcast domain. Interchangeable with “VXLAN Segment ID”.

VTEP (Virtual Tunnel Endpoint) – This is the device that does the encapsulation and de-encapsulation.

NVE (Network Virtual Interface) – Logical interface where the encapsulation and de-encapsulation occur.

What is VXLAN?
VXLAN is a technology which allows overlaying a Layer 2 (L2) network over a Layer 3 (L3) underlay with use of any IP routing protocol. It uses MAC-in-UDP Encapsulation.

VXLAN solves three main problems:

1. 16M VNIs (broadcast domains) versus the 4094 offered by traditional VLANs.
2. Allows L2 to be extended anywhere in an IP network.
3. Optimized flooding.

Why VXLAN?

1. VLAN Scalability – VXLAN extends the L2 Segment ID field to 24-bits, which potentially allows up to 16 million unique L2 segments over the same network.
2. L2 Segment Elasticity over L3 Boundary – VXLAN encapsulates an L2 frame in an IP-UDP header, which allows L2 adjacency across router boundaries.
3. Leverages multicast in the transport network in order to simulate flooding behavior for broadcast, unknown unicast, and multicast in the L2 segment.
4. Leverage Equal Cost Multi-pathing (ECMP) in order to achieve optimal path usage over the transport network.

Figure 1.2 | Overlay Network(VXLAN) and Underlay Network (VTEP)

Below is the existing configuration for Point-to-Point Interlink between the Core Switches

Core Switch 1: Interface A1

Core-1# 

interface A1
   name "10GB Interlink Leased Link"
   untagged vlan 100
   exit
Core-1#

Core Switch 2: Interface A1

Core-2# 
interface A1
   name "10GB Interlink Leased Link"
   untagged vlan 100
   exit

Core Switch 1: Interface Configuration

Core-1#
vlan 100
   name "Intersite Leased 10 Gbps Pipe"
   untagged A1
   ip address 10.10.10.1 255.255.255.252
   ip ospf 10.10.10.1 area backbone
   ip ospf 10.10.10.1 network-type point-to-point
   exit
Core-1#

Core Switch 2: Interface Configuration

Core-2# 
vlan 100
   name "Intersite Leased 10 Gbps Pipe"
   untagged A1
   ip address 10.10.10.2 255.255.255.252
   ip ospf 10.10.10.2 area backbone
   ip ospf 10.10.10.2 network-type point-to-point
   exit
Core-2#

A VTEP performs these two roles:

Receive Layer 2 traffic from a source, such as a VM, in an Ethernet broadcast domain, encapsulating it within a VXLAN frame and sending it to the destination
VTEP. Receive the VXLAN frame, stripping the encapsulation to reveal the encapsulated Ethernet frame, and forwarding the frame toward the destination included in the encapsulated Ethernet frame.
VXLAN tunnel which carries the encapsulated data is called an overlay network.
The IP physical network that switches and routes the tunnels that make up the overlay is called the underlay network.(This is your traditional network, mostly your existing vlans and trunks.)

Configuration Steps:

1. Disable v2 modules

Cannot create a Virtual Network when the switch is in compatibility mode. The compatibility mode can be disabled by executing the [no] allow-v2-modules command.

2. Enable VXLAN
3. Create a Virtual-network (VNI) and associate a vlan
4. Create a Vxlan tunnel and define source and destination VTEP’s
5. Map the tunnel to a overlay network

Core Switch-1: Configure a loopback interface for the tunnel’s source.

interface loopback 0
ip address 1.1.1.1

Enable VXLAN – Disallow V2 Modules(Will not work with allowed v2 modules)

no allow-v2-modules
vxlan enable

Configure Virtual Network Interface (VNI)

virtual-network 100 20 "Server 1" (vlan is 20 and VNI is 100)

Configuring GRE Tunnels and VXLAN for VLAN Overlays

interface tunnel 1
   tunnel name "Server-1 Network"
   tunnel mode vxlan

Specify the source and destination of the tunnel:

   tunnel source 1.1.1.1
   tunnel destination 2.2.2.2
   exit

Overlay Tunnel with VLAN 20

vxlan tunnel 1 overlay-vlan 20

Configure Static Route towards Tunnel Loopback Interface on Core B

ip route 2.2.2.2 255.255.255.255  10.10.10.2

Core Switch-2:Configure a loopback interface for the tunnel’s source

interface loopback 0
ip address 2.2.2.2

Enable VXLAN - Disallow V2 Modules(Will not work with allowed v2 modules)

no allow-v2-modules
vxlan enable

Configure Virtual Network Interface (VNI)

virtual-network 100 20 "Server 2" (vlan is 20 and VNI is 100)

Configure Tunnel Interface

interface tunnel 2
   tunnel name "Server-2 Network"
   tunnel mode vxlan

Specify the source and destination of the tunnel:

   tunnel source 2.2.2.2
   tunnel destination 1.1.1.1
   exit

Overlay Tunnel with VLAN 20

vxlan tunnel 2 overlay-vlan 20

Configure Static Route towards Tunnel Loopback Interface on Core A

ip route 1.1.1.1 255.255.255.255  10.10.10.1

Verify Tunnel Interface States

Core-1# show interfaces tunnel brief

 Status - Tunnel Information Brief


  Tunnel                   : 251659490
  Mode                     : VXLAN Tunnel
  Source Address           : 1.1.1.1
  Destination Address      : 2.2.2.2
  Configured Tunnel Status : Enabled
  Current Tunnel State     : Up

Core-1#

Core-1# show interfaces tunnel

 Tunnel Configuration :

  Tunnel               : 251659490
  Tunnel Name          : VXLAN_Tunnel01
  Tunnel Status        : Enabled
  Source Address       : 1.1.1.1
  Destination Address  : 2.2.2.2
  Mode                 : VXLAN Tunnel
  TOS                  : -1
  TTL                  : 64
  IPv6                 : n/a
  MTU                  : 1450


 Current Tunnel Status :

  Tunnel State              : Up
  Destination Address Route : 2.2.2.2/32
  Next Hop IP               : 10.10.10.2
  Next Hop Interface        : vlan-10
  Next Hop IP Link Status   : Up
  Source Address            : 1.1.1.1


Core-1#

Server Configuration for Server 1:

Ethernet adapter Ethernet 1:

   Connection-specific DNS Suffix  . : expertnetworkconsultant.com
   Link-local IPv6 Address . . . . . : fe80::8051:832b:99:60c7%20
   Link-local IPv6 Address . . . . . : fe80::ab47:d863:23d2:5ffd%20
   IPv4 Address. . . . . . . . . . . : 10.100.200.99
   Subnet Mask . . . . . . . . . . . : 255.255.255.0
   Default Gateway . . . . . . . . . : ::

Server Configuration for Server 2:

Ethernet adapter Ethernet 1:

   Connection-specific DNS Suffix  . : expertnetworkconsultant.com
   Link-local IPv6 Address . . . . . : fe80::8051:832b:99:60c7%20
   Link-local IPv6 Address . . . . . : fe80::ab47:d863:23d2:5ffd%20
   IPv4 Address. . . . . . . . . . . : 10.100.200.100
   Subnet Mask . . . . . . . . . . . : 255.255.255.0
   Default Gateway . . . . . . . . . : ::

Perform Verification:

Core-Switch-1(config)# show interfaces tunnel 1
 Tunnel Configuration :

  Tunnel               : 251659491
  Tunnel Name          : Storage Server 1
  Tunnel Status        : Enabled
  Source Address       : 1.1.1.1
  Destination Address  : 2.2.2.2
  Mode                 : VXLAN Tunnel
  TOS                  : -1
  TTL                  : 64
  IPv6                 : n/a
  MTU                  : 1280

Core-Switch-1# show interfaces tunnel type vxlan statistics 251659490
  Tunnel Name                                  : Storage Server 1
  Rx Packets                                   : 0
  Tx Packets                                   : 0
  Rx 5 Minute Weighted Average Rate (Pkts/sec) : 0
  Tx 5 Minute Weighted Average Rate (Pkts/sec) : 0
Core-Switch-2:

Core-Switch-2# show interfaces tunnel 2
 Tunnel Configuration :

  Tunnel               : 251659491
  Tunnel Name          : Storage Server 2
  Tunnel Status        : Enabled
  Source Address       : 2.2.2.2
  Destination Address  : 1.1.1.1
  Mode                 : VXLAN Tunnel
  TOS                  : -1
  TTL                  : 64
  IPv6                 : n/a
  MTU                  : 1280

Core-Switch-2# show interfaces tunnel type vxlan statistics 251659491
  Tunnel Name                                  : Storage Server 2
  Rx Packets                                   : 0
  Tx Packets                                   : 0
  Rx 5 Minute Weighted Average Rate (Pkts/sec) : 0
  Tx 5 Minute Weighted Average Rate (Pkts/sec) : 0
Core-Switch-2:

Now that you are satisfied with the configuration of both tunnels, let us test connectivity between Server-1 and Server-2.

Ping Server 1:

Ping Server 2:

Applications:
Storage Replications to DR Site(s). Imagine your HQ has a 10Gbps link to another site which serves as the Disaster Recovery Site. NetApp SnapMirror® for example replicates between sites. Sometimes, the link between the sites may be a routed link which will not permit the extension of a layer 2 vlan across. Creating a Tunnel tied with a VXLAN ensures that the NetApp SnapMirror® would work over layer 2 as an overlay over the routed inter-site link.

Download Cisco VXLAN Document BRKDCT-2404

Microsoft has an excellent article which applies to Hyper-V over VXLAN technology which I find might be a good read for you.

The next steps will be to secure the point to point GRE over IPSEC. Ciso has a good article on Point-to-Point GRE over IPSEC which is a good read.

More Cisco OTV Resources.
Some useful Cisco VXLAN Labs