Tag Archives: Direct Connect

Watch a Failover from Direct Connect to Backup VPN for VMware Cloud on AWS

This post demonstrates a simulated failure of Amazon Direct Connect, with VMware Cloud (VMC) on Amazon Web Services (AWS). In this setup the standby VPN has been configured to provide connectivity in the event of a Direct Connect failure. The environment consists of a 6 host stretched cluster in the eu-west-2 (London) region, across Availability Zones eu-west-2a and eu-west-2b.

In this instance a pair of hosted private Virtual Interfaces (VIFs) are provided by a Cloud Connect service from a single third party provider. A Route-Based VPN has been configured. Direct Connect with VPN as standby was introduced in SDDC v1.7. For more information see Nico Vibert’s post here.

AWS Direct Connect: “Using AWS Direct Connect, you can establish private connectivity between AWS and your datacenter, office, or colocation environment, which in many cases can reduce your network costs, increase bandwidth throughput, and provide a more consistent network experience than Internet-based connections.”

AWS VPN: “AWS Virtual Private Network (AWS VPN) lets you establish a secure and private tunnel from your network or device to the AWS global network.”

DX_VPN_Setup

Direct Connect Outage

Before beginning it is worth re-iterating that the following screenshots do not represent a process. Providing the backup VPN is configured correctly then the customer / consumer of the service does not need to intervene; in the event of a real world outage everything highlighted below happens automatically. You may also want to review further reading: How to Deploy and Configure VMware Cloud on AWS (Part 1), How to Migrate VMware Virtual Machines to VMware Cloud on AWS (Part 2), plus additional demo posts: Watch VMware vSphere HA Recover Virtual Machines Across AWS Availability Zones and Watch a Virtual Machine Live Migration to VMware Cloud on AWS.

Taking down the primary and secondary VIFs was carried out by the hosting third party, to help with providing evidence of network resilience. When we start out in this particular environment the VIFs are attached and available. Servers in VMware Cloud are contactable from on-premise across the Direct Connect. The backup VPN is enabled.

DX_VPN_1DX_VPN_2

Following disabling of the interfaces by our third party provider the BGP status changes to down, along with the Direct Connect status for both VIFs.

DX_VPN_3DX_VPN_5

This is confirmed in the AWS console as both the BGP status and therefore the VIF state are down.

DX_VPN_4

With the Direct Connect down routes are redistributed using the backup VPN. The Direct Connect BGP hold timer is 90 seconds and the BGP keep alive is 30 seconds. After 90 seconds the VIF(s) BGP hold time expires and traffic starts to flow through the VPN connection.

In the screenshot below you can see an on-premise monitoring solution reporting on a server hosted in VMware Cloud on AWS. The server is available over the Direct Connect, drops, and is then available over the backup VPN after we disable the interfaces to simulate a failure. The test was conducted twice.

VPN_Monitor

How to Configure AWS Direct Connect with VMware Cloud on AWS

This post talks about the setup of AWS Direct Connect with VMware Cloud (VMC) on AWS. Direct Connect provides a high-speed, low latency connection between Amazon services and your on-premises environment. Direct Connect is useful for those who want dedicated private connectivity with a consistent network experience in comparison with internet-based VPN connections.

Direct Connect traffic travels over one or more virtual interfaces that you create in your customer AWS account. For SDDCs in which networking is supplied by NSX-T, all Direct Connect traffic, including vMotion, management traffic, and compute gateway traffic, uses a private virtual interface. This establishes a private connection between your on-premises data center and a single Amazon VPC.

You can create multiple interfaces to allow for redundancy and greater availability.”

Using AWS Direct Connect with VMware Cloud on AWS

Make sure you understand the terminology around a Virtual Interface (VIF) and the difference between a Standard VIF, Hosted VIF, and Hosted Connection: What’s the difference between a hosted virtual interface (VIF) and a hosted connection? It is important to consider that VMware Cloud on AWS requires a dedicated Virtual Interface (VIF) – or a pair of VIFs for resilience. If you have a standard 1Gbps or 10Gbps connection direct from Amazon then you can create and allocate VIFs for this purpose. If you are using a hosted connection from an Amazon Partner Network (APN) for sub-1G connectivity then you may need to procure additional VIFs, or a dedicated Direct Connect with the ability to have multiple VIFs on a single circuit. This is a discussion you should have with your APN partner.

Firstly review the pre-requisites and steps to request an AWS Direct Connection connection at Getting Started with AWS Direct Connect. The steps below will walk through configuring Direct Connect for use with VMware Cloud on AWS once the initial connection with Amazon or Amazon partner has been setup. Also review Direct Connect Pricing.

Direct Connect VMC Setup

Log into the VMware on AWS Console, from the SDDCs tab locate the appropriate SDDC and click View Details. Select the Networking & Security tab. Under System click Direct Connect. Make a note of the AWS Account ID, this is the shadow AWS account setup for VMC, you will need this account ID to associate with the Direct Connect.

VMC_DX_1

Log into the AWS console and navigate to the Direct Connect service. If you have not already accepted the connection from your third party provider then review the Amazon documentation referenced above.

AWS_DX_1

Select Virtual Interfaces and click Create Virtual Interface. In this instance we are creating a private VIF. Select the physical connection to use and give the virtual interface a name. Change the virtual interface owner to Another AWS Account and enter the VMC shadow AWS account ID. Fill in the VLAN and BGP ASN information provided by your connection provider. Repeat the process if you are assigning more than one VIF.

AWS_DX_2

Once the VIF or VIFs are created you will see a message that they need to be accepted by the account we have set as owner.

AWS_DX_3

Go back to the VMC portal and the Direct Connect page, click Refresh if necessary. Any interfaces associated with the shadow AWS account will now be listed as available.

VMC_DX_2

Attach the virtual interfaces and confirm acknowledgement that you will be responsible for any data transfer charges that are incurred.

VMC_DX_3

At this point it will take up to 10 minutes for the state of each interface to change from Attaching to Attached, and the BGP status to change from Down to Up. You should now see Advertised BGP Routes listing the network segments you have configured, and Learned BGP Routes listing the subnets peering from your on-premises network.

Click Overview. The Direct Connect shows green, the corresponding VIFs in the AWS Direct Connect page show green and available.

Direct_Connect_Up_VMC

For Direct Connect deep dives review the following blog posts by Nico Vibert: AWS Direct Connect – Deep Dive and Integration with VMware Cloud on AWS, and Direct Connect with VMware Cloud on AWS with VPN as a back-up.

Further Reading: How to Deploy and Configure VMware Cloud on AWS (Part 1), How to Migrate VMware Virtual Machines to VMware Cloud on AWS (Part 2).