VMware Cloud on AWS Live Migration Demo

This post will demonstrate a live migration of a virtual machine from an on-premise VMware infrastructure, to VMware Cloud on AWS. The steps below will demonstrate how quick and easy it is to move virtual machines between VMware Cloud on AWS and on-premise VMware environments using HCX, without the need to re-IP or re-architect services.

You may also want to review VMware Cloud on AWS Deployment Planning, and VMware Cloud on AWS Live Migration Demo. For more information on Stretched Clusters for VMware Cloud on AWS see Overview and Documentation, as well as the following external links:

VMware FAQ | AWS FAQ | Roadmap | Product Documentation | Technical Overview | VMware Product Page | AWS Product Page | Try first @ VMware Cloud on AWS – Getting Started Hands-on Lab

The VMware Cloud environment in this demo is setup as follows:

  • SDDC deployed consisting of a 6 host stretched cluster in eu-west-2
  • On-premise connectivity provided by Direct Connect
  • On-premise vCenter and SDDC vCenter in hybrid linked mode
  • HCX Cloud add-on enabled, and appliances deployed on-premise
  • On-premise networks are VLAN backed port groups in a distributed switch
  • VLAN_98 has been stretched for the purposes of this demo


The virtual machine I am going to migrate is the web server from a 3-tier application: VMC-DEMO-WEB-01, with a private IPv4 address of

In the on-premise vCenter Server I have selected the HCX plugin from the Menu drop-down. The dashboard shows my site pairing and cloud overview. Under Network Extension I can see that VLAN_98 has been stretched to VMware Cloud on AWS.


From the Migration screen I can see previous migration history, and I select Migrate Virtual Machines.


The migration interface loads and I search for the virtual machine.


Having selected the virtual machine to migrate I can now go ahead and select the folder, resource pool, and datastore to use. In this example the machine is already thin provisioned and I am using the vMotion migration type. The network has automatically been populated with the stretched network VLAN_98.


HCX will perform some validation checks and then I click Finish to start the migration.


The virtual machine migration progress is now underway.


After 4 minutes, the migration is complete.


The virtual machine did not drop any pings during the migration, the web site is still accessible and able to pull data from the database.


A HTTP monitor setup in Solarwinds shows that there was no loss of service during the migration.


The virtual machine is now running in VMware Cloud on AWS and is visible in the SDDC vCenter.


Should the machine need moving back on-premise the same process can be followed, with the Reverse Migration tick-box.


Once virtual machines are running in VMware Cloud on AWS they have access to native AWS services using the 25 Gbps Elastic Network Interface (ENI): Connecting VMware Cloud on AWS to Amazon EC2Load Balancing VMware Cloud on AWS with Amazon ELB.

Configuring AWS Direct Connect with VMware Cloud on AWS

This post talks about the setup of AWS Direct Connect with VMware Cloud (VMC) on AWS. Direct Connect provides a high-speed, low latency connection between Amazon services and your on-premises environment. Direct Connect is useful for those who want dedicated private connectivity with a consistent network experience in comparison with internet-based VPN connections.

Direct Connect traffic travels over one or more virtual interfaces that you create in your customer AWS account. For SDDCs in which networking is supplied by NSX-T, all Direct Connect traffic, including vMotion, management traffic, and compute gateway traffic, uses a private virtual interface. This establishes a private connection between your on-premises data center and a single Amazon VPC.

You can create multiple interfaces to allow for redundancy and greater availability.”

Using AWS Direct Connect with VMware Cloud on AWS

Make sure you understand the terminology around a Virtual Interface (VIF) and the difference between a Standard VIF, Hosted VIF, and Hosted Connection: What’s the difference between a hosted virtual interface (VIF) and a hosted connection? It is important to consider that VMware Cloud on AWS requires a dedicated Virtual Interface (VIF) – or a pair of VIFs for resilience. If you have a standard 1Gbps or 10Gbps connection direct from Amazon then you can create and allocate VIFs for this purpose. If you are using a hosted connection from an Amazon Partner Network (APN) for sub-1G connectivity then you may need to procure additional VIFs, or a dedicated Direct Connect with the ability to have multiple VIFs on a single circuit. This is a discussion you should have with your APN partner.

Firstly review the pre-requisites and steps to request an AWS Direct Connection connection at Getting Started with AWS Direct Connect. The steps below will walk through configuring Direct Connect for use with VMware Cloud on AWS once the initial connection with Amazon or Amazon partner has been setup. Also review Direct Connect Pricing.

Direct Connect VMC Setup

Log into the VMware on AWS Console, from the SDDCs tab locate the appropriate SDDC and click View Details. Select the Networking & Security tab. Under System click Direct Connect. Make a note of the AWS Account ID, this is the shadow AWS account setup for VMC, you will need this account ID to associate with the Direct Connect.


Log into the AWS console and navigate to the Direct Connect service. If you have not already accepted the connection from your third party provider then review the Amazon documentation referenced above.


Select Virtual Interfaces and click Create Virtual Interface. In this instance we are creating a private VIF. Select the physical connection to use and give the virtual interface a name. Change the virtual interface owner to Another AWS Account and enter the VMC shadow AWS account ID. Fill in the VLAN and BGP ASN information provided by your connection provider. Repeat the process if you are assigning more than one VIF.


Once the VIF or VIFs are created you will see a message that they need to be accepted by the account we have set as owner.


Go back to the VMC portal and the Direct Connect page, click Refresh if necessary. Any interfaces associated with the shadow AWS account will now be listed as available.


Attach the virtual interfaces and confirm acknowledgement that you will be responsible for any data transfer charges that are incurred.


At this point it will take up to 10 minutes for the state of each interface to change from Attaching to Attached, and the BGP status to change from Down to Up. You should now see Advertised BGP Routes listing the network segments you have configured, and Learned BGP Routes listing the subnets peering from your on-premises network.

Click Overview. The Direct Connect shows green, the corresponding VIFs in the AWS Direct Connect page show green and available.


For Direct Connect deep dives review the following blog posts by Nico Vibert: AWS Direct Connect – Deep Dive and Integration with VMware Cloud on AWS, and Direct Connect with VMware Cloud on AWS with VPN as a back-up.

Load Balancing VMware Cloud on AWS with Amazon ELB

This post demonstrates the connectivity between VMware Cloud (VMC) on AWS and native AWS services. In the example below we will be using Amazon Elastic Load Balancing (ELB) to provide highly available, scaleable, and secure load balancing backed by virtual machines hosted in the VMware Cloud Software-Defined Data Centre (SDDC). There is an assumption you have a basic understanding of both platforms.

When integrating with Amazon ELB there are 2 options: Application Load Balancer (ALB) which operates at the request layer (7), or Network Load Balancer (NLB) which operates at the connection layer (4). The Amazon Classic Load Balancer is for Amazon EC2 instances only. For assistance with choosing the correct type of load balancer review Details for Elastic Load Balancing Products and Product Comparisons. Amazon load balancers and their targets can be monitored using Amazon Cloud Watch.

Connectivity Overview

  • VMware Cloud on AWS links with your existing AWS account to provide access to native services. During provisioning a Cloud Formation template will grant AWS permissions using the Identity Access Management (IAM) service. This allows your VMC account to create and manage Elastic Network Interfaces (ENI) as well as auto-populate Virtual Private Cloud (VPC) route tables.
  • An Elastic Network Interface (ENI) dedicated to each physical host connects the VMware Cloud to the corresponding Availability Zone in the native AWS VPC. There is no charge for data crossing the 25 Gbps ENI between the VMC VPC and the native AWS VPC, however it is worth remembering that data crossing Availability Zones is charged at $0.01 per GB (at the time of writing).
  • An example architecture below shows a stretched cluster in VMware on AWS with web services running on virtual machines across multiple Availability Zones. The load balancer sits in the customers native AWS VPC and connects to the web servers using the ENI connectivity. Amazon’s DNS service Route 53 routes users accessing a custom domain to the web service.
  • Remember to consider the placement of your target servers when deploying the Amazon load balancer. For more information see VMware Cloud on AWS Deployment Planning. See also Elastic Load Balancing Pricing.


VMC Gateway Firewall

Before configuring the ELB we need to make sure it can access the target servers. Log into the VMware on AWS Console, from the SDDCs tab locate the appropriate SDDC and click View Details. Select the Networking & Security tab, under Security click Gateway Firewall and Compute Gateway.


In this example I have added a rule for inbound access to my web servers. The source is AWS Connected VPC Prefixes (this can be tied down to only allow access from the load balancer if required). The destination is a user defined group which contains the private IPv4 addresses for the web servers in VMC, and the allowed service is set to HTTP (TCP 80).

If you are using the Application Load Balancer then you also need to consider the security group attached to the ALB. If the default group is not used, or the security group attached to the Elastic Network Interfaces has been changed, then you may need to make additional security group changes to allow traffic between the ALB and the ENIs. Review the Security Group Configuration section of Connecting VMware Cloud on AWS to EC2 Instances for more information. The Network Load Balancer does not use security groups. The gateway firewall rule outlined above will be needed regardless of the load balancer type.

ELB Deployment

Log into the VMware on AWS Console, from the SDDCs tab locate the appropriate SDDC and click View Details. Select the Networking & Security tab. Under System click Connected VPC. Make a note of the AWS Account ID and the VPC ID. You will need to deploy the load balancer into this account and VPC.

Log into the AWS Console and navigate to the EC2 service. Locate the Load Balancing header in the left hand navigation pane and click Load Balancers. Click Create Load Balancer. Select the load balancer type and click Create.


Typically for HTTP/HTTPS the Application Load Balancer will be used. In this example since I want to deploy the load balancer to a single Availability Zone for testing I am using a Network Load Balancer, which can also have a dedicated Elastic (persistent public) IP.

Enter the load balancer configuration. I am configuring an internet-facing load balancer with listeners on port 80 for HTTP traffic. Scroll down and specify the VPC and Availability Zones to use. Ensure you use the VPC connected to your VMware on AWS VPC. In this example I have selected a subnet in the same Availability Zone as my VMware Cloud SDDC.


In the routing section configure the target group which will contain the servers behind the load balancer. The target type needs to be IP.


In this instance since I am creating a new target group I need to specify the IP addresses of the web servers which are VMs sitting in my VMC SDDC. The Network column needs to be set to Other private IP address.


Once the load balancer and target group are configured review the settings and deploy. You can review the basic configuration, listeners, and monitoring by selecting the newly deployed load balancer.


Click the Description tab to obtain the DNS name of the load balancer. You can add a CNAME to reference the load balancer using Amazon Route 53 or another DNS service.


Finally, navigate to Target Groups. Here you can view the health status of your registered targets, and configure health checks, monitoring, and tags.

Connecting VMware Cloud on AWS to Amazon EC2

This post demonstrates the connectivity between VMware Cloud (VMC) on AWS and native AWS services. In the example below we will be using Amazon Elastic Compute Cloud (EC2) to provision a virtual instance backed by Amazon Elastic Block Store (EBS) storage. To complete the use case we will install Veeam and use the EC2 instance to backup virtual machines hosted in the VMware Cloud Software-Defined Data Centre (SDDC).

Connectivity Overview

  • VMware Cloud on AWS links with your existing AWS account to provide access to native services. During provisioning a Cloud Formation template will grant AWS permissions using the Identity Access Management (IAM) service. This allows your VMC account to create and manage Elastic Network Interfaces (ENI) as well as auto-populate Virtual Private Cloud (VPC) route tables.
  • An Elastic Network Interface (ENI) dedicated to each physical host connects the VMware Cloud to the corresponding Availability Zone in the native AWS VPC. There is no charge for data crossing the 25 Gbps ENI between the VMC VPC and the native AWS VPC, however it is worth remembering that data crossing Availability Zones is charged at $0.01 per GB (at the time of writing).
  • The example architecture we will be using is shown below. For more information see VMware Cloud on AWS Deployment Planning.


Security Group Configuration

AWS Security Groups will be attached to your EC2 instances and ENIs, it is therefore vital that you fully understand the concepts and configuration you are implementing. Please review Understanding AWS Security Groups with VMware Cloud on AWS by Brian Graf.

In the AWS console Security Groups can be accessed from the EC2 service. In this example I have created a security group allowing all protocols (any port) inbound from the source CIDR block used in VMC for both my compute and management subnets. In other words this is allowing connectivity into the EC2 instance from VM in my VMC SDDC. You may want to lock this down to specific IP addresses or ports to provide a more secure operating model. Outbound access from the EC2 instance is defined as any IPv4 destination ( on any port.


I have also changed the default security group associated with the ENIs used by VMC to a custom security group. The security group allows inbound access on the ENI (which is inbound access to VMC as explained in the article below) on all ports from the source CIDR block of my native AWS VPC. Outbound access which is from VMC into AWS is defined as any IPv4 destination ( on any port.


EC2 Deployment

Log into the VMware on AWS Console, from the SDDCs tab locate the appropriate SDDC and click View Details. Select the Networking & Security tab. Under System click Connected VPC. Make a note of the AWS Account ID and the VPC ID. You will need to deploy an EC2 instance into this account and VPC.

Log into the AWS Console and navigate to the EC2 service. Launch an EC2 instance that meets the System Requirements for Veeam. In this example I have used the t2.medium instance and Microsoft Windows Server 2019 Base AMI. When configuring network the EC2 instance must be in the VPC connected to VMC. I have added an additional EBS volume for the backup repository using volume type General Purpose SSD (gp2). Ensure the security group selected or created allows the relevant access.

Gateway Firewall

In addition to security group settings inbound access also needs allowing on the VMC Gateway Firewall. In this instance as we are connecting the EC2 instance to the vCenter we define the rule on the Management Gateway. If we were connecting to a workload in one of the compute subnets the rule would be defined on the Compute Gateway. You may have noticed that although I allowed any port in the AWS Security Groups, the actual ports allowed can also be defined on the Gateway Firewall.

In this example I have added a new user defined group which contains the private IPv4 address for the EC2 instance and added it as a source in the vCenter Inbound Rule. The allowed port is set to HTTPS (TCP 443) – I have also allowed ICMP. I have added the same source group to the ESXi Inbound Rule which allows Provisioning (TCP 902). Both these rules are needed to allow Veeam to backup virtual machines in VMC.


Veeam Setup

Now that connectivity between the EC2 instance and the VMC vCenter has been configured I can hop onto the EC2 instance and begin the setup of Veeam. I will, of course, need an inbound rule for RDP (TCP 3389) adding to the security group of the EC2 instance, specifying the source I am connecting from.

Follow the installation steps outlined in the Veeam Backup & Replication 9.5 Update 4 User Guide for VMware vSphere.


In the VMC console navigate to the Settings tab of the SDDC and make a note of the  password for the cloudadmin@vmc.local account. Open the Veeam Backup & Replication console and add the vCenter private IP address, use the vCenter cloud admin credentials.


Add the backup repository using the EBS volume and create a backup job as normal. Refer to the Veeam Backup Guide if you need assistance with Veeam.


To make use of S3 object storage AWS you will need an IAM Role granting S3 access, and an S3 VPC Endpoint. In the case of VMC, as an alternative design, you can host the Veeam B&R server inside your VMC SDDC to make use of the built in S3 endpoint. In testing we found backup speeds to be faster but you will likely still need an EBS backed EC2 instance for your backup repository. It goes without saying you should make sure backup data is not held solely on the same physical site as the servers you are backing up. See Veeam KB2414: VMware Cloud on AWS Support for further details.

Add a new Scale-Out Backup Repository and follow the steps to add account and bucket details.

Set an appropriate policy for moving backups to object based storage, once this threshold is met you will start to see Veeam files populating the S3 bucket.


VMware Cloud on AWS Deployment Planning

This post pulls together the notes I have made during the planning of VMware Cloud (VMC) on AWS (Amazon Web Services) deployment, and migrations of virtual machines from traditional on-premise vSphere infrastructure. It is intended as a generic list of considerations and useful links, and is not a comprehensive guide. Cloud, more-so than traditional infrastructure, is constantly changing. Features are implemented regularly and transparently so always validate against official documentation. This post was last updated on August 6th 2019.

Part 1: SDDC Deployment

Part 2: Migration Planning & Lessons Learned

1. Capacity Planning

You can still use existing tools or methods for basic capacity planning, you should also consult the VMware Cloud on AWS Sizer and TCO Calculator provided by VMware. There is a What-If Analysis built into both vRealize Business and vRealize Operations, which is similar to the sizer tool and can also help with cost comparisons. Additional key considerations are:

  • Egress costs are now a thing! Use vRealize Network Insight to understand network egress costs and application topology in your current environment. Calculate AWS Egress Fees Proactively for VMware Cloud on AWS is a really useful resource.
  • You do not need to factor in N+1 when planning capacity. If there is a host failure VMware will automatically add a new host to the cluster, allowing you to utilise more of the available resource.
  • Export a list of Virtual Machines (VMs) from vCenter and review each VM. Contact service owners, application owners, or super users to understand if there is still a requirement for the machine and what it is used for. This ties in to the migration planning piece but crucially allows you to better understand capacity requirements. Most environments have VM sprawl and identifying services that are either obsolete, moved to managed services, or were simply test machines no longer required will clearly reduce capacity requirements.
  • Consider you are now on a ‘metered’ charging model, so don’t set the meter going; in other words don’t deploy the SDDC, until you are ready to start using the platform. Common sense, but internal service reviews or service acceptance and approvals can take longer than expected.
  • You can make savings using reserved instances, by committing to 1 or 3 years. Pay as you go pricing may be sufficient for evaluation or test workloads, but for production workloads it is much more cost effective to use reserved instances.
  • At the time of writing up to 2 SDDC’s can be deployed per organisation (soft limit), each SDDC supporting up to 20 vSphere clusters and each cluster up to 16 physical nodes.
  • The standard i3 bare metal instance currently offers 2 sockets, 36 cores, 512 GiB RAM, 10.7 TB vSAN storage, a 16-node cluster provides 32 sockets, 576 cores, 8192 GiB RAM, 171.2 TB.
  • New R5 bare metal instances are deployed with 2.5 GHz Intel Platinum 8000 series (Skylake-SP) processors; 2 sockets, 48 cores, 768 GiB RAM and AWS Elastic Block Storage (EBS) backed capacity scaling up to 105 TB for 3-node resources and 560 TB for 16-node resources. For up to date configuration maximums see Configuration Maximums for VMware Cloud on AWS.



2. Placement and Availability

Ultimately placement of your SDDC is going to be driven by specific use cases, and any regulations for the data type you are hosting. How VMware is Accelerating NHS Cloud Adoption uses the UK National Health Service (NHS) and Information Governance as an example. Additional placement and availability considerations are:

  • An SDDC can be deployed to a single Availability Zone (AZ) or across multiple AZ’s, otherwise known as a stretched cluster. For either configuration if a problem is identified with a host in the cluster High Availability (HA) evacuation takes place as normal, an additional host is then automatically provisioned and added as a replacement.
  • The recommendation for workload availability is to use a stretched cluster which distributes workloads across 2 Availability Zones with a third hosting a witness node. In this setup data is written to both Availability Zones in an active active setup. In the event of an outage to an entire Availability Zone vSphere HA brings virtual machines back online in the alternative AZ: VMware Cloud on AWS Stretched Cluster Failover Demo.
  • Stretched clusters have an SLA Availability Commitment of 99.99% (99.9% for single AZ), and provide a Recovery Point Objective (RPO ) of zero by using synchronous data replication. Note that there are additional cross-AZ charges for stretched clusters. The Recovery Time Objective (RTO) is a vSphere HA failover, usually sub 5 minutes.
  • The decision on whether to use single or multiple Availability Zones needs to be taken at the time of deployment. An existing SDDC cannot be upgraded to multi-AZ or downgraded to a single AZ.
  • An Elastic Network Interface (ENI) dedicated to each physical host connects the VMware Cloud to the corresponding Availability Zone in the native AWS Virtual Private Cloud (VPC). There is no charge for data crossing the 25 Gbps ENI between the VMware Cloud VPC and the native AWS VPC.
  • Data that crosses Availability Zones is chargeable, therefore it is good practise to deploy the SDDC to the same region and AZ as your current or planned native AWS services.


3. Networks and Connectivity

  • VMware Cloud on AWS links with your existing AWS account to provide access to native services. During provisioning a Cloud Formation template will grant AWS permissions using the Identity Access Management (IAM) service. This allows your VMC account to create and manage Elastic Network Interfaces (ENI’s) as well as auto-populate Virtual Private Cloud (VPC) route tables when NSX subnets are created.
  • It is good practise to enable Multi-Factor Authentication (MFA) for your accounts in both VMC and AWS. VMware Cloud can also use Federated Identity Management, for example with Azure AD. This currently needs to be facilitated by your VMware Customer Success team, but once setup means you can control accounts using Active Directory and enforce MFA or follow your existing user account policies.
  • It is important to ensure proper planning of your IP addressing scheme, if the IP range used overlaps with anything on-premise or in AWS then routes will not be properly distributed and the SDDC needs destroying and reinstalling with an updated subnet to resolve.
  • You will need to allocate a CIDR block for SDDC management, as well as network segments for your SDDC compute workloads to use. Review Selecting IP Subnets for your SDDC for assistance with selecting IP subnets for your VMC environment.
  • Connectivity to the SDDC can be achieved using either AWS Direct Connect (DX) or VPN, see Connectivity Options for VMware Cloud on AWS Software Defined Data Centers. From SDDC v1.7 onwards it is possible to use DX with a backup VPN for resilience.
  • Traffic between VMC and your native AWS VPC is handled by the 25 Gbps Elastic Network Interfaces (ENI) referenced in the section above. To connect to additional VPCs or accounts you can setup an IPsec VPN. The Amazon Transit Gateway feature is available for some regions and configurations, if you are using DX then the minimum requirement is 1Gbps.
  • Access to native AWS services needs to be setup on the VMC Gateway Firewall, for example: Connecting VMware Cloud on AWS to EC2 Instances, as well as Amazon security groups; this is explained in How AWS Security Groups Work With VMware Cloud on AWS.
  • To migrate virtual machines from your on-premise data centre review Hybrid Linked Mode Prerequisites and vMotion across hybrid cloud: performance and best practices. In addition you will need to know the Required Firewall Rules for vMotion and for Cold Migration.
  • For virtual machines to keep the same IP addressing layer 2 networks can be stretched with HCX, review VMware HCX Documentation. HCX is included with VMC licensing but is a separate product in its own right so should be planned accordingly and is not covered in this post. Review VMware Cloud on AWS Live Migration Demo to see HCX in action.
  • VMware Cloud on AWS: Internet Access and Design Deep Dive is a useful resource for considering virtual machines that may require internet access.

4. Operational Readiness

The SDDC is deployed but before you can start migrating virtual machines you need to make sure the platform is fully operational. There are some key aspects but in general make sure you cover everything you do currently on premise:

  • You will likely still have a need for Active Directory, DNS, DHCP, and time synchronisation. Either use native cloud services, or build new Domain Controllers for example in VMC.
  • If you have a stretched-cluster and build Domain Controllers, or other management servers, consider building these components in each Availability Zone, then using compute policies to control the virtual machine placement. This is similar to anti-affinity rules on-premise, see VMware Cloud on AWS Compute Policies for more information.
  • Remember Disaster Recovery (DR) still needs to be factored in. DR as a Service (DRaaS) is offered through Site Recovery Manager (SRM) between regions in the cloud or on-premise. A stretched-cluster may be sufficient but again, this is dependent on the organisation or service requirements.
  • Anti-Virus, monitoring, and patching (OS / application) solutions need to be implemented. Depending on your licensing model you should be able to continue using the same products and tool-set, and carry the license over, but check with the appropriate vendor. Also start thinking about integrating cloud monitoring and management where applicable.
  • VMware Cloud Log Intelligence is a SaaS offering for log analytics, it can forward to an existing syslog solution or integrate with AWS CloudTrail.
  • Backups are still a crucial part of VMware Cloud on AWS and it is entirely the customers responsibility to ensure backups are in place. Unless you have a specific use case to backup machines from VMware Cloud to on-premise, it probably makes sense to move or implement backup tooling in the cloud, for example using Veeam in Native AWS.
  • Perform full backups initially to create a new baseline. Try native cloud backup products that will backup straight to S3, or continue with traditional backup methods that connect into vCenter. The reference architecture below uses Elastic Block Storage (EBS) backed Elastic Compute Cloud (EC2) instances running Veeam as a backup solution, then archiving out to Simple Storage Services (S3). Druva are able to backup straight to S3 from VMC. Veeam are also constantly updating functionality so as mentioned at the start of the post this setup may not stay up to date for long:


  • Customers must be aware of the shared security model that exists between: VMware; delivering the service, Amazon Web Services (the IaaS provider); delivering the underlying infrastructure, and customers; consuming the service.
  • VMware Cloud on AWS meets a number of security standards such as NIST, ISO, and CIS. You can review VMware’s security commitments in the VMware Cloud Services on AWS Security Overview.
  • When using native AWS services you must always follow Secure by Design principals to make sure you are not leaving the environment open or vulnerable to attack.

Part 2: VMware Cloud on AWS Migration Planning & Lessons Learned

Additional resources: VMware Cloud On AWS On-Boarding Handbook | VMware Cloud on AWS Operating Principles | Resources | Documentation | Factbook

VMware Cloud on AWS VideosYouTube PlaylistsVMworld 2018 Recorded Sessions

VMware Site Recovery Manager 8.x Upgrade Guide

This post will walk through an inplace upgrade of VMware Site Recovery Manager (SRM) to version 8.1, which introduces support for the vSphere HTML5 client and recovery / migration to VMware on AWS. Read more about what’s new in this blog post. The upgrade is relatively simple but we need to cross-check compatibility and perform validation tests after running the upgrade installer.



  • The Site Recovery Manager upgrade retains configuration and information such as recovery plans and history but does not preserve any advanced settings
  • Protection groups and recovery plans also need to be in a valid state to be retained, any invalid configurations or not migrated
  • Check the upgrade path here, for Site Recovery Manager 8.1 we can upgrade from 6.1.2 and later
  • If vSphere Replication is in use then upgrade vSphere Replication first, following the steps outlined here
  • Site Recovery Manager 8.1 is compatible with vSphere 6.0 U3 onwards, and VMware Tools 10.1 and onwards, see the compatibility matrices page here for full details
  • Ensure the vCenter and Platform Services Controller are running and available
  • In Site Recovery Manager 8.1 the version number is decoupled from vSphere, however check that you do not need to perform an upgrade for compatibility
  • For other VMware products check the product interoperability site here
  • If you are unsure of the upgrade order for VMware components see the Order of Upgrading vSphere and Site Recovery Manager Components page here
  • Make a note of any advanced settings you may have configured under Sites > Site > Manage > Advanced Settings
  • Confirm you have Platform Services Controller details, the administrator@vsphere.local password, and the database details and password

Download the VMware Site Recovery Manager self extracting installer here to the server, and if applicable; the updated Storage Replication Adapter (SRA) – for storage replication. Review the release notes here, and SRM upgrade documentation centre here.

Database Backup

Before starting the upgrade make sure you take a backup of the embedded vPostgres database, or the external database. Full instructions can be found here, in summary:

  • Log into the SRM Windows server and stop the VMware Site Recovery Manager service
  • From command prompt run the following commands, replacing the db_username and srm_backup_name parameters, and the install path and port if they were changed from the default settings
cd C:\Program Files\VMware\VMware vCenter Site Recovery Manager Embedded Database\bin
pg_dump -Fc --host --port 5678 --username=db_username srm_db > srm_backup_name
  • If you need to restore the vPostgres database follow the instructions here

In addition to backing up the database check the health of the SRM servers and confirm there are no pending reboots. Log into the vSphere web client and navigate to the Site Recovery section, verify there are no pending cleanup operations or configuration issues, all recovery plans and protection groups should be in a Ready state.


As identified above, vSphere Replication should be upgraded before Site Recovery Manager. In this instance we are using Nimble storage replication, so the Storage Replication Adapter (SRA) should be upgraded first. Download and run the installer for the SRA upgrade, in most cases it is a simple next, install, finish.

We can now commence the Site Recovery Manager upgrade, it is advisable to take a snapshot of the server and ensure backups are in place. On the SRM server run the executable downloaded earlier.

  • Select the installer language and click Ok, then Next
  • Click Next on the patent screen, accept the EULA and click Next again
  • Double-check you have performed all pre-requisite tasks and click Next
  • Enter the FQDN of the Platform Services Controller and the SSO admin password, click Next
  • The vCenter Server address is auto-populated, click Next
  • The administrator email address and local host ports should again be auto-populated, click Next
  • Click Yes when prompted to overwrite registration
  • Select the appropriate certificate option, in this case keeping the existing certificate, click Next
  • Check the database details and enter the password for the database account, click Next
  • Configure the service account to run the SRM service, again this will be retain the existing settings by default, click Next
  • Click Install and Finish once complete


After Site Recovery Manager is upgraded log into the vSphere client. If the Site Recovery option does not appear immediately you may need to clear your browser cache, or restart the vSphere client service.


On the summary page confirm both sites are connected, you may need to reconfigure the site pair if you encounter connection problems.


Validate the recovery plan and run a test to confirm there are no configuration errors.


The test should complete successfully.


I can also check the replication status and Storage Replication Adapter status.


Configuring vCenter 6.7 High Availability

The vCenter Server Appliance has provided vCenter High Availability (HA) with vSphere 6.5 onwards. In the fully functioning HTML5 release of vCenter 6.7 Update 1 onwards the setup of vCenter HA was hugely simplified. Read more about the improvements made in vSphere 6.7U1 in this blog post. By implementing vCenter HA you can protect your vCenter from host and hardware failures, and significantly reduce down time during patching due to the active / standby nature of the vCenter cluster.

The vCenter HA architecture is made up of the components in the vSphere image below. The vCenter Server Appliance is cloned out to create passive and witness nodes. Updated data is replicated between the active and passive nodes. In the event of an outage to the active vCenter the passive vCenter automatically assumes the active role and identity. Management connections still route to the same IP address and FQDN, however they have now failed over to the replica node. When the outage is resolved and the vCenter that failed comes back online; it then takes on the role of the passive node, and receives replication data from the active vCenter Server.



  • vCenter HA was introduced with the vCenter Server Appliance 6.5
  • The vCenter deployment size should be at least small, and therefore 4 vCPU 16 GB RAM
  • A minimum of three hosts
  • The hosts should be running at least ESXi 5.5
  • The management network should be configured with a static IP address and reachable FQDN
  • SSH should be enabled on the VCSA
  • A port group for the HA network is required on each ESXi host
  • The HA network must be on a different subnet to the management network
  • Network latency between the nodes must be less than 10ms
  • vCenter HA is compatible with both embedded deployment model and external PSC
  • For further information on vCenter HA performance and best practises see this post

If you are configuring vCenter HA on a version of vCenter prior to 6.7 Update 1 then see this post. If you are configuring vCenter HA in a cluster with less than the required number of physical hosts, such as in a home lab, you can add a parameter to override the anti-affinity setting; see this post by William Lam.

Configuring vCenter HA

Log into the vSphere client and select the top level vCenter Server in the inventory. Click the Configure tab and vCenter HA. The vCenter HA summary page is displayed with a list of prerequisites, ensure these are met along with the requirements above. Click Setup vCenter HA.


Select the vCenter HA network by clicking Browse. Scroll down the vCenter HA resource settings, review the network and resource settings of the active node of the vCenter Server. Scroll down to the passive node and click Edit. Follow the on-screen prompts to select a folder location, compute and storage resources. Select the management and HA networks for the passive node, review the settings once complete and click Finish. Follow the same steps for the witness node.


On the IP settings page enter the HA network settings for the active, passive, and witness nodes. Click Finish.


The vCenter Server will now be cloned and the HA network settings applied, this can be monitored from the tasks pane. Once complete the vCenter HA state will show Healthy, and all nodes in the cluster will show Up.


You can edit the status of vCenter HA at any time by going back into the vCenter HA menu and clicking Edit. You also have the option of removing the vCenter HA configuration or manually initiating a failover.


For more information on vCenter 6.7 High Availability see the vCenter Documentation Centre here.