Category Archives: VMware Cloud on AWS

VMware Cloud on AWS Deployment Planning

This post pulls together the notes I have made during the planning of VMware Cloud (VMC) on AWS (Amazon Web Services) deployment, and migrations of virtual machines from traditional on-premise vSphere infrastructure. It is intended as a generic list of considerations and useful links, and is not a comprehensive guide. Cloud, more-so than traditional infrastructure, is constantly changing. Features are implemented regularly and transparently so always validate against official documentation. This post was last updated on August 6th 2019.

Part 1: SDDC Deployment

1. Capacity Planning

You can still use existing tools or methods for basic capacity planning, you should also consult the VMware Cloud on AWS Sizer and TCO Calculator provided by VMware. There is a What-If Analysis built into both vRealize Business and vRealize Operations, which is similar to the sizer tool and can also help with cost comparisons. Additional key considerations are:

  • Egress costs are now a thing! Use vRealize Network Insight to understand network egress costs and application topology in your current environment. Calculate AWS Egress Fees Proactively for VMware Cloud on AWS is a really useful resource.
  • You do not need to factor in N+1 when planning capacity. If there is a host failure VMware will automatically add a new host to the cluster, allowing you to utilise more of the available resource.
  • Export a list of Virtual Machines (VMs) from vCenter and review each VM. Contact service owners, application owners, or super users to understand if there is still a requirement for the machine and what it is used for. This ties in to the migration planning piece but crucially allows you to better understand capacity requirements. Most environments have VM sprawl and identifying services that are either obsolete, moved to managed services, or were simply test machines no longer required will clearly reduce capacity requirements.
  • Consider you are now on a ‘metered’ charging model, so don’t set the meter going; in other words don’t deploy the SDDC, until you are ready to start using the platform. Common sense, but internal service reviews or service acceptance and approvals can take longer than expected.
  • You can make savings using reserved instances, by committing to 1 or 3 years. Pay as you go pricing may be sufficient for evaluation or test workloads, but for production workloads it is much more cost effective to use reserved instances.
  • At the time of writing up to 2 SDDC’s can be deployed per organisation (soft limit), each SDDC supporting up to 20 vSphere clusters and each cluster up to 16 physical nodes.
  • The standard i3 bare metal instance currently offers 2 sockets, 36 cores, 512 GiB RAM, 10.7 TB vSAN storage, a 16-node cluster provides 32 sockets, 576 cores, 8192 GiB RAM, 171.2 TB.
  • New R5 bare metal instances are deployed with 2.5 GHz Intel Platinum 8000 series (Skylake-SP) processors; 2 sockets, 48 cores, 768 GiB RAM and AWS Elastic Block Storage (EBS) backed capacity scaling up to 105 TB for 3-node resources and 560 TB for 16-node resources. For up to date configuration maximums see Configuration Maximums for VMware Cloud on AWS.

2. Placement and Availability

Ultimately placement of your SDDC is going to be driven by specific use cases, and any regulations for the data type you are hosting. How VMware is Accelerating NHS Cloud Adoption uses the UK National Health Service (NHS) and Information Governance as an example. Additional placement and availability considerations are:

  • An SDDC can be deployed to a single Availability Zone (AZ) or across multiple AZ’s, otherwise known as a stretched cluster. For either configuration if a problem is identified with a host in the cluster High Availability (HA) evacuation takes place as normal, an additional host is then automatically provisioned and added as a replacement.
  • The recommendation for workload availability is to use a stretched cluster which distributes workloads across 2 Availability Zones with a third hosting a witness node. In this setup data is written to both Availability Zones in an active active setup. In the event of an outage to an entire Availability Zone vSphere HA brings virtual machines back online in the alternative AZ: VMware Cloud on AWS Stretched Cluster Failover Demo.
  • Stretched clusters have an SLA Availability Commitment of 99.99% (99.9% for single AZ), and provide a Recovery Point Objective (RPO ) of zero by using synchronous data replication. Note that there are additional cross-AZ charges for stretched clusters. The Recovery Time Objective (RTO) is a vSphere HA failover, usually sub 5 minutes.
  • The decision on whether to use single or multiple Availability Zones needs to be taken at the time of deployment. An existing SDDC cannot be upgraded to multi-AZ or downgraded to a single AZ.
  • An Elastic Network Interface (ENI) dedicated to each physical host connects the VMware Cloud to the corresponding Availability Zone in the native AWS Virtual Private Cloud (VPC). There is no charge for data crossing the 25 Gbps ENI between the VMware Cloud VPC and the native AWS VPC.
  • Data that crosses Availability Zones is chargeable, therefore it is good practise to deploy the SDDC to the same region and AZ as your current or planned native AWS services.

3. Networks and Connectivity

  • VMware Cloud on AWS links with your existing AWS account to provide access to native services. During provisioning a Cloud Formation template will grant AWS permissions using the Identity Access Management (IAM) service. This allows your VMC account to create and manage Elastic Network Interfaces (ENI’s) as well as auto-populate Virtual Private Cloud (VPC) route tables when NSX subnets are created.
  • It is good practise to enable Multi-Factor Authentication (MFA) for your accounts in both VMC and AWS. VMware Cloud can also use Federated Identity Management, for example with Azure AD. This currently needs to be facilitated by your VMware Customer Success team, but once setup means you can control accounts using Active Directory and enforce MFA or follow your existing user account policies.
  • It is important to ensure proper planning of your IP addressing scheme, if the IP range used overlaps with anything on-premise or in AWS then routes will not be properly distributed and the SDDC needs destroying and reinstalling with an updated subnet to resolve.
  • You will need to allocate a CIDR block for SDDC management, as well as network segments for your SDDC compute workloads to use. Review Selecting IP Subnets for your SDDC for assistance with selecting IP subnets for your VMC environment.
  • Connectivity to the SDDC can be achieved using either AWS Direct Connect (DX) or VPN, see Connectivity Options for VMware Cloud on AWS Software Defined Data Centers. From SDDC v1.7 onwards it is possible to use DX with a backup VPN for resilience.
  • Traffic between VMC and your native AWS VPC is handled by the 25 Gbps Elastic Network Interfaces (ENI) referenced in the section above. To connect to additional VPCs or accounts you can setup an IPsec VPN. The Amazon Transit Gateway feature is available for some regions and configurations, if you are using DX then the minimum requirement is 1Gbps.
  • Access to native AWS services needs to be setup on the VMC Gateway Firewall, for example: Connecting VMware Cloud on AWS to EC2 Instances, as well as Amazon security groups; this is explained in How AWS Security Groups Work With VMware Cloud on AWS.
  • To migrate virtual machines from your on-premise data centre review Hybrid Linked Mode Prerequisites and vMotion across hybrid cloud: performance and best practices. In addition you will need to know the Required Firewall Rules for vMotion and for Cold Migration.
  • For virtual machines to keep the same IP addressing layer 2 networks can be stretched with HCX, review VMware HCX Documentation. HCX is included with VMC licensing but is a separate product in its own right so should be planned accordingly and is not covered in this post. Review VMware Cloud on AWS Live Migration Demo to see HCX in action.
  • VMware Cloud on AWS: Internet Access and Design Deep Dive is a useful resource for considering virtual machines that may require internet access.

4. Operational Readiness

The SDDC is deployed but before you can start migrating virtual machines you need to make sure the platform is fully operational. There are some key aspects but in general make sure you cover everything you do currently on premise:

  • You will likely still have a need for Active Directory, DNS, DHCP, and time synchronisation. Either use native cloud services, or build new Domain Controllers for example in VMC.
  • If you have a stretched-cluster and build Domain Controllers, or other management servers, consider building these components in each Availability Zone, then using compute policies to control the virtual machine placement. This is similar to anti-affinity rules on-premise, see VMware Cloud on AWS Compute Policies for more information.
  • Remember Disaster Recovery (DR) still needs to be factored in. DR as a Service (DRaaS) is offered through Site Recovery Manager (SRM) between regions in the cloud or on-premise. A stretched-cluster may be sufficient but again, this is dependent on the organisation or service requirements.
  • Anti-Virus, monitoring, and patching (OS / application) solutions need to be implemented. Depending on your licensing model you should be able to continue using the same products and tool-set, and carry the license over, but check with the appropriate vendor. Also start thinking about integrating cloud monitoring and management where applicable.
  • VMware Cloud Log Intelligence is a SaaS offering for log analytics, it can forward to an existing syslog solution or integrate with AWS CloudTrail.
  • Backups are still a crucial part of VMware Cloud on AWS and it is entirely the customers responsibility to ensure backups are in place. Unless you have a specific use case to backup machines from VMware Cloud to on-premise, it probably makes sense to move or implement backup tooling in the cloud, for example using Veeam in Native AWS.
  • Perform full backups initially to create a new baseline. Try native cloud backup products that will backup straight to S3, or continue with traditional backup methods that connect into vCenter. The reference architecture below uses Elastic Block Storage (EBS) backed Elastic Compute Cloud (EC2) instances running Veeam as a backup solution, then archiving out to Simple Storage Services (S3). Druva are able to backup straight to S3 from VMC. Veeam are also constantly updating functionality so as mentioned at the start of the post this setup may not stay up to date for long:


  • Customers must be aware of the shared security model that exists between: VMware; delivering the service, Amazon Web Services (the IaaS provider); delivering the underlying infrastructure, and customers; consuming the service.
  • VMware Cloud on AWS meets a number of security standards such as NIST, ISO, and CIS. You can review VMware’s security commitments in the VMware Cloud Services on AWS Security Overview.
  • When using native AWS services you must always follow Secure by Design principals to make sure you are not leaving the environment open or vulnerable to attack.

Part 2 of this post will cover the planning and migration of virtual machine workloads.

Additional resources: VMware Cloud On AWS On-Boarding Handbook | VMware Cloud on AWS Operating Principles | Resources | Documentation | Factbook

VMware Cloud on AWS VideosYouTube PlaylistsVMworld 2018 Recorded Sessions

VMware Cloud on AWS Deployment Demo

This opening post will give an overview and demo of VMware Cloud on AWS. VMware Cloud on AWS provides on-demand, scaleable cloud environments based on existing vSphere Software-Defined Data Centre (SDDC) products. VMware and AWS have worked together to optimise running vSphere, vSAN and NSX, directly on dedicated, elastic, bare-metal AWS infrastructure without the need for nested virtualization. A SDDC cloud can be deployed in a few hours and then capacity scaled up and down within minutes; either manually or automatically using elastic DRS.

Key Benefits

There are a number of benefits and use cases for extending on-premise data centres to the cloud with VMware Cloud on AWS:

  • VMware maintains software updates, emergency software patches, and auto-remediation of hardware failures
  • Increasing capacity in the cloud is generally quicker, easier, and sometimes more cost effective than increasing physical capacity in the data centre
  • Scale capacity to protect services when met with temporary or unplanned demand
  • Improve business continuity by using the cloud for Disaster Recovery (DR) with Site Recovery
  • Consistent operating environments allows for simplified cloud migrations with minimal re-training for system administrators
  • Transfer your existing operating system and third party licensing to the cloud and make use of existing support contracts with VMware
  • Expand into additional geographical locations without needing to provision new data centres

Key Details

Update 18/01/2019 – see also VMware Cloud on AWS Deployment Planning. As with all cloud services functionality and limitations are constantly changing, I have updated some of this content but make sure you review the links below for the most up to date information.

VMware FAQ | AWS FAQRoadmap | AWS Pricing

The following links contain enough reading to plan your VMware Cloud on AWS implementation and cloud migration strategy, the points below should also be enough to get you started.

Product Documentation | Technical Overview | VMware Product Page | AWS Product Page | | Case Study | Try first @ VMware Cloud on AWS – Getting Started Hands-on Lab

  • At the time of writing up to 2 SDDC’s can be deployed per organisation (soft limit), each SDDC supporting up to 20 vSphere clusters and each cluster up to 16 physical nodes.
  • The standard i3 bare metal instance currently offers 2 sockets, 36 cores, 512 GiB RAM, 10.7 TB vSAN storage, a 16-node cluster provides 32 sockets, 576 cores, 8192 GiB RAM, 171.2 TB.
  • New R5 bare metal instances are deployed with 2.5 GHz Intel Platinum 8000 series (Skylake-SP) processors; 2 sockets, 48 cores, 768 GiB RAM and AWS Elastic Block Storage (EBS) backed capacity scaling up to 105 TB for 3-node resources and 560 TB for 16-node resources. For up to date configuration maximums see Configuration Maximums for VMware Cloud on AWS.
  • Each ESXi host is connected to an Amazon Virtual Private Cloud (VPC) through Elastic Networking Interfaces (ENI’s), which supports throughput up to 25 Gbps
  • Hybrid Cloud Extension allows stretched subnets between on-premise and cloud data centres for live migration of virtual machines
  • Hybrid Linked Mode allows administrators to connect vCenter Server running in VMware Cloud on AWS to an on-premises vCenter server to view both cloud and on-premises resources from a single interface
  • VMware Cloud on AWS complies with ISO 27001, ISO 27017, ISO 27018, SOC 1, SOC 2, SOC 3, HIPAA, and GDPR, find the full list of compliance certification here
  • VMware Cloud on AWS is managed from a web-based console or RESTful API
  • At the time of writing VMware Cloud on AWS is available in the AWS Europe (Frankfurt and London), AWS US East (N. Virginia) and AWS US West (Oregon) Regions
  • Basic pricing before discount can be calculated here


Product Demo

The demo below creates a SDDC in the cloud for lab purposes. Before deploying your own environment you should review all the above linked documentation and do your own research to plan your cloud strategy as well as the following:

  • Identify or create an AWS account and ensure that all technical personnel have access to the account
  • Identify a VPC and subnet by cross-linking the AWS account to the SDDC
  • Allocate IP ranges for the SDDC, and determine a DNS strategy
  • Identify the authentication model for the SDDC
  • Plan connectivity to the SDDC
  • Develop a network security policy for the SDDC

Browse to the VMware Cloud Services portal ( and login using your VMware ID. At the time of writing to access VMware Cloud on AWS you need to be invited or you can register for a 30 day single host trial here.


Select VMware Cloud on AWS. If you have not used the service before you will be prompted to create a new organisation. Enter a name for your new organisation and accept the terms of service, click Continue.


Add a credit card to be billed if you use the service.


After you have created the organisation and added payment information you will be sent to the VMware Cloud on AWS dashboard. The first step is to create our SDDC in the cloud, click Create SDDC.

Billing: annual subscriptions are listed under the Subscriptions tab, you can see other billing information from the drop-down menu next to your organisation name: select Organisation Settings, View Organisation. From here you have services, identity and access management, billing and subscriptions, and support options.


Select a region and deployment model for the SDDC, enter a name and the number of hosts if you are not using the single host deployment. Click Next.


Follow the instructions to connect an AWS account and assign the relevant capabilities.


Once the connection is successfully established click Next.


Select the VPC and subnet to use then click Next.


Specify a private subnet range for the management subnet or leave blank to use default addressing. As mentioned above ensure you have planned accordingly and are not using any ranges that will conflict with other networks you may connect in the future. Click Deploy SDDC.


The SDDC will now be deployed, it takes around 2 hours to provision the ESXi hosts and all management components.


Once the deployment is complete the dashboard will show the new SDDC and assigned resources. Click View Details (you can toggle the web portal theme using the Dark/Light options in the top right hand corner).


From either the SDDC Summary tab or back on the SDDC dashboard you can seamlessly add additional hosts or clusters at any time.


If needed the chat bubble in the bottom right hand corner of the screen will take you through to support.


The Network tab shows the network topology and is where you can configure firewall rules, NAT rules, VPN, Direct Connect, etc.


To access the vCenter Server through the vSphere client the port needs opening, a VPN can also be used. Under Management Gateway select Firewall Rules, click Add Rule. Configure the rule to allow access to the vCenter on port 443 and click Save.


Click Open vCenter from either the Summary or Network tab, if access is in place you are given the cloudadmin@vmc.local credentials to open vCenter. Active Directory can also be configured as an identity source later on.

Once you are logged into the vSphere client you will see the familiar vSphere layout.


It is also possible to see your on-premise vCenter Server(s) in the same pane of glass using Hybrid Linked Mode, click here for more information.

Back in the VMware Cloud on AWS portal the Add Ons tab features Site Recovery and Hybrid Cloud Extension for protecting and migrating workloads to your SDDC in the cloud.


You can delete a SDDC from the Actions drop-down menu in either the SDDC Summary tab or the SDDC dashboard. Once a SDDC is deleted all workloads, data, and interfaces are destroyed and any public IP addresses released.