VMware Cloud on AWS Outposts Overview

Introduction

Managed and as-a-service models are a growing trend across infrastructure consumers. Customers in general want ease and consistency within both IT and finance, for example opting to shift towards OpEx funding models.

For large or enterprise organisations with significant investments in existing technologies, processes, and skills, refactoring everything into cloud native services can be complex and expensive. For these types of environments the strategy has sharpened from Cloud-First to Cloud-Smart. A Cloud-Smart approach enables customers to transition to the cloud quickly where it makes sense to do so, without tearing up roots on existing live services, and workloads or data that do not have a natural progression to traditional cloud.

In addition to the operational complexities of rearchitecting services, many industries have strict regulatory and compliance rules that must be adhered to. Customers may have specific security standards or customised policies requiring sensitive data to be located on-premises, under their own physical control. Applications may also have low latency requirements or the need to be located in close proximity to data processing or back end systems. This is where VMware Local Cloud as a Service (LCaaS) can help combine the key benefits from both public cloud and on-premises environments.

What is VMware Cloud on AWS Outposts?

VMware Cloud on AWS Outposts is a jointly engineered solution, bringing AWS hardware and the VMware Software Defined Data Centre (SDDC) to the customer premises. The relationship with AWS is VMware’s longest standing hyperscaler partnership; with VMware Cloud on AWS the maturest of the multi-cloud offerings from VMware, having been available since August 2017. In October 2021, at VMworld, VMware announced general availability of VMware Cloud on AWS Outposts.

VMware Cloud on AWS Outposts is a fully managed service, as if it were in an AWS location, with consistent APIs. It is built on the same AWS-designed bare metal infrastructure using the AWS Nitro System, assembled into a dedicated rack, and then installed in the customer site ready to be plumbed into power and networking. The term Outpost is a logical construct that is used to pool capacity from 1 or more racks of servers.

The VMware SSDDC overlay, and hardware underlay, comprises of:

  • VMware vSphere and vCenter for compute virtualisation and management
  • VMware vSAN for storage virtualisation
  • VMware NSX-T for network virtualisation
  • VMware HCX for live migration of virtual machines with stretched Layer 2 capability
  • 3-8 AWS managed dedicated Nitro-based i3.en metal EC2 instances with local SSD storage
  • Non-chargeable standby node in each rack for service continuity
  • Fully assembled standard 42U rack
  • Redundant Top of Rack (ToR) data plane switches
  • Redundant power conversion unit and DC distribution system (with support for redundant power feeds)

At the time of writing the i3.en metal is the only node type available with VMware Cloud on AWS Outposts. The node specification is as follows:

  • 48 physical CPU cores, with hyperthreading enabled delivering 96 logical cores
  • 768 GiB RAM
  • 45.84 TiB (50 TB) raw capacity per host, delivering up to 40.35 TiB of usable storage capacity per host depending on RAID and FTT configuration

Both scale-out and multi-rack capabilities are currently not available, but are expected. It is also expected that the maximum node count will increase over time, check with your VMware or AWS teams for the most up to date information.

Once the rack is installed on-site, the customer is responsible for power, connectivity into the LAN, and environmental prerequisites such as temperature, humidity, and airflow. The customer is also responsible for the physical security of the Outpost location, however each rack has a lockable door and tamper detection features. Each server is protected by a removable and destroyable Nitro hardware security key. Data on the Outpost is encrypted both at-rest, and in-transit between nodes in the Outpost and back to the AWS region.

Inside the rack, all the hardware is managed and maintained by AWS and VMware, this includes things like firmware updates and failure replacements. VMware are the single support contact for the service regardless of whether the issue is hardware or software related. Additionally, VMware take on the lifecycle management of the full SDDC stack. Customers can run virtual machines using familiar tooling without having to worry about vSphere, vSAN, and NSX upgrades or security patches. Included in the cost ‘per node’ is all hardware within the rack, the VMware SDDC licensing, and the managed service and support.

Existing vCenter environments running vSphere 6.5 or later can be connected in Hybrid Linked Mode for ease of management. Unfortunately for consumers of Microsoft licensing, such as Windows and SQL, Outposts are still treated as AWS cloud infrastructure (in other words not customer on-premises).

Why VMware Cloud on AWS Outposts?

VMware Cloud on AWS Outposts provides a standardised platform with built-in availability and resiliency, continuous lifecycle management, proactive monitoring, and enhanced security. VMware Cloud on AWS delivers a developer ready infrastructure that can now be stood up in both AWS and customer locations in a matter of weeks. Using VMware Cloud on AWS, virtual machines can be moved bi-directionally across environments without the need for application refactoring or conversion.

The initial use case for VMware Cloud on AWS Outposts is existing VMware or VMware Cloud on AWS customers with workloads that must remain on-premises. This could be for regulatory and compliance reasons, or app/data proximity and latency requirements. As features and configurations start to scale, further use cases will no doubt become more prominent.

You can also use other AWS services with Outposts, however you have to make a decision on a per-rack basis whether you are running VMware Cloud on AWS for that rack, or native AWS services. The deployment of the rack is dedicated to one or the other.

VMware Cloud on AWS Outposts Network Connectivity

VMware Cloud on AWS Outposts requires a connection back to a parent VMware Cloud on AWS supported region, or more specifically an availability zone. Conceptually, you can think of the physical VMware Cloud on AWS Outposts installation as an extension of that availability zone. The connection back to AWS is used for the VMware Cloud control plane, also known as the service link.

The service link needs to be a minimum of 1Gbps with a maximum 150ms latency, either using a Direct Connect, or over the public internet using a VPN. Public Amazon Elastic IPs are used for the service link endpoint. Although the VMware Cloud on AWS Outposts service is not designed to operate in environments with limited or no connectivity, in the event of a service link outage the local SDDC will continue functioning as normal. This includes vCenter access and VM operations. A service link outage will prevent monitoring and access to configurations or other functionality from the VMware Cloud portal.

There is no charge for data transfer from VMware Cloud on AWS Outposts back to the connected region. Data transfer from the parent availability zone to the VMware Cloud on AWS Outposts environment will incur the standard AWS inter-AZ VPC data transfer charges.

Customers can use the connected VPC in the customer managed AWS account to access native AWS services in the cloud, either using the Elastic Network Interface (ENI) or VMware Transit Connect.

The Local Gateway (LGW) is an Outposts-specific logical construct used to route traffic to and from the existing on-premises network. This traffic stays within the local site allowing for optimised traffic flow and low latency communication. There is no data transfer cost for data traversing the LGW, either out to the internet or to your local network.

For more information on network connectivity and VMware Cloud on AWS Outposts in general, take a look at the AWS re:Invent 2021 session – A technical deep dive on VMware Cloud on AWS Outposts.

VMware Cloud on AWS Outposts LGW example

Getting Started with VMware Cloud on AWS Outposts

You can view a demo of the steps in the VMware Cloud on AWS Outposts: Order Flow video. At a high level, the process is as follows:

  • Extensive workshops are carried out between VMware and/or AWS and the customer
  • If the customer is a new VMware Cloud customer then a new org is created with a unique org ID
    • Customer pre-req: a VMware Cloud account and org is required
  • The customer receives an invite to join the VMware Cloud on AWS Outposts service through email
  • The customer places an order via the VMware Cloud console
    • Customer pre-req: customer AWS account with VPC and dedicated subnet, if using a private VIF for Direct Connect, then the VIF should already be created in the customer AWS account
    • Customer pre-req: knowledge of the facility, power, and network setup*
    • Customer pre-req: knowledge of desired instance count and configuration
  • The customer receives and responds to the request to submit logical networking information
    • This information will be gathered during the customer workshop, the service link requires a dedicated VLAN and /26 subnet, the SDDC management network requires a dedicated /23 minimum, and an additional CIDR block needs allocating for compute networks
  • AWS schedule and carry out a site survey
  • AWS builds and delivers the rack
  • Final onsite provisioning is carried out by AWS and validated by VMware
  • VMware notify the customer the environment is ready to use
  • The SDDC is provisioned through automated workflows as instructed by the customer

*full details of the facility, power, and network requirements for the local site can be found in the AWS Outposts requirements page

The VMware Cloud on AWS Outposts solution brief provides more information, and you can find an overview, pricing, and FAQ section on the VMware Cloud on AWS Outposts solution page. AWS also have their own version of the VMware Cloud on AWS Outposts FAQ page.

Another great place to get started is the VMware Cloud Tech Zone, and for AWS specifically the VMware Cloud on AWS Tech Zone.

VMware Cloud on AWS Tech Zone

How CloudHealth Optimises and Secures Your Cloud Assets

Introduction

Over the past 12 months we have seen further growth within the cloud, as many organisations scale or create new digital services in response to the coronavirus pandemic. Improved speed and agility has allowed businesses to pivot where traditional siloed infrastructure may have caused them to stall.

As the usage of cloud services expands, standardising and consolidating cloud tooling becomes important for financial management, operational governance, and security and compliance. Visibility into distributed system architectures across many accounts or subscriptions, or even multi-cloud, is another key challenge. For some customers cloud workloads are not optimised or configured to best standards, many will spend more than their anticipated budget, and others may accidentally expose data or services.

Those with an established cloud strategy may decide to implement a Cloud Centre of Excellence (CCoE); responsible for cloud operations, security, and financial management. The CCoE will navigate the security and configuration landscape of cloud assets, automating response and remediation to configuration drift or threats. As the team grows in maturity optimisations are made continuously and automatically, inline with the key drivers of the business. This is where CloudHealth comes in.

CloudHealth by VMware is a multi-cloud SaaS solution managing more than $11B of public cloud spend for over 10,000 customers. CloudHealth accelerates business transformation in the cloud by providing a single platform solution for visibility into AWS, Microsoft Azure, Google Cloud Platform, Oracle Cloud Infrastructure, VMware Cloud on AWS, and on-premises VMware based environments. The key functionality is broken down into the 2 products we’ll look at below.

CloudHealth Multicloud Platform

CloudHealth takes data from cloud platforms, data centres, and third party tools for application, security, and configuration management. Data is ingested and aggregated using CloudHealth’s integrated data layer, which performs analysis on usage, performance, cost, and security posture. CloudHealth becomes a single source for multi-cloud management across environments, strengthening security and compliance, consolidating management, and improving collaboration between previously siloed teams of people and tools.

Data and assets can be categorised by tags or other metadata, and viewed in logical business groups known as perspectives . Perspectives provide a breakdown for cost allocation using dynamic groups such as line of business, department, cost centre, or project. The output can be used to identify trends and build dashboards and reports. This approach simplifies financial management, saves time, aids with budgeting and forecasting, and encourages accountability through accurate chargeback or showback.

CloudHealth Cost Dashboard

Whilst visibility is great, to really have a positive impact on operations we need to know what to do with the data collected. CloudHealth presents back cost optimisation recommendations and security risks, but can also carry out remediation actions automatically.

Cost optimisation is where you can save money, using AWS as an example, based on things like; EC2 instances that are oversized or on an inefficient purchase plan, elastic IP addresses or EBS volumes that are not attached to any resources, snapshots that have not been deleted. In the physical on-premises world all of these issues were common as part of VM sprawl, they impacted capacity planning and resource consumption but were mostly hidden or swallowed as part of the wider infrastructure cost. As organisations shift from large capital investments to ongoing revenue and consumption based pricing, oversized or unused resources literally convert to money going out of the door every single month.

CloudHealth Health Check

Recommendations and actions are where CloudHealth carries out remediation for incorrectly configured or under-utilised resources. Policies can also be used to define desired states and ensure operational compliance. For example, an organisation may want to report on untagged resources, connected accounts, or open ports. The number of available actions currently appears to only cover AWS and Azure, but with support recently added for Oracle Cloud Infrastructure, and Google Cloud Platform before that, hopefully this functionality will continue to be built out.

CloudHealth Remediation Actions

At the time of writing CloudHealth is priced based on cloud spend, and can be purchased as a 1, 2, or 3 year prepaid commitment, or variable pricing based on the previous months cloud spend. A free trial is available to uncover ROI in your own environment from CloudHealth here.

Where VMware environments are in use with vRealize Operations, the CloudHealth management pack for vRealize Operations can be installed. Bringing CloudHealth dashboards and prospects into vROps allows IT ops teams to track on-premises infrastructure and public cloud costs from a single interface. The CloudHealth management pack for vROps can be downloaded from the VMware Marketplace, instructions are here.

CloudHealth Secure State

By default CloudHealth provides real-time information on security risk exposure, but for deep-dive visibility and remediation those who are serious about security will want to look at Secure State. CloudHealth Secure State is available with CloudHealth or standalone, and currently supports AWS, Azure, and GCP.

Dashboards within CloudHealth Secure State enable at-a-glance checks on security posture and compliance. There are over 700 built-in security rules and compliance frameworks that can be used as security guardrails, with the ability to add custom rules and frameworks on top.

As systems become distributed over multiple accounts, subscriptions, or even clouds, the dynamics of securing an organisations assets shift significantly. Previously all services were contained within a data centre, firstly using perimeter firewalls and then with micro-segmentation. IT teams were generally in control and had visibility throughout the corporate network. Nowadays a developer or user responsible for a service can potentially open applications or data to the public, either on purpose or by accident. Cloud security guardrails form an important baseline for security posture and cloud strategy. Security guardrails are made up of critical must-have configurations in policies with auto-remediation actions attached, they help avoid mistakes or configuration drift to ultimately reduce security risk.

CloudHealth Secure State gives further visibility into resource relationships and context, using the Explore UI. Explore enables a powerful model of multi-cloud or account architectures, with visual topology diagrams of complex environments. Cyber security analysts or operations centres can drill down into individual resources with all interoperable components and dependencies already mapped out.

CloudHealth Secure State Dashboard
CloudHealth Secure State Compliance

Featured image by Scott Webb on Unsplash

AWS Native Services Integration With VMware Cloud on AWS

This post lists some of the available options for connecting VMware Cloud on Amazon Web Services (AWS) with native AWS services for hybrid cloud deployments. Read more about multi-account and VPC management at Building AWS Environments for VMware Cloud Customers.

The most common way of consuming AWS services with VMware Cloud on AWS is to use the built-in Elastic Network Interface (ENI) functionality. Each VMware Cloud Software-Defined Data Center (SDDC) can be connected to another AWS Virtual Private Cloud (VPC) during the deployment phase. A VPC is Amazon’s logical separation of virtual networks. At scale, you may choose to have many VPCs and many accounts for different applications and environments. Multiple VPCs can be connected together using an AWS Transit Gateway (TGW). A further option we will look at is VPC Endpoints, enabling you to privately connect to supported AWS services and endpoints.

1. Connected VPC

The AWS bare metal hosts deployed for VMware Cloud on AWS use a redundant 25 Gbps physical interface or Elastic Network Adaptor (ENA). The physical interface uses a trunk port to carry multiple VLANs for services like management, vMotion, NSX, and connectivity to the AWS backbone network.

The cross-linked VPC architecture is provided by a series of ENIs. Each host in the vSphere cluster uses the network adaptor outlined above to provide an individual cross-VPC ENI per physical host; supporting high-bandwidth, low latency connectivity to native AWS services.

VMC_Host_Connectivity

VMware Cloud on AWS uses the AWS VPC  as an underlay for NSX-T. The NSX Edge (Tier 0) router is a virtual router acting as the uplink to the connected VPC. The active ENI in use is the physical ESXi host where the virtual router is running. The connected VPC is owned and managed by the customer, any native services deployed are billed separately by AWS. When deploying the SDDC, the connected account and VPC is required along with a private subnet in each applicable Availability Zone (AZ). A static route is created for the defined subnets adding the connected VPC router as the next hop.

Traffic that traverses the ENI is not chargeable; however, cross-AZ charges do need to be taken into consideration if a Stretched Cluster is in use. During provisioning of the SDDC, and connection of the customer-managed AWS account, a CloudFormation template is deployed creating the necessary AWS Identity Access Management (IAM) roles and ENI configuration.

SDDC_AWS_1SDDC_AWS_2

Following the SDDC deployment, you can view the connected account, VPC, ENI, and subnet details in the Connected VPC menu under the Networking & Security tab of the SDDC, from the VMware Cloud Services Portal.

Access to and from native AWS services can be controlled and needs to be opened, using the NSX firewalls (gateway and distributed) and AWS Security Groups. To see an example configuration see the Connecting VMware Cloud on AWS to Amazon EC2 post or the Access an EC2 Instance section of the VMware Cloud on AWS Docs page.

2. VPC Endpoints

VPC Endpoints allow private connectivity between your VPC and supported AWS services or custom applications. Network traffic traversing a VPC endpoint does not leave the AWS backbone network and therefore does not require Internet Gateway, Direct Connect, or VPN.

The Access an S3 Bucket Using an S3 Endpoint section of the VMware Cloud on AWS Docs page details the process for configuring a Gateway VPC Endpoint to access AWS Simple Storage Services (S3) from VMware Cloud on AWS, without having to go out to the Internet. Furthermore, you can use Interface VPC Endpoints to connect to supported AWS services in another VPC, or VPC Endpoint Services (AWS PrivateLink) to connect to custom applications in another VPC. Here are some examples:

AWS_Integration_Examples

The general process for creating an endpoint is the same across these VPC Endpoint types. In the example below, we are connecting to a VPC Endpoint Service for Splunk, fronted by a Network Load Balancer (NLB) in another VPC. The administrator of the VPC Endpoint Service needs to grant IAM service consumer permissions and accept the incoming connection, as detailed in the AWS documentation here.

In the AWS console, I log into the connected account and select the VPC service. I choose Endpoints and Create Endpoint. To create a Gateway VPC Endpoint, e.g. for S3, or an Interface VPC Endpoint, e.g. for DynamoDB or other services, I would select the appropriate service from the AWS services service category. In this instance, I use Find service by name and enter the endpoint private service name. Either way, the key point is that I select the connected VPC from the VPC drop-down and the subnets that match up with those used for the ENI when deploying the VMware Cloud on AWS SDDC.

VPCEndpoint1VPCEndpoint1cont

By using the cross-VPC linked subnets the Virtual Machines in the SDDC will utilise the static route across the ENI outlined in the Connected VPC section above. AWS Security Groups can be used to limit this to certain source IP addresses from within the SDDC or the wider VPC if required. In this instance, we can successfully test the connection over port 443 following the creation of the VPC Endpoint.

3. Additional VPC Connectivity

Traditionally VPC Peering has been used to provide one to one private network connectivity between VPCs, including VPCs in different accounts. VPC Peering cannot be configured in the SDDC as we do not have access to the underlying AWS account. VPN connections between additional VPCs and the SDDC router (Tier 0) can be configured from the VMware Cloud Services Portal, enabling VMware Cloud on AWS connectivity with other VPC environments. As the number of VPCs and accounts begins to scale the VPN approach becomes harder to manage.

This predicament is resolved with a relatively new addition to AWS; the Transit Gateway (TGW). The native AWS TGW is available now and acts as a transit network hub allowing you to connect multiple VPCs and on-premise networks (using Direct Connect or VPN attachments). A Managed Transit Gateway is being developed by VMware to assist with multi-SDDC and multi-VPC connectivity. You can review how the native AWS Transit Gateway fits into the VMware Cloud on AWS architecture on the VMware Network Virtualization blog: VMware Cloud on AWS with Transit Gateway Demo:

VMC_TGW_Examples

Image VMware Cloud on AWS with Transit Gateway Demo. Further Resources: