The uptake and importance of both cloud and data services has increased significantly over recent years. Since VMware Cloud on AWS was first introduced in 2017, the customer base has followed a similar trajectory, with new use cases being uncovered. As a result, customer expectations have also risen. Consumers now want elastic compute; the ability to scale compute independent of storage, elastic performance; the ability to scale IOPS independent of capacity, and elastic capacity; the ability to scale capacity independent of IOPS.
The very nature of VMware Cloud on AWS Hyper-Converged Infrastructure (HCI) nodes has afforded its users greater consolidation and compression ratios, and more flexibility around virtual machine sizing. The first node type introduced was the i3.metal with around 10.37 TiB raw storage, all based on local host NVMe SSD devices. VMware then experimented with an r5.metal node type, offering elastic vSAN backed by the automatic provisioning and management of AWS EBS volumes. The r5 was decommissioned shortly after, and the i3 node was joined by the i3en.metal. The i3enhanced boasted a more powerful specification, including 45.84 TiB of raw capacity per host.
Whilst the i3en.metal node boosted the available storage capacity, it boosted compute with it. For large environments the power of the i3en node is great, but there was still a gap for those environments which were really heavily skewed in the direction of storage consumption, without the need for matching compute overhead.
What is VMware Cloud Flex Storage?
Enter VMware Cloud Flex Storage. Announced today, March 29; a fully VMware-managed and integrated cloud storage service for VMware Cloud on AWS.
VMware Cloud Flex Storage provides disaggregated and elastic storage, catering for capacity-heavy workloads and lower performance needs. By augmenting vSAN as supplemental storage, customers can scale out attached storage without the growth in compute capacity.
NFS datastores of up to 400 TB and up to 150K IOPS can be attached to an SDDC cluster, comprising of either i3 or i3en nodes. Multiple datastores can be attached to a cluster, and a datastore can also be shared across clusters, enabling an initial target into the petabytes. As with the vSAN datastore, data is encrypted at rest by default. Customers can choose between a pay-as-you-go consumption model, or an optional subscription term with a minimum capacity buy in.
VMware Cloud Flex Storage Overview
How Does VMware Cloud Flex Storage Work?
The traditional block storage used by vSAN in the hyper-converged nodes is designed for latency sensitive applications, it handles random transactional workloads well, and is simple to manage or consume. As outlined above, HCI block storage scales per host, as does the cost.
In the back end, VMware Cloud Flex Storage uses AWS S3 with a VMware-managed front end. Object based storage is typically not for latency sensitive workloads, however the additional architecture built around the file system adds caching and I/O optimisation. The scale out file system itself it built on the same technology as VMware Cloud Disaster Recovery. Object storage is also highly durable, resilient across availability zones, low cost, and allows for more granular scaling of capacity, whilst only paying for consumption.
VMware Cloud Flex Storage mounts directly to the hosts using NFS. It resides in a separate VPC maintained by VMware, but connects in using a cross-VPC Elastic Network Interface (ENI). Writes are synchronously written to a HA pair of back end nodes, before being moved into thin-provisioned cloud object storage. There are no customer managed VMs, or cloud gateway type connectivity to worry about. Furthermore, existing vCenter and vROps tooling will be able to monitor performance, latency, IOPS, and availability.
VMware Cloud Flex Storage High Level Architecture
How Do I Access VMware Cloud Flex Storage?
Customers can apply for the early access program by emailing vmcfs_ea at vmware.com. Those accepted onto the program will be provided with a VMware Cloud on AWS SDDC with VMware Cloud Flex Storage for testing at no cost. Although there has been no official date published for general availability, more information can be found on todays blog post Announcing Preview of VMware Cloud Flex Storage.
Managed and as-a-service models are a growing trend across infrastructure consumers. Customers in general want ease and consistency within both IT and finance, for example opting to shift towards OpEx funding models.
For large or enterprise organisations with significant investments in existing technologies, processes, and skills, refactoring everything into cloud native services can be complex and expensive. For these types of environments the strategy has sharpened from Cloud-First to Cloud-Smart. A Cloud-Smart approach enables customers to transition to the cloud quickly where it makes sense to do so, without tearing up roots on existing live services, and workloads or data that do not have a natural progression to traditional cloud.
In addition to the operational complexities of rearchitecting services, many industries have strict regulatory and compliance rules that must be adhered to. Customers may have specific security standards or customised policies requiring sensitive data to be located on-premises, under their own physical control. Applications may also have low latency requirements or the need to be located in close proximity to data processing or back end systems. This is where VMware Local Cloud as a Service (LCaaS) can help combine the key benefits from both public cloud and on-premises environments.
What is VMware Cloud on AWS Outposts?
VMware Cloud on AWS Outposts is a jointly engineered solution, bringing AWS hardware and the VMware Software Defined Data Centre (SDDC) to the customer premises. The relationship with AWS is VMware’s longest standing hyperscaler partnership; with VMware Cloud on AWS the maturest of the multi-cloud offerings from VMware, having been available since August 2017. In October 2021, at VMworld, VMware announced general availability of VMware Cloud on AWS Outposts.
VMware Cloud on AWS Outposts is a fully managed service, as if it were in an AWS location, with consistent APIs. It is built on the same AWS-designed bare metal infrastructure using the AWS Nitro System, assembled into a dedicated rack, and then installed in the customer site ready to be plumbed into power and networking. The term Outpost is a logical construct that is used to pool capacity from 1 or more racks of servers.
The VMware SSDDC overlay, and hardware underlay, comprises of:
VMware vSphere and vCenter for compute virtualisation and management
VMware vSAN for storage virtualisation
VMware NSX-T for network virtualisation
VMware HCX for live migration of virtual machines with stretched Layer 2 capability
3-8 AWS managed dedicated Nitro-based i3.en metal EC2 instances with local SSD storage
Non-chargeable standby node in each rack for service continuity
Fully assembled standard 42U rack
Redundant Top of Rack (ToR) data plane switches
Redundant power conversion unit and DC distribution system (with support for redundant power feeds)
At the time of writing the i3.en metal is the only node type available with VMware Cloud on AWS Outposts. The node specification is as follows:
48 physical CPU cores, with hyperthreading enabled delivering 96 logical cores
768 GiB RAM
45.84 TiB (50 TB) raw capacity per host, delivering up to 40.35 TiB of usable storage capacity per host depending on RAID and FTT configuration
Both scale-out and multi-rack capabilities are currently not available, but are expected. It is also expected that the maximum node count will increase over time, check with your VMware or AWS teams for the most up to date information.
Once the rack is installed on-site, the customer is responsible for power, connectivity into the LAN, and environmental prerequisites such as temperature, humidity, and airflow. The customer is also responsible for the physical security of the Outpost location, however each rack has a lockable door and tamper detection features. Each server is protected by a removable and destroyable Nitro hardware security key. Data on the Outpost is encrypted both at-rest, and in-transit between nodes in the Outpost and back to the AWS region.
Inside the rack, all the hardware is managed and maintained by AWS and VMware, this includes things like firmware updates and failure replacements. VMware are the single support contact for the service regardless of whether the issue is hardware or software related. Additionally, VMware take on the lifecycle management of the full SDDC stack. Customers can run virtual machines using familiar tooling without having to worry about vSphere, vSAN, and NSX upgrades or security patches. Included in the cost ‘per node’ is all hardware within the rack, the VMware SDDC licensing, and the managed service and support.
Existing vCenter environments running vSphere 6.5 or later can be connected in Hybrid Linked Mode for ease of management. Unfortunately for consumers of Microsoft licensing, such as Windows and SQL, Outposts are still treated as AWS cloud infrastructure (in other words not customer on-premises).
Why VMware Cloud on AWS Outposts?
VMware Cloud on AWS Outposts provides a standardised platform with built-in availability and resiliency, continuous lifecycle management, proactive monitoring, and enhanced security. VMware Cloud on AWS delivers a developer ready infrastructure that can now be stood up in both AWS and customer locations in a matter of weeks. Using VMware Cloud on AWS, virtual machines can be moved bi-directionally across environments without the need for application refactoring or conversion.
The initial use case for VMware Cloud on AWS Outposts is existing VMware or VMware Cloud on AWS customers with workloads that must remain on-premises. This could be for regulatory and compliance reasons, or app/data proximity and latency requirements. As features and configurations start to scale, further use cases will no doubt become more prominent.
You can also use other AWS services with Outposts, however you have to make a decision on a per-rack basis whether you are running VMware Cloud on AWS for that rack, or native AWS services. The deployment of the rack is dedicated to one or the other.
VMware Cloud on AWS Outposts Network Connectivity
VMware Cloud on AWS Outposts requires a connection back to a parent VMware Cloud on AWS supported region, or more specifically an availability zone. Conceptually, you can think of the physical VMware Cloud on AWS Outposts installation as an extension of that availability zone. The connection back to AWS is used for the VMware Cloud control plane, also known as the service link.
The service link needs to be a minimum of 1Gbps with a maximum 150ms latency, either using a Direct Connect, or over the public internet using a VPN. Public Amazon Elastic IPs are used for the service link endpoint. Although the VMware Cloud on AWS Outposts service is not designed to operate in environments with limited or no connectivity, in the event of a service link outage the local SDDC will continue functioning as normal. This includes vCenter access and VM operations. A service link outage will prevent monitoring and access to configurations or other functionality from the VMware Cloud portal.
There is no charge for data transfer from VMware Cloud on AWS Outposts back to the connected region. Data transfer from the parent availability zone to the VMware Cloud on AWS Outposts environment will incur the standard AWS inter-AZ VPC data transfer charges.
Customers can use the connected VPC in the customer managed AWS account to access native AWS services in the cloud, either using the Elastic Network Interface (ENI) or VMware Transit Connect.
The Local Gateway (LGW) is an Outposts-specific logical construct used to route traffic to and from the existing on-premises network. This traffic stays within the local site allowing for optimised traffic flow and low latency communication. There is no data transfer cost for data traversing the LGW, either out to the internet or to your local network.
Extensive workshops are carried out between VMware and/or AWS and the customer
If the customer is a new VMware Cloud customer then a new org is created with a unique org ID
Customer pre-req: a VMware Cloud account and org is required
The customer receives an invite to join the VMware Cloud on AWS Outposts service through email
The customer places an order via the VMware Cloud console
Customer pre-req: customer AWS account with VPC and dedicated subnet, if using a private VIF for Direct Connect, then the VIF should already be created in the customer AWS account
Customer pre-req: knowledge of the facility, power, and network setup*
Customer pre-req: knowledge of desired instance count and configuration
The customer receives and responds to the request to submit logical networking information
This information will be gathered during the customer workshop, the service link requires a dedicated VLAN and /26 subnet, the SDDC management network requires a dedicated /23 minimum, and an additional CIDR block needs allocating for compute networks
AWS schedule and carry out a site survey
AWS builds and delivers the rack
Final onsite provisioning is carried out by AWS and validated by VMware
VMware notify the customer the environment is ready to use
The SDDC is provisioned through automated workflows as instructed by the customer
*full details of the facility, power, and network requirements for the local site can be found in the AWS Outposts requirements page
Managed and as-a-service models are a growing trend across infrastructure consumers. Customers in general want ease and consistency within both IT and finance, for example opting to shift towards OpEx funding models.
For large or enterprise organisations with significant investments in existing technologies, processes, and skills, refactoring everything into cloud native services can be complex and expensive. For these types of environments the strategy has sharpened from Cloud-First to Cloud-Smart. A Cloud-Smart approach enables customers to transition to the cloud quickly where it makes sense to do so, without tearing up roots on existing live services, and workloads or data that do not have a natural progression to traditional cloud.
In addition to the operational complexities of rearchitecting services, many industries have strict regulatory and compliance rules that must be adhered to. Customers may have specific security standards or customised policies requiring sensitive data to be located on-premises, under their own physical control. Applications may also have low latency requirements or the need to be located in close proximity to data processing or back end systems. This is where VMware Local Cloud as a Service (LCaaS) can help combine the key benefits from both public cloud and on-premises environments.
What is VMware Cloud on Dell EMC?
VMware Cloud on Dell EMC is a fully managed Infrastructure-as-a-Service (IaaS) local-cloud deployment. A dedicated rack with all supporting hardware and equipment is wheeled into the customer site where it is maintained directly by VMware Site Reliability Engineering (SRE). The customer provides the physical location for the rack to sit, the power source, and the existing network for the data plane switches to plug into.
VMware Cloud on Dell EMC delivers a fully integrated software and hardware stack, jointly engineered by VMware and Dell EMC.
VMware Cloud on Dell EMC Overview
The VMware Software Defined Data Centre (SDDC) overlay, and hardware underlay, comprises of:
VMware vSphere and vCenter for compute virtualisation and management
VMware vSAN for storage virtualisation
VMware NSX-T for network virtualisation
VMware HCX for live migration of virtual machines with stretched Layer 2 capability
3-26 Dell VxRail Hyper-Converged Infrastructure (HCI) nodes per full-height rack (and currently up to 3 racks per SDDC)
1 non-chargeable standby VxRail node per rack for service continuity
Redundant Power Distribution Units (PDUs)
Uninterruptible Power Supply (UPS) for half-height rack configurations
Redundant Top of Rack (ToR) data plane switches
Redundant VMware SD-WAN appliances for remote management
All of this is delivered in a dedicated rack, as a fully managed service, with a single point of support directly with VMware. VMware SRE will take care of updating and maintaining all components of the software overlay, firmware updates, and management or repair of the underlying hardware. The customer maintains responsibility for the virtual machines they run on the infrastructure, plus configuration like network and storage policies. Let’s take a deeper dive.. you can also find out more from the VMware Cloud on Dell EMC product page, or the VMware Cloud on Dell EMC Solution Overview Brief.
VMware Cloud on Dell EMC can be used in any location the customer has authority to land equipment into. A site survey needs to be carried out before kit is shipped and installed. VMware is the single point of contact for support (unless you are purchasing through Dell APEX, more on that at the end of this post). For support issues that require an on-site fix, a Dell engineer will attend, but VMware will manage that support case directly. The subscription price per-node is inclusive of all hardware, software, licensing, support, and services, outlined in the graphic below.
VMware Cloud on Dell EMC What’s Included
The VMware SRE boundary ends at the LAN link into the customers network (beyond the ToR switches), VMware teams have no access beyond this point. Equally, the customer boundary ends at the LAN link between the SDDC and the VeloCloud Edge devices in the rack. The VeloCloud Edge devices provide connectivity over VMware’s SD-WAN using a secure IPSEC tunnel, and will need outbound connectivity on ports TCP 443 and UDP 2426.
There are multiple security processes in place to protect against unauthorised access. For example, in order to access a customer environment, a support engineer must generate one-time, time-sensitive credentials, which require a support case to be raised in the system. All activity is logged and monitored by VMware’s Cyber Security Operations Centre (CSOC), and can also be logged into a similar customer setup. Further references and information can be found in the VMware Cloud on Dell EMC Shared Responsibility Model Overview.
VMware Cloud on Dell EMC hosts come in standardised ‘T-Shirt’ sizes to optimise CPU, memory, and storage resources. Currently there are 6 different node sizes from extra small through extra large. You can find full specifications of the node sizes and rack types in the VMware Cloud on Dell EMC Service Data Sheet. Here is a quick run down of the sizing naming convention:
VMware Cloud on Dell EMC Node Sizing Guide
Why VMware Cloud on Dell EMC?
You’ll see me advocate public cloud a lot on this blog, but on-premises infrastructure often has its use cases. Data sovereignty, regulatory and compliance, workload to data proximity, latency requirements, local control, and existing investments all spring to mind. Running infrastructure at the edge is also becoming more prominent and overlaps with some of these use cases. As systems are more distributed, and consumers have more choice, there are many benefits in creating consistent application, infrastructure, and operating experiences across private cloud, public cloud, and edge locations.
VMware Cloud on Dell EMC benefits from a cloud operating and delivery model, whilst being classed as an on-premises service. This means that regulatory and data sovereignty requirements can be satisfied as all customer data is held on the local hardware. The VMware SD-WAN appliances and VMware Cloud portal are only used for management, without any further access into the customers network. VI admins continue to use vCenter Server as normal to manage virtual machines, however they no longer need to worry about maintaining the underlying infrastructure. IT teams now benefit from a managed service operating model with a predictable subscription-based monthly or annual outgoing, without the hardware ownership depreciation and management overhead.
VMware Cloud on Dell EMC Use Cases
A great use case for VMware Cloud on Dell EMC is VDI. Whether or not you have data or application proximity requirements, the Hyper-Converged Infrastructure (HCI) and node size configurations fit exceptionally well with virtual desktops utilising hyper-threading and instant clone technology. The SDDC can be built as a brand new pod, or used to extend an existing pod within the customers environment.
At the time of writing Horizon perpetual licenses can be used to run virtual desktops on VMware Cloud on Dell EMC, along with existing Microsoft licensing. A common consideration of moving VDI to the cloud is around Microsoft license mobility for Windows, Office 365, and SQL, and the requirement for Horizon Universal. Microsoft treat this solution as customer on-premises, which means that implementing VMware LCaaS delivers the best of both worlds. You can read more about the VDI use case in the VMware Horizon Deployed on VMware Cloud on Dell EMC technical overview.
As well as VDI, other popular use cases for VMware Cloud on Dell EMC include data centre modernisation, a change in IT funding model, application modernisation, and services with low latency, sensitive data, or data sovereignty requirements. VMware Cloud on Dell EMC integrates seamlessly with existing on-premises environments, with continuity of third party tools and processes already in place, such as backups, monitoring, and security. Hybrid Linked Mode allows single pane of glass management of vCenter Servers across IaaS and self-managed infrastructure. You can find out more about the benefits of VMware Cloud on Dell EMC, including Total Cost of Ownership (TCO) improvements, in the VMware Cloud Economics data sheet.
VMware Local Cloud as a Service (LCaaS)
Getting Started with VMware Cloud on Dell EMC
VMware Cloud on Dell EMC can be ordered, customised, and scaled through the VMware Cloud portal. Delivery and installation takes place in a matter of weeks, including the site survey. Check with your VMware or Dell account team for up to date time timelines, I have been quoted between 4-8 weeks at the time of writing (early 2022) which may fluctuate depending on hardware availability. The service is available in the UK, USA, France, and Germany, with plans to roll out to further regions.
When ordering the service, the customer can select the rack type and see full details of the host capacity, network bandwidth, height in rack units, and power configuration. The customer will be asked to confirm that the site location meets the rack requirements, including rack dimensions, power source, and environmental variables such as temperature and humidity.
VMware Cloud on Dell EMC Example Requirements
Next the customer will be asked to select the host type, the number of hosts, and provide the networking settings. A CIDR block is needed for the management subnets, including rack out-of-band management, SDDC management, and the VMware SD-WAN appliances. It is very important that the IP ranges are correct and do not overlap with any existing networks. Changing these values post-order will cause additional complexity and delays.
Ports TCP 443 and UDP 2426 will need to be open outbound to connect to VMware Cloud. The term commitment is also selected during the order process, and the term begins when the SDDC is deployed and activated from the VMware Cloud console. You can track the status of the order at any time from the portal.
VMware Cloud on Dell EMC Example SDDC Order
When the rack arrives on-site it is fully cabled and ready to be connected to the customer environment. The ToR switches are physically connected to the existing upstream network using customer provided SFP adapters and copper or fibre cables. Dynamic routing can be configured using eBGP, facilitating fast routing failover in the event of a ToR switch failure or upstream switch failure. Static routing can also be used but is less optimal.
Once the SDDC is deployed the L3 ECMP uplink connectivity between the ToR switches and the existing upstream network can be configured from the VMware Cloud console.
VMware Cloud on Dell EMC Example SDDC Summary
After setup is complete the service maintains operational consistency with existing VMware environments; for example virtual machines are managed using vCenter Server, and new networks are created using NSX-T. For more information review the VMware Cloud on Dell EMC Data Sheet, or the more comprehensive VMware Cloud on Dell EMC Technical Overview.
Another great place to get started is the VMware Cloud Tech Zone. You can find detailed white papers, reference architectures, technical demos, and hands on labs for VMware Cloud on Dell EMC specifically at the VMware Cloud on Dell EMC Tech Zone.
VMware Cloud on Dell EMC vs Dell APEX Cloud Services
As outlined in the introduction of this post, many organisation are moving to as-a-service and subscription services. Dell, along with VMware, have recognised this shift and made many of their compute and storage platforms available on managed and subscription based plans. Dell APEX Cloud Services is the self-service portal where Dell customers can configure and order such solutions.
Dell APEX Cloud Services with VMware Cloud, allows Dell customers to order VMware Cloud on Dell EMC directly through Dell. Although this may seem confusing, it gives customers an alternative purchasing route which can help leverage existing commercial agreements, credits, partners, and relationships.
The core technical concepts of the solution outlined above all remain the same. The key difference is that when purchasing through Dell APEX, the customer is buying directly from Dell (instead of VMware), and Dell are the single point of contact for all support and maintenance (instead of VMware). Whilst the order process remains fundamentally the same, the screenshots above are of the VMware Cloud portal, and so the Dell APEX portal will look slightly different.
esxsi.com 