vSphere 7 and vSAN 7 Headline New Features

vSphere 7 Cloud Infrastructure for Modern Applications Part 1:

vSphere 7 Cloud Infrastructure for Modern Applications Part 2

vSphere 7 with Kubernetes

Now and over the next 5 years, we will see a shift in how applications are built and run. In 2019 Line of Business (LOB) IT, or shadow IT, spend exceeded Infrastructure and Operations IT spend for the first time*. Modern applications are distributed systems built across serverless functions or managed services, containers, and Virtual Machines (VMs), replacing typical monolithic VM application and database deployments. The VMware portfolio is expanding to meet the needs of customers building modern applications, with a portfolio of services from Pivotal, Wavefront, Cloud Health, bitnami, heptio, bitfusion, and more. In the container space, VMware is strongly positioned to address modern application challenges for developers, business leaders, and infrastructure administrators.

Launched on March 10 2020, with expected April 2020 availability, vSphere 7 with Kubernetes is powering VMware Cloud Foundation 4. vSphere 7 with Kubernetes integration, the first product including capabilities announced as part of Project Pacific, provides real-time access to infrastructure in the Software-Defined Data Centre (SDDC) through familiar Kubernetes APIs, delivering security and performance benefits even over bare-metal hardware. The Kubernetes integration enables the full SDDC stack to utilise the Hybrid Infrastructure Services from ESXi, vSAN, and NSX-T, which provide the Storage Service, Registry Service, Network Service, and Container Service. Developers do not need to translate applications to infrastructure, instead of leveraging existing APIs to provision micro-services, while infrastructure administrators use existing vCenter Server tooling to support Kubernetes workloads along with side Virtual Machines.

At this point, on-premises Kubernetes orchestration is available through VMware Cloud Foundation 4. You can read more about Kubernetes with vSphere 7 in vSphere 7 with Kubernetes and Tanzu on VMware Cloud Foundation. Continue reading this post to review the additional functionality introduced with vSphere 7 and vSAN 7 around lifecycle management, scalability, security and compliance, you can also check the full vSphere 7 introduction here.

vSphere 7 Headline New Features

vCenter Server Profiles

vCenter Server Profiles are introduced in vSphere 7; enabling consistent configuration across the vCenter Server estate. vCenter Server Profiles export management, network, authentication, and user configurations into JSON format. The configurations can be edited, validated, and imported or pushed to up to 100 vCenter Servers, providing version control and a consistent last-known-good state. vCenter Server Profiles are accessible via 4 new REST APIs; list, export, validate, and import.  This also means they can be consumed with DCLI, PowerCLI, or other automation tools such as Ansible, Puppet, and Chef. Behind the scenes, vCenter Server Profiles are known as Infrastructure Profiles and can be found under infra-profiles in the vCenter Developer Center  API Explorer. Note that vCenter Server profiles do not replace file-based backups for vCenter, the profile exports do not contain GUIDs etc. that would be required for a full and supported vCenter Server restore.

vCenter Server Update Planner

The new vCenter Server Update Planner provides native tooling to help with discovering, planning and upgrading vCenter Server and connected products successfully. VMware administrators can receive notifications in the vSphere client when an upgrade or update is available. VMware product interoperability is built-in and automatically detects installed products to provide monitoring and checks against the current vCenter Server version; showing compatible upgrades and removing guesswork and complicated interoperability questions for complex environments. To further validate upgrade paths ‘what-if’ workflows, and pre-update checks can be run against the selected target vCenter. The vCenter Server Update Planner also links to applicable release notes and Knowledge Base (KB) articles. An extra benefit of the vCenter Server 7 upgrade process is the automation of the external Platform Services Controller (PSC) which is now built into the upgrade, more on this further down the post.

vCenter Server Cluster Image Management

Cluster Image is the new model for management of the ESXi lifecycle, providing consistency of ESXi hosts across a cluster. The cluster image comprises of specific firmware, drivers, or vendor software add-ons, to create a desired state model with multi-host remediation capabilities. The Cluster Image feature is exposed through the vSphere client, REST API, and also integrates with third-party vendor management tools such as Dell OpenManage and HPE OneView. This means host firmware can now be managed and upgraded from within vSphere, removing the risk of unsupported drivers and firmware. To use this feature, all hosts in a cluster must be the same hardware type and must all be running ESXi 7.0.

New vSphere DRS Improvements

Distributed Resource Scheduler (DRS) is evolving to meet Virtual Machine needs and has undergone several new improvements. DRS now makes workload centric placement decisions based on VM data gathered every minute, as opposed to cluster centric decisions based on 5 minutes of data. Placement decisions are now based on the individual VM DRS scores and granted memory. This shifts the focus onto the workload resource fulfilment, rather than the balance of the whole cluster. The VM DRS score is calculated using CPU %RDY (Ready) time, memory swap, CPU cache behaviour, headroom for the workload to burst, and migration cost. VM DRS scores are grouped into buckets of 20% increments.

Improved DRS in vSphere 7 now includes Scalable Shares, providing relative resource entitlements across Resource Pools. Setting a share level to ‘high’ now ensures prioritisation over lower share VM entitlements, whereas previously, the higher share level did not guarantee a higher resource entitlement. Scalable Shares need to be enabled and can be configured on a cluster level and/or Resource Pool level. Share allocations are dynamically changed depending on the number of VMs in a Resource Pool. The only exception to this rule is vSphere with Kubernetes where a Resource Pool is used as a Namespace, in this instance, Scalable Shares are used by default.

DRS placement now includes assignable hardware – support for hardware accelerators, for both DRS initial placement and vSphere High Availability (HA). When adding a new device dynamic DirectPath IO or NVIDIA GRID vGPU devices are supported. DRS works with the assignable hardware framework to find a host with an available PCIe device configured, or hardware profile when making initial placement decisions. The functionality requires the new VM hardware version 17.

New vSphere vMotion Improvements

Increased workload resource consumption as applications change-over-time has started presenting performance challenges during vMotion and stun times for large or monster VMs. To address these challenges, vMotion has been refactored as part of vSphere 7, bringing back vMotion capabilities for workloads like SAP HANA or Oracle.

During vMotion a page tracer is installed so vSphere can keep track of the memory pages that are overwritten by the guest OS while the VM is in a vMotion state. To install the page tracer, the vCPU is stopped (for micro-seconds), allowing the monitoring of memory page overwrites. These overwrites are referred to as page fires, which are replicated to the destination ESXi host. The page tracer was previously installed on all vCPUs in a VM. In vSphere 7 only one vCPU is claimed and dedicated to all the page tracing work during a vMotion operation. This improves the efficiency of page tracing and dramatically reduces the performance impact on a workload. When all memory pages have been migrated the last memory bitmap is transferred, in previous versions the entire bitmap was transferred, in vSphere 7 the bitmap is compacted, and only the last pages are sent, cutting down the switch over and stun time.

Enhanced vMotion Capability (EVC) has been updated with new baselines for CPU packages: Intel Cascade Lake generation and AMD Zen2 generation (EPYC Rome).

New vSphere Security & Compliance Features

vSphere 7 now supports Intel Softguard Extensions (SGX) which allows applications to work within the underlying hardware to create a secure enclave that cannot be viewed by the guest OS or hypervisor. The application can store secrets or data in the enclave, which is an important feature for risk management, although currently there is minimal hardware support. Intel Ice Lake CPUs will have dual socket implementations of SGX. If implementing SGX, remember that you will lose certain features such as vMotion, snapshots, etc. if the hypervisor cannot see everything in the VM, this becomes very much an application design decision.

vSphere 7 introduces vSphere Trust Authority (vTA), providing trusted hosts and encryption key management. Previous trust models in vSphere had the potential for running secure workload on untrusted hosts, with no repercussions for failing secure baselines. Attestation and key management were done by vCenter Server, which itself could not be encrypted. The dependencies on the vCenter Server itself made it difficult to implement the principle of least privilege. With vTA a hardware root of trust is created using a separate ESXi host cluster, this can also be your management cluster. The key manager only talks directly to trusted hosts, rather than the vCenter Server. Workloads running on the trusted cluster, now including vCenter Server, can be encrypted. A smaller number of administrators can be given access to the trusted hosts, with regular admins maintaining access to the workload hosts. Currently, vTA is still foundational, so expect more functionality to be available in future releases. It is important to note that to use the trusted host model, the physical server must have the TMP 2.0 chip, which is cryptographically bonded to the host.

Identity Federation is introduced in vSphere 7 to modernise vSphere Authentication utilising standards-based federated authentication with enterprise Identity Providers. Using Identity Federation organisations can benefit from reduced audit scope and administrative workload, as well as security enhancements such as Multi-Factor Authentication (MFA). Initial integration will be with Active Directory Federation Services (ADFS) / Azure Active Directory, which alongside MFA is great for compliance and security. Identity Federation will also work with the Supervisor Cluster for Kubernetes, which inherits a lot of the security and functional controls from vCenter to help bridge the gap between developing modern applications and existing processes and infrastructure.

There are hundreds of improvements in vSphere 7 to drive consistency and trust in the environment. For example, the default settings for the vSwitch now includes SecurebyDefault to enforce security settings, the Certificate Management UI has been consolidated and simplified, and so on. You can review the vSphere 7 release notes for full information.

Additional Noteworthy vSphere 7 Features

• vCenter Server Content Library: a new interface provides vast improvements in template management. Virtual Machine templates are now checked out to edit and checked in to save, facilitating version control, quick historical view of edits, and ability to restore to previous versions. You can switch between the new view and classic view in the vSphere client. Additional features such as versioning are only available when the VM template is stored in a Content Library. Advanced configuration now allows an update of auto-sync frequency and performance optimisation.
• vCenter Server Multi-Homing: vCenter Server 7 now supports multiple network adaptors, the maximum supported vNIC limit is 4 per vCenter Server, with NIC1 reserved for vCenter HA.
• vCenter Server SSO Domain Consolidation: vSphere SSO domain or external PSC consolidation has been simplified with new tooling commands for domain re-pointing or un-registering: cmsso-util unregister and domain-repoint.
• vCenter Server External PSC Consolidation: the Upgrade and Migration setup no longer allows the deployment of an external PSC. Furthermore, the external PSC consolidation process is now automatically built into the upgrade, reducing administrative time and effort during the upgrade process. This means the vCenter Server Converge Tool has been removed from the ISO. The external PSC consolidation during an upgrade is also a supported configuration in JSON format when upgrading using the CLI.
• VM Hardware v17: the new VM hardware version features a virtual Watchdog Timer providing guest OS and application monitoring, especially important for clustered applications like databases and filesystems. Precision Time Protocol (PTP) now provides sub-millisecond timekeeping, helpful for financial and scientific applications. PTP requires both an in-guest device and the ESXi service to be enabled.

• vCenter Server Configuration Maximums: further enhancements to vCenter Server scalability:
  • vCenter Server (standalone) number of hosts per vCenter Server: 2500, powered-on VMs per vCenter Server: 30,000
  • Linked Mode vCenter Servers (15 per SSO domain) hosts: 15,000, powered-on VMs: 150,000
  • vCenter Server latency requirements for vCenter Server to vCenter Server: 150ms, vCenter Server to ESXi Hosts: 150ms, vSphere Client to vCenter Server: 100ms

You can read the full vSphere 7 release information at Introducing vSphere 7: Essential Services for the Modern Hybrid Cloud as well as the vSphere 7 Data Sheet and vSphere 7 Product Page.

vSAN 7 Headline New Features

Several new features have been added to vSAN 7 alongside the vSphere 7 announcement, here are the key product enhancements:

Simplified Lifecycle Management

vSphere Lifecycle Manager (vLCM) is a new approach to unified software and firmware management, increasing reliability and decreasing the number of update tools. vLCM is built around the desired state model and monitors and remediates compliance drift. Desired state and desired images are applied at cluster level and manage the server stack as a whole, across hypervisor, drivers, and firmware. Furthermore, the modular framework supports vendor firmware plugins such as Dell and HPE.

Unified Block and File Storage

Fully Integrated File Services provides a native file service built into the hypervisor through vSAN. Cluster capacity for vSAN can be provisioned into file shares with support for NFS v4.1 & v3, and file share quotas, unifying management of block and file storage. vSAN file shares are aimed at ease of use for both cloud-native and traditional workloads running in the cluster, it is not necessarily a replacement for large scale filers.

Expanded Data Services

Continued Integration of Cloud Native Storage provides the control plan and storage service for vSphere with Kubernetes integration, and offers file-based persistent volumes easily accessible and managed within vCenter. This now includes support for vVols, persistent volume encryption, and snapshots, volume resizing, and a mixture of tooling such as application monitoring with Wavefront, next-generation monitoring solutions like Prometheus, and infrastructure analytic solutions like vRealize Operations, providing an advanced level of visibility for vSphere administrators.

Improved Efficiency and Operations

• Enhancements for Stretched Cluster and 2-Node Topologies: such as support for overriding the default gateway used by vSAN hosts to simplify deployments and routed topologies, immediate repair operation after a witness host appliance is replaced.
• Intelligent capacity management for stretched cluster topologies; when the cluster is in a capacity-constrained state, for example, due to host failure, objects in a critical condition are marked by vSAN as absent, allowing I/O to be processed at another site. The degraded state of the object in terms of resilience still stands, but the VM uptime is improved by allowing the continuation of read/write operations. The object is updated when the capacity strain condition is removed.
• Stretched cluster awareness for DRS placement decisions; enables prioritisation of I/O read locality over VM site affinity rules, completion of vSAN resync before DRS migrations,  and a reduction in I/O across ISL in recovery conditions.
• Improved accuracy in VM capacity reporting across vCenter UI and APIs when working with thin-provisioned VMs, swap objects, and namespace objects; reducing confusion and inconsistency over provided and used space for a given VM.
• A new vSAN memory metric has been added in the vSAN performance service to display memory consumption of vSAN operations such as hardware and software configuration changes. The additional vSAN memory metric shows time-based memory consumption per host and is available in the vCenter UI and API.
• New vSphere Replication object identity types to easily identify objects created by or using vSphere Replication, replacing the previous unknown object type.
• Additional support for larger storage devices; up to 32 TB physical drives, and up to 1 PB in logical capacity. This gives the potential for improved deduplication ratios when using larger devices for the capacity tier and deduplication domain.
• Native support for NVMe hot-plug through vSAN and vSphere for selected OEM platforms. This feature reduces host restarts and administrative complexity when carrying out planned or unplanned maintenance.
• Removal of Eager Zero Thick (EZT) requirement for vSAN shared disks, improving application consumption and flexibility.
• The full vSAN announcement can be found here

vSphere 7 with Kubernetes and vSAN 7 are built into VMware Cloud Foundation 4, read more on the March 10 2020 announcement in vSphere 7 with Kubernetes and Tanzu on VMware Cloud Foundation.

*LOB spend 51% to infrastructure operations spend 49% – source IDC WW Semiannual IT Spending Guide: Line of Business, 09 April 2018 (HW, SW and services; excludes Telecom)