NSX 6.4.x Upgrade Coordinator

This post will walk through an upgrade to NSX 6.4.1 using the new Upgrade Coordinator feature allowing simultaneous upgrade planning of multiple NSX components. From version 6.4 onwards upgrade plans can be used to upgrade host clusters, controller clusters, Edge Service Gateways (ESGs), Distributed Logical Routers – including Universal (DLRs and UDLRs), and Service Virtual Machines such as Guest Introspection. Upgrade plans consist of either a one click system managed upgrade, or planning your own upgrade where objects and options can be customised.

NSXUpgradeCoorindator

Review the operational impacts of NSX upgrades for each component here when planning your upgrade, it is best practise to limit all operations in the environment until the upgrade is complete. Make sure NSX Manager is backed up before starting an upgrade, and be aware that after a successful upgrade NSX cannot be downgraded. You should also review the VMware NSX for vSphere 6.4.1 Release Notes here and NSX for vSphere Documentation Center here.

Requirements

Requirements specific to NSX 6.4.1 are listed below. As we are doing an upgrade the assumption is that the vSphere and NSX environment is already setup and working, you can validate the existing NSX configuration here. You should also ensure an underlying network with IP connectivity and an MTU size of 1600 or above, FQDN resolution, connectivity, and time synchronisation between NSX and vSphere components, syslog, monitoring, and backups are all in place. In addition review the basic system requirements for NSX here and the full list of network port requirements here.

  • NSX 6.4.1 is compatible with vSphere versions 6.0 U2 and above, also note; if you are using 6.0 then U3 is recommended, the minimum supported version for 6.5 is 6.5a, support for 5.5 has now been removed
  • Supported upgrade paths to NSX 6.4.1 are from 6.2.4 onwards, there is a workaround for upgrading from 6.2.0, 6.2.1, or 6.2.2 which can be found here
  • Review the VMware Upgrade Path page here and also fully review the NSX 6.4.1 Release Notes here, as there are a number of things to be aware of when upgrading from 6.2.x or 6.3.x
  • Check compatibility with VMware products using the VMware Interoperability page here
  • Check compatibility with other third party products such as partner services for Guest Introspection using the VMware Compatibility Guide here
  • Before starting the upgrade make sure existing appliances meet the recommended hardware requirements:
    • NSX Manager 16 GB RAM (24 GB for large deployments), 4 vCPU (8 vCPU for large deployments), and 60 GB disk, a large deployment is typically 256+ hosts or 2000+ VMs
    • NSX Controllers 4 GB RAM, 4 vCPU, and 28 GB disk
    • NSX Edge Compact: 512 MB RAM, 1 vCPU, 584 MB + 512 MB disks. Large: 1 GB RAM, 2 vCPU, 584 MB + 512 MB disks. Quad Large: 2 GB RAM, 4 vCPU, 584 MB + 512 MB disks. X-Large: 8 GB RAM, 6 vCPU, 584 MB + 2 GB + 256 MB disks.
  • Verify the existing NSX Manager has sufficient space by connecting to the CLI (if using SSH service may need starting on the summary page of NSX Manager appliance page) and running show filesystems
  • Maximum latency between NSX components and NSX and vSphere components should be 150 ms RTT or below
  • NSX Data Security is no longer supported, it should be removed if installed prior to the upgrade
  • If you are using Cross-vCenter NSX then each component should be upgraded in the order listed here
  • A completed upgrade can be validated following the steps listed here

Backups

Before we start take a backup of the vCenter Server and NSX Manager. NSX configuration can be backed up using FTP/SFTP, see this post for more information. From version 6.4.1 a configuration backup is automatically taken at the start of the upgrade process, this is intended as a fall back and you should still take your own backup before beginning. You can also take a snapshot of the NSX Manager incase we need to revert back the NSX Manager upgrade. For extra peace of mind export the vSphere Distributed Switch configuration by following the instructions here.

In the event you do need to restore from an NSX backup a new appliance should be deployed and the configuration restored, click here for further details.

Upgrade Process

Download the NSX for vSphere 6.4.1 Upgrade Bundle from the download page here to a location accessible from the NSX Manager. Browse to the NSX Manager and log in as admin. From the home page click Upgrade.

Click Upload Bundle and browse to the upgrade bundle downloaded earlier, click Continue. Once the bundle is uploaded you can (optional) select to enable SSH and/or join the Customer Experience Improvement Program. Click Upgrade to start the upgrade.

NSX64_1

The installer will now upgrade NSX Manager, once complete you will be returned to the login page.

NSX64_2

Log back into NSX Manager and click Upgrade. Verify the upgrade state is complete and the version number is correct. Click Summary and verify the health of the NSX Manager.

NSX64_3

Log into the vSphere Client, if you were already logged in then log out and back in, or you may need to clear your browser cache. From the Menu drop-down select Networking and Security. The steps below can be performed in either the vSphere web client or the HTML5 client.

NSX64_5

Click Installation and Upgrade and select the Upgrade tab. Review the components, any warnings, and current and target version details.

NSX64_4

To start an upgrade plan click Plan Upgrade.

Upgrade Coordinator puts objects of the same type in default upgrade groups when planning an upgrade. These groups and other settings can be modified by planning your own upgrade (controller upgrades are mandatory) or you can allow the system to upgrade everything using a one click upgrade. Select the desired upgrade plan and click Next.

NSX64_7

The default options for the one click upgrade are to upgrade Host Clusters and Service VMs individually (serial), and to upgrade NSX Edges all together (parallel). There is no pause between components or pause on error. If you are happy with these settings then click Start Upgrade to being the upgrade process, otherwise go back to Plan Your Upgrade.

NSX64_8

Select your own upgrade to choose which components are upgraded, controller upgrades are mandatory and are done first. You can also pause the upgrade between components or pause the upgrade if an error is returned.

NSX64_9

The next 3 pages of the Upgrade Coordinator allow you to manage upgrade groups for Host Clusters, NSX Edges, and Service VMs. When planning your upgrade take into consideration the following:

  • Objects of the same type can be added to or removed from an upgrade group
  • The order of object upgrades within a group can be changed
  • All components included in an upgrade group must be upgraded before the next component type can be upgraded, e.g. all hosts included in an upgrade plan must be upgraded before moving onto Edges, and so on
  • Excluding an object within an upgrade group is useful for multiple maintenance windows, where you want to add an object to an upgrade plan but exclude them from this upgrade session
  • If the upgrade order within group is set to Serial then each object is upgraded one at a time, if it is Parallel then multiple objects within that group are upgraded at the same time

NSX64_10

Configure your upgrade plan based on the components you want to upgrade in this session, and review the final plan. When you’re read click Start Upgrade to begin the upgrade process.

NSX64_13

Monitor the status of the upgrade on the Upgrade page. If any warnings or errors are displayed during the upgrade process see the Monitor and Troubleshoot Your Upgrade page here. An in-progress upgrade plan can be paused to make modifications; when paused the object currently being upgraded will continue and the upgrade plan pauses when this object upgrade succeeds or fails.

NSX64_14

After the upgrade is complete verify the Upgrade page shows the system upgrade status successful. Verify the NSX health from the Dashboard page. You should now take a further backup of NSX Manager. Any third party appliances for Guest Introspection or Network Introspection can now be upgraded. If you are using stateless images with Auto Deploy you should also update your ESXi image with the latest NSX VIBs or they will be lost at next reboot, for guidance see this post.

VMware Cloud on AWS Demo

This opening post will give an overview and demo of VMware Cloud on AWS. VMware Cloud on AWS provides on-demand, scalable cloud environments based on existing vSphere Software-Defined Data Center (SDDC) products. VMware and AWS have worked together to optimise running vSphere, vSAN and NSX, directly on dedicated, elastic, bare-metal AWS infrastructure without the need for nested virtualization. A SDDC cloud can be deployed in a few hours and then capacity scaled up and down within minutes.

Key Benefits

There are a number of benefits and use cases for extending on-premise data centers to the cloud with VMware Cloud on AWS:

  • VMware maintains software updates, emergency software patches, and auto-remediation of hardware failures
  • Increasing capacity in the cloud is generally quicker, easier, and sometimes more cost effective than increasing physical capacity in the data center
  • Scale capacity to protect services when met with temporary or unplanned demand
  • Improve business continuity by using the cloud for Disaster Recovery (DR) with VMware Site Recovery
  • Consistent operating environments allows for simplified cloud migrations with minimal re-training for system administrators
  • Transfer your existing operating system and third party licensing to the cloud and make use of existing support contracts with VMware
  • Expand footprint into additional geographical locations without needing to provision new data centers

Key Details

The following links contain enough reading to plan your VMware Cloud on AWS implementation and cloud migration strategy, the points below should be enough to get you started.

VMware Cloud on AWS: Product Documentation | Technical Overview | VMware Product Page | VMware FAQ| AWS Product Page | AWS FAQRoadmap | Case Study

  • Each SDDC supports 4 to 32 hosts, each with 512 GB of memory and 2.3 GHz CPUs (custom-built Intel Xeon Processor E5-2686 v4 CPU package) with 18 cores per socket for a total of 36 cores
  • Each SDDC cluster uses an all-flash vSAN configuration utilising NVMe storage
  • An initial 4 host cluster provides roughly 21 TB usable capacity
  • Each ESXi host is connected to an Amazon Virtual Private Cloud (VPC) through Elastic Networking Adapter (ENA), which supports throughput up to 25 Gbps
  • Hybrid Cloud Extension allows stretched subnets between on-premise and cloud data centers for live migration of virtual machines
  • Hybrid Linked Mode allows administrators to connect vCenter Server running in VMware Cloud on AWS to an on-premises vCenter server to view both cloud and on-premises resources from a single interface
  • VMware Cloud on AWS complies with ISO 27001, ISO 27017, ISO 27018, SOC 1, SOC 2, SOC 3, HIPAA, and GDPR
  • VMware Cloud on AWS is managed from a web-based console or RESTful API
  • At the time of writing VMware Cloud on AWS is available in the AWS Europe (Frankfurt and London), AWS US East (N. Virginia) and AWS US West (Oregon) Regions
  • Basic pricing before discount can be calculated here

VMware_AWS

Product Demo

The demo below creates a SDDC in the cloud for lab purposes. Before deploying your own environment you should review all the above linked documentation and do your own research to plan your cloud strategy as well as the following:

  • Identify or create an AWS account and ensure that all technical personnel have access to the account
  • Identify a VPC and subnet by cross-linking the AWS account to the SDDC
  • Allocate IP ranges for the SDDC, and determine a DNS strategy
  • Identify the authentication model for the SDDC
  • Plan connectivity to the SDDC
  • Develop a network security policy for the SDDC

Browse to the VMware Cloud Services portal (https://console.cloud.vmware.com) and login using your VMware ID. At the time of writing to access VMware Cloud on AWS you need to be invited or you can register for a 30 day single host trial here.

VMware_Cloud

Select VMware Cloud on AWS. If you have not used the service before you will be prompted to create a new organisation. Enter a name for your new organisation and accept the terms of service, click Continue.

AWS_1

Add a credit card to be billed if you use the service. If you are using one of the free or trial methods outlined above you will not be billed.

aws_2.png

After you have created the organisation and added payment information you will be sent to the VMware Cloud on AWS dashboard. The first step is to create our SDDC in the cloud, click Create SDDC.

Billing: annual subscriptions are listed under the Subscriptions tab, you can see other billing information from the drop-down menu next to your organisation name: select Organisation Settings, View Organisation. From here you have services, identity and access management, billing and subscriptions, and support options.

AWS_3

Select a region and deployment model for the SDDC, enter a name and the number of hosts if you are not using the single host deployment. Click Next.

AWS_4

Follow the instructions to connect an AWS account and assign the relevant capabilities.

AWS_5

Once the connection is successfully established click Next.

AWS_7

Select the VPC and subnet to use then click Next.

AWS_8

Specify a private subnet range for the management subnet or leave blank to use default addressing. As mentioned above ensure you have planned accordingly and are not using any ranges that will conflict with other networks you may connect in the future. Click Deploy SDDC.

AWS_9

The SDDC will now be deployed, it takes around 2 hours to provision the ESXi hosts and all management components.

AWS_10

Once the deployment is complete the dashboard will show the new SDDC and assigned resources. Click View Details (you can toggle the web portal theme using the Dark/Light options in the top right hand corner).

AWS_14

From either the SDDC Summary tab or back on the SDDC dashboard you can seamlessly add additional hosts or clusters at any time.

AWS_15

If needed the chat bubble in the bottom right hand corner of the screen will take you through to support.

AWS_Support

The Network tab shows the network topology and is where you can configure firewall rules, NAT rules, VPN, Direct Connect, etc.

AWS_12

To access the vCenter Server through the vSphere client the port needs opening, a VPN can also be used. Under Management Gateway select Firewall Rules, click Add Rule. Configure the rule to allow access to the vCenter on port 443 and click Save.

AWS_11

Click Open vCenter from either the Summary or Network tab, if access is in place you are given the cloudadmin@vmc.local credentials to open vCenter. Active Directory can also be configured as an identity source later on.

Once you are logged into the vSphere client you will see the familiar vSphere layout.

vCenter_AWS

It is also possible to see your on-premise vCenter Server(s) in the same pane of glass using Hybrid Linked Mode, click here for more information.

Back in the VMware Cloud on AWS portal the Add Ons tab features Site Recovery and Hybrid Cloud Extension for protecting and migrating workloads to your SDDC in the cloud.

AWS_16

You can delete a SDDC from the Actions drop-down menu in either the SDDC Summary tab or the SDDC dashboard. Once a SDDC is deleted all workloads, data, and interfaces are destroyed and any public IP addresses released.

AWS_17

Migrating Windows vCenter Server to VCSA 6.7

VMware vCenter Server pools ESXi host resources to provide a rich feature set delivering high availability and fault tolerance to virtual machines. The vCenter Server is a centralised management application and can be deployed as a virtual appliance or Windows machine. It should be noted that vCenter 6.7 is the final release where Windows modules will be available, see here for more information. All future releases will only be available as vCenter Server Appliance (VCSA) which is the preferred deployment method of vCenter Server. This post gives a walk through on migrating from a Windows based vCenter Server (VCS) to the Photon OS based vCenter Server Appliance (VCSA).

vCenter 6.7: Download | Release Notes | What’s New | VMware DocsvSphere Central

About VCSA

migrate2vcsaThe VCSA is a pre-configured virtual appliance built on Project Photon OS. Since the OS has been developed by VMware it benefits from enhanced performance and boot times over the previous Linux based appliance. Furthermore the embedded vPostgres database means VMware have full control of the software stack, resulting in significant optimisation for vSphere environments and quicker release of security patches and bug fixes. The VCSA scales up to 2000 hosts and 35,000 virtual machines. A couple of releases ago the VCSA reached feature parity with its Windows counterpart, and is now the preferred deployment method for vCenter Server. Features such as Update Manager are bundled into the VCSA, as well as file based backup and restore, and vCenter High Availability. The appliance also saves operating system license costs and is quicker and easier to deploy and patch.

Migrating to VCSA involves the deployment of a new appliance and migration of all configuration (including distributed switches) and historical data using the upgrade installer. The VCSA uses a temporary IP address during migration before switching to the IP and host name of the VCS, the Windows box is then powered off.

Software Considerations

  • The Windows VCS must be v.6.0 or v6.5 (any build / patch) to migrate to VCSA 6.7. Both physical and virtual vCenter Server installations are compatible.
  • Any database, internal or external, supported by VCS can be migrated to the embedded vPostgres database within the target VCSA.
  • The ESXi host or vCenter where VCSA will be deployed must be running v5.5 or above. However, all hosts you intend to connect to vCenter Server 6.7 should be running ESXi 6.0 or above, hosts running 5.5 and earlier cannot be managed by vCenter 6.7 and do not have a direct upgrade path to 6.7.
  • The Windows server is powered off once the VCSA is brought online, this means any other components, VMware or third party, need to be migrated off the Windows server in advance or they will no longer work (don’t forget to move and update any scripts that may live on the Windows server).
  • If you are using Update Manager the VCSA now includes an embedded Update Manager instance.
  • You must check compatibility of any third party products and plugins that might be used for backups, anti-virus, monitoring, etc. as these may need upgrading for vSphere 6.7 compatibility.
  • To check version compatibility with other VMware products see the Product Interoperability Matrix.
  • The points above are especially important since at the time of writing vSphere 6.7 is new enough that other VMware and third party products may not have released compatible versions. Verify before installing vSphere 6.7 and review the Release Notes and Important information before upgrading to vSphere 6.7 KB.

Hardware Considerations

  • The VCSA with embedded PSC requires the following hardware resources (disk can be thin provisioned)
    • Tiny (up to 10 hosts, 100 VMs) – 2 CPUs, 10 GB RAM.
    • Small (up to 100 hosts, 1000 VMs) – 4 CPUs, 16 GB RAM.
    • Medium (up to 400 hosts, 4000 VMs) – 8 CPUs, 24 GB RAM.
    • Large (up to 1000 hosts, 10,000 VMs) – 16 CPUs, 32 GB RAM.
    • X-Large (up to 2000 hosts, 35,000 VMs) – 24 CPUs, 48 GB RAM – new to v6.5.
  • Storage requirements for the smallest environments start at 250 GB and increase depending on your specific database requirements. See the Storage Requirements document for further details.
  • Where the PSC is deployed as a separate appliance this requires 2 CPUs, 4 GB RAM, 60 GB disk.
  • Environments with ESXi host(s) with more than 512 LUNs and 2048 paths should be sized large or x-large.
  • To help with selecting the appropriate storage size for the appliance calculate the size of your existing VCS database here.

Architectural Considerations

  • The migration tool supports different deployment topologies but can not, make changes to the topology and SSO domain configuration.
  • For more information on the deployment topologies available with vCenter 6.x see vCenter Server and Platform Services Controller Deployment Types.
  • A series of videos covering vCenter Server and Platform Services Architecture can be found here. If you require further assistance with vCenter planning see also the vSphere Topology and Upgrade Planning Tool here,
  • Most deployments will include the vCenter Server and PSC in one appliance, following the embedded deployment model, which I will use in this guide.
  • Consider if the default self-signed certificates are sufficient or if you want to replace with custom CA or VMware CA signed certs, see Installing vCenter Internal CA signed SSL Certificates for more information.

embedded

Other Considerations

  • Ensure you have a good backup of the vCenter Server and the database.
  • Variables such as FQDN resolution, database permissions and access to the licensing portal should all be in place since we are upgrading an existing vCenter solution.
  • All vSphere components should be configured to use an NTP server. The installation can fail or the vCenter Server Appliance vpxd service may not be able to start if the clocks are unsynchronized.
  • The ESXi host on which you deploy the VCSA should not be in lockdown or maintenance mode.
  • You will need the SSO administrator login details and if the Windows VCS service runs as a service account then the account must have replace a process level token permission.
  • Local Windows users that have vSphere permissions are not migrated since they are specific to the Windows server, all SSO users and permissions are migrated.
  • The upgrade can be easily rolled back by following this KB.
  • Migration of vCenter using DHCP, or services with custom ports, is not supported. The settings of only one physical network adapter are migrated.
  • Downtime varies depending on the amount of data you are migrating and is calculated when running the migration wizard.
  • A list of Required Ports for vCenter Server and PSC can be found here.
  • The configuration maximums for vSphere 6.7 can be found here.
  • In vSphere 6.7 TLS 1.2 is enabled by default. TLS 1.0 and TLS 1.1 are disabled by default, review the Release Notes for more information.
  • There are a number of Intel and AMD CPUs no longer supported with vSphere 6.7, review the Release Notes for a full list of unsupported processors.

Process

Before we begin if your existing Windows vCenter is virtual it may be beneficial to rename the vCenter virtual machine name in the vSphere inventory to include -old or equivalent. While the hostname and IP are migrated the vSphere inventory name of the VM cannot be a duplicate. The old server is powered down but not deleted so that we have a back out.

Download the VMware vCenter Server Appliance 6.7 ISO from VMware downloads: v6.7.0. Mount the ISO on your computer. The VCSA 6.5 installer is compatible with Mac, Linux, and Windows. Copy the migration-assistant folder to the Windows vCenter Server (and PSC server if external). If the PSC is running on a different Windows server then you must run the Migration Assistant on the PSC server first and migrate following the instructions below, then complete the same process on the Windows vCenter Server.

Start the VMware-Migration-Assistant and enter the SSO Administrator credentials to start running pre-checks.

VCSA_Migration_1

If all checks complete successfully the Migration Assistant will finish at ‘waiting for migration to start’.

On a different machine from your Windows vCenter and PSC server(s) open the vcsa-ui-installer folder file located on the root of the ISO. Browse to the corresponding directory for your operating system, e.g. \vcsa-ui-installer\win32. Right click Installer and select Run as administrator. The vCenter Server Appliance Installer will open, click Migrate.

VCSA_Migration_2

The migration is split into 2 stages; stage 1 deploys the new appliance with temporary network settings, there is no outage to the Windows vCenter at this stage. Stage 2 migrates data and network settings over to the new appliance and shuts down the Windows server. We begin with deploying the appliance. Click Next.

VCSA_Migration_3

Accept the license terms and click Next.

VCSA_Migration_4

Enter the details of the vCenter Server to migrate, then click Next.

VCSA_Migration_5

Enter the FQDN or IP address of the host, or vCenter upon which you wish to deploy the new VCSA. Enter the credentials of an administrative or root user and click Next. The installer will validate access, if prompted with an untrusted SSL certificate message click Yes to continue. Tip – connect to the vCenter for visibility of any networks using a distributed switch, connecting to the host direct will only pull back networks using a standard switch.

VCSA_Migration_6

Enter the virtual appliance VM name, this is the name that appears in the vSphere inventory as mentioned earlier. The host name of the vCenter Server will automatically be migrated. Click Next.

VCSA_Migration_7

Select the appropriate deployment size for your environment and click Next.

VCSA_Migration_8

Select the datastore to locate the virtual appliance and click Next. Configure the temporary network settings for the appliance. These will only be used during migration of the data, once complete the temporary settings are discarded and the VCSA assumes the identity, including IP settings, of the Windows vCenter Server. Click Next.

VCSA_Migration_9

Review the settings on the summary page and click Finish. The VCSA will now be deployed. Once complete click Continue to being the second stage of the migration.

VCSA_Migration_10

Click Next to begin the migration wizard.

VCSA_Migration_11

The source vCenter details are imported from stage 1.

VCSA_Migration_12

As my source Windows vCenter was joined to a domain I am prompted for credentials to join the VCSA to the domain.

VCSA_Migration_13

Select the data to migrate and click Next.

VCSA_Migration_14

Select whether or not to join the VMware Customer Experience Improvement Program and click Next.

VCSA_Migration_15

Review the summary page and click Finish. Data will now be migrated to the VCSA, once complete the Windows vCenter Server will be powered off and the network settings transferred to the VCSA. If you urgently need to power back on the Windows server to retrieve files or such like, then do so with the vNICs disconnected, otherwise you will cause an IP/host name conflict on the network.

VCSA_Migration_16

Post-Installation

Connect to the vCenter post install using the IP or FQDN of the vCenter. Access vSphere by clicking either Launch vSphere Client (HTML5) or Launch vSphere Web Client (FLEX). As the web client will be depreciated in future versions, and the HTML5 client is now nearly at full feature parity, we will use the HTML5 vSphere client.

Windows_vCenter67_14

Management features of the VCSA can be accessed by browsing to the IP or FQDN of the vCenter on port 5480. The login is the root account we configured a password for during the migration wizard.

VCSA_Management

vCenter Server Appliance 6.7 Install Guide

VMware vCenter Server pools ESXi host resources to provide a rich feature set delivering high availability and fault tolerance to virtual machines. The vCenter Server is a centralised management application and can be deployed as a virtual appliance or Windows machine. It should be noted that vCenter 6.7 is the final release where Windows modules will be available, see here for more information. All future releases will only be available as vCenter Server Appliance (VCSA) which is the preferred deployment method of vCenter Server. An existing Windows vCenter can be migrated to VCSA by following the steps in Migrating Windows vCenter Server to VCSA 6.7 This post gives a walk through on a clean installation of VCSA 6.7.

vCenter 6.7: Download | Release Notes | What’s New | VMware DocsvSphere Central

468x60

About VCSA

migrate2vcsaThe VCSA is a pre-configured virtual appliance built on Project Photon OS. Since the OS has been developed by VMware it benefits from enhanced performance and boot times over the previous Linux based appliance. Furthermore the embedded vPostgres database means VMware have full control of the software stack, resulting in significant optimisation for vSphere environments and quicker release of security patches and bug fixes. The VCSA scales up to 2000 hosts and 35,000 virtual machines. A couple of releases ago the VCSA reached feature parity with its Windows counterpart, and is now the preferred deployment method for vCenter Server. Features such as Update Manager are bundled into the VCSA, as well as file based backup and restore, and vCenter High Availability. The appliance also saves operating system license costs and is quicker and easier to deploy and patch.

Software Considerations

  • VCSA 6.7 must be deployed to an ESXi host or vCenter running v5.5 or above. However, all hosts you intend to connect to vCenter Server 6.7 should be running ESXi 6.0 or above, hosts running 5.5 and earlier cannot be managed by vCenter 6.7 and do not have a direct upgrade path to 6.7.
  • You must check compatibility of any third party products and plugins that might be used for backups, anti-virus, monitoring, etc. as these may need upgrading for vSphere 6.7 compatibility.
  • To check version compatibility with other VMware products see the Product Interoperability Matrix.
  • The points above are especially important since at the time of writing vSphere 6.7 is new enough that other VMware and third party products may not have released compatible versions. Verify before installing vSphere 6.7 and review the Release Notes and Important information before upgrading to vSphere 6.7 KB.

Architectural Considerations

  • When implementing a new vSphere 6.7 environment you should plan your topology in accordance with the VMware vCenter Server and Platform Services Controller Deployment Types.
  • A series of videos covering vCenter Server and Platform Services Architecture can be found here. If you require further assistance with vCenter planning see also the vSphere Topology and Upgrade Planning Tool here,
  • Most deployments will include the vCenter Server and PSC in one appliance, following the embedded deployment model, which I will use in this guide.
  • Greenfield deployments of vSphere 6.7 can take advantage of Embedded PSC with Enhanced Linked Mode, providing native vCenter Server HA support, and removal of SSO site boundaries.
  • Consider if the default self-signed certificates are sufficient or if you want to replace with custom CA or VMware CA signed certs, see Installing vCenter Internal CA signed SSL Certificates for more information.

embedded

Other Considerations

  • The VCSA with embedded PSC requires the following hardware resources (disk can be thin provisioned)
    • Tiny (up to 10 hosts, 100 VMs) – 2 CPUs, 10 GB RAM.
    • Small (up to 100 hosts, 1000 VMs) – 4 CPUs, 16 GB RAM.
    • Medium (up to 400 hosts, 4000 VMs) – 8 CPUs, 24 GB RAM.
    • Large (up to 1000 hosts, 10,000 VMs) – 16 CPUs, 32 GB RAM.
    • X-Large (up to 2000 hosts, 35,000 VMs) – 24 CPUs, 48 GB RAM – new to v6.5.
  • Storage requirements for the smallest environments start at 250 GB and increase depending on your specific database requirements. See the Storage Requirements document for further details.
  • Where the PSC is deployed as a separate appliance this requires 2 CPUs, 4 GB RAM, 60 GB disk.
  • Environments with ESXi host(s) with more than 512 LUNs and 2048 paths should be sized large or x-large.
  • The ESXi host on which you deploy the VCSA should not be in lockdown or maintenance mode.
  • All vSphere components should be configured to use an NTP server. The installation can fail or the vCenter Server Appliance vpxd service may not be able to start if the clocks are unsynchronized.
  • FQDN resolution should be in place when deploying vCenter Server.
  • A list of Required Ports for vCenter Server and PSC can be found here.
  • The configuration maximums for vSphere 6.7 can be found here.
  • In vSphere 6.7 TLS 1.2 is enabled by default. TLS 1.0 and TLS 1.1 are disabled by default, review the Release Notes for more information.
  • There are a number of Intel and AMD CPUs no longer supported with vSphere 6.7, review the Release Notes for a full list of unsupported processors.

Installation

Download the VMware vCenter Server Appliance 6.7 ISO from VMware downloads: v6.7.0.

Mount the ISO on your computer. The VCSA 6.7 installer is compatible with Mac, Linux, and Windows. Browse to the corresponding directory for your operating system, e.g. \vcsa-ui-installer\win32. Right click Installer and select Run as administrator. As we are installing a new instance click Install.

VCSA_1

The installation is split into 2 stages, we begin with deploying the appliance. Click Next.

VCSA_2

Accept the license agreement and click Next.

VCSA_3

Select the deployment model, in this example we will be using an embedded deployment combining the vCenter Server and Platform Services Controller in one appliance, click Next.

VCSA_4

Enter the FQDN or IP address of the host, or vCenter upon which you wish to deploy the new VCSA. Enter the credentials of an administrative or root user and click Next. The installer will validate access, if prompted with an untrusted SSL certificate message click Yes to continue. Tip – connect to the vCenter for visibility of any networks using a distributed switch, connecting to the host direct will only pull back networks using a standard switch.

VCSA_5

Enter the VM name for the VCSA and a root password, click Next.

VCSA_6

Select the deployment size in line with the number of hosts and virtual machines that will be managed, click Next.

VCSA_7

Select the datastore where the VCSA will be deployed, select thin provisioning if required, and click Next. Configure the network settings for the appliance and click Next.

VCSA_8

On the summary page click Finish. The appliance will now be deployed.

VCSA_9

With the VCSA now deployed we can move on to stage 2, click Continue.

VCSA_10

Click Next to being the VCSA setup.

VCSA_11

Configure the NTP servers, enable SSH access if required, and click Next.

VCSA_12

Enter a unique SSO domain name, the default is vsphere.local. The SSO domain name should not be the same as your Active Directory Domain. Configure a password for the SSO administrator, click Next.

VCSA_13

Select or deselect the customer experience improvement program box and click Next.

VCSA_14

Review the details on the summary page and click Finish. Click Ok to acknowledge that the VCSA setup cannot be paused or stopped once started. When the installer is complete click Close to close the wizard.

Post-Installation

Connect to the vCenter post install using the IP or FQDN of the vCenter. Access vSphere by clicking either Launch vSphere Client (HTML5) or Launch vSphere Web Client (FLEX). As the web client will be depreciated in future versions, and the HTML5 client is now nearly at full feature parity, we will use the HTML5 vSphere client.

Windows_vCenter67_14

You must apply a new vCenter license key within 60 days. If you have purchased vCenter Server then log into your licensing portal here. If the license key does not appear then check with your VMware account manager. Log in to the vSphere Web Client using the SSO administrator login.  From the Menu drop-down click Administration,

Windows_vCenter67_16

Under Licensing select Licenses. First we need to add a new license key, click Add New Licenses. Enter the new license key for vCenter Server, click Next. If applicable assign a name to the licence, click Next. Click Finish to add the license key.

Windows_vCenter67_15

Switch to Assets, the vCenter Server is listed in evaluation mode. Highlight the vCenter and click Assign License. Select the license key and click Ok.

Windows_vCenter67_17

If you have an Active Directory domain then vCenter can use this as an identity source. First ensure the vCenter is joined to the domain; from the Menu drop-down click Administration. Under Single Sign On click Configuration. Select the Active Directory Domain tab and verify the vCenter is domain joined. Change to the Identity Sources tab and click Add Identity Source. Fill in the Active Directory details for your domain and click Ok.

Windows_vCenter67_18

You can now add permissions to vCenter objects such as datacenters, clusters, folders, individual virtual machines, etc. for Active Directory users and groups. To learn more about vSphere permissions click here.

To start adding ESXi hosts to vCenter click the Menu drop-down and select Hosts and Clusters. Right click the vCenter and select New Datacenter, give the datacenter a name and click Ok. Right click the datacenter and select Add Host. Follow the onscreen wizard to add a host. Creating clusters and configuring vCenter is beyond the scope of this post, for assistance follow the documentation links at the top of the page.

Windows_vCenter67_19

Windows vCenter Server 6.7 Install Guide

VMware vCenter Server pools ESXi host resources to provide a rich feature set delivering high availability and fault tolerance to virtual machines. The vCenter Server is a centralised management application and can be deployed as a virtual appliance or Windows machine. It should be noted that vCenter 6.7 is the final release where Windows modules will be available, see here for more information. All future releases will only be available in VCSA form, if you have not already started planning migration to VCSA see vCenter Server Appliance 6.7 Install Guide and Migrating Windows vCenter Server to VCSA 6.7. This post gives a walk through on a clean installation of vCenter Server 6.7 on Windows Server 2016.

vCenter 6.7: Download | Release Notes | What’s New | VMware DocsvSphere Central

Software Considerations

  • The operating system should be 64 bit and Windows Server 2008 SP2 or above.
  • For environments with up to 20 hosts and 200 VMs the bundled internal PostgreSQL database can be used.
  • If an external database is used it should be Microsoft SQL Server 2008 R2 SP2 or above, or Oracle 11g or 12c. You can review a full list of compatible versions at the Database Interoperability Matrix.
  • The account used for external database authentication requires Oracle DBA role, or SQL sysadmin server role, or db_owner fixed database role. For a full list of explicit permissions review the Database Permission Requirements.
  • You must check compatibility of any third party products and plugins that might be used for backups, anti-virus, monitoring, etc. as these may need upgrading for vSphere 6.7 compatibility.
  • Any hosts you want to add to vCenter 6.7 should be running version 6.0 or above, 5.5 and earlier will not work and do not have a direct upgrade path to 6.7.
  • To check version compatibility with other VMware products see the Product Interoperability Matrix.
  • The points above are especially important since at the time of writing vSphere 6.7 is new enough that other VMware and third party products may not have released compatible versions. Verify before installing vSphere 6.7 and review the Release Notes and Important information before upgrading to vSphere 6.7 KB.

Architectural Considerations

  • As noted above the Windows modules will not be included for future versions, therefore the recommended installation method for vCenter 6.7 is the vCenter Server Appliance (VCSA).
  • When implementing a new vSphere 6.7 environment you should plan your topology in accordance with the VMware vCenter Server and Platform Services Controller Deployment Types.
  • A series of videos covering vCenter Server and Platform Services Architecture can be found here. If you require further assistance with vCenter planning see also the vSphere Topology and Upgrade Planning Tool here,
  • Most deployments will include the vCenter Server and PSC on one server, following the embedded deployment model, which I will use in this guide.
  • Greenfield deployments of vSphere 6.7 can take advantage of Embedded PSC with Enhanced Linked Mode, providing native vCenter Server HA support, and removal of SSO site boundaries.

embedded

Hardware Considerations

  • A Windows based vCenter Server can be installed on either a physical or virtual machine. Windows vCenter Server with embedded PSC requires the following hardware resources:
    • Tiny (up to 10 hosts, 100 VMs) – 2 CPUs, 10 GB RAM.
    • Small (up to 100 hosts, 1000 VMs) – 4 CPUs, 16 GB RAM.
    • Medium (up to 400 hosts, 4000 VMs) – 8 CPUs, 24 GB RAM.
    • Large (up to 1000 hosts, 10,000 VMs) – 16 CPUs, 32 GB RAM.
    • X-Large (up to 2000 hosts, 35,000 VMs) – 24 CPUs, 48 GB RAM – new to v6.5.
  • Where the PSC is deployed on a separate machine this requires 2 CPUs, 4 GB RAM.
  • Environments with ESXi host(s) with more than 512 LUNs and 2048 paths should be sized large or x-large.
  • The Windows vCenter Server requires the following free disk space for installation: (the first 2 may not necessarily be the system drive depending on installation location) Program Files 6 GB, Program Data 8 GB, System folder 3 GB. The PSC machine requires; Program Files 1 GB, Program Data 2 GB, System folder 1 GB.
  • There are a number of Intel and AMD CPUs no longer supported with vSphere 6.7, review the Release Notes for a full list of unsupported processors.

Other Considerations

  • It may be necessary to temporarily stop any third party software which could interfere with the installer, such as anti-virus scanner.
  • If the vCenter Server services are running as a user other than the Local System account then the user must be a member of the administrators group and have the following permissions; log on as a service, act as part of the operating system.
  • Verify that the local machine policy allows assigning Log on as a batch job rights to new local users.
  • All vSphere components should be configured to use the same NTP server.
  • FQDN resolution should be in place when deploying vCenter Server.
  • A list of Required Ports for vCenter Server and PSC can be found here.
  • The configuration maximums for vSphere 6.7 can be found here.
  • In vSphere 6.7 TLS 1.2 is enabled by default. TLS 1.0 and TLS 1.1 are disabled by default, review the Release Notes for more information.

Create Data Source

Before beginning if you intend to use vCenter Server with an external SQL database you must configure a 64-bit ODBC data source for external databases. You may also need to install the Microsoft ODBC Driver for SQL Server. ODBC Data Source Administrator can be accessed via Control Panel > Administrative Tools. Click System DNS, Add and input the details for the external database, test the data source before continuing. If you are using the internal Postgres database then the System DSN is added automatically during installation.

odbc

Installation

Download the VMware vCenter Server and Modules for Windows ISO from VMware downloads: v6.7.0.

Mount the ISO and right click autorun.exe, select Run as administrator. The VMware vCenter Installer will open. Ensure vCenter Server for Windows is selected and click Install.

Windows_vCenter67_1

The vCenter Server 6.7 Installer will open in a separate window, click Next.

Windows_vCenter67_2

Accept the end user license agreement and click Next.

Windows_vCenter67_3

In this guide we will be using an embedded deployment model. If you are using an external deployment model the PSC component must be installed first before the vCenter. Select the deployment type and click Next. If the Windows server does not have sufficient resources allocated the installer will error at this stage.

Windows_vCenter67_4

Enter the FQDN in the System Name field and click Next.

Windows_vCenter67_5

Create a new Single Sign-On domain, or join the vCenter to an existing SSO domain. If you are creating a new SSO domain either leaves as the default vsphere.local or create a new SSO domain name, (not the same as your Active Directory name). Configure a password for the SSO administrator account and a vCenter specific site name, click Next. Note: vCenter 6.7 is the last release where a SSO site name will need to be provided.

Windows_vCenter67_6

Select whether to run vCenter services as the local system account or enter details of a service account and click Next. Ensure the account running vCenter services has been granted permissions as per the other considerations section of this guide.

Windows_vCenter67_7

Select an embedded Postgre database or point the installer to the DSN for an external database, click Next.

Windows_vCenter67_8

Accept the default port configuration and click Next.

Windows_vCenter67_9

Select the directory to install vCenter services and click Next.

Windows_vCenter67_10

Tick or untick the VMware Customer Experience Improvement Program as appropriate and click Next.

Windows_vCenter67_11

Check the configuration on the review page and click Install to begin the installation process.

Windows_vCenter67_12

Once the installation has completed click Finish.

Windows_vCenter67_13

Post-Installation

Connect to the vCenter post install using the IP or FQDN of the vCenter. Access vSphere by clicking either Launch vSphere Client (HTML5) or Launch vSphere Web Client (FLEX). As the web client will be depreciated in future versions, and the HTML5 client is now nearly at full feature parity, we will use the HTML5 vSphere client.

Windows_vCenter67_14

You must apply a new vCenter license key within 60 days. If you have purchased vCenter Server then log into your licensing portal here. If the license key does not appear then check with your VMware account manager. Log in to the vSphere Web Client using the SSO administrator login.  From the Menu drop-down click Administration,

Windows_vCenter67_16

Under Licensing select Licenses. First we need to add a new license key, click Add New Licenses. Enter the new license key for vCenter Server, click Next. If applicable assign a name to the licence, click Next. Click Finish to add the license key.

Windows_vCenter67_15

Switch to Assets, the vCenter Server is listed in evaluation mode. Highlight the vCenter and click Assign License. Select the license key and click Ok.

Windows_vCenter67_17

If you have an Active Directory domain then vCenter can use this as an identity source. First ensure the vCenter is joined to the domain; from the Menu drop-down click Administration. Under Single Sign On click Configuration. Select the Active Directory Domain tab and verify the vCenter is domain joined. Change to the Identity Sources tab and click Add Identity Source. Fill in the Active Directory details for your domain and click Ok.

Windows_vCenter67_18

You can now add permissions to vCenter objects such as datacenters, clusters, folders, individual virtual machines, etc. for Active Directory users and groups. To learn more about vSphere permissions click here.

To start adding ESXi hosts to vCenter click the Menu drop-down and select Hosts and Clusters. Right click the vCenter and select New Datacenter, give the datacenter a name and click Ok. Right click the datacenter and select Add Host. Follow the onscreen wizard to add a host. Creating clusters and configuring vCenter is beyond the scope of this post, for assistance follow the documentation links at the top of the page.

Windows_vCenter67_19

Updating vCenter Server with External PSC

The following post demonstrates the update process for applying minor updates to a vSphere environment running multiple vCenter Server appliances and external Platform Services Controllers.

In this instance we are updating vCenter to 6.5 U1e as one of the remediation actions for the Branch Target Injection issue (CVE-2017-5715) commonly known as Spectre. For more information on Meltdown and Spectre see this blog post, VMwares responses can be found here, on the VMware Security & Compliance Blog here, as well as VMware Security Announcement VMSA-2018-0004.2 here.

meltdown-spectre-vmware

Pre-Update Checks

When upgrading vSphere with an external Platform Services Controller (PSC), upgrade the PSC first, then the vCenter Server, then the ESXi hosts, and finally the virtual machines (hardware versions, VMware Tools).

Prior to updating vCenter ensure you have verified the compatibility of any third party products such as backups, anti-virus, monitoring, etc. Also cross-check the compatibility of other VMware products using the Product Interoperability Matrix. Since we are applying a minor update to vCenter Server the usual pre-requisites such as FQDN resolution, time synchronization, relevant ports open, etc. should already be in place. For vCenter 6.5 U1e all hosts must be running at least ESXi version 5.5. For more information on the requirements for vCenter Server 6.5, or if you are upgrading from an earlier version, the following posts may be of use:

Before beginning the update process take a backup and snapshot of the vCenter Server Appliance. There is downtime during the update but this is minimal – around 10 mins to update and reboot using an ISO as an update source, when using the online repository the update time may vary depending on your internet connection.

Review the version release notes and the VMware Docs site here.

VAMI Update

Platform Services Controller (PSC) appliances that are replicating should all be updated before the vCenter Server appliances. The easiest way of updating the vCenter Servers and Platform Services Controllers is through the VAMI (vCenter Server Appliance Management Interface). Browse to https://PSC:5480, where PSC is the FQDN or IP address of the external Platform Services Controller. Log in as the root user.

VAMI1

Select the Update option from the navigator.

vcupgrade2

Click the Check Updates drop-down. If the VCSA has internet access then select Check Repository to pull the update direct from the VMware online repository.

If the VCSA does not have internet access, or you’d prefer to provide the update manually then download the relevant update from VMware here (in this case VMware-vCenter-Server-Appliance-6.5.0.14000-7515524-patch-FP.iso) and attach the ISO to the CD/DVD drive of the VCSA in the virtual machine settings. Back in the VAMI update page select the Check Updates drop-down and click Check CDROM.

VAMI3

Details of the available update from either the online repository or attached ISO are displayed. Click Install Updates. Accept the EULA and click Install to begin the installation.

vcupgrade3

When the update process has completed click OK. From an attached ISO the update took around 5 minutes. The updated version and release date should now be displayed in the current version details. Finally, to complete the upgrade reboot the vCenter Server Appliance. Select Summary from the navigator and click Reboot.

vcupgrade4

If you are running multiple external PSCs then repeat the above process for each PSC in the SSO domain. Do not update the vCenter Server appliances until all PSC appliances are running the same updated version.

Once all external PSC appliances that replicate between one another have been upgraded then move on to the vCenter Server appliances. Repeat the above process for each vCenter Server in the SSO domain.

CLI Update

Alternatively the vCenter Server Appliance can be updated from the command line. Again, either using the online repository or by downloading the update from VMware here (VMware-vCenter-Server-Appliance-6.5.0.10000-5973321-patch-FP.iso or latest version) and attaching the ISO to the CD/DVD drive of the VCSA in the virtual machine settings. For more information on updating the vCenter Server Appliance using the appliance shell see this section of VMware docs.

Platform Services Controller (PSC) appliances that are replicating should all be updated before the vCenter Server appliances. Log in to the external Platform Services Controller appliance as root. First stage the patches from your chosen source using either:

  • software-packages stage --iso --acceptEulas stages software packages from ISO and accepts EULA.
  •  software-packages stage --url --acceptEulas stages software packages from the default VMware online repository and accepts EULA.

Next, review the staged packages, install the update, and reboot the VCSA.

  • software-packages list --staged lists the details of the staged software package.
  • software-packages install --staged installs the staged software package.
  • shutdown reboot -r update reboots the VCSA where ‘update’ is the reboot reason. Use -d to add a delay.

CLI4

If you are running multiple external PSCs then repeat the above process for each PSC in the SSO domain. Do not update the vCenter Server appliances until all PSC appliances are running the same updated version.

Once all external PSC appliances that replicate between one another have been upgraded then move on to the vCenter Server appliances. Repeat the above process for each vCenter Server in the SSO domain.

Configuring VMware Cross-vCenter NSX

This post provides an overview of cross-vCenter NSX and walks through the configuration steps. Cross-vCenter NSX allows central management of network virtualization and security policies across multiple vCenter Server systems. Cross vCenter NSX introduces universal objects; such as universal logical switches, universal logical routers, and universal distributed firewall rules. Universal objects are able to span multiple sites or vCenter Server instances, enhancing workload mobility by allowing cross vCenter and long distance vMotion for virtual machines, whilst keeping the same network settings and firewall rules. This improves DR capabilities, overcomes scale limits of vCenter Server, and gives administrators more control over resource pooling and the separation of environments.

Cross vCenter-NSX was introduced in NSX v6.2 and requires vSphere v6.0 or later. As normal NSX Manager is deployed with vCenter server in a 1:1 pairing.  In a single site NSX deployment the NSX Manager is given the standalone role by default. When configuring cross-vCenter NSX one NSX Manager is assigned the primary role, and up to seven other NSX Managers are assigned the secondary role. NSX Managers configured for cross-vCenter NSX must all be running the same version. The primary NSX Manager is responsible for deploying the Universal Controller Cluster; forming the control plane across the NSX Managers. The Universal Controller Cluster runs in the site of the primary NSX Manager. Universal objects are created on the primary NSX Manager and automatically synchronized across the multi-site NSX environment.

Configuring Cross-vCenter NSX

The steps below assume you have already deployed and registered the NSX Managers, and have a good understanding of NSX. This post is intended as add on to the NSX Install Guide to provide an outline of the additional or different steps required for a cross-vCenter NSX install, further resources are listed at the bottom of the page. If you are using vCenter enhanced linked mode then multiple NSX Manager instances are displayed within the same interface, or single pane of glass, when managing the Network & Security section of the vSphere web client. Enhanced linked mode is not a requirement for cross-vCenter NSX however, and vCenter Server systems not in enhanced linked mode can still be configured for cross-vCenter NSX.

From the Networking & Security page of the vSphere web client select Installation, highlight the NSX Manager in the primary site, from the Actions menu select Assign Primary Role.

NSX_Promote

The secondary NSX Manager(s) synchronize with the primary using the Universal Synchronization Service. These sites do not run any NSX Controllers, although they can be redeployed easily in the event of a primary site outage. Before assigning the secondary role you should ensure there are no existing NSX Controllers deployed in the associated vCenter. If you have already assigned a segment ID pool to the NSX Managers then ensure the segment ID pools do not overlap. Select the primary NSX Manager and from the Actions menu click Add Secondary NSX Manager. Enter the secondary NSX Manager information and admin password.

NSX2

Review the table of NSX Managers, the roles have now changed accordingly.

NSX_Roles

The universal controller cluster is formed by individually deploying the NSX controllers from the primary NSX Manager, the method of deploying the controllers is the same (see NSX Install Guide Part 1 – Mgmt and Control Planes for further assistance). Once the controllers are deployed you will notice placeholder controllers listed against the secondary NSX Manager, these are not connected or deployed. In the event of a site failure the configuration is synchronized between NSX Managers so you can simply re-deploy the controllers in the DR site. To see the failover process review this blog post. VMware recommend deploying 3 controllers on different hosts with anti-affinity rules.

NSX_Controllers

The next part of the install process is to follow the host preparation and VXLAN configuration steps as normal (see NSX Install Guide Part 2 – Data Plane for further assistance). Create the segment ID pools for each NSX Manager, making sure they do not overlap. On the primary NSX Manager you will also assign a universal segment ID pool.

In order for us to deploy universal logical switches we need to create a universal transport zone. A universal transport zone determines which hosts a universal logical switch can reach, spanning multiple vCenters. From the Logical Network Preparation tab open Transport Zones, ensure the primary NSX Manager is selected and click the plus symbol. Select Mark this object for Universal Synchronization, and enter the configuration for the universal transport zone. All universal objects must be created on the primary NSX Manager, change the NSX Manager to the secondary site and you will see the universal transport zone has synchronized there also.

NSX_TZ

Next we will create a universal logical switch for the transit network. Local objects such as logical switches, logical routers, and Edge Services Gateways can still be deployed from each NSX Manager, although by design they are only local to the vCenter linked to that specific NSX Manager, and cannot be deployed or edited elsewhere. From the left hand navigation pane in Networking & Security select Logical Switches, ensure the primary NSX Manager is selected and click the plus symbol. Enter a name for the transit network and select the universal transport zone we created earlier.

NSX_Universal_Transit

At this stage you can also deploy another universal logical switch, connecting a couple of test VMs on a private subnet, and have them ping one another to confirm connectivity. Now that we have a transit network and test universal logical switches connected to our universal transport zone we can go ahead and create a universal DLR. In this particular environment we have already deployed an ESG in each site. For further assistance with deploying an ESG and DLR see NSX Install Guide Part 3 – Edge and DLR.

From the Networking & Security page click NSX Edges, ensure the primary NSX Manager is selected and click the plus symbol. The control VM for the DLR is deployed to the primary site, again the configuration is synchronized and this can be re-deployed to the DR site in the event of a primary site outage. Select Universal Logical Router and follow the wizard as normal, if local egress is required then check the appropriate box. Sites configured in a cross-vCenter NSX environment can use the same physical routers for egress traffic, or have the local egress feature enabled within a universal logical router. The local egress feature allows routes to be customized at host, cluster, or router level.

NSX_UDLR

From the NSX Edges page double click the new universal DLR, select Manage, Settings, Interfaces and click the add button. In order for traffic to route from the universal DLR to the ESG(s) we must add an uplink interface connecting them to the universal transit network. Change the logical router interface to Uplink, in the Connected To field select the transit network universal logical switch we created earlier. Configure the IP and MTU settings of the interface per your own environment.

NSX_UDLR_Interface

You can also add Internal interfaces here corresponding with universal logical switches for virtual machine subnets. Before these subnets can route out follow the same process to add an Internal interface to the ESG(s) connecting them to the same transit network.

A virtual machine connected to the test universal logical switch can now vMotion between sites keeping the same IP addressing, providing L2 over L3 capability. As well as remaining on the same logical network a virtual machine can also be migrated across sites without any additional firewall rules, this is achieved with the use of universal firewall rules. Universal firewall rules require a dedicated section creating under the Firewall section of Networking & Security, you must select Mark this section for Universal Synchronization. For assistance with creating universal firewall rules see here.

NSX_Universal_Firewall

Additional Resources

To plan a cross-vCenter NSX installation review the VMware Cross-vCenter NSX Design Guide, Cross-vCenter NSX Topologies Guide, and the VMware Cross-vCenter Installation Guide. For more information on cross-vCenter NSX design see the following blog posts: