vCenter Server Appliance Integrated TFTP Server

This post covers the steps required to use the vCenter Server Appliance for Auto Deploy, with the built in TFTP server in vSphere 6.5. For more information on Auto Deploy, and to see the process for creating ESXi images and deploy rules to boot hosts, see the VMware Auto Deploy 6.x Guide. This post assumes that you have a working vCenter Server Appliance, and may be of use if you have recently migrated from Windows vCenter to VCSA.

Enable Auto Deploy

Open the vSphere web client and click System Configuration, Nodes. Select the vCenter Server and open the Related Objects tab. The Auto Deploy, ImageBuilder Service, and VMware vSphere ESXi Dump Collector services should all be set to Automatic and Running.

To start a service right click and select Start, then select Edit Startup Type and choose Automatic.

servicesLog out of the web client and log back in. You should now see the Auto Deploy icon on the home page.

autodeploy1

Enable TFTP

Now that Auto Deploy is enabled we can configure the TFTP server. Enable SSH on the VCSA by browsing to the Appliance Management page: https://VCSA:5480 where VCSA is the IP or FQDN of your appliance.

Log in as the root account. From the Access page enable SSH Login and Bash Shell.

SSH

SSH onto the vCenter Appliance, using a client such as Putty, and log in with the root account. First type shell and hit enter to launch Bash.

To start the TFTP service enter service atftpd start. Check the service is started using service atftpd status.

startservice

To allow TFTP traffic through the firewall on port 69; we must run iptables -A port_filter -p udp -m udp –dport 69 -j ACCEPT. Validate traffic is being accepted over port 69 using iptables -nL | grep 69.

firewall

The TFTP server will now work, however we need to make a couple of additional changes to make the configuration persistent after the VCSA is rebooted. There isn’t an official VMware way of doing this, and as it’s done in Linux there may be more than one way of achieving what we want. Basically I am going to backup iptables and create a script to restore iptables and start the TFTP service when the appliance boots. The steps are outlined below and this worked for me, however as a reminder this is not supported by VMware, and if you are a Linux expert you’ll probably find a better way round it.

The following commands are all run in Bash on the vCenter Server Appliance, you can stay in the existing session we were using above.

First make a copy of the existing iptables config by running iptables-save > /etc/iptables.rules.

Next change the directory by running cd /etc/init.d, and create a new script: vi scriptname.sh, for example: vi starttftp.sh.

Press I to begin typing. I used the following, which was copied from the Image Builder Service startup script, and modified for TFTP.

#! /bin/sh
#
# TFTP Start/Stop the TFTP service and allow port 69
#
# chkconfig: 345 80 05
# description: atftpd

### BEGIN INIT INFO
# Provides: atftpd
# Required-Start: $local_fs $remote_fs $network
# Required-Stop:
# Default-Start: 3 5
# Default-Stop: 0 1 2 6
# Description: TFTP
### END INIT INFO

service atftpd start
iptables-restore -c < /etc/iptables.rules

The file must be in the above format to be compatible with chkconfig which runs the script at startup. I left the defaults in from the Image Builder Service as it made sense they started at the same time and had the same dependencies. If you wish to modify further see the following sources: Bash commands, Script Options, Startup, and vmwarebits.com for the iptables commands.

Press escape to leave the editor and :wq to save the file and quit.

Next set execute permissions on the script by running chmod +x scriptname.sh, for example: chmod +x starttftp.sh.

To set the script to run at startup use chkconfig –add scriptname.sh, for example: chkconfig –add starttftp.sh.

Reboot the vCenter appliance to test the script is running. If successful the atftpd service will be started and port 69 allowed, you can check these with service atftpd status and iptables -nL | grep 69.

Close the session and disable SSH if required.

Configure DHCP

In this example I will be using PXE boot to boot the ESXi hosts using a DHCP reservation. On the DHCP scope that will be serving the hosts I have configured reservations and enabled options 066 and 067. In the value for option 066 (Boot Server Host Name) goes the IP address or FQDN of the vCenter Server where TFTP is running. In the value for option 067 (Bootfile Name) I have entered the BIOS DHCP File Name (undionly.kpxe.vmw-hardwired).

DHCP

Now that Auto Deploy is up and running using the built-in components of VCSA 6.5 you can begin creating ESXi images and deploy rules to boot hosts; using the Auto Deploy GUI. See the VMware Auto Deploy 6.x Guide.

Configuring vCenter 6.5 High Availability

VMware vSphere 6.5 is scheduled to reach end of general support 15 October 2022, referenced in the VMware Lifecycle Matrix. See also How to Install vSphere 7.0. Upgrade to vSphere 7 can be achieved directly from vSphere 6.5.0 and above, whereas vSphere 6.0 requires an intermediate upgrade to 6.5 or 6.7 first. For more information see the VMware Upgrade Matrix. Finally, the Windows vCenter Server and external PSC deployment models are now depreciated and not available with vSphere 7.0.

The vCenter Server Appliance now provides vCenter High Availability (HA) with vSphere 6.5 onwards. By implementing vCenter HA you can protect your vCenter from host and hardware failures, and significantly reduce down time during patching due to the active / standby nature of the vCenter cluster. In vSphere 6.7 Update 1 onwards the vCenter HA configuration is simplified, see Configuring vCenter 6.7 High Availability for more information.

The vCenter HA architecture is made up of the components in the vSphere image below. The vCenter Server Appliance is cloned out to create passive and witness nodes. Updated data is replicated between the active and passive nodes. In the event of an outage to the active vCenter the passive vCenter automatically assumes the active role and identity. Management connections still route to the same IP address and FQDN, however they have now failed over to the replica node. When the outage is resolved and the vCenter that failed comes back online; it then takes on the role of the passive node, and receives replication data from the active vCenter Server.

vCenter_HA_1
728x90

Requirements

  • vCenter HA was introduced with the vCenter Server Appliance 6.5
  • The vCenter deployment size should be at least small, and therefore 4 vCPU 16 GB RAM
  • A minimum of three hosts
  • The hosts should be running at least ESXi 5.5
  • The management network should be configured with a static IP address and reachable FQDN
  • SSH should be enabled on the VCSA
  • A port group for the HA network is required on each ESXi host
  • The HA network must be on a different subnet to the management network
  • Network latency between the nodes must be less than 10ms
  • vCenter HA is compatible with both embedded deployment model and external PSC
  • For further information on vCenter HA performance and best practises see this post

Configuration Types

When setting up vCenter HA we are given the option of basic configuration or advanced. The correct deployment type depends on your environment. If the VCSA is managing its own ESXi host and virtual machine, or is managed by another vCenter Server in the same SSO domain then the basic deployment method should be used. This automatically clones the vCenter, and creates DRS anti-affinity rules.

If the VCSA is on a separate vCenter in a different SSO domain then the advanced deployment method should be used. In this case we need to manually add an additional NIC and clone the VCSA. The basic and advanced configuration types produce the same end result, but with a different process for different environments.

Both the embedded PSC and external PSC deployment models are supported. In this post we will walk through the advanced and basic configuration steps for vCenter with embedded PSC. For external PSC a load balancer can be implemented to provide HA, you can read more about implementing vCenter HA with the external deployment model here. If you are configuring vCenter HA in a cluster with less than the required number of physical hosts, such as in a home lab, you can add a parameter to override the anti-affinity setting; see this post by William Lam.

Basic Configuration Process

Log into the vSphere web client. Right click the top level vCenter Server in the inventory and select vCenter HA Settings. Click Configure in the top right hand corner.

ha1

Select the configuration type, in this example we are going to use Basic. Click Next.

basic1

An additional NIC will automatically be added to the active VCSA. Select the HA network to use and enter an IP address, remember this must be a separate subnet to the management network or the configuration wizard will error. Click Next.

basic2

Once the configuration wizard is complete the active VCSA will be cloned to create passive, and witness nodes. On this page we need to specify the HA IP addresses to use for each node, then click Next. You do not need to manually add any NICs during the basic configuration, this is all done for you. However as per the pre-requisites you do need to make sure a network is available to use for HA traffic.

basic3

Review the deployment page, if applicable you may need to change the compute or datastore locations by clicking Edit to ensure each component is spread across the vSphere cluster.

basic4

As you can see on the final page clone tasks will automatically be created. The new VMs are named VCSA-peer and VCSA-witness, where VCSA is the VM name of your current vCenter Server Appliance. Click Finish.

basic5

Monitor the tasks pane, vCenter HA may take around 5 minutes to clone and deploy the cluster nodes, depending on the speed of your underlying infrastructure. Once complete the vCenter HA status will show Enabled, and all nodes in the cluster will show Up.

basic6

You can edit the status of vCenter HA at any time by going back into the vCenter HA menu and clicking Edit. These are the available options.

edit

Advanced Configuration Process

The advanced deployment process takes longer as it involves much more manual configuration. The first thing we need to do is add an additional network adapter to our existing vCenter Server Appliance, and configure a vCenter HA IP address. Log into the vSphere web client of the vCenter managing the VCSA. Locate and right click the VCSA, select Edit Settings. From the New device drop down select Network and click Add. Select the port group to use, remember this needs to be a separate subnet to the management network, ensure Connect is ticked and click Ok.

Now we can configure the network settings using the Appliance Management portal. Browse to https:// :5480 where is the IP address or FQDN of your vCenter Server Appliance. Log in with the root password.

backup1

Select Networking on the left hand navigation menu.

backup2

Open the Manage tab and click Edit next to the Networking Interfaces box. Expand nic1, note that the status is down, configure the IP settings and click Ok.

ip

Verify that nic1 is now showing a status of Up.

interfaces

We can now start the vCenter HA configuration wizard. Open the vSphere web client of the VCSA for which you want to configure HA. Right click the top level vCenter Server in the inventory and select vCenter HA Settings. Click Configure in the top right hand corner.

ha1

Select the configuration type, in this example we are going to use Advanced. Click Next.

advanced1

Enter the IP address settings for the passive and witness nodes, on the HA network, then click Next.

advanced2

Now we need to do some manual cloning, go back to the vSphere client of the vCenter Server managing the VCSA. Locate the VCSA in the inventory, right click and select Clone, Clone to Virtual Machine.

Run through the clone wizard, let’s create the passive node first. During the clone wizard we configure all settings, including management IP address and host name, to be the same as the active VCSA except for the HA IP address. Each node has a unique IP address on the HA network.

  • Enter a name and location for the virtual appliance.
  • Select different compute resource and datastores to the active VCSA if possible.
  • On the clone options page select Customise the operating system, Power on virtual machine after creation.
clone1
  • On the customise guest OS page click the create new specification icon.
  • Enter a name and description for the new customisation.
clone2
  • Enter the same OS host name and domain as the active node.
  • Configure the same time zone as the active node.
  • On the network page edit the settings for NIC1, select use the following IP settings, and enter the management network settings of the active vcsa. This network adapter will be used to assume the identity of the active VCSA in the event of a fail over.
nic1
  • Edit the settings for NIC2, select prompt the user for an address when the specification is used. Enter the subnet mask and leave the gateway blank. This adapter will be used for the HA network, we will configure the unique IP address shortly.
nic2
  • On the DNS and domain settings page of the wizard enter the domain name and DNS server(s) that the interface will connect to, click Finish.
  • You will be returned to the clone virtual machine wizard. Select the newly created customisation profile.
  • Enter the IP address for NIC1. This is the HA IP for the passive node we input during the vCenter HA configuration wizard earlier.
usersettings
  • Accept the default virtual hardware and vApp properties.

The VCSA will now be cloned to create the passive node. Repeat the steps above for the witness node, however this time select the existing guest customisation that we created first time round.

customisation

Enter the unique HA IP address for the witness node that we specified during the vCenter HA configuration wizard.

usersettings

When these manual steps have been completed go back to the vCenter HA configuration wizard and click Finish. Monitor the Configure a vCenter HA Cluster task in the recent tasks pane.

clone

Once complete the vCenter HA status will show Enabled, and all nodes in the cluster will show Up.

basic6

For more information on vCenter HA or configuring different aspects of the advanced deployment; see the vCenter High Availability section of the vSphere 6.5 Documentation Centre.

The final step is to configure an anti-affinity rule to stop the vCenter Server appliances from running on the same hosts. Log into the vSphere web client and browse to Hosts and Clusters. Click the vSphere cluster and select the Manage tab. Under Configuration click VM/Host Rules. Under VM/Host Rules click Add.

Enter a name for the rule, such as vCenter HA, ensure Enable rule is ticked and select Separate Virtual Machines as the rule type. Click Add and select the vCenter Server nodes. Click Ok.

drs

This rule will ensure DRS does not place nodes on the same hosts in a vSphere cluster.

vCenter Appliance 6.5 Upgrade

VMware vSphere 6.5 is scheduled to reach end of general support 15 October 2022, referenced in the VMware Lifecycle Matrix. See also How to Install vSphere 7.0. Upgrade to vSphere 7 can be achieved directly from vSphere 6.5.0 and above, whereas vSphere 6.0 requires an intermediate upgrade to 6.5 or 6.7 first. For more information see the VMware Upgrade Matrix. Finally, the Windows vCenter Server is now depreciated and not available with vSphere 7.0.

VMware vCenter Server pools ESXi host resources to provide a rich feature set delivering high availability and fault tolerance to virtual machines. The vCenter Server is a centralised management application and can be deployed as a virtual appliance or Windows machine. This post walks through an upgrade of the vCenter Server Appliance from v5.5 or v6.0 to v6.5. See also vCenter Server Appliance 6.5 Install Guide, or Migrating Windows vCenter Server.

About VCSA

The VCSA is a pre-configured virtual appliance; as of v6.5 the operating environment is built on Project Photon OS 1.0. Since the OS has been developed by VMware it benefits from enhanced performance and boot times over the previous Linux based appliance. Furthermore the embedded Postgre database means VMware have full control of the software stack, resulting in significant optimisation for vSphere environments and quicker release of security patches and bug fixes. The VCSA scales up to 2000 hosts and 35,000 virtual machines.

In vSphere 6.0 the VCSA reached feature parity with its Windows counterpart, 6.5 begins to pave the way for VCSA to become the preferred deployment method for vCenter Server. One key addition is the inclusion of Update Manager bundled into the VCSA, as well as vCenter High Availability, Backup and Restore, and other features. The appliance also saves operating system license costs and is quicker and easier to deploy and patch.

Upgrading to VCSA 6.5 involves the deployment of a new appliance and migration of all configuration and historical data (optional) using the upgrade installer. The VCSA uses a temporary IP address during migration before switching to the IP and host name of the new VCSA, the old appliance is then powered off.

vcs

Software Considerations

  • VCSA 6.5 must be deployed to an ESXi host running v5.5 or above. All hosts you intend to connect to vCenter Server 6.5 should also be running ESXi v5.5 or above.
  • The VCSA to be upgraded can be either v5.5 or v6.0.
  • VCSA 6.5 does not support the use of an external database. Where a system using an external database is upgraded, the data is imported into the internal Postgres database within VCSA 6.5.
  • You must check compatibility of any third party products and plugins that might be used for backups, anti-virus, monitoring, etc. as these may need upgrading for use with vSphere 6.5.
  • If you are unsure check the Product Interoperability Matrix.

Architectural Considerations

  • From vSphere 6 onwards the Platform Services Controller (PSC) was introduced to the vSphere architecture. The PSC contains infrastructure services such as Single Sign On, Certificate Authority, licensing, etc. The PSC is deployed internally with vCenter Server or as an external component. Read more about the PSC in this KB.
  • When implementing a new vSphere 6.5 environment you should plan your topology in accordance with the VMware vCenter Server and PSC Deployment Types. Larger environments may require an external PSC.
  • When upgrading vCenter the deployment model already in place will be migrated, the upgrade supports different deployment topologies but can not make changes to the topology and SSO domain configuration.
  • In this post we will be upgrading vCenter Server 6.0 using the embedded deployment model. If you are using an external deployment model the PSC appliance must be upgraded before the vCenter Server.
  • Consider if the default self-signed certificates are sufficient or if you want to replace with custom CA or VMware CA signed certs, see Installing vCenter Internal CA signed SSL Certificates for more information.
embedded

Other Considerations

  • VCSA 6.5 with embedded PSC requires the following hardware resources (disk can be thin provisioned)
    • Tiny (up to 10 hosts, 100 VMs) – 2 CPUs, 10 GB RAM.
    • Small (up to 100 hosts, 1000 VMs) – 4 CPUs, 16 GB RAM.
    • Medium (up to 400 hosts, 4000 VMs) – 8 CPUs, 24 GB RAM.
    • Large (up to 1000 hosts, 10,000 VMs) – 16 CPUs, 32 GB RAM.
    • X-Large (up to 2000 hosts, 35,000 VMs) – 24 CPUs, 48 GB RAM – new to v6.5.
  • Storage requirements for the smallest environments start at 250 GB and increase depending on your specific database requirements. See the Storage Requirements document for further details.
  • Where the PSC is deployed as a separate appliance this requires 2 CPUs, 4 GB RAM, 60 GB disk.
  • Environments with ESXi host(s) with more than 512 LUNs and 2048 paths should be sized large or x-large.
  • The ESXi host on which you deploy the new appliance should not be in lockdown or maintenance mode.
  • All vSphere components should be configured to use an NTP server. The installation can fail or the vCenter Server Appliance vpxd service may not be able to start if the clocks are unsynchronized.
  • FQDN resolution should be in place when deploying vCenter Server.
  • Review the list of Required Ports for vCenter Server and PSC.
  • Official resources – vSphere 6.5 Documentation Centre, vSphere 6.5 Release Notes.
  • Read the Important information before upgrading to vSphere 6.5 KB.

Installation

Download the VMware vCenter Server Appliance 6.5 ISO from VMware downloads: v6.5.0 | v6.5.0 U1.

Unlike the Windows vCenter installer, which hasn’t changed much in v6.5; the VCSA installer has had a complete overhaul. You’ll notice straight away that the GUI is much cleaner, and multiple deployment options (install, upgrade, migrate, restore) are now bundled into one installer.

Mount the ISO on your computer. The VCSA 6.5 installer is compatible with Mac, Linux, and Windows. Browse to the corresponding directory for your operating system, e.g. \vcsa-ui-installer\win32. Right click Installer and select Run as administrator. As we are upgrading an existing system click Upgrade.

vcsa1

The installation is split into 2 stages, we begin with deploying a new appliance. The second stage migrates all data and settings. Click Next.

upgrade1

Accept the EULA and click Next.

upgrade2

Enter the details for the existing vCenter Server Appliance and the host or vCenter it is managed by. Click Next.

upgrade3

Enter the FQDN or IP address of the host, or vCenter upon which you wish to deploy the new VCSA, then click Next. The installer will validate access, if prompted with an untrusted SSL certificate message click Yes to continue. Tip – connect to the vCenter for visibility of any networks using a distributed switch, connecting to the host direct will only pull back networks using a standard switch.

upgrade4

Enter a VM name and root password for the new appliance, and click Next.

upgrade5

Configure the deployment size of the new appliance and click Next.

upgrade6

Select the datastore to locate the virtual appliance and click Next. Configure the temporary network settings for the appliance. These will only be used during migration of the data, once complete the temporary settings are discarded and the VCSA assumes the identity, including IP settings, of the old appliance. Click Next.

upgrade7

The new VCSA will now be deployed, once complete click Finish.

upgrade8

Stage 2 migrates data and identity across to the new VCSA, click Next.

upgrade9

Select the data to migrate and click Next.

upgrade10

Select whether or not to join the VMware Customer Experience Improvement Program and click Next.

upgrade11

Review the summary page, confirm you have taken a backup of the vCenter and click Finish.Click OK to the shut down warning.

upgrade12

Data will now be migrated to the new VCSA, once complete the old VCSA will be powered off and the network settings transferred.

upgrade13

When finished click Close, the vCenter upgrade is complete.

upgrade14

Post-Installation

Connect to the vCenter post install using the IP or FQDN of the vCenter. Access vSphere by clicking either the vSphere Web Client (Flash) or the vSphere Client (HTML5). Connect to the vSphere Web Client to manage your system, the thick client (Windows) is no longer supported.

vsphereweb

Log in to the vSphere Web Client using the SSO administrator login. Verify the installed version is correct under the Summary tab when selecting the vCenter, you can also go to Help > About.

vsphereclient

You must apply a new vCenter license key within 60 days. From the Hosts and Clusters view select the vCenter Server. Click Actions and Assign License. Select a license or use the green plus button to add a new license and click Ok.

You can obtain a 60 day trial license for vCenter Server here. If you have purchased vCenter Server then log into your licensing portal here. If the license key does not appear then check with your VMware account manager.

client