Tag Archives: Veeam

Connecting VMware Cloud on AWS to Amazon EC2

This post demonstrates the connectivity between VMware Cloud (VMC) on AWS and native AWS services. In the example below we will be using Amazon Elastic Compute Cloud (EC2) to provision a virtual instance backed by Amazon Elastic Block Store (EBS) storage. To complete the use case we will install Veeam and use the EC2 instance to backup virtual machines hosted in the VMware Cloud Software-Defined Data Centre (SDDC). Further Reading: How to Deploy and Configure VMware Cloud on AWS (Part 1), How to Migrate VMware Virtual Machines to VMware Cloud on AWS (Part 2).

Connectivity Overview

  • VMware Cloud on AWS links with your existing AWS account to provide access to native services. During provisioning a Cloud Formation template will grant AWS permissions using the Identity Access Management (IAM) service. This allows your VMC account to create and manage Elastic Network Interfaces (ENI) as well as auto-populate Virtual Private Cloud (VPC) route tables.
  • An Elastic Network Interface (ENI) dedicated to each physical host connects the VMware Cloud to the corresponding Availability Zone in the native AWS VPC. There is no charge for data crossing the 25 Gbps ENI between the VMC VPC and the native AWS VPC, however it is worth remembering that data crossing Availability Zones is charged at $0.01 per GB (at the time of writing).
  • The example architecture we will be using is shown below.

VMC_Connectivity

Security Group Configuration

AWS Security Groups will be attached to your EC2 instances and ENIs, it is therefore vital that you fully understand the concepts and configuration you are implementing. Please review Understanding AWS Security Groups with VMware Cloud on AWS by Brian Graf.

In the AWS console Security Groups can be accessed from the EC2 service. In this example I have created a security group allowing all protocols (any port) inbound from the source CIDR block used in VMC for both my compute and management subnets. In other words this is allowing connectivity into the EC2 instance from VM in my VMC SDDC. You may want to lock this down to specific IP addresses or ports to provide a more secure operating model. Outbound access from the EC2 instance is defined as any IPv4 destination (0.0.0.0/0) on any port.

Veeam_SG

I have also changed the default security group associated with the ENIs used by VMC to a custom security group. The security group allows inbound access on the ENI (which is inbound access to VMC as explained in the article below) on all ports from the source CIDR block of my native AWS VPC. Outbound access which is from VMC into AWS is defined as any IPv4 destination (0.0.0.0/0) on any port.

ENI_SG

EC2 Deployment

Log into the VMware on AWS Console, from the SDDCs tab locate the appropriate SDDC and click View Details. Select the Networking & Security tab. Under System click Connected VPC. Make a note of the AWS Account ID and the VPC ID. You will need to deploy an EC2 instance into this account and VPC.

Log into the AWS Console and navigate to the EC2 service. Launch an EC2 instance that meets the System Requirements for Veeam. In this example I have used the t2.medium instance and Microsoft Windows Server 2019 Base AMI. When configuring network the EC2 instance must be in the VPC connected to VMC. I have added an additional EBS volume for the backup repository using volume type General Purpose SSD (gp2). Ensure the security group selected or created allows the relevant access.

Gateway Firewall

In addition to security group settings inbound access also needs allowing on the VMC Gateway Firewall. In this instance as we are connecting the EC2 instance to the vCenter we define the rule on the Management Gateway. If we were connecting to a workload in one of the compute subnets the rule would be defined on the Compute Gateway. You may have noticed that although I allowed any port in the AWS Security Groups, the actual ports allowed can also be defined on the Gateway Firewall.

In this example I have added a new user defined group which contains the private IPv4 address for the EC2 instance and added it as a source in the vCenter Inbound Rule. The allowed port is set to HTTPS (TCP 443) – I have also allowed ICMP. I have added the same source group to the ESXi Inbound Rule which allows Provisioning (TCP 902). Both these rules are needed to allow Veeam to backup virtual machines in VMC.

VMC_GW_FW

Veeam Setup

Now that connectivity between the EC2 instance and the VMC vCenter has been configured I can hop onto the EC2 instance and begin the setup of Veeam. I will, of course, need an inbound rule for RDP (TCP 3389) adding to the security group of the EC2 instance, specifying the source I am connecting from.

Follow the installation steps outlined in the Veeam Backup & Replication 9.5 Update 4 User Guide for VMware vSphere.

Veeam_1

In the VMC console navigate to the Settings tab of the SDDC and make a note of the  password for the cloudadmin@vmc.local account. Open the Veeam Backup & Replication console and add the vCenter private IP address, use the vCenter cloud admin credentials.

Veeam_2

Add the backup repository using the EBS volume and create a backup job as normal. Refer to the Veeam Backup Guide if you need assistance with Veeam.

Veeam_3

To make use of S3 object storage AWS you will need an IAM Role granting S3 access, and an S3 VPC Endpoint. In the case of VMC, as an alternative design, you can host the Veeam B&R server inside your VMC SDDC to make use of the built in S3 endpoint. In testing we found backup speeds to be faster but you will likely still need an EBS backed EC2 instance for your backup repository. It goes without saying you should make sure backup data is not held solely on the same physical site as the servers you are backing up. See Veeam KB2414: VMware Cloud on AWS Support for further details.

Add a new Scale-Out Backup Repository and follow the steps to add account and bucket details.

 

Set an appropriate policy for moving backups to object based storage, once this threshold is met you will start to see Veeam files populating the S3 bucket.

S3_repo

Veeam Integration with vRA Part 2: Restore

In this 2 part series we will walk through integrating Veeam with vRealize Automation and vRealize Orchestrator. Part 1 focused on giving users the ability to add virtual machines to existing Veeam backup jobs from within the vRA self-service portal. In Part 2 we will add the ability to restore virtual machines from a list of available restore points in vRA. The versions used are Veeam 9.5 and vRA 7.2 / 7.3.

The steps outlined below assume that you have already installed and configured Veeam Backup and Replication, and vRealize Automation with either embedded or external vRealize Orchestrator instance, as well as having a basic knowledge of both areas. The following process and the sample workflows we will import are not endorsed by, or supported by Veeam. Finally, Veeam Enterprise Manager is required to use Veeam RESTful API. For further reading material see the Veeam RESTful API Reference here. Alternative sample workflows and reading provided by The IT Hollow here, and another useful article by vRatpack here with vRA 6.2.

Add the REST Host

If you have already added your Veeam backup server as a REST host in part 1 then skip this step. Otherwise, open the vRealize Orchestrator client and log in as an administrator, change the view to Design from the drop down menu. The first thing we will do is add the Veeam server as a REST host. From the Workflows tab expand Library, HTTP-REST, Configuration.

REST_host_1

Right click Add a REST host and click Start workflow. Enter the name and URL of the Veeam server, the default URL uses port 9399, for example http://VeeamServer:9399. Review the default options and click Next.

REST_host_2

Configure the host authentication options as required. Here I have used Basic authentication, and entered the credentials for a service account with administrative access to Veeam.

REST_host_3

Configure proxy and advanced settings if required, then click Submit. The workflow will run and add the Veeam server as a REST host. There are also Update a REST host, and Remove a REST host, workflows if you want to make any changes. Existing REST hosts can be viewed from the inventory tab under expand HTTP-REST.

Import the Sample Workflows

If you have already imported the sample workflows in part 1 then skip this step. In this example I am using sample workflows provided here, again these are not supported by Veeam. Download and extract the ZIP file to a location accessible from the vRO client. Change to the packages tab and click the Import Package icon. When prompted browse to the downloaded package file and click Import.

Veeam_Package_1

Ensure all the required elements are included and click Import selected elements.

Veeam_Package_2

We have now imported the backup workflow and action, and the restore workflow and action. The final element is a settings file which we will use to determine the REST host. Open the configurations tab and expand Library, Veeam. Click the Settings file and the pencil icon to edit. Select Attributes and locate the restHost attribute, click the Not set value and expand HTTP-REST, select the Veeam server we added earlier from the list of REST hosts and click Select. Click Save and close. The value of the restHost attribute should now be the Veeam backup server.

The restore jobs users select from are pulled using the getVMRestorePoints action under com.veeam.library in the actions tab. If you want to examine the workflow in more detail go to the workflows tab and expand Library, Veeam. Select the Restore VM workflow and go through the tabs in the right hand pane. From the General tab you can see the restHost attribute is using the settings configuration file we have just configured. The Inputs for the workflow are Date (the Veeam restore point) and vmObj (virtual machine name). Under the Schema tab you can view the Scripting task which is making the API calls.

Restore_VM

Update Sample Script

If you are using the sample script referenced in this post then there are further steps required to fix the date formatting with later versions of Veeam. If you are using alternative or custom workflows then the following is not required.

  • Edit the Restore VM workflow, open the Schema tab and click the Find Restore Point script. Update the date and time format on line 25 to: var rpDateLocale = System.getDateFromFormat(restorePointNodes.item(i).getElementsByTagName(“CreationTimeUTC”).item(0).textContent,”yyyy-MM-dd’T’HH:mm:ss.sss’Z'”).toLocaleString();

Find_Restore_Point_OldFind_Restore_Point_New

  • Edit the getVMRestorePoints action, open the Scripting tab. Update the date and time format on line 26 to: var rpDateLocale = System.getDateFromFormat(restorePointNodes.item(i).getElementsByTagName(“CreationTimeUTC”).item(0).textContent + ” UTC”,”yyyy-MM-dd’T’HH:mm:ss.sss’Z’ ZZZ”).toLocaleString();

Restore_Point_Action_OldRestore_Point_Action_New

  • You can test the API calls are successfully bringing back restore points by running the workflow in vRO and selecting a virtual machine, a list of available restore points should be displayed.

Run_vRO

vRA Integration

The final step is to hook the vRO workflow into vRA. Log into the vRealize Automation portal as a user with service architect permissions. From the Design tab select XaaS and Resource Actions. Any existing resource actions are listed. Click New.

Existing_Resource

Map the resource action to the relevant vRO workflow. In this case we need to expand Library, Veeam and select the Restore VM workflow. Click Next.

Restore_VM_Resource

The input mappings should already be populated; the resource type is IaaS VC Virtual Machine, the input parameter matches up with the parameter configured in the vRO workflow (vmObj which passes the virtual machine name), and this maps to the VC:VirtualMachine orchestrator type.

Restore_VM_Input

Accept the default values for the resource action form and click Finish.

Restore_VM_Form

The new resource action is now listed as a draft. To start using the action select it and click Publish.

New_Resource

Now select the Administration tab and Catalog Management. Open the Actions page, the new resource action we created should now be displayed.

If you want to change the icon of the resource action you can do so by selection the action and clicking Configure. There are a number of useful vRA icons available here, including sample icons for day 2 actions. Note for users of vRA 7.2 there is a known issue with changing the icon for custom actions, resolved in 7.3 as per this KB article.

Restore_VM_Action

The next step is to assign our custom action to an entitlement. Open the Entitlements page and select the relevant entitlement. Click the Items & Approvals tab, under Entitled Actions click the green plus symbol. Locate the new resource action and select the check box to add it to the entitled actions. Click Ok and Finish.

Restore_VM_Entitlement

To confirm the configuration has worked browse to the Items tab and select Machines. Any virtual machines that have the custom resource action added to the entitlement will show the new action in the drop-down Actions menu.

restore_vm_item

When selecting the new action I am presented with the action form as per the design canvas we saw earlier. In this example I select the restore point from the drop-down list that the getVMRestorePoints vRO action has pulled from the Veeam backup server, and click Submit.

restore_request

The virtual machine name is then passed through to the next stage of the workflow, along with the restore point ID. You can check the status of the job in vRA under the Requests tab, check the Restore VM workflow has run successfully in the vRO console, and check the restore task that will be running as normal in the Veeam Backup & Replication console.

_______________

Veeam Integration with vRA Part 1: Backup

Veeam Integration with vRA Part 2: Restore

Veeam Integration with vRA Part 1: Backup

In this 2 part series we will walk through integrating Veeam with vRealize Automation and vRealize Orchestrator. Part 1 will focus on giving users the ability to add virtual machines to existing Veeam backup jobs from within the vRA self-service portal. In Part 2 we will add the ability to restore virtual machines from a list of available restore points in vRA. The versions used are Veeam 9.5 and vRA 7.2 / 7.3.

The steps outlined below assume that you have already installed and configured Veeam Backup and Replication, and vRealize Automation with either embedded or external vRealize Orchestrator instance, as well as having a basic knowledge of both areas. The following process and the sample workflows we will import are not endorsed by, or supported by Veeam. Finally, Veeam Enterprise Manager is required to use Veeam RESTful API. For further reading material see the Veeam RESTful API Reference here. Alternative sample workflows and reading provided by The IT Hollow here, and another useful article by vRatpack here with vRA 6.2.

Add the REST Host

Open the vRealize Orchestrator client and log in as an administrator, change the view to Design from the drop down menu. The first thing we will do is add the Veeam server as a REST host. From the Workflows tab expand Library, HTTP-REST, Configuration.

REST_host_1

Right click Add a REST host and click Start workflow. Enter the name and URL of the Veeam server, the default URL uses port 9399, for example http://VeeamServer:9399. Review the default options and click Next.

REST_host_2

Configure the host authentication options as required. Here I have used Basic authentication, and entered the credentials for a service account with administrative access to Veeam.

REST_host_3

Configure proxy and advanced settings if required, then click Submit. The workflow will run and add the Veeam server as a REST host. There are also Update a REST host, and Remove a REST host, workflows if you want to make any changes. Existing REST hosts can be viewed from the inventory tab under expand HTTP-REST.

Import the Sample Workflows

In this example I am using sample workflows provided here, again these are not supported by Veeam. Download and extract the ZIP file to a location accessible from the vRO client. Change to the packages tab and click the Import Package icon. When prompted browse to the downloaded package file and click Import.

Veeam_Package_1

Ensure all the required elements are included and click Import selected elements.

Veeam_Package_2

We have now imported the backup workflow and action, and the restore workflow and action. The final element is a settings file which we will use to determine the REST host. Open the configurations tab and expand Library, Veeam. Click the Settings file and the pencil icon to edit. Select Attributes and locate the restHost attribute, click the Not set value and expand HTTP-REST, select the Veeam server we added earlier from the list of REST hosts and click Select. Click Save and close. The value of the restHost attribute should now be the Veeam backup server.

The backup jobs users select from are pulled using the getBackupJobs action under com.veeam.library in the actions tab. If you want to examine the workflow in more detail go to the workflows tab and expand Library, Veeam. Select Add VM to Backup Job and go through the tabs in the right hand pane. From the General tab you can see the restHost attribute is using the settings configuration file we have just configured. The Inputs for the workflow are jobname (Veeam backup job) and vmObj (virtual machine name). Under the Schema tab you can view the Scripting task which is making the API calls.

Add_VM

vRA Integration

The final step is to hook the vRO workflow into vRA. Log into the vRealize Automation portal as a user with service architect permissions. From the Design tab select XaaS and Resource Actions. Any existing resource actions are listed. Click New.

Existing_Resource

Map the resource action to the relevant vRO workflow. In this case we need to expand Library, Veeam and select the Add VM to Backup Job workflow.

Backup_VM_Resource

The input mappings should already be populated; the resource type is IaaS VC Virtual Machine, the input parameter matches up with the parameter configured in the vRO workflow (vmObj which passes the virtual machine name), and this maps to the VC:VirtualMachine orchestrator type.

Backup_VM_Input

Accept the default values for the resource action and click Finish.

Backup_VM_Form

The new resource action is now listed as a draft. To start using the action select it and click Publish.

vra6

Now select the Administration tab and Catalog Management. Open the Actions page, the new resource action we created should now be displayed.

If you want to change the icon of the resource action you can do so by selection the action and clicking Configure. There are a number of useful vRA icons available here, including sample icons for day 2 actions. Note for users of vRA 7.2 there is a known issue with changing the icon for custom actions, resolved in 7.3 as per this KB article.

vra7

The next step is to assign our custom action to an entitlement. Open the Entitlements page and select the relevant entitlement. Click the Items & Approvals tab, under Entitled Actions click the green plus symbol. Locate the new resource action and select the check box to add it to the entitled actions. Click Ok and Finish.

vra8

To confirm the configuration has worked browse to the Items tab and select Machines. Any virtual machines that have the custom resource action added to the entitlement will show the new action in the drop-down Actions menu.

vra9

When selecting the new action I am presented with the action form as per the design canvas we saw earlier. In this example I select the backup job from the drop-down list of jobs that the getBackupJobs vRO action has pulled from the Veeam backup server, and click Submit.

vra10

The virtual machine name is then passed through to the next stage of the workflow, which adds the virtual machine to the selected backup job. You can check the status of the job in vRA under the Requests tab, check the Add VM to Backup Job workflow has run successfully in the vRO console, and check the backup job itself has been updated using the Veeam Backup & Replication console.

_______________

Veeam Integration with vRA Part 1: Backup

Veeam Integration with vRA Part 2: Restore