Veeam Integration with vRA Part 2: Restore

In this 2 part series we will walk through integrating Veeam with vRealize Automation and vRealize Orchestrator. Part 1 focused on giving users the ability to add virtual machines to existing Veeam backup jobs from within the vRA self-service portal. In Part 2 we will add the ability to restore virtual machines from a list of available restore points in vRA. The versions used are Veeam 9.5 and vRA 7.2 / 7.3.

The steps outlined below assume that you have already installed and configured Veeam Backup and Replication, and vRealize Automation with either embedded or external vRealize Orchestrator instance, as well as having a basic knowledge of both areas. The following process and the sample workflows we will import are not endorsed by, or supported by Veeam. Finally, Veeam Enterprise Manager is required to use Veeam RESTful API. For further reading material see the Veeam RESTful API Reference here. Alternative sample workflows and reading provided by The IT Hollow here, and another useful article by vRatpack here with vRA 6.2.

Add the REST Host

If you have already added your Veeam backup server as a REST host in part 1 then skip this step. Otherwise, open the vRealize Orchestrator client and log in as an administrator, change the view to Design from the drop down menu. The first thing we will do is add the Veeam server as a REST host. From the Workflows tab expand Library, HTTP-REST, Configuration.

REST_host_1

Right click Add a REST host and click Start workflow. Enter the name and URL of the Veeam server, the default URL uses port 9399, for example http://VeeamServer:9399. Review the default options and click Next.

REST_host_2

Configure the host authentication options as required. Here I have used Basic authentication, and entered the credentials for a service account with administrative access to Veeam.

REST_host_3

Configure proxy and advanced settings if required, then click Submit. The workflow will run and add the Veeam server as a REST host. There are also Update a REST host, and Remove a REST host, workflows if you want to make any changes. Existing REST hosts can be viewed from the inventory tab under expand HTTP-REST.

Import the Sample Workflows

If you have already imported the sample workflows in part 1 then skip this step. In this example I am using sample workflows provided here, again these are not supported by Veeam. Download and extract the ZIP file to a location accessible from the vRO client. Change to the packages tab and click the Import Package icon. When prompted browse to the downloaded package file and click Import.

Veeam_Package_1

Ensure all the required elements are included and click Import selected elements.

Veeam_Package_2

We have now imported the backup workflow and action, and the restore workflow and action. The final element is a settings file which we will use to determine the REST host. Open the configurations tab and expand Library, Veeam. Click the Settings file and the pencil icon to edit. Select Attributes and locate the restHost attribute, click the Not set value and expand HTTP-REST, select the Veeam server we added earlier from the list of REST hosts and click Select. Click Save and close. The value of the restHost attribute should now be the Veeam backup server.

The restore jobs users select from are pulled using the getVMRestorePoints action under com.veeam.library in the actions tab. If you want to examine the workflow in more detail go to the workflows tab and expand Library, Veeam. Select the Restore VM workflow and go through the tabs in the right hand pane. From the General tab you can see the restHost attribute is using the settings configuration file we have just configured. The Inputs for the workflow are Date (the Veeam restore point) and vmObj (virtual machine name). Under the Schema tab you can view the Scripting task which is making the API calls.

Restore_VM

Update Sample Script

If you are using the sample script referenced in this post then there are further steps required to fix the date formatting with later versions of Veeam. If you are using alternative or custom workflows then the following is not required.

  • Edit the Restore VM workflow, open the Schema tab and click the Find Restore Point script. Update the date and time format on line 25 to: var rpDateLocale = System.getDateFromFormat(restorePointNodes.item(i).getElementsByTagName(“CreationTimeUTC”).item(0).textContent,”yyyy-MM-dd’T’HH:mm:ss.sss’Z'”).toLocaleString();

Find_Restore_Point_OldFind_Restore_Point_New

  • Edit the getVMRestorePoints action, open the Scripting tab. Update the date and time format on line 26 to: var rpDateLocale = System.getDateFromFormat(restorePointNodes.item(i).getElementsByTagName(“CreationTimeUTC”).item(0).textContent + ” UTC”,”yyyy-MM-dd’T’HH:mm:ss.sss’Z’ ZZZ”).toLocaleString();

Restore_Point_Action_OldRestore_Point_Action_New

  • You can test the API calls are successfully bringing back restore points by running the workflow in vRO and selecting a virtual machine, a list of available restore points should be displayed.

Run_vRO

vRA Integration

The final step is to hook the vRO workflow into vRA. Log into the vRealize Automation portal as a user with service architect permissions. From the Design tab select XaaS and Resource Actions. Any existing resource actions are listed. Click New.

Existing_Resource

Map the resource action to the relevant vRO workflow. In this case we need to expand Library, Veeam and select the Restore VM workflow. Click Next.

Restore_VM_Resource

The input mappings should already be populated; the resource type is IaaS VC Virtual Machine, the input parameter matches up with the parameter configured in the vRO workflow (vmObj which passes the virtual machine name), and this maps to the VC:VirtualMachine orchestrator type.

Restore_VM_Input

Accept the default values for the resource action form and click Finish.

Restore_VM_Form

The new resource action is now listed as a draft. To start using the action select it and click Publish.

New_Resource

Now select the Administration tab and Catalog Management. Open the Actions page, the new resource action we created should now be displayed.

If you want to change the icon of the resource action you can do so by selection the action and clicking Configure. There are a number of useful vRA icons available here, including sample icons for day 2 actions. Note for users of vRA 7.2 there is a known issue with changing the icon for custom actions, resolved in 7.3 as per this KB article.

Restore_VM_Action

The next step is to assign our custom action to an entitlement. Open the Entitlements page and select the relevant entitlement. Click the Items & Approvals tab, under Entitled Actions click the green plus symbol. Locate the new resource action and select the check box to add it to the entitled actions. Click Ok and Finish.

Restore_VM_Entitlement

To confirm the configuration has worked browse to the Items tab and select Machines. Any virtual machines that have the custom resource action added to the entitlement will show the new action in the drop-down Actions menu.

restore_vm_item

When selecting the new action I am presented with the action form as per the design canvas we saw earlier. In this example I select the restore point from the drop-down list that the getVMRestorePoints vRO action has pulled from the Veeam backup server, and click Submit.

restore_request

The virtual machine name is then passed through to the next stage of the workflow, along with the restore point ID. You can check the status of the job in vRA under the Requests tab, check the Restore VM workflow has run successfully in the vRO console, and check the restore task that will be running as normal in the Veeam Backup & Replication console.

_______________

Veeam Integration with vRA Part 1: Backup

Veeam Integration with vRA Part 2: Restore

Veeam Integration with vRA Part 1: Backup

In this 2 part series we will walk through integrating Veeam with vRealize Automation and vRealize Orchestrator. Part 1 will focus on giving users the ability to add virtual machines to existing Veeam backup jobs from within the vRA self-service portal. In Part 2 we will add the ability to restore virtual machines from a list of available restore points in vRA. The versions used are Veeam 9.5 and vRA 7.2 / 7.3.

The steps outlined below assume that you have already installed and configured Veeam Backup and Replication, and vRealize Automation with either embedded or external vRealize Orchestrator instance, as well as having a basic knowledge of both areas. The following process and the sample workflows we will import are not endorsed by, or supported by Veeam. Finally, Veeam Enterprise Manager is required to use Veeam RESTful API. For further reading material see the Veeam RESTful API Reference here. Alternative sample workflows and reading provided by The IT Hollow here, and another useful article by vRatpack here with vRA 6.2.

Add the REST Host

Open the vRealize Orchestrator client and log in as an administrator, change the view to Design from the drop down menu. The first thing we will do is add the Veeam server as a REST host. From the Workflows tab expand Library, HTTP-REST, Configuration.

REST_host_1

Right click Add a REST host and click Start workflow. Enter the name and URL of the Veeam server, the default URL uses port 9399, for example http://VeeamServer:9399. Review the default options and click Next.

REST_host_2

Configure the host authentication options as required. Here I have used Basic authentication, and entered the credentials for a service account with administrative access to Veeam.

REST_host_3

Configure proxy and advanced settings if required, then click Submit. The workflow will run and add the Veeam server as a REST host. There are also Update a REST host, and Remove a REST host, workflows if you want to make any changes. Existing REST hosts can be viewed from the inventory tab under expand HTTP-REST.

Import the Sample Workflows

In this example I am using sample workflows provided here, again these are not supported by Veeam. Download and extract the ZIP file to a location accessible from the vRO client. Change to the packages tab and click the Import Package icon. When prompted browse to the downloaded package file and click Import.

Veeam_Package_1

Ensure all the required elements are included and click Import selected elements.

Veeam_Package_2

We have now imported the backup workflow and action, and the restore workflow and action. The final element is a settings file which we will use to determine the REST host. Open the configurations tab and expand Library, Veeam. Click the Settings file and the pencil icon to edit. Select Attributes and locate the restHost attribute, click the Not set value and expand HTTP-REST, select the Veeam server we added earlier from the list of REST hosts and click Select. Click Save and close. The value of the restHost attribute should now be the Veeam backup server.

The backup jobs users select from are pulled using the getBackupJobs action under com.veeam.library in the actions tab. If you want to examine the workflow in more detail go to the workflows tab and expand Library, Veeam. Select Add VM to Backup Job and go through the tabs in the right hand pane. From the General tab you can see the restHost attribute is using the settings configuration file we have just configured. The Inputs for the workflow are jobname (Veeam backup job) and vmObj (virtual machine name). Under the Schema tab you can view the Scripting task which is making the API calls.

Add_VM

vRA Integration

The final step is to hook the vRO workflow into vRA. Log into the vRealize Automation portal as a user with service architect permissions. From the Design tab select XaaS and Resource Actions. Any existing resource actions are listed. Click New.

Existing_Resource

Map the resource action to the relevant vRO workflow. In this case we need to expand Library, Veeam and select the Add VM to Backup Job workflow.

Backup_VM_Resource

The input mappings should already be populated; the resource type is IaaS VC Virtual Machine, the input parameter matches up with the parameter configured in the vRO workflow (vmObj which passes the virtual machine name), and this maps to the VC:VirtualMachine orchestrator type.

Backup_VM_Input

Accept the default values for the resource action and click Finish.

Backup_VM_Form

The new resource action is now listed as a draft. To start using the action select it and click Publish.

vra6

Now select the Administration tab and Catalog Management. Open the Actions page, the new resource action we created should now be displayed.

If you want to change the icon of the resource action you can do so by selection the action and clicking Configure. There are a number of useful vRA icons available here, including sample icons for day 2 actions. Note for users of vRA 7.2 there is a known issue with changing the icon for custom actions, resolved in 7.3 as per this KB article.

vra7

The next step is to assign our custom action to an entitlement. Open the Entitlements page and select the relevant entitlement. Click the Items & Approvals tab, under Entitled Actions click the green plus symbol. Locate the new resource action and select the check box to add it to the entitled actions. Click Ok and Finish.

vra8

To confirm the configuration has worked browse to the Items tab and select Machines. Any virtual machines that have the custom resource action added to the entitlement will show the new action in the drop-down Actions menu.

vra9

When selecting the new action I am presented with the action form as per the design canvas we saw earlier. In this example I select the backup job from the drop-down list of jobs that the getBackupJobs vRO action has pulled from the Veeam backup server, and click Submit.

vra10

The virtual machine name is then passed through to the next stage of the workflow, which adds the virtual machine to the selected backup job. You can check the status of the job in vRA under the Requests tab, check the Add VM to Backup Job workflow has run successfully in the vRO console, and check the backup job itself has been updated using the Veeam Backup & Replication console.

_______________

Veeam Integration with vRA Part 1: Backup

Veeam Integration with vRA Part 2: Restore

Veeam Backup Error: Out of the Vector Bound

When running a backup job using Veeam Backup & Replication v8 or v9 the job fails with Error: Out of the vector bound. Record index: [0]. Vector Size: [1] Job finished with error. Running an active full produces the same result. In our case this issue was caused by corruption to the metadata file. This can occur when the metadata file is not properly closed and breaks the chain, potentially down to a file system filling up, or server failure.

To resolve we start a new chain to re-create both full data and metadata. This is done by cleanly deleting records about the backup job from the Veeam Backup & Replication console and configuration database, and deleting backup files themselves from the destination storage. The job itself remains so does not need recreating.

  • First disable the job; open the Veeam Backup & Replication client. Ensure Backup & Replication is selected on the task pane on the left hand side and select Jobs. Right click the failed job and click Disable.

veeamfix1

  • Next we need to remove the corrupted files.  Still in the Backup & Replication task pane select Backups. Right click the failed job and click Delete from disk to remove the backup files and records.

veeamfix2

  • Now go back to the Jobs page and enable the job. Run an Active Full to create new data and metadata files.

veeamfix3