The use case we are talking about is the ability to recover your backups from maybe on premises up into Microsoft Azure.

I was asked “what about AWS?” and yes of course if you are using the capacity tier option within Veeam Backup & Replication on premises and you are using the copy mode function to land a copy of your backups on AWS S3 or IBM Cloud or any S3 Compatible storage then there could be possible synergies in doing this in AWS, why I chose Microsoft Azure was simply because there is an Azure Marketplace offering we can take advantage of.

If you would like to see a similar series with AWS then let me know either on twitter or in the comments below. This will involve a different way of automating the provisioning of a Windows OS and the installation of Veeam Backup & Replication, but not too hard as we already have this functionality using Terraform & CHEF but only for vSphere but the code can be changed to work with AWS and really any platform that requires this functionality.

Veeam Configuration

As I said if you followed Part 1 of this series then you will have your Veeam server now running in Azure with no Veeam configuration.

In order for us to automate the direct restore process we need to provide some details in the script which i will share in stages and in full at the end of the post. But as a high level we need to

Add Azure Storage Account
Import Backups
Add Azure Compute Account

Then we will take those backups and run the Direct Restore to Microsoft Azure on the appropriate backups in a converted state ready to be powered on, or you can choose to power them on as part of this script process.

Firstly we need to add the Veeam snap in and connect to the local Veeam Backup & Replication Server, depending on where you run this script you will need to change the appropriate localhost below to the relevant DNS or IP Address. It is my recommendation that this is done on the server itself, but I am exploring how this PowerShell script could be hosted on your network and not publicly and used that way to fill in the secure details.

Add-PSSnapin VeeamPSSnapin



#Connects to Veeam backup server.

Connect-VBRServer -server "localhost"

Next we will add the Microsoft Azure Compute Account, this command will prompt you to login and authenticate into Microsoft Azure. I use MFA so this was the only way I could find to achieve this.

#Add Azure Compute Account



Add-VBRAzureAccount -Region Global

Next we will add the storage account, You will need to update the script with the requirements below.

Access Key – this will be based on a storage account that you have already created and you will need the long access key for authentication.

Azure Blob Account – this is the name of the storage blob account you have previously created. This is the same blob account and process that you used for adding Microsoft Azure Blob Storage to Veeam Backup & Replication on premises.

#Add Azure Storage Account 



$accesskey = "ADD AZURE ACCESS KEY"

 

$blob1 = Add-VBRAzureBlobAccount -Name "AZUREBLOBACCOUT" -SharedKey $accesskey

Now we need to add our capacity tier, this is where you have been sending those backups.

#Add Capacity Tier (Microsoft Azure Blob Storage) Repository 



$account = Get-VBRAzureBlobAccount -Name "AZUREBLOBACCOUNT"

 

$connect = Connect-VBRAzureBlobService -Account $account -RegionType Global -ServiceType CapacityTier



$container = Get-VBRAzureBlobContainer -Connection $connect | where {$_.name -eq 'AZURECONTAINER'}



$folder = Get-VBRAzureBlobFolder -Container $container -Connection $connect

The next part to adding capacity tier is important and I have also added this into the script, this repository needs to be added with exactly the same name that you have in your production Veeam Backup & Replication.

#The name needs to be exactly the same as you find in your production Veeam Backup & Replication server

$repositoryname = "REPOSITORYNAME"



Add-VBRAzureBlobRepository -AzureBlobFolder $folder -Connection $connect -Name $repositoryname

Next we need to import and rescan those backups that are in the Azure Blob Storage.

#Import backups from Capacity Tier Repository 



$repository = Get-VBRObjectStorageRepository -Name $repositoryname



Mount-VBRObjectStorageRepository -Repository $repository 

Rescan-VBREntity -AllRepositories

Now if you are using encryption then you will need the following commands instead of the one above.

#if you have used an encryption key then configure this section 



$key = Get-VBREncryptionKey -Description "Object Storage Key"

Mount-VBRObjectStorageRepository -Repository $repository -EncryptionKey $key

At this point if we were to jump into the Veeam Backup & Replication console we would see our Storage and Compute accounts added to the Cloud Credential Manager, we would see the Microsoft Azure Blob Storage container added to our backup repositories and on the home screen you will see the object storage (imported) which is where you will also see the bakcups that reside there.

Next we need to create the variables in order to start our Direct Restore scenarios to Microsoft Azure.

A lot of the variables are quite self explanatory, but as a brief overview you will need to change the following to suit your backups.

VMBACKUPNAME = Which VM is it you want to restore

AZURECOMPUTEACCOUNT = this is the Azure Compute Account you added to Veeam Backup & Replication at the beginning of the script.

SUBSCRIPTIONNAME = you may have multiple subscriptions on one Azure compute account pick the appropriate one here.

STORAGEACCOUNTFORRESTOREDMACHINE = we are going to be converting that backup to your Azure Storage Group

REGION = Which Azure region would you like this to be restored to

$vmsize = this is where you will define what size Azure VM you wish to use here. In this example Basic_A0 is being used, you can change this to suit your workload.

AZURENETWORK = define the Azure Virtual Network you wish this converted machine to live.

SUBNET = Which subnet should the machine live

AZURERESOURCEGROUP = the Azure Resource Group you wish the VM to live

NAMEFORRESTOREDMACHINEINAZURE = Maybe a different naming conversion but this is what you wish to call your machine in Azure.

 #This next section will enable you to automate the Direct Restore to Microsoft Azure 



$restorepoint = Get-VBRRestorePoint -Name "VMBACKUPNAME" | Sort-Object $_.creationtime -Descending | Select -First 1 



$account = Get-VBRAzureAccount -Type ResourceManager -Name "AZURECOMPUTEACCOUNT"



$subscription = Get-VBRAzureSubscription -Account $account -name "SUBSCRIPTIONNAME"



$storageaccount = Get-VBRAzureStorageAccount -Subscription $subscription -Name "STORAGEACCOUNTFORRESTOREDMACHINE"



$location = Get-VBRAzureLocation -Subscription $subscription -Name "REGION"



$vmsize = Get-VBRAzureVMSize -Subscription $subscription -Location $location -Name Basic_A0



$network = Get-VBRAzureVirtualNetwork -Subscription $subscription -Name "AZURENETWORK"



$subnet = Get-VBRAzureVirtualNetworkSubnet -Network $network -Name "SUBNET"



$resourcegroup = Get-VBRAzureResourceGroup -Subscription $subscription -Name "AZURERESOURCEGROUP"



$RestoredVMName1 = "NAMEOFRESTOREDMACHINEINAZURE"

Now we have everything added to Veeam Backup & Replication, We have all the variables for our machines that we wish to convert and recover to Microsoft Azure VMs. Next is to start the restore process.

Start-VBRVMRestoreToAzure -RestorePoint $restorepoint -Subscription $subscription -StorageAccount $storageaccount -VmSize $vmsize -VirtualNetwork $network -VirtualSubnet $subnet -ResourceGroup $resourcegroup -VmName $RestoredVMName1 -Reason "Automated DR to the Cloud Testing"

The full script can be found here

#This script will automate the configuration steps of adding the following steps

#Add Azure Compute Account

#Add Azure Storage Account 

#Add Capacity Tier (Microsoft Azure Blob Storage) Repository 

#Import backups from Capacity Tier Repository 

#This will then enable you to perform Direct Restore to Azure the image based backups you require. 



Add-PSSnapin VeeamPSSnapin



#Connects to Veeam backup server.

Connect-VBRServer -server "localhost"



#Add Azure Compute Account



#Need to think of a better way to run this as this will close down PowerShell when installing

msiexec.exe /I "C:\Program Files\Veeam\Backup and Replication\Console\azure-powershell.5.1.1.msi"



Add-VBRAzureAccount -Region Global



#Add Azure Storage Account 



$accesskey = "ADD AZURE ACCESS KEY"

 

$blob1 = Add-VBRAzureBlobAccount -Name "AZUREBLOBACCOUT" -SharedKey $accesskey



#Add Capacity Tier (Microsoft Azure Blob Storage) Repository 



$account = Get-VBRAzureBlobAccount -Name "AZUREBLOBACCOUNT"

 

$connect = Connect-VBRAzureBlobService -Account $account -RegionType Global -ServiceType CapacityTier



$container = Get-VBRAzureBlobContainer -Connection $connect | where {$_.name -eq 'AZURECONTAINER'}



$folder = Get-VBRAzureBlobFolder -Container $container -Connection $connect



#The name needs to be exactly the same as you find in your production Veeam Backup & Replication server

$repositoryname = "REPOSITORYNAME"



Add-VBRAzureBlobRepository -AzureBlobFolder $folder -Connection $connect -Name $repositoryname



#Import backups from Capacity Tier Repository 



$repository = Get-VBRObjectStorageRepository -Name $repositoryname



Mount-VBRObjectStorageRepository -Repository $repository 

Rescan-VBREntity -AllRepositories



#if you have used an encryption key then configure this section 



#$key = Get-VBREncryptionKey -Description "Object Storage Key"

#Mount-VBRObjectStorageRepository -Repository $repository -EncryptionKey $key



 #This next section will enable you to automate the Direct Restore to Microsoft Azure 



$restorepoint = Get-VBRRestorePoint -Name "VMBACKUPNAME" | Sort-Object $_.creationtime -Descending | Select -First 1 



$account = Get-VBRAzureAccount -Type ResourceManager -Name "AZURECOMPUTEACCOUNT"



$subscription = Get-VBRAzureSubscription -Account $account -name "SUBSCRIPTIONNAME"



$storageaccount = Get-VBRAzureStorageAccount -Subscription $subscription -Name "STORAGEACCOUNTFORRESTOREDMACHINE"



$location = Get-VBRAzureLocation -Subscription $subscription -Name "REGION"



$vmsize = Get-VBRAzureVMSize -Subscription $subscription -Location $location -Name Basic_A0



$network = Get-VBRAzureVirtualNetwork -Subscription $subscription -Name "AZURENETWORK"



$subnet = Get-VBRAzureVirtualNetworkSubnet -Network $network -Name "SUBNET"



$resourcegroup = Get-VBRAzureResourceGroup -Subscription $subscription -Name "AZURERESOURCEGROUP"



$RestoredVMName1 = "NAMEOFRESTOREDMACHINEINAZURE"





Start-VBRVMRestoreToAzure -RestorePoint $restorepoint -Subscription $subscription -StorageAccount $storageaccount -VmSize $vmsize -VirtualNetwork $network -VirtualSubnet $subnet -ResourceGroup $resourcegroup -VmName $RestoredVMName1 -Reason "Automated DR to the Cloud Testing"

You will also find the most up to date and committed PowerShell script here within the GitHub repository.

Feedback is key on this one and would love to make this work better and faster. Feedback welcome below in the comments as well as getting hold of me on Twitter.

Installing Azure PowerShell

In order for us to start this automated deployment we need to install locally on our machine the Azure PowerShell module.

More details of that can be found here.

Run the following code on your system.

if ($PSVersionTable.PSEdition -eq 'Desktop' -and (Get-Module -Name AzureRM -ListAvailable)) {

    Write-Warning -Message ('Az module not installed. Having both the AzureRM and ' +

      'Az modules installed at the same time is not supported.')

} else {

    Install-Module -Name Az -AllowClobber -Scope CurrentUser

Select either [Y] Yes or [A] Yes to All as this is an untrusted repository. You can also change currentuser to allusers if you wish to enable for all users on the local machine.

Breaking down the code

This section is going to talk through the steps taken in the code, the way in which this will work though is by taking this code from the GitHub Repository you will be able to modify the variables and begin testing yourself without any actual code changes.

First we need to connect to our Azure account, this will provide you with a web browser to login to your Azure Portal, if you are using MFA then this will enable you to authenticate this way also.

# Connect to Azure with a browser sign in token

Connect-AzAccount

Next we want to start defining what, where and how we want this to look in our Azure accounts. It should be pretty straight forward to understand the following but

locName = Azure Location

Publisher Name = Veeam

Offer Name = is the particular offering we wish to deploy from the publisher, there are quite a few so expect to see other options using this method.

SkuName = what product sku of the offering do you wish to use

version = what version of the product

# Set the Marketplace image

$locName="EASTUS"

$pubName="veeam"

$offerName="veeam-backup-replication"

$skuName="veeam-backup-replication-v10"

$version = "10.0.1"

The following are aligned to the environment.

resourcegroup = which resource group do you wish to use this can be an existing resource group or a new name

vmname = what name do you wish your Veeam Backup & Replication server to have within your Azure environment

vmsize = this is the image that will be used, my advice to pick the supported sizes, this is the default size used for production environments.

# Variables for common values

$resourceGroup = "CadeTestingVBR"

$vmName = "CadeVBR"

$vmSize = "Standard_F4s_v2"

Next we need to agree to the license terms of deploying from the marketplace for this specific VM Image. The following commands will do this.

Get-AzVMImage -Location $locName -PublisherName $pubName -Offer $offerName -Skus $skuName -Version $version



$agreementTerms=Get-AzMarketplaceterms -Publisher "veeam" -Product "veeam-backup-replication" -Name "10.0.1"



Set-AzMarketplaceTerms -Publisher "veeam" -Product "veeam-backup-replication" -Name "10.0.1" -Terms $agreementTerms -Accept

If you wish to review the terms then you can do by running the following command. Spoiler alert the command will give you a link to a txt file to save you the hassle here is the link in the txt file where you will find the Veeam EULA – https://www.veeam.com/eula.html

Get-AzMarketplaceTerms -Publisher "veeam" -Product "veeam-backup-replication" -Name "10.0.1"

Next we need to start defining how our Veeam Backup & Replication server will look in regards to configuration of network, authentication and security.

I also wanted to keep this script following best practice and not containing any usernames or passwords so the first config setting is to gather the username and password for your deployed machine in a secure string.

# Create user object

$cred = Get-Credential -Message "Enter a username and password for the virtual machine."

Create a resource group

# Create a resource group



New-AzResourceGroup -Name $resourceGroup -Location $locname -force

Create a subnet configuration

# Create a subnet configuration

$subnetConfig = New-AzVirtualNetworkSubnetConfig -Name "cadesubvbr" -AddressPrefix 10.0.0.0/24

Create a virtual network

# Create a virtual network

$vnet = New-AzVirtualNetwork -ResourceGroupName $resourceGroup -Location $locName `

  -Name CadeVBRNet -AddressPrefix 10.0.0.0/24 -Subnet $subnetConfig

Create a public IP Address

# Create a public IP address and specify a DNS name

$pip = New-AzPublicIpAddress -ResourceGroupName $resourceGroup -Location $locName `

  -Name "CadeVBR$(Get-Random)" -AllocationMethod Static -IdleTimeoutInMinutes 4

Create inbound security group rule for RDP

# Create an inbound network security group rule for port 3389

$nsgRuleRDP = New-AzNetworkSecurityRuleConfig -Name CadeVBRSecurityGroupRuleRDP  -Protocol Tcp `

  -Direction Inbound -Priority 1000 -SourceAddressPrefix * -SourcePortRange * -DestinationAddressPrefix * `

  -DestinationPortRange 3389 -Access Allow

Create network security group

# Create a network security group

$nsg = New-AzNetworkSecurityGroup -ResourceGroupName $resourceGroup -Location $locName `

  -Name CadeVBRNetSecurityGroup -SecurityRules $nsgRuleRDP

Create a virtual network

# Create a virtual network card and associate with public IP address and NSG

$nic = New-AzNetworkInterface -Name CadeVBRNIC -ResourceGroupName $resourceGroup -Location $locName `

  -SubnetId $vnet.Subnets[0].Id -PublicIpAddressId $pip.Id -NetworkSecurityGroupId $nsg.Id

Next we need to define what the virtual machine configuration is going to look in our environment using the above environment configurations.

#Create a virtual machine configuration



$vmConfig = New-AzVMConfig -VMName "$vmName" -VMSize $vmSize 

$vmConfig = Set-AzVMPlan -VM $vmConfig -Publisher $pubName -Product $offerName -Name $skuName

$vmConfig = Set-AzVMOperatingSystem -Windows -VM $vmConfig -ComputerName $vmName -Credential $cred

$vmConfig = Set-AzVMSourceImage -VM $vmConfig -PublisherName $pubName -Offer $offerName -Skus $skuName -Version $version

$vmConfig = Add-AzVMNetworkInterface -Id $nic.Id -VM $vmConfig

Then now we have everything we need we can now begin deploying the machine.

# Create a virtual machine

New-AzVM -ResourceGroupName $resourceGroup -Location $locName -VM $vmConfig

If you saw the video demo you would have seen that the deployment really does not take long at all, I actually think using this method is a little faster either way less than 5 minutes to quickly deploy a Veeam Backup & Replication server in Microsoft Azure.

Now that we have our machine there is one thing we want to do to ensure the next stages of configuration run smoothly. Out of the box there is a requirement for Azure PowerShell to be installed to be able to use the Azure Compute accounts and Direct Restore to Microsoft Azure. The installer is already on the deployed box and if we go through manually you would have to just install that msi instead in this script we remote run a powershell script from GitHub that will do it for you.

# Start Script installation of Azure PowerShell requirement for adding Azure Compute Account

Set-AzVMCustomScriptExtension -ResourceGroupName $resourceGroup `

    -VMName $vmName `

    -Location $locName `

    -FileUri https://raw.githubusercontent.com/MichaelCade/veeamdr/master/AzurePowerShellInstaller.ps1 `

    -Run 'AzurePowerShellInstaller.ps1' `

    -Name DemoScriptExtension

At this stage the PowerShell installation for me has required a reboot but it is very fast and generally up within 10-15 seconds. So we run the following command to pause the command before then understanding what that public IP is and then start a Windows Remote Desktop to that IP address.

Start-Sleep -s 15



Write-host "Your public IP address is $($pip.IpAddress)"

mstsc /v:$($pip.IpAddress)

Now, this might seem like a long winded approach to getting something up and running but with this combined into one script and you having the ability to create all of this on demand brings a powerful story to being able to recover workloads into Microsoft Azure.

In the next parts to this post will concentrate on a configuration script which is where we will configure Veeam Backup & Replication to attach the Microsoft Azure Blob Storage where our backups reside, Our Azure Compute Account and then we can look at how we could automate end to end this process to bring your machines up in Microsoft Azure when you need them or before you need them.

here is the complete script

# Connect to Azure with a browser sign in token

Connect-AzAccount



# Set the Marketplace image

$locName="EASTUS"

$pubName="veeam"

$offerName="veeam-backup-replication"

$skuName="veeam-backup-replication-v10"

$version = "10.0.1"



# Variables for common values

$resourceGroup = "CadeTestingVBR"

$vmName = "CadeVBR"

$vmSize = "Standard_F4s_v2"

$StorageSku = "Premium_LRS"

$StorageName = "cadestorage"



Get-AzVMImage -Location $locName -PublisherName $pubName -Offer $offerName -Skus $skuName -Version $version



$agreementTerms=Get-AzMarketplaceterms -Publisher "veeam" -Product "veeam-backup-replication" -Name "10.0.1"



Set-AzMarketplaceTerms -Publisher "veeam" -Product "veeam-backup-replication" -Name "10.0.1" -Terms $agreementTerms -Accept





# Create user object

$cred = Get-Credential -Message "Enter a username and password for the virtual machine."



# Create a resource group



New-AzResourceGroup -Name $resourceGroup -Location $locname -force



# Create a subnet configuration

$subnetConfig = New-AzVirtualNetworkSubnetConfig -Name "cadesubvbr" -AddressPrefix 10.0.0.0/24



# Create a virtual network

$vnet = New-AzVirtualNetwork -ResourceGroupName $resourceGroup -Location $locName `

  -Name CadeVBRNet -AddressPrefix 10.0.0.0/24 -Subnet $subnetConfig



# Create a public IP address and specify a DNS name

$pip = New-AzPublicIpAddress -ResourceGroupName $resourceGroup -Location $locName `

  -Name "CadeVBR$(Get-Random)" -AllocationMethod Static -IdleTimeoutInMinutes 4



# Create an inbound network security group rule for port 3389

$nsgRuleRDP = New-AzNetworkSecurityRuleConfig -Name CadeVBRSecurityGroupRuleRDP  -Protocol Tcp `

  -Direction Inbound -Priority 1000 -SourceAddressPrefix * -SourcePortRange * -DestinationAddressPrefix * `

  -DestinationPortRange 3389 -Access Allow



# Create a network security group

$nsg = New-AzNetworkSecurityGroup -ResourceGroupName $resourceGroup -Location $locName `

  -Name CadeVBRNetSecurityGroup -SecurityRules $nsgRuleRDP



# Create a virtual network card and associate with public IP address and NSG

$nic = New-AzNetworkInterface -Name CadeVBRNIC -ResourceGroupName $resourceGroup -Location $locName `

  -SubnetId $vnet.Subnets[0].Id -PublicIpAddressId $pip.Id -NetworkSecurityGroupId $nsg.Id



# Create a virtual machine configuration

#vmConfig = New-AzVMConfig -VMName $vmName -VMSize $vmSize | `

#Set-AzVMOperatingSystem -Windows -ComputerName $vmName -Credential $cred | `

#Set-AzVMSourceImage -VM $vmConfig -PublisherName $pubName -Offer $offerName -Skus $skuName -Version $version | `

#Add-AzVMNetworkInterface -Id $nic.Id



#Create a virtual machine configuration



$vmConfig = New-AzVMConfig -VMName "$vmName" -VMSize $vmSize 

$vmConfig = Set-AzVMPlan -VM $vmConfig -Publisher $pubName -Product $offerName -Name $skuName

$vmConfig = Set-AzVMOperatingSystem -Windows -VM $vmConfig -ComputerName $vmName -Credential $cred

$vmConfig = Set-AzVMSourceImage -VM $vmConfig -PublisherName $pubName -Offer $offerName -Skus $skuName -Version $version

$vmConfig = Add-AzVMNetworkInterface -Id $nic.Id -VM $vmConfig



# Create a virtual machine

New-AzVM -ResourceGroupName $resourceGroup -Location $locName -VM $vmConfig



# Start Script installation of Azure PowerShell requirement for adding Azure Compute Account

Set-AzVMCustomScriptExtension -ResourceGroupName $resourceGroup `

    -VMName $vmName `

    -Location $locName `

    -FileUri https://raw.githubusercontent.com/MichaelCade/veeamdr/master/AzurePowerShellInstaller.ps1 `

    -Run 'AzurePowerShellInstaller.ps1' `

    -Name DemoScriptExtension



Start-Sleep -s 15



Write-host "Your public IP address is $($pip.IpAddress)"

mstsc /v:$($pip.IpAddress)

You can also find this version and updated versions of this script here in my GitHub repository.

Any comments feedback either down below here, twitter or on GitHub.

So far in this series I have shown you hopefully how powerful the Replication side of the house is within Veeam Backup & Replication. It is by no means the whole topic covered, we will have CDP as mentioned as a feature, Veeam also have the Veeam Availability Orchestrator which I consider the next level Veeam Replication on Steroids and that alone deserves its own series.

There is another form of replication though that in some environments can be driven by Veeam to allow for data to reside in that secondary location. This comes in the form of the storage snapshot integration. This was also mentioned in regard to transport modes used for your replication jobs but when I see storage snapshot orchestration I mean that Veeam is capable of using the storage replication technology and drive that through the Veeam user interface.

Overview

Veeam’s Integration allows Veeam Backup & Replication to sit on top of the storage systems from NetApp & HPE Nimble and control application consistent Snapshots for local fast Recovery Point Objectives. Veeam also enables the ability to orchestrate and schedule those app aware snapshots to be sent to a secondary storage array from the same vendor.

We also have similar capabilities with HPE 3PAR with their Peer Persistence replication technology. The screen below one step further also in that we are able to then take a backup of the data from the secondary location.

Configuration

Let’s run through some of the options for all of these firstly how does this look from a HPE Nimble and NetApp point of view.

For this orchestration, it’s a backup job first, we need to create a new backup job and this is also after the storage arrays have been configured and added to the console. Notice how from the drop down this allows me to choose a Primary Storage Snapshot.

Next, we can select the configure secondary destination for this job.

Next, we have the options to choose where the secondary destination should be. It is worth noting that this feature will only talk to the same vendor and model specific i.e HPE Nimble to HPE Nimble and NetApp ONTAP to NetApp ONTAP.

All software versions should be checked within the system requirements before attempting this.

The retention set here is how many snapshots on the secondary array should we be keeping.

When this job runs providing licensing and network replication has been configured on the storage arrays this will orchestrate the storage snapshots and replication to the target location.

HPE 3PAR

Is a similar process to the one above, I need to clarify some of the next points though as these were put together during a beta phase. Thanks to Pierre and Niels for their work on the diagrams and screen grabs.

Much the same if we want to leverage the storage replication for that secondary target then we can select the following configuration with the required remote copies of snapshots to keep.

from a recovery point of view, the Veeam explorers can be used for recovery tasks however the failover and failback from Veeam replication will not work.

In this day and age, we want to be able to do more with less, seems to be everywhere, or we want to be able to do something additional with what we have already got. Get more ROI out of an asset. Leveraging data is no difference, for years as technical customers we have been backing up our data to some form of media, this could be classed as dumb storage. We send our backups there then we update again at the next iteration. We only come back to that backup or replicated data if we need something in the form of a recovery.

Why not leverage it, that’s your production assets in the form of a backup or replication state. In the last post, I discussed SureReplica which is the Veeam way in which those replicated virtual machines can be verified for recovery. To do that we power on those machines in an isolated network away from production and in this instance in our secondary site for disaster recovery. Fully automated to make sure that those files are good for if we need to failover our workloads.

Verification is one thing leverage is another, by leveraging that data we are getting more use from that target storage array that traditionally may have just been sat there holding those replicated VMs.

SandBox Labs via SureBackup Job

By clicking a simple step within the SureBackup job wizard you can keep that same application group up and running for as long as you need. A good use for this is patching, if you are responsible for patching your key application servers or all servers then this could help that process.

Options

One thing to note here is that this is going to start that same application groups it’s going to adhere to the configuration of that application group about memory assigned and boot time delay etc. be sure to configure those tests and settings to ensure you have the optimal configuration when running a Sandbox environment.

You also have the ability to run your own scripts here this is part of the verification also, this could be a script that you may run that runs a windows update via script to test updates faster.

Scheduling & Stopping

When you have that in line you are able to start your job or schedule that job. What I mean by scheduling is that this will run a scheduled time and spin up these replica VMs in your secondary location the difference here is that they will keep running. Whilst they are running no further replication jobs can be completed. I want to show you how you can stop the session so that you don’t face this issue, when you are finished with your sandbox testing then be sure to stop that session.

By stopping the session, you are going to shut down them virtual machines and you are going to lose anything you placed on that we are not going to write anything to those replica files or backups.

Tips – Data Sources

A useful tip here is that you may have these VMs in your application group that are being replicated over to the secondary location however you may have a Domain Controller or something alike already running in that location, with that you may or I should say should be taking a local backup of that Domain Controller. Within your application group configuration, you can mix your workloads, we could use some of the replicated virtual machines and we can use some of the local backup files.

There is also one more option of sources and that’s the ability to leverage storage snapshots. I will touch on more later on in the series on this one.

Other Resources

I also did a vBrownBag session at VMworld in 2017 that allowed me to demonstrate the power of this cool unsung feature within Veeam.

<Link to VMworld YouTube Session>

What if I told you that the same tool that you purchased for Backup & Replication could also perform a level of verification against your replication jobs. Oh and did I mention that this test would automatically test the verification of Virtual Machine, Operating System and Application.

SureReplica gives you the ability to spin up an isolated network in your secondary location to confirm that the restore point is in a good state should you require to actually recover from it, rather than waiting for a failure scenario to occur and then you are sat there with crossed fingers hoping the backup or replica is in a good state to recover from.

This same feature allows for some further lab environment, a sandbox that can be accessed whereas this is fully automated. This will be coming up next in the series.

Components required for SureReplica

There are some one-time configuration components that we need to consider before we get going. These really don’t take long to configure.

Virtual Lab

Most of the time we would not want our verification tests to interfere with the production network, this is why we create a Virtual Lab, a virtual lab consists of a virtual network switch on your remote site ESXi host.

This can be on a single host or a multi-site environment, depending on the size of the testing group of Virtual Machines.

The most basic requirement here requires:

• A resource pool
• VM Folder
• A standard vSwitch

The VMs are started in this virtual lab and this is the only use case for the vSwitch, there is a proxy appliance also deployed at this stage on the same host, resource pool and VM folder. This is used to bridge connectivity between the production network and this isolated network.

Some more detail on this configuration can be found here – https://helpcenter.veeam.com/docs/backup/vsphere/surereplica_vlab_config.html?ver=95

Application Group

An Application Group is a group of Applications, kidding, this is where we can define our virtual machines that we would like to verify in our secondary location. Remember here that it might be one mission critical system that you want to test, but you need to make sure within that application group you have all dependencies so we can truly test if that application is in a good state or not.

I will also add that you can mix where your VMs come from this could be from Storage Snapshots, Replication Jobs and Backup Jobs all mixed accordingly to achieve the requirements that you have.

SureBackup Job

In the Veeam Backup & Replication console this type of job is called SureBackup, this offers that same level of verification to the Veeam backup files.

To piece the Application Group and Virtual Lab together comes the SureBackup Job. Let’s run through this configuration with our SQL example we used in the last post.

Firstly, find the SureBackup job wizard in the console.

Set the name and description for the job, this wizard is assuming you have been through and created your virtual lab and application group.

Select the required virtual lab that you have created. If you need pointers on this then there is a demo video at the end of the post.

Select the Application Group that you have also just created.

Additional notification settings can be added at this next screen to determine SNMP traps or emails as well as some integrity checks. This is going to give you the audit trail for that DR testing plan though that normally takes weeks and weeks to perform.

If you are wanting to run an automated Disaster Recovery test, which Is completely possible then you may want to make sure this runs on a scheduled basis. Remember this will consume resource, within the application group creation you can determine the associated ram in terms of a percentage to the live production system, just be cautious in choosing when this should take place.

Finally, a summary page to show you the configuration, also the option to run that job.

Now that you have this created and scheduled you can now wait for the scheduled time to come by and then once this job has completed you will either receive an email notification if this has been configured or you can see from within the console.

https://youtu.be/LHoTua_fxNg

We are into the home stretch on this series now, this post will cover the failover and failback of our replicated virtual machines.

In the previous posts, we setup our replication jobs and got our data going from one site to another, so this one will cover how easy it is to failover for a reason I will let you imagine, for the story though let’s use power outage but we only need to failover our SQL VM for a few hours whilst this host power issue is resolved.

For the purposes of this post lets expect that all the replication jobs have been successful, of course they are if you have been following. Oh, and we have some restore points in our secondary location.

The screen above shows the ready replicas that we have powered off in our secondary sites. Notice in the above though that I have two replica locations and in some instances, I have our Domain Controllers and SQL boxes going to both locations. This could achieve a multi-site failover plan if required. Or also it could be that one site is a development site on the same campus as live and these replicas can be used for a SureReplica job (VMware only) This will be covered in a forthcoming post.

When you right click on the VMs you have some options.

The ones we are interested in are the top three.

Failover Now – This is the panic stations button, disaster scenario and we need everything over and up and running as soon as possible, we will cover this shortly.

Planned Failover – A little less panic station, maybe a power situation on the host and we only need a subset of machines to failover to a secondary but the difference being here is we have the planned time for the maintenance. We will get to this later as well.

Add to Failover Plan – A failover plan allows you to group a number of virtual machines and set some basic configuration on how they get started up in that second location, in this scenario we are only going to failover one VM but let’s walk through the process as it would still work with 1 VM.

Failover Plan

When you select through the wizard above you get the simple steps to create a new failover plan. On the first screen, you can give a name and description and you are able to add pre-and post-failover scripts. Script files formats allowed: BAT, CMD, EXE and PS1. An example here could be that you have a tertiary application that is linked to this particular VM that is not part of the failover so we may just want to trigger a PowerShell script to stop a certain service on the application server.

The following screen is where you can choose the VMs you require to be part of this failover plan. You have the ability to set a delay this is to delay the VM boot time as part of the failover trigger, on this same screen you are able to move the VMs up and down according to the order in which you need them to boot.

That’s it once you complete that step its then time to see the summary of the configuration for the failover plan, simple stuff here as we only have one VM. I do quite like the command function it gives you in this last summary screen. Meaning you could in theory run this from another location that can talk to the VBR server.

In the console, you will then see the newly created failover plan appear

I am not sure how many people are actually aware of this fact but if you then navigate to the failover plan you have two options to choose from, you can start and that’s going to kick off a failover plan, starting the VMs at the secondary site. And then you have Start to, this allows you to schedule the most recent replicated restore point to be powered on. One thing to note here is this will not shut down the primary VM. If you select the undo option here when the replicas are running this allows you undo without reverting any changes to the primary VM.

Planned Failover

Planned Failover is the ability to deal with scheduled disruptive maintenance. Going back to that replica screen and right clicking on the VM you have this option. You can choose further VMs here if required.

We can then add our audit reason for performing this.

Finally, the summary.

The next stage in this option is a further incremental from production to secondary will take place. The great thing about this option is that the planned failover job will shut down the production system as well. It will then perform another update to the secondary site.

At this point we are now running our SQL workload on the secondary site. When I first open up the SQL management studio you can see I only have one very important DB.

The power outage has taken a little longer than expected so much so we had to create another database for a new application. We may have rushed the naming convention.

Failback

We have been told it won’t take long now for the power company to switch things back on at our main data centre. This is the view from the Veeam Backup & Replication console.

The options that you get when you right click on that VM are shown below, this gives us some options to consider, if we thought performance is much better now it’s over in our secondary location then we could choose permanent failover, or we could use undo failover if we just want to bring things up in the primary forgetting all actions performed on the secondary.

We got the phone call to say that things were all back online from a power perspective so we tested connectivity and it was good to failback with all of our changes. We selected failback to production and a new wizard appears.

We have a few options as part of the failback wizard. Be sure to check out the options here. Quick Rollback is a great feature that is not mentioned enough. I will try and write up something on that another time.

Last page although a summary it’s important to note that this is where you can trigger the power on of the VM.

Steps taken on failback.

and finally showing the failed back instance with the newly created database still in place.

Finally we have to tell Veeam that we are ready to commit this failback.

It’s worth noting before closing out that the failover now option, assumes that the production VM is already powered off and we are getting straight into powering on the secondary VM, no incremental replication jobs happen. Recovery options are the same as we went through above.

Veeam Replication – Advanced features and functionality

Welcome back, as we move through this series where we have touched on the basic replication configuration, or I should say the easy to use deployment, the walkthrough and some more advance configurations or ways in which to achieve the same outcome from PowerShell commands.

This post will hopefully share some of the more advanced features, options or even some of the outcomes you might not be able to find any detail on.

Remote replica from backup

The clue is in the title, this gives the ability to create a replica VM from a backup that has already traversed the WAN or already resides on that secondary location. This method has some benefits in that it will not have to touch the production estate and you obviously do not need to send the data twice if you have a backup copy configuration to the same site. More information can be found here –

https://helpcenter.veeam.com/docs/backup/vsphere/replica_from_backup.html?ver=95

Low Connection Bandwidth

Another option when creating your replication job was “Low Connection Bandwidth” Introducing Replica Seeding, this is the ability to use the man in the van bandwidth. Poor links between sites or large datasets then if possible take advantage of seeding. This could be used in some cases for getting replicas to our cloud connect providers.

Basically, if you have a backup for the replicated VM on the backup repository located in the DR site. You can point the replication copy job to this backup. The first run of the replication job will seed the data from the backup file, i.e. it will create a VM replica image, once complete it will resync with the production VM for any changes since the backup file. Notice how it only does that seed once where as remote replica continues to use that same method.

Giving a shout out to Luca here on his blog a while back he wrote about seeding to a service provider (Cloud Connect – https://www.virtualtothecore.com/en/seeding-veeam-cloud-connect-part-3-replication-jobs/)

Separate Virtual Networks

Not really an advanced setting as such but something I should cover here, if your target environment does not use the same networking, this setting allows you to map the production (source network) and the secondary (target network)

Calculating Digests

For those of you already running Veeam replication you may have seen this, let’s cover what this process is and what do we expect to see from a calculation point of view. This task is checking the difference between the source VM and the target VM (Replica VM)

A good rule of thumb here is 1GB per minute, just bare that in mind when you are calculating how long your replication jobs are going to take.

Why does this happen? Well you are probably going to know why this is happening but just to run through some of them.

Most commonly it’s if you change something on that source VM, add a disk for example probably the most common update or change here and there is very little you can do to avoid this then wanting to calculate the digests during the job.

You will also find this message in your job log if you reinstall vCentre or something happens to your vCentre and something changes your VM IDs. Datastore disconnections could cause it, oh and my favourite is if you remove the source VM from the inventory and then re add it back in. it’s going to get a new VM ID and that’s also going to prompt for this process.

There is very little we can do with the above scenarios but sit it out, although if you find that something is outside of those timeframes then a call to support might be worth the clarification of the setup.

This job running at the moment from @HammondRhys shows what can happen if you run out of space on the repository which is used for that replication metadata another use case or fail case for the calculating digests.

Different IP addressing Scheme

More common that not these days is the requirement for different IP addressing in a secondary location. This exact setting or configuration is available as part of the Veeam replication wizard.

By setting this different IP addressing scheme when a failover plan is triggered it will apply these different addresses to the replica VM.

A big note here is that this is only applicable to Windows virtual machines.

Storage Integration Settings

I touched on this very briefly in the transport modes post but wanted to touch again on an example of when this setting needs to be considered.

When configuring such large jobs, it is advised to configure the maximum number of VMs within one storage snapshot. The setting is available in the advanced job settings within the Integration tab.

Example: When creating a job with 100 VMs and setting the limit to 10, the Backup from Storage Snapshot job will instruct the job manager to process the first 10 VMs, issue the storage snapshot, and proceed with the backup. When that step has successfully completed for the first 10 VMs, the job will repeat the above for the following 10 VMs in the job.

Replica Metadata

Finally, and I know this post is now getting on the long side but given some recent conversations I wanted to add at least something in here that may help when it comes to metadata location and sizing.

The latest information I can find is from one of our product managers. Outlining a good rule when it comes to sizing for metadata space.

“Typical metadata size can be calculated as 128MB for each 1TB of VM data (per each replica restore point).”

Next post we will touch on the WAN Accelerator options and benefits.

Veeam Replication – Walkthrough

Now we have set the scene with our previous posts I wanted to share a complete walkthrough on Veeam Replication and show the simple steps it takes to send your virtual machines from Site A to Site B.

This post is aimed for those environments that have a Veeam Backup & Replication server installed and configured. It will also expect that you have deployed an additional Veeam Proxy on that secondary site.

When I mention secondary site, this could be considered either onsite or offsite. If it is the same onsite location then the same source proxy can be classed and used as the target proxy for data movement, however it must have access to the source and target environments.

It is also required that the backup management server can connect to both source and target hosts.

Once you have that deployed its then time to create your replication job. The license file I have is only for VMware but Microsoft Hyper-V is also an option.

Create Replication Job

Job Name and Advanced Configuration

A wizard will then open and you can begin to configure how the replication job should look. On the first screen as well as the Job name and description there are some advanced features that we will cover in later posts.

Add Objects

The following screen is probably the most important, this is where you will select the VMs you want to replicate, this can be done by either selecting the individual VM or you can use different construct options in the top right-hand corner, these could be based on cluster or host, datastore or the ability to use vSphere tags or Microsoft SCVMM tags.

Another useful tip here is if you have a large environment and you want to pin point certain named VMs then you have the ability to search and filter at the bottom of the screen.

Where are we going to get that data from? We have two options, we can choose to actually take directly from the production storage which is going to take the most recent state of the VM. Or we can choose to take from a backup location. We may cover this second option off in a later post.

On this same screen, we also have the ability to exclude certain objects from the replication job, this could be certain attached disks that are not required in a failure scenario. But for the sake of this walkthrough I am going to keep things very simple. It is worth noting that Veeam will already exclude VM log files and templates to reduce the size of the replicated data.

You may have noticed some additional buttons here on this data source screen. The Up and Down function. This allows us to set an order. Mission critical VMs should be at the top because we want to make sure we process them first. (only relevant if you have added individual VMs)

Destination Options

Now we must choose where we want our VMs to be replicated to, we choose our destination cluster or host, we can choose a resource pool or VM folder if we want. Finally, we choose the datastore we would like this replicated VM to be stored in.

Job Settings

This first screen requires a backup repository that has been configured within the management server where the source side metadata can be stored, a replica suffix also should be specified this is how the VM will look within the secondary location i.e VMNAME_replica in this instance, and then restore points. How many restore points would you like or need on that secondary location.

Under the advanced tab we have some additional settings that you may choose to configure depending on your environment and infrastructure.

Traffic settings – By default this is what the compression and deduplication configuration will look like, to understand more about these options then you can find more information here – https://helpcenter.veeam.com/docs/backup/vsphere/replica_advanced_traffic_vm.html?ver=95

Notification settings – This screen allows us to set specific notifications for this specific job either via email or via SNMP or both. https://helpcenter.veeam.com/docs/backup/vsphere/replica_advanced_notify_vm.html?ver=95

vSphere settings – By default it is set that CBT will be enabled but this can be changed if required. You also have the option to enable VMware tools quiescence. https://helpcenter.veeam.com/docs/backup/vsphere/replica_advanced_vsphere_vm.html?ver=95

Integration settings – If adding over 100 VMs to a backup or replication job. This could cause a very high VM snapshot lifetime for the first VMs in the job list.

Script settings – This allows you to run pre-or post-scripts before or after the job if you would like to run specific scripts within a VM then you have that option later on in the process under the guest processing wizard.

Data Transfer

Now to choose how the VM data is going to be transferred to the secondary site. The appropriate proxies at this stage should be selected and you should also determine if WAN Acceleration is used or not this is something we will again cover later on.

Guest Processing

Depending on the workload it may require your VMs are application consistent. Options can be configued here by providing the appropriate requirements.

In this instance, I have a backup job dealing with truncation of logs. I have configured the job to copy only the logs to the secondary site. There is also the ability to exclude OS files from the VM and to run specific scripts from within the VM.

Scheduling

Lastly in the configuration wizard is when we want to schedule this job to run.

Summary

Summary of the configuration and then we are good to go and run this job against that schedule we just set.

Next up we will look at this same process through PowerShell.

Veeam Replication – Transport Modes

I decided to give this topic its own post. This is probably the most important consideration when it comes to replication and how best to utilise the environment. If you are using Veeam you will most likely know that there are multiple options when it comes to how Veeam will take the backup and send to the repository. This is the same with the replication engine.

In this post, I want to outline all available transport modes and how possibly each one will be best in the environment.

This post will only reflect VMware estates and not that of Hyper-V.

VMware

For all transport modes, Veeam leverages the VMware vStorage APIs for Data Protection (VADP)

There is no “Best” transport modes as such but being able to leverage Storage integration is going to give you the benefits of not having to hold a VMware snapshot open for so long. But this and all modes depend on the environment.

Each transport mode depends really on the type of datastore being used at source. Also, the backup proxy depending if that is physical or virtual.

Before we continue I want to highlight one more thing on this diagram.

Associated to the transport modes are the following steps:

5. The source Veeam Data Mover establishes a connection with the target Veeam Data Mover, and Veeam Data Mover.
6. The source Veeam Data Mover reads the VM data from the read-only VM disk and copies it. During incremental job sessions, the source Veeam Data Mover uses CBT to retrieve only those data blocks that have changed since the previous job session. If CBT is not available, the source Veeam Data Mover interacts with the Veeam Data Mover on the backup repository to obtain replica metadata, and uses this metadata to detect blocks that have changed since the previous job session.
7. The target Veeam Data Mover decompresses VM data and writes the result to the destination datastore.

These steps and diagram were taken from the Veeam user guide – https://helpcenter.veeam.com/docs/backup/vsphere/replication_chain.html?ver=95

Network Mode

The beauty of network mode is that it can be used in any infrastructure regardless of storage presented. All the data is going to be retrieved via the vSphere environment over the LAN using the Network Block Device Protocol (NBD) or NBDSSL in vSphere 6.5 or above.

The impact on of this mode is that of low data transfers over the LAN this is why we would recommend where possible to move down this list to find a more performant and efficient transport mode.

Virtual Appliance (Hot-Add)

This mode is not as efficient as the following Direct Storage modes but can be more efficient than Network mode that we just discussed. Ideal if you are looking to achieve a 100% virtualised footprint in your data centre.

The appliance (A windows VM) is deployed within the environment. It uses the VMware SCSI HotAdd capability that allows attaching devices to a VM while the VM is running. During replication, the disks of the required VM are attached to the Veeam Proxy, data is retrieved directly from the datastore.

Direct Storage Access

This mode reads and writes data directly from and to the storage system where VM data or backups are located.

Direct SAN

If you are running your virtual machines hosted on VMFS SAN LUNs. Then this is the right transport mode for you, apart from if you have storage integration and even then, this is underlining the same process but it will leverage storage snapshots rather than the live LUN from the storage array.

Rather than step 7 in this instance transferring to a backup repository this will transfer to another proxy on a remote site and into the secondary remote datastore as a replica VM.

Direct NFS

If you are running virtual machines hosted on an NFS export. This mode allows for the proxy to bypass the vSphere environment and reads directly from the NFS datastore or NFS storage array. VM data still travels over LAN but there is no load on the ESXi host.

Storage Integration

If this can be achieved as your transport mode then this is the route to take. The ability to leverage the same benefits of Direct Storage access but now not from the live volume or LUN but to a storage snapshot, which is going to benefit from the fast removal of VMware snapshot as well as no contention when pulling data from the production LUN or volume. The snapshot will be presented to the Veeam Proxy and data retrieved.

More on Storage integration will come later in the series.

I hope this has given you a good understanding of what each transport mode represents and also some use cases on how they can be used in certain environments.

More detail on all things Veeam and Replication can be found here – User Guide for VMware vSphere

The Workflow & Components

In the last post, I covered a 101 of replication in general and mentioned where Veeam has a play in A-Synchronous and will have Near Synchronous Replication in v10 in the form of CDP.

In this post, I wanted to walk through the steps taken when Veeam is used for A-Synchronous replication in your virtualised estate.

First, I want to touch on the components that are required for any sort of Veeam replication, these can double up for the backup elements also.

Veeam Backup Server

• The “configuration and control center”
• Coordinates tasks, controls job scheduling and resource allocation
• Set up and manage backup infrastructure components
• Windows-based physical or virtual machine on which Veeam Backup & Replication is installed

Veeam Proxy Server

• Used to process jobs and deliver backup traffic
• Takes data processing off the Veeam Backup server
• Controls Deduplication and compression tasks on the data
• A dedicated Windows server (physical or virtual)
• You can deploy backup proxies both in the primary site and remote sites

Veeam Replication Architecture

The Snapshot

During the first run of a replication job, Veeam Backup & Replication copies the VM running on the source host and creates its full replica on the target host. The replica is stored uncompressed, in a native hypervisor format.

All subsequent replication jobs are incremental. Veeam Backup & Replication copies only those data blocks that have changed since the last replication cycle.

These image level replicas are stored as restore points in the target destination as snapshots, there are some small differences between VMware and Microsoft Hyper-V, I will differentiate those below.

VMware

If you are leveraging Veeam replication on VMware then you will still something like this in your secondary location.

VMware has a supported limit of 32 snapshots, minus the working snapshots that Veeam requires and a bit of overhead, this would allow for a 28 snapshots/restore points limitation.

Hyper-V

For those using Microsoft Hyper-V you will see something like this on the target destination.

Microsoft Hyper-V has a supported limit of 50 snapshots, minus the working snapshots Veeam requires and a bit of overhead, this would allow for 47 snapshots/restore points.

The Process

VM data is moved block by block, with multiple processing cycles. The replication process is performed in the following way.

1. The Source Data Mover speaks with the Target Data Mover to start any data collection.
2. The Source Data Mover then checks the VM and copies the data using the correct Veeam transport mode. This is related to how the Proxy has been configured to access the virtual infrastructure. During scheduled incremental job runs the Veeam data mover service will only take the changed blocks since the last replication job. As well as copying the data the source data mover will perform some additional tasks, it will consolidate and filter out zero data block and swaps files from the virtual disks.
3. Those changed blocks are compressed and transferred from the source data mover to the target data mover.
4. The target data move decompresses the data and writes the result to the destination datastore.

How do we get that first bit of data over?

When planning for offsite replication, we should consider available features to reduce the amount of replication traffic and streamline replica configuration.

Seeding

Replica Seeding allows us to send over the bulk of data via physical methods, this could be by placing the copy of data onto moveable hardware and shipping that to the target destination. USB Drive, Storage Array etc. Don’t underestimate the bandwidth of a man in a van. It will save a lot of time and bandwidth over those expensive links.

WAN Accelerator

Another Veeam component, software only and it’s there to optimize VM data transfer, this is achieved by Global data caching and deduplication. The configuration for the WAN Accelerator is a One to many configuration, or can be used in pairs if you have an active pair of sites, A single WAN accelerator in the primary location, multiple WAN accelerators at branches or targets destinations.

There are some great best practices noted in the Veeam Best Practice guide around WAN Acceleration.

I will be sure to cover off in more detail some of these components and highlight some best practices, sizing and deployment facts from the field.