So far in this series I have shown you hopefully how powerful the Replication side of the house is within Veeam Backup & Replication. It is by no means the whole topic covered, we will have CDP as mentioned as a feature, Veeam also have the Veeam Availability Orchestrator which I consider the next level Veeam Replication on Steroids and that alone deserves its own series.

There is another form of replication though that in some environments can be driven by Veeam to allow for data to reside in that secondary location. This comes in the form of the storage snapshot integration. This was also mentioned in regard to transport modes used for your replication jobs but when I see storage snapshot orchestration I mean that Veeam is capable of using the storage replication technology and drive that through the Veeam user interface.

Overview

Veeam’s Integration allows Veeam Backup & Replication to sit on top of the storage systems from NetApp & HPE Nimble and control application consistent Snapshots for local fast Recovery Point Objectives. Veeam also enables the ability to orchestrate and schedule those app aware snapshots to be sent to a secondary storage array from the same vendor.

We also have similar capabilities with HPE 3PAR with their Peer Persistence replication technology. The screen below one step further also in that we are able to then take a backup of the data from the secondary location.

Configuration

Let’s run through some of the options for all of these firstly how does this look from a HPE Nimble and NetApp point of view.

For this orchestration, it’s a backup job first, we need to create a new backup job and this is also after the storage arrays have been configured and added to the console. Notice how from the drop down this allows me to choose a Primary Storage Snapshot.

Next, we can select the configure secondary destination for this job.

Next, we have the options to choose where the secondary destination should be. It is worth noting that this feature will only talk to the same vendor and model specific i.e HPE Nimble to HPE Nimble and NetApp ONTAP to NetApp ONTAP.

All software versions should be checked within the system requirements before attempting this.

The retention set here is how many snapshots on the secondary array should we be keeping.

When this job runs providing licensing and network replication has been configured on the storage arrays this will orchestrate the storage snapshots and replication to the target location.

HPE 3PAR

Is a similar process to the one above, I need to clarify some of the next points though as these were put together during a beta phase. Thanks to Pierre and Niels for their work on the diagrams and screen grabs.

Much the same if we want to leverage the storage replication for that secondary target then we can select the following configuration with the required remote copies of snapshots to keep.

from a recovery point of view, the Veeam explorers can be used for recovery tasks however the failover and failback from Veeam replication will not work.

In this day and age, we want to be able to do more with less, seems to be everywhere, or we want to be able to do something additional with what we have already got. Get more ROI out of an asset. Leveraging data is no difference, for years as technical customers we have been backing up our data to some form of media, this could be classed as dumb storage. We send our backups there then we update again at the next iteration. We only come back to that backup or replicated data if we need something in the form of a recovery.

Why not leverage it, that’s your production assets in the form of a backup or replication state. In the last post, I discussed SureReplica which is the Veeam way in which those replicated virtual machines can be verified for recovery. To do that we power on those machines in an isolated network away from production and in this instance in our secondary site for disaster recovery. Fully automated to make sure that those files are good for if we need to failover our workloads.

Verification is one thing leverage is another, by leveraging that data we are getting more use from that target storage array that traditionally may have just been sat there holding those replicated VMs.

SandBox Labs via SureBackup Job

By clicking a simple step within the SureBackup job wizard you can keep that same application group up and running for as long as you need. A good use for this is patching, if you are responsible for patching your key application servers or all servers then this could help that process.

Options

One thing to note here is that this is going to start that same application groups it’s going to adhere to the configuration of that application group about memory assigned and boot time delay etc. be sure to configure those tests and settings to ensure you have the optimal configuration when running a Sandbox environment.

You also have the ability to run your own scripts here this is part of the verification also, this could be a script that you may run that runs a windows update via script to test updates faster.

Scheduling & Stopping

When you have that in line you are able to start your job or schedule that job. What I mean by scheduling is that this will run a scheduled time and spin up these replica VMs in your secondary location the difference here is that they will keep running. Whilst they are running no further replication jobs can be completed. I want to show you how you can stop the session so that you don’t face this issue, when you are finished with your sandbox testing then be sure to stop that session.

By stopping the session, you are going to shut down them virtual machines and you are going to lose anything you placed on that we are not going to write anything to those replica files or backups.

Tips – Data Sources

A useful tip here is that you may have these VMs in your application group that are being replicated over to the secondary location however you may have a Domain Controller or something alike already running in that location, with that you may or I should say should be taking a local backup of that Domain Controller. Within your application group configuration, you can mix your workloads, we could use some of the replicated virtual machines and we can use some of the local backup files.

There is also one more option of sources and that’s the ability to leverage storage snapshots. I will touch on more later on in the series on this one.

Other Resources

I also did a vBrownBag session at VMworld in 2017 that allowed me to demonstrate the power of this cool unsung feature within Veeam.

<Link to VMworld YouTube Session>

What if I told you that the same tool that you purchased for Backup & Replication could also perform a level of verification against your replication jobs. Oh and did I mention that this test would automatically test the verification of Virtual Machine, Operating System and Application.

SureReplica gives you the ability to spin up an isolated network in your secondary location to confirm that the restore point is in a good state should you require to actually recover from it, rather than waiting for a failure scenario to occur and then you are sat there with crossed fingers hoping the backup or replica is in a good state to recover from.

This same feature allows for some further lab environment, a sandbox that can be accessed whereas this is fully automated. This will be coming up next in the series.

Components required for SureReplica

There are some one-time configuration components that we need to consider before we get going. These really don’t take long to configure.

Virtual Lab

Most of the time we would not want our verification tests to interfere with the production network, this is why we create a Virtual Lab, a virtual lab consists of a virtual network switch on your remote site ESXi host.

This can be on a single host or a multi-site environment, depending on the size of the testing group of Virtual Machines.

The most basic requirement here requires:

• A resource pool
• VM Folder
• A standard vSwitch

The VMs are started in this virtual lab and this is the only use case for the vSwitch, there is a proxy appliance also deployed at this stage on the same host, resource pool and VM folder. This is used to bridge connectivity between the production network and this isolated network.

Some more detail on this configuration can be found here – https://helpcenter.veeam.com/docs/backup/vsphere/surereplica_vlab_config.html?ver=95

Application Group

An Application Group is a group of Applications, kidding, this is where we can define our virtual machines that we would like to verify in our secondary location. Remember here that it might be one mission critical system that you want to test, but you need to make sure within that application group you have all dependencies so we can truly test if that application is in a good state or not.

I will also add that you can mix where your VMs come from this could be from Storage Snapshots, Replication Jobs and Backup Jobs all mixed accordingly to achieve the requirements that you have.

SureBackup Job

In the Veeam Backup & Replication console this type of job is called SureBackup, this offers that same level of verification to the Veeam backup files.

To piece the Application Group and Virtual Lab together comes the SureBackup Job. Let’s run through this configuration with our SQL example we used in the last post.

Firstly, find the SureBackup job wizard in the console.

Set the name and description for the job, this wizard is assuming you have been through and created your virtual lab and application group.

Select the required virtual lab that you have created. If you need pointers on this then there is a demo video at the end of the post.

Select the Application Group that you have also just created.

Additional notification settings can be added at this next screen to determine SNMP traps or emails as well as some integrity checks. This is going to give you the audit trail for that DR testing plan though that normally takes weeks and weeks to perform.

If you are wanting to run an automated Disaster Recovery test, which Is completely possible then you may want to make sure this runs on a scheduled basis. Remember this will consume resource, within the application group creation you can determine the associated ram in terms of a percentage to the live production system, just be cautious in choosing when this should take place.

Finally, a summary page to show you the configuration, also the option to run that job.

Now that you have this created and scheduled you can now wait for the scheduled time to come by and then once this job has completed you will either receive an email notification if this has been configured or you can see from within the console.

https://youtu.be/LHoTua_fxNg

We are into the home stretch on this series now, this post will cover the failover and failback of our replicated virtual machines.

In the previous posts, we setup our replication jobs and got our data going from one site to another, so this one will cover how easy it is to failover for a reason I will let you imagine, for the story though let’s use power outage but we only need to failover our SQL VM for a few hours whilst this host power issue is resolved.

For the purposes of this post lets expect that all the replication jobs have been successful, of course they are if you have been following. Oh, and we have some restore points in our secondary location.

The screen above shows the ready replicas that we have powered off in our secondary sites. Notice in the above though that I have two replica locations and in some instances, I have our Domain Controllers and SQL boxes going to both locations. This could achieve a multi-site failover plan if required. Or also it could be that one site is a development site on the same campus as live and these replicas can be used for a SureReplica job (VMware only) This will be covered in a forthcoming post.

When you right click on the VMs you have some options.

The ones we are interested in are the top three.

Failover Now – This is the panic stations button, disaster scenario and we need everything over and up and running as soon as possible, we will cover this shortly.

Planned Failover – A little less panic station, maybe a power situation on the host and we only need a subset of machines to failover to a secondary but the difference being here is we have the planned time for the maintenance. We will get to this later as well.

Add to Failover Plan – A failover plan allows you to group a number of virtual machines and set some basic configuration on how they get started up in that second location, in this scenario we are only going to failover one VM but let’s walk through the process as it would still work with 1 VM.

Failover Plan

When you select through the wizard above you get the simple steps to create a new failover plan. On the first screen, you can give a name and description and you are able to add pre-and post-failover scripts. Script files formats allowed: BAT, CMD, EXE and PS1. An example here could be that you have a tertiary application that is linked to this particular VM that is not part of the failover so we may just want to trigger a PowerShell script to stop a certain service on the application server.

The following screen is where you can choose the VMs you require to be part of this failover plan. You have the ability to set a delay this is to delay the VM boot time as part of the failover trigger, on this same screen you are able to move the VMs up and down according to the order in which you need them to boot.

That’s it once you complete that step its then time to see the summary of the configuration for the failover plan, simple stuff here as we only have one VM. I do quite like the command function it gives you in this last summary screen. Meaning you could in theory run this from another location that can talk to the VBR server.

In the console, you will then see the newly created failover plan appear

I am not sure how many people are actually aware of this fact but if you then navigate to the failover plan you have two options to choose from, you can start and that’s going to kick off a failover plan, starting the VMs at the secondary site. And then you have Start to, this allows you to schedule the most recent replicated restore point to be powered on. One thing to note here is this will not shut down the primary VM. If you select the undo option here when the replicas are running this allows you undo without reverting any changes to the primary VM.

Planned Failover

Planned Failover is the ability to deal with scheduled disruptive maintenance. Going back to that replica screen and right clicking on the VM you have this option. You can choose further VMs here if required.

We can then add our audit reason for performing this.

Finally, the summary.

The next stage in this option is a further incremental from production to secondary will take place. The great thing about this option is that the planned failover job will shut down the production system as well. It will then perform another update to the secondary site.

At this point we are now running our SQL workload on the secondary site. When I first open up the SQL management studio you can see I only have one very important DB.

The power outage has taken a little longer than expected so much so we had to create another database for a new application. We may have rushed the naming convention.

Failback

We have been told it won’t take long now for the power company to switch things back on at our main data centre. This is the view from the Veeam Backup & Replication console.

The options that you get when you right click on that VM are shown below, this gives us some options to consider, if we thought performance is much better now it’s over in our secondary location then we could choose permanent failover, or we could use undo failover if we just want to bring things up in the primary forgetting all actions performed on the secondary.

We got the phone call to say that things were all back online from a power perspective so we tested connectivity and it was good to failback with all of our changes. We selected failback to production and a new wizard appears.

We have a few options as part of the failback wizard. Be sure to check out the options here. Quick Rollback is a great feature that is not mentioned enough. I will try and write up something on that another time.

Last page although a summary it’s important to note that this is where you can trigger the power on of the VM.

Steps taken on failback.

and finally showing the failed back instance with the newly created database still in place.

Finally we have to tell Veeam that we are ready to commit this failback.

It’s worth noting before closing out that the failover now option, assumes that the production VM is already powered off and we are getting straight into powering on the secondary VM, no incremental replication jobs happen. Recovery options are the same as we went through above.

Veeam Replication – WAN Accelerator

The Veeam WAN Accelerator is an advanced feature but I decided to give it its own spotlight because how powerful it is.

Licensing wise it requires Enterprise Plus licensing unless you are leveraging cloud connect to a service provider where Enterprise is sufficient.

What is it for? Well it’s for reducing the bandwidth required for transferring either backups or replicas over a WAN.

So, when should it be considered? Let’s start by saying it’s not suitable for every environment. In fact, if your environment has some of the things we are going to talk about later then the WAN Accelerator could actually be a burden.

Do you have a link between 3MB/s and 40MB/s? – This could be the sweet spot and where the WAN Acceleration could work for you and bring you some benefits.

However, although the Veeam WAN Accelerator is included in that Enterprise Plus license you really need to consider that bandwidth availability over the WAN connection. You should see how much it’s going to cost first as with the reduction in bandwidth requirement comes cost on resources to make that happen.

Requirements

If you have determined from the above best practices outlined in the Veeam Best Practices guide here – http://bp.veeam.expert then you will need consider the requirements for your WAN accelerator deployment.

The Veeam WAN accelerator works in pairs which is what we will cover, there is also a solution, many to one.

We will have a WAN Accelerator at the source site, this will require access to the Veeam Management server, at least one source Veeam proxy and the remote site WAN Accelerator.

The target WAN accelerator on the remote site. This one will require access to the same management server, target side Veeam proxy and the source WAN accelerator. It is worth noting here that if you are also using the WAN accelerator for Backup Copy operations the target will also need access to the target repository.

The source Veeam proxy must have access to the Veeam management server, source hosts, source WAN accelerator and backup repository holding replica metadata.

The target Veeam proxy must have access to the Veeam management server, target hosts and target WAN accelerator.

We have also mentioned earlier in the series regarding the replication metadata, this is to be stored within a backup repository on the source site. The closer to the Veeam proxy the better.

Global Cache (6 behaviours of a Global Cache)

• The global cache is a folder on the target WAN accelerator.
  
• The global cache data is stored in the VeeamWAN folder on the disk with the most amount of space available. (This is default and can be changed)
  
• The size of the global cache is 100 GB (This is default and can be changed)
  
• The global cache size is specified per source WAN accelerator.
  
• The WAN global cache is a “library” that holds data blocks repeatedly going from the source side to the target side.
  
• The global cache is populated at the first cycle of a remote job.
  

The Veeam management server is checking the cache every now and again for certain things this is done by monitoring the replicated data and going to the target against the global cache. During this check, it is checking for two different outcomes.

1. Any New Data that is required to be sent over the WAN should be added to the cache.
   
2. Any Old Data that is no longer required or is not used as much, this makes room for number 1.
   

Veeam Backup & Replication will also be checking for consistency against corruption, if blocks are corrupted and found in the cache they will be removed.

There is so much information on the Global Cache this can be found here – https://helpcenter.veeam.com/docs/backup/vsphere/wan_global_cache.html?ver=95

Efficiencies

The whole point to WAN acceleration is to reduce the amount of data going from one site to another or over that WAN.

Deduplication takes place takes place in three areas:

1. Virtual Machine Disks – analyse data blocks within the VM disk. If identical blocks are found, duplicates are removed. (step 2 below, Workflow)
   
2. Restore Points – analyse data in the restore point that is about to be replicated against the already replicated restore points. If identical blocks are found, duplicates are removed. (step 5, Workflow)
   
3. Global Cache – analyse data blocks to be sent and compares them with data blocks stored in the global cache. If identical blocks are found, the duplicate on the source is removed. (Step 6, Workflow)
   

Workflow

Once deployed, head back to your replication job, configure the below setting and determine the Source and Destination WAN Accelerators.

1. When this job runs the Source WAN Accelerator will analyse blocks and create a digest that is stored in that VeeamWAN folder.
   
2. Compress Virtual Machine data and copies to target.
   
3. The target WAN accelerator populates the global cache storage with data blocks during the transfer process.
   
4. Each job cycle after will analyse and create new digests.
   
5. Compare the created digests with the digests that have been previously stored on source.
   
6. Analyses restore points that have been previously copied to the target.
   

Veeam Replication – Advanced features and functionality

Welcome back, as we move through this series where we have touched on the basic replication configuration, or I should say the easy to use deployment, the walkthrough and some more advance configurations or ways in which to achieve the same outcome from PowerShell commands.

This post will hopefully share some of the more advanced features, options or even some of the outcomes you might not be able to find any detail on.

Remote replica from backup

The clue is in the title, this gives the ability to create a replica VM from a backup that has already traversed the WAN or already resides on that secondary location. This method has some benefits in that it will not have to touch the production estate and you obviously do not need to send the data twice if you have a backup copy configuration to the same site. More information can be found here –

https://helpcenter.veeam.com/docs/backup/vsphere/replica_from_backup.html?ver=95

Low Connection Bandwidth

Another option when creating your replication job was “Low Connection Bandwidth” Introducing Replica Seeding, this is the ability to use the man in the van bandwidth. Poor links between sites or large datasets then if possible take advantage of seeding. This could be used in some cases for getting replicas to our cloud connect providers.

Basically, if you have a backup for the replicated VM on the backup repository located in the DR site. You can point the replication copy job to this backup. The first run of the replication job will seed the data from the backup file, i.e. it will create a VM replica image, once complete it will resync with the production VM for any changes since the backup file. Notice how it only does that seed once where as remote replica continues to use that same method.

Giving a shout out to Luca here on his blog a while back he wrote about seeding to a service provider (Cloud Connect – https://www.virtualtothecore.com/en/seeding-veeam-cloud-connect-part-3-replication-jobs/)

Separate Virtual Networks

Not really an advanced setting as such but something I should cover here, if your target environment does not use the same networking, this setting allows you to map the production (source network) and the secondary (target network)

Calculating Digests

For those of you already running Veeam replication you may have seen this, let’s cover what this process is and what do we expect to see from a calculation point of view. This task is checking the difference between the source VM and the target VM (Replica VM)

A good rule of thumb here is 1GB per minute, just bare that in mind when you are calculating how long your replication jobs are going to take.

Why does this happen? Well you are probably going to know why this is happening but just to run through some of them.

Most commonly it’s if you change something on that source VM, add a disk for example probably the most common update or change here and there is very little you can do to avoid this then wanting to calculate the digests during the job.

You will also find this message in your job log if you reinstall vCentre or something happens to your vCentre and something changes your VM IDs. Datastore disconnections could cause it, oh and my favourite is if you remove the source VM from the inventory and then re add it back in. it’s going to get a new VM ID and that’s also going to prompt for this process.

There is very little we can do with the above scenarios but sit it out, although if you find that something is outside of those timeframes then a call to support might be worth the clarification of the setup.

This job running at the moment from @HammondRhys shows what can happen if you run out of space on the repository which is used for that replication metadata another use case or fail case for the calculating digests.

Different IP addressing Scheme

More common that not these days is the requirement for different IP addressing in a secondary location. This exact setting or configuration is available as part of the Veeam replication wizard.

By setting this different IP addressing scheme when a failover plan is triggered it will apply these different addresses to the replica VM.

A big note here is that this is only applicable to Windows virtual machines.

Storage Integration Settings

I touched on this very briefly in the transport modes post but wanted to touch again on an example of when this setting needs to be considered.

When configuring such large jobs, it is advised to configure the maximum number of VMs within one storage snapshot. The setting is available in the advanced job settings within the Integration tab.

Example: When creating a job with 100 VMs and setting the limit to 10, the Backup from Storage Snapshot job will instruct the job manager to process the first 10 VMs, issue the storage snapshot, and proceed with the backup. When that step has successfully completed for the first 10 VMs, the job will repeat the above for the following 10 VMs in the job.

Replica Metadata

Finally, and I know this post is now getting on the long side but given some recent conversations I wanted to add at least something in here that may help when it comes to metadata location and sizing.

The latest information I can find is from one of our product managers. Outlining a good rule when it comes to sizing for metadata space.

“Typical metadata size can be calculated as 128MB for each 1TB of VM data (per each replica restore point).”

Next post we will touch on the WAN Accelerator options and benefits.

Veeam Replication – PowerShell

There is a large adoption of people using PowerShell to make their lives easier and reduce time spent on performing lengthy repeatable tasks. Veeam Replication could be one of those where in the last post I showed the simple steps to setup your replication jobs. If you have many different groups or even sites, then to do that a number of times is going to be a pain.

What I want to show you in this post is how easy it is to use the Veeam PowerShell Snap-In and the full set of capabilities to run and manage those replication tasks.

Before I begin there is a more in-depth resource on all different PowerShell commands that can be found here – https://helpcenter.veeam.com/docs/backup/powershell/replication.html?ver=95

For the purpose of this post I am going to be using VMware as my platform the command structure for Hyper-V is slightly different and the above link can also help there to determine the required steps.

Add-VBRViReplicaJob

Resource – https://helpcenter.veeam.com/docs/backup/powershell/add-vbrvireplicajob.html?ver=95

There are many parameters available for this cmdlet these can be found in more detail at the above link. I am going to be using the same parameters that we used during the walkthrough in the last post.

First, we need to add the snap in and connect to our Veeam Backup & Replication Server this can be done with the following. All code will be available in the end script.

#Add the Veeam PSSnapIn to get access to the Veeam Backup & Replication cmdlets library. Run the following command:
Add-PSSnapin VeeamPSSnapin
#Connects to Veeam backup server.
Connect-VBRServer -server "10.0.40.10"

This command will set the server you want the replica to be sent to, it will also save it to a variable of $Server.

#Set Destination ESXi Server 
$server = Get-VBRServer -Name "tpm03-131.aperaturelabs.biz"

Next, we need to determine which VMs or objects we would like to replicate we can do this with the following command. And this will set the VM to a variable.

#Define Source Virtual Machine(s)
$vm = Find-VBRViEntity -Server vc03.aperaturelabs.biz -Name "TPM04-DC-01"

Now we need to determine the resource pool that we would want the replica to be located in and set that as a variable.

#Define destination resource pool 
$pool = Find-VBRViResourcePool -Server $server -Name "TPM04-MC"

The final task of finding and setting variables comes in the form of which datastore do we want our replica to be stored in.

#Define destination datastore
$datastore = Find-VBRViDatastore -Server "tpm03-131.aperaturelabs.biz" -Name "SolidFire005_iSCSI"

I will share the commands that I used to find out the below information at the end of the post. Now that we have set our variables and we know the:

• What we are replication
  
• Where are we replicating
  

Now that we have set the variables for our script it might be worth just checking that they all return the right objects this can be done with the following code.

#Now that you have defined your variables I would suggest running the following to ensure all have been populated.
$server
$vm
$pool
$datastore

Creating the Replication Job

We can now create the replication job using the above information.

#This command will create the backup job with no schedule defined.
Add-VBRViReplicaJob -Name "PS Replication Job" -Server $server -Entity $vm -ResourcePool $pool -Datastore $datastore -Suffix "_replicated"

Now that we have the job created you can also see the job created in the Veeam Backup & Replication console.

At this point or later we also have the option to add additional VMs to the replication job. The first one will add the additional VMs to the variable. The second is how you would update the job.

#If you then wanted to add additional VMs to the job then you could do by adding to this line of code 
$vm = Find-VBRViEntity -Server vc03.aperaturelabs.biz -Name "TPM04-DC-01", "TPM04-SQL-02" 

#This command will then update the job with the newly added Virtual Machines. 
Set-VBRViReplicaJob -Job $job -Server $server -Entity $vm

See below the console view now with the newly added extra VM in the job,

If we had a large number of Virtual Machines to add to the job maybe with a similar naming convention such as the one I have used here, then we also have the ability to add against a wildcard to the job.

#This command will also allow you to add wildcards to your replication job Note that this will add all objects with this name. 
Find-VBRViEntity -Name TPM04-* | Add-VBRViJobObject -Job $job 

We can then use the following command to give us the configuration for the job. As well as setting a variable for the job.

#This command will show the job configuration 
Get-VBRJob -name "PS Replication Job" 

Setting the schedule

For those that have noticed we have the job created but no schedule so next up is creating a schedule for the requirements. The below command will set the schedule on the job.

#This command schedules the job represented by the $job variable to run every 1 hour
Set-VBRJobSchedule -Job $job -Periodicaly -FullPeriod 1 -PeriodicallyKind Hours

As well as setting the schedule we need to enable the schedule on the job this can be done like this.

#This will enable the job schedule
Enable-VBRJobSchedule -Job $job 

Navigating back into the job in the console then you will now see the schedule is now configured.

Starting the Job

We have come this far so let’s continue and start our job from the PowerShell script as well.

#The following command will allow us to start the job 
Start-VBRJob -Job $job

within the console

As the job progresses we will see on the destination ESXi host our replicated virtual machine.

When the job completes we get the following summary back in our PowerShell window.

Obviously, we can get that same information and some more from the console.

The complete script

#Add the Veeam PSSnapIn to get access to the Veeam Backup & Replication cmdlets library. Run the following command:
Add-PSSnapin VeeamPSSnapin
#Connects to Veeam backup server.
Connect-VBRServer -server "10.0.40.10"
#Returns hosts connected to Veeam Backup & Replication.
Get-VBRServer 
#This will show all objects 
Find-VBRViEntity
#This will show all resource pools available on the defined server either use variable or add in your ESXi host name/ip
Find-VBRViResourcePool -Server $server
#This will show available datastores on host 
Find-VBRViDatastore -Server "tpm03-131.aperaturelabs.biz"
#This will show availabel VMs 
Find-VBRViEntity -Server vc03.aperaturelabs.biz -name "TPM04-*"

#Set Destination ESXi Server 
$server = Get-VBRServer -Name "tpm03-131.aperaturelabs.biz"
#Define Source Virtual Machine(s)
$vm = Find-VBRViEntity -Server vc03.aperaturelabs.biz -Name "TPM04-DC-01"
#Define destination resource pool 
$pool = Find-VBRViResourcePool -Server $server -Name "TPM04-MC"
#Define destination datastore
$datastore = Find-VBRViDatastore -Server "tpm03-131.aperaturelabs.biz" -Name "SolidFire005_iSCSI"
 

#Now that you have defined your variables I would suggest running the following to ensure all have been populated.
$server
$vm
$pool
$datastore

#This command will create the backup job with no schedule defined.
Add-VBRViReplicaJob -Name "PS Replication Job" -Server $server -Entity $vm -ResourcePool $pool -Datastore $datastore -Suffix "_replicated"

#This command will show the job configuration 
Get-VBRJob -name "PS Replication Job"

#If you then wanted to add additional VMs to the job then you could do by adding to this line of code 
$vm = Find-VBRViEntity -Server vc03.aperaturelabs.biz -Name "TPM04-DC-01", "TPM04-SQL-02"

#Setting a variable to the newly created replication job
$job = Get-VBRJob -name "PS Replication Job"

#Confirming that the variable has populated
$job

#This command will then update the job with the newly added Virtual Machines. 
Set-VBRViReplicaJob -Job $job -Server $server -Entity $vm
#This command will also allow you to add wildcards to your replication job Note that this will add all objects with this name. 
Find-VBRViEntity -Name TPM04-* | Add-VBRViJobObject -Job $job

#This command schedules the job represented by the $job variable to run every 1 hour
Set-VBRJobSchedule -Job $job -Periodicaly -FullPeriod 1 -PeriodicallyKind Hours

#This will enable the job schedule
Enable-VBRJobSchedule -Job $job

#The following command will allow us to start the job 
Start-VBRJob -Job $job

Reference – https://helpcenter.veeam.com/docs/backup/powershell/create_replica_vmware.html?ver=95

Veeam Replication – Walkthrough

Now we have set the scene with our previous posts I wanted to share a complete walkthrough on Veeam Replication and show the simple steps it takes to send your virtual machines from Site A to Site B.

This post is aimed for those environments that have a Veeam Backup & Replication server installed and configured. It will also expect that you have deployed an additional Veeam Proxy on that secondary site.

When I mention secondary site, this could be considered either onsite or offsite. If it is the same onsite location then the same source proxy can be classed and used as the target proxy for data movement, however it must have access to the source and target environments.

It is also required that the backup management server can connect to both source and target hosts.

Once you have that deployed its then time to create your replication job. The license file I have is only for VMware but Microsoft Hyper-V is also an option.

Create Replication Job

Job Name and Advanced Configuration

A wizard will then open and you can begin to configure how the replication job should look. On the first screen as well as the Job name and description there are some advanced features that we will cover in later posts.

Add Objects

The following screen is probably the most important, this is where you will select the VMs you want to replicate, this can be done by either selecting the individual VM or you can use different construct options in the top right-hand corner, these could be based on cluster or host, datastore or the ability to use vSphere tags or Microsoft SCVMM tags.

Another useful tip here is if you have a large environment and you want to pin point certain named VMs then you have the ability to search and filter at the bottom of the screen.

Where are we going to get that data from? We have two options, we can choose to actually take directly from the production storage which is going to take the most recent state of the VM. Or we can choose to take from a backup location. We may cover this second option off in a later post.

On this same screen, we also have the ability to exclude certain objects from the replication job, this could be certain attached disks that are not required in a failure scenario. But for the sake of this walkthrough I am going to keep things very simple. It is worth noting that Veeam will already exclude VM log files and templates to reduce the size of the replicated data.

You may have noticed some additional buttons here on this data source screen. The Up and Down function. This allows us to set an order. Mission critical VMs should be at the top because we want to make sure we process them first. (only relevant if you have added individual VMs)

Destination Options

Now we must choose where we want our VMs to be replicated to, we choose our destination cluster or host, we can choose a resource pool or VM folder if we want. Finally, we choose the datastore we would like this replicated VM to be stored in.

Job Settings

This first screen requires a backup repository that has been configured within the management server where the source side metadata can be stored, a replica suffix also should be specified this is how the VM will look within the secondary location i.e VMNAME_replica in this instance, and then restore points. How many restore points would you like or need on that secondary location.

Under the advanced tab we have some additional settings that you may choose to configure depending on your environment and infrastructure.

Traffic settings – By default this is what the compression and deduplication configuration will look like, to understand more about these options then you can find more information here – https://helpcenter.veeam.com/docs/backup/vsphere/replica_advanced_traffic_vm.html?ver=95

Notification settings – This screen allows us to set specific notifications for this specific job either via email or via SNMP or both. https://helpcenter.veeam.com/docs/backup/vsphere/replica_advanced_notify_vm.html?ver=95

vSphere settings – By default it is set that CBT will be enabled but this can be changed if required. You also have the option to enable VMware tools quiescence. https://helpcenter.veeam.com/docs/backup/vsphere/replica_advanced_vsphere_vm.html?ver=95

Integration settings – If adding over 100 VMs to a backup or replication job. This could cause a very high VM snapshot lifetime for the first VMs in the job list.

Script settings – This allows you to run pre-or post-scripts before or after the job if you would like to run specific scripts within a VM then you have that option later on in the process under the guest processing wizard.

Data Transfer

Now to choose how the VM data is going to be transferred to the secondary site. The appropriate proxies at this stage should be selected and you should also determine if WAN Acceleration is used or not this is something we will again cover later on.

Guest Processing

Depending on the workload it may require your VMs are application consistent. Options can be configued here by providing the appropriate requirements.

In this instance, I have a backup job dealing with truncation of logs. I have configured the job to copy only the logs to the secondary site. There is also the ability to exclude OS files from the VM and to run specific scripts from within the VM.

Scheduling

Lastly in the configuration wizard is when we want to schedule this job to run.

Summary

Summary of the configuration and then we are good to go and run this job against that schedule we just set.

Next up we will look at this same process through PowerShell.

Veeam Replication – Transport Modes

I decided to give this topic its own post. This is probably the most important consideration when it comes to replication and how best to utilise the environment. If you are using Veeam you will most likely know that there are multiple options when it comes to how Veeam will take the backup and send to the repository. This is the same with the replication engine.

In this post, I want to outline all available transport modes and how possibly each one will be best in the environment.

This post will only reflect VMware estates and not that of Hyper-V.

VMware

For all transport modes, Veeam leverages the VMware vStorage APIs for Data Protection (VADP)

There is no “Best” transport modes as such but being able to leverage Storage integration is going to give you the benefits of not having to hold a VMware snapshot open for so long. But this and all modes depend on the environment.

Each transport mode depends really on the type of datastore being used at source. Also, the backup proxy depending if that is physical or virtual.

Before we continue I want to highlight one more thing on this diagram.

Associated to the transport modes are the following steps:

5. The source Veeam Data Mover establishes a connection with the target Veeam Data Mover, and Veeam Data Mover.
6. The source Veeam Data Mover reads the VM data from the read-only VM disk and copies it. During incremental job sessions, the source Veeam Data Mover uses CBT to retrieve only those data blocks that have changed since the previous job session. If CBT is not available, the source Veeam Data Mover interacts with the Veeam Data Mover on the backup repository to obtain replica metadata, and uses this metadata to detect blocks that have changed since the previous job session.
7. The target Veeam Data Mover decompresses VM data and writes the result to the destination datastore.

These steps and diagram were taken from the Veeam user guide – https://helpcenter.veeam.com/docs/backup/vsphere/replication_chain.html?ver=95

Network Mode

The beauty of network mode is that it can be used in any infrastructure regardless of storage presented. All the data is going to be retrieved via the vSphere environment over the LAN using the Network Block Device Protocol (NBD) or NBDSSL in vSphere 6.5 or above.

The impact on of this mode is that of low data transfers over the LAN this is why we would recommend where possible to move down this list to find a more performant and efficient transport mode.

Virtual Appliance (Hot-Add)

This mode is not as efficient as the following Direct Storage modes but can be more efficient than Network mode that we just discussed. Ideal if you are looking to achieve a 100% virtualised footprint in your data centre.

The appliance (A windows VM) is deployed within the environment. It uses the VMware SCSI HotAdd capability that allows attaching devices to a VM while the VM is running. During replication, the disks of the required VM are attached to the Veeam Proxy, data is retrieved directly from the datastore.

Direct Storage Access

This mode reads and writes data directly from and to the storage system where VM data or backups are located.

Direct SAN

If you are running your virtual machines hosted on VMFS SAN LUNs. Then this is the right transport mode for you, apart from if you have storage integration and even then, this is underlining the same process but it will leverage storage snapshots rather than the live LUN from the storage array.

Rather than step 7 in this instance transferring to a backup repository this will transfer to another proxy on a remote site and into the secondary remote datastore as a replica VM.

Direct NFS

If you are running virtual machines hosted on an NFS export. This mode allows for the proxy to bypass the vSphere environment and reads directly from the NFS datastore or NFS storage array. VM data still travels over LAN but there is no load on the ESXi host.

Storage Integration

If this can be achieved as your transport mode then this is the route to take. The ability to leverage the same benefits of Direct Storage access but now not from the live volume or LUN but to a storage snapshot, which is going to benefit from the fast removal of VMware snapshot as well as no contention when pulling data from the production LUN or volume. The snapshot will be presented to the Veeam Proxy and data retrieved.

More on Storage integration will come later in the series.

I hope this has given you a good understanding of what each transport mode represents and also some use cases on how they can be used in certain environments.

More detail on all things Veeam and Replication can be found here – User Guide for VMware vSphere

The Workflow & Components

In the last post, I covered a 101 of replication in general and mentioned where Veeam has a play in A-Synchronous and will have Near Synchronous Replication in v10 in the form of CDP.

In this post, I wanted to walk through the steps taken when Veeam is used for A-Synchronous replication in your virtualised estate.

First, I want to touch on the components that are required for any sort of Veeam replication, these can double up for the backup elements also.

Veeam Backup Server

• The “configuration and control center”
• Coordinates tasks, controls job scheduling and resource allocation
• Set up and manage backup infrastructure components
• Windows-based physical or virtual machine on which Veeam Backup & Replication is installed

Veeam Proxy Server

• Used to process jobs and deliver backup traffic
• Takes data processing off the Veeam Backup server
• Controls Deduplication and compression tasks on the data
• A dedicated Windows server (physical or virtual)
• You can deploy backup proxies both in the primary site and remote sites

Veeam Replication Architecture

The Snapshot

During the first run of a replication job, Veeam Backup & Replication copies the VM running on the source host and creates its full replica on the target host. The replica is stored uncompressed, in a native hypervisor format.

All subsequent replication jobs are incremental. Veeam Backup & Replication copies only those data blocks that have changed since the last replication cycle.

These image level replicas are stored as restore points in the target destination as snapshots, there are some small differences between VMware and Microsoft Hyper-V, I will differentiate those below.

VMware

If you are leveraging Veeam replication on VMware then you will still something like this in your secondary location.

VMware has a supported limit of 32 snapshots, minus the working snapshots that Veeam requires and a bit of overhead, this would allow for a 28 snapshots/restore points limitation.

Hyper-V

For those using Microsoft Hyper-V you will see something like this on the target destination.

Microsoft Hyper-V has a supported limit of 50 snapshots, minus the working snapshots Veeam requires and a bit of overhead, this would allow for 47 snapshots/restore points.

The Process

VM data is moved block by block, with multiple processing cycles. The replication process is performed in the following way.

1. The Source Data Mover speaks with the Target Data Mover to start any data collection.
2. The Source Data Mover then checks the VM and copies the data using the correct Veeam transport mode. This is related to how the Proxy has been configured to access the virtual infrastructure. During scheduled incremental job runs the Veeam data mover service will only take the changed blocks since the last replication job. As well as copying the data the source data mover will perform some additional tasks, it will consolidate and filter out zero data block and swaps files from the virtual disks.
3. Those changed blocks are compressed and transferred from the source data mover to the target data mover.
4. The target data move decompresses the data and writes the result to the destination datastore.

How do we get that first bit of data over?

When planning for offsite replication, we should consider available features to reduce the amount of replication traffic and streamline replica configuration.

Seeding

Replica Seeding allows us to send over the bulk of data via physical methods, this could be by placing the copy of data onto moveable hardware and shipping that to the target destination. USB Drive, Storage Array etc. Don’t underestimate the bandwidth of a man in a van. It will save a lot of time and bandwidth over those expensive links.

WAN Accelerator

Another Veeam component, software only and it’s there to optimize VM data transfer, this is achieved by Global data caching and deduplication. The configuration for the WAN Accelerator is a One to many configuration, or can be used in pairs if you have an active pair of sites, A single WAN accelerator in the primary location, multiple WAN accelerators at branches or targets destinations.

There are some great best practices noted in the Veeam Best Practice guide around WAN Acceleration.

I will be sure to cover off in more detail some of these components and highlight some best practices, sizing and deployment facts from the field.