Replica – vZilla https://vzilla.co.uk One Step into Kubernetes and Cloud Native at a time, not forgetting the world before Thu, 11 Jan 2018 20:12:48 +0000 en-GB hourly 1 https://wordpress.org/?v=6.8.1 https://vzilla.co.uk/wp-content/uploads/2018/01/cropped-profile_picture_symbol-32x32.png Replica – vZilla https://vzilla.co.uk 32 32 Veeam Replication – Advanced Features and Functionality https://vzilla.co.uk/vzilla-blog/veeam-replication-advanced-features-functionality https://vzilla.co.uk/vzilla-blog/veeam-replication-advanced-features-functionality#comments Fri, 12 Jan 2018 07:48:51 +0000 https://vzilla.co.uk/?p=757 veeam replication series data protection

Veeam Replication – Advanced features and functionality

Welcome back, as we move through this series where we have touched on the basic replication configuration, or I should say the easy to use deployment, the walkthrough and some more advance configurations or ways in which to achieve the same outcome from PowerShell commands.

Veeam Replication Series

1VeeamReplication 101  2VeeamReplication workflowcomponents e1515696389784  3VeeamReplication transportmodes  

4VeeamReplication walkthrough 1  5VeeamReplication PowerShell  6VeeamReplication advft

7VeeamReplication wan  8VeeamReplication failover  9VeeamReplication surereplica

10VeeamReplication sandbox  12VeeamReplication cdp  11VeeamReplication storage

This post will hopefully share some of the more advanced features, options or even some of the outcomes you might not be able to find any detail on.

Remote replica from backup

The clue is in the title, this gives the ability to create a replica VM from a backup that has already traversed the WAN or already resides on that secondary location. This method has some benefits in that it will not have to touch the production estate and you obviously do not need to send the data twice if you have a backup copy configuration to the same site. More information can be found here –

https://helpcenter.veeam.com/docs/backup/vsphere/replica_from_backup.html?ver=95

011018 0746 VeeamReplic1

Low Connection Bandwidth

Another option when creating your replication job was “Low Connection Bandwidth” Introducing Replica Seeding, this is the ability to use the man in the van bandwidth. Poor links between sites or large datasets then if possible take advantage of seeding. This could be used in some cases for getting replicas to our cloud connect providers.

Basically, if you have a backup for the replicated VM on the backup repository located in the DR site. You can point the replication copy job to this backup. The first run of the replication job will seed the data from the backup file, i.e. it will create a VM replica image, once complete it will resync with the production VM for any changes since the backup file. Notice how it only does that seed once where as remote replica continues to use that same method.

Giving a shout out to Luca here on his blog a while back he wrote about seeding to a service provider (Cloud Connect – https://www.virtualtothecore.com/en/seeding-veeam-cloud-connect-part-3-replication-jobs/)

Separate Virtual Networks

Not really an advanced setting as such but something I should cover here, if your target environment does not use the same networking, this setting allows you to map the production (source network) and the secondary (target network)

011018 0746 VeeamReplic2

Calculating Digests

For those of you already running Veeam replication you may have seen this, let’s cover what this process is and what do we expect to see from a calculation point of view. This task is checking the difference between the source VM and the target VM (Replica VM)

A good rule of thumb here is 1GB per minute, just bare that in mind when you are calculating how long your replication jobs are going to take.

Why does this happen? Well you are probably going to know why this is happening but just to run through some of them.

Most commonly it’s if you change something on that source VM, add a disk for example probably the most common update or change here and there is very little you can do to avoid this then wanting to calculate the digests during the job.

You will also find this message in your job log if you reinstall vCentre or something happens to your vCentre and something changes your VM IDs. Datastore disconnections could cause it, oh and my favourite is if you remove the source VM from the inventory and then re add it back in. it’s going to get a new VM ID and that’s also going to prompt for this process.

There is very little we can do with the above scenarios but sit it out, although if you find that something is outside of those timeframes then a call to support might be worth the clarification of the setup.

This job running at the moment from @HammondRhys shows what can happen if you run out of space on the repository which is used for that replication metadata another use case or fail case for the calculating digests.

011018 0746 VeeamReplic3

Different IP addressing Scheme

More common that not these days is the requirement for different IP addressing in a secondary location. This exact setting or configuration is available as part of the Veeam replication wizard.

By setting this different IP addressing scheme when a failover plan is triggered it will apply these different addresses to the replica VM.

A big note here is that this is only applicable to Windows virtual machines.

011018 0746 VeeamReplic4

Storage Integration Settings

I touched on this very briefly in the transport modes post but wanted to touch again on an example of when this setting needs to be considered.

When configuring such large jobs, it is advised to configure the maximum number of VMs within one storage snapshot. The setting is available in the advanced job settings within the Integration tab.

Example: When creating a job with 100 VMs and setting the limit to 10, the Backup from Storage Snapshot job will instruct the job manager to process the first 10 VMs, issue the storage snapshot, and proceed with the backup. When that step has successfully completed for the first 10 VMs, the job will repeat the above for the following 10 VMs in the job.

011018 0746 VeeamReplic5

Replica Metadata

Finally, and I know this post is now getting on the long side but given some recent conversations I wanted to add at least something in here that may help when it comes to metadata location and sizing.

The latest information I can find is from one of our product managers. Outlining a good rule when it comes to sizing for metadata space.

“Typical metadata size can be calculated as 128MB for each 1TB of VM data (per each replica restore point).”

Next post we will touch on the WAN Accelerator options and benefits.

]]>
https://vzilla.co.uk/vzilla-blog/veeam-replication-advanced-features-functionality/feed 1
Veeam Replication – PowerShell https://vzilla.co.uk/vzilla-blog/veeam-replication-powershell https://vzilla.co.uk/vzilla-blog/veeam-replication-powershell#respond Thu, 11 Jan 2018 08:30:46 +0000 https://vzilla.co.uk/?p=743 veeam replication series data protection

Veeam Replication – PowerShell

There is a large adoption of people using PowerShell to make their lives easier and reduce time spent on performing lengthy repeatable tasks. Veeam Replication could be one of those where in the last post I showed the simple steps to setup your replication jobs. If you have many different groups or even sites, then to do that a number of times is going to be a pain.

Veeam Replication Series

1VeeamReplication 101  2VeeamReplication workflowcomponents e1515696389784  3VeeamReplication transportmodes  

4VeeamReplication walkthrough 1  5VeeamReplication PowerShell  6VeeamReplication advft

7VeeamReplication wan  8VeeamReplication failover  9VeeamReplication surereplica

10VeeamReplication sandbox  12VeeamReplication cdp  11VeeamReplication storage

What I want to show you in this post is how easy it is to use the Veeam PowerShell Snap-In and the full set of capabilities to run and manage those replication tasks.

Before I begin there is a more in-depth resource on all different PowerShell commands that can be found here – https://helpcenter.veeam.com/docs/backup/powershell/replication.html?ver=95

For the purpose of this post I am going to be using VMware as my platform the command structure for Hyper-V is slightly different and the above link can also help there to determine the required steps.

Add-VBRViReplicaJob

Resource – https://helpcenter.veeam.com/docs/backup/powershell/add-vbrvireplicajob.html?ver=95

There are many parameters available for this cmdlet these can be found in more detail at the above link. I am going to be using the same parameters that we used during the walkthrough in the last post.

First, we need to add the snap in and connect to our Veeam Backup & Replication Server this can be done with the following. All code will be available in the end script.

#Add the Veeam PSSnapIn to get access to the Veeam Backup & Replication cmdlets library. Run the following command:
Add-PSSnapin VeeamPSSnapin
#Connects to Veeam backup server.
Connect-VBRServer -server "10.0.40.10"

010618 2121 VeeamReplic1 

This command will set the server you want the replica to be sent to, it will also save it to a variable of $Server.

#Set Destination ESXi Server 
$server = Get-VBRServer -Name "tpm03-131.aperaturelabs.biz"

Next, we need to determine which VMs or objects we would like to replicate we can do this with the following command. And this will set the VM to a variable.

#Define Source Virtual Machine(s)
$vm = Find-VBRViEntity -Server vc03.aperaturelabs.biz -Name "TPM04-DC-01"

Now we need to determine the resource pool that we would want the replica to be located in and set that as a variable.

#Define destination resource pool 
$pool = Find-VBRViResourcePool -Server $server -Name "TPM04-MC"

The final task of finding and setting variables comes in the form of which datastore do we want our replica to be stored in.

#Define destination datastore
$datastore = Find-VBRViDatastore -Server "tpm03-131.aperaturelabs.biz" -Name "SolidFire005_iSCSI"

I will share the commands that I used to find out the below information at the end of the post. Now that we have set our variables and we know the:

  • What we are replication
  • Where are we replicating

010618 2121 VeeamReplic2

Now that we have set the variables for our script it might be worth just checking that they all return the right objects this can be done with the following code.

#Now that you have defined your variables I would suggest running the following to ensure all have been populated.
$server
$vm
$pool
$datastore

010618 2121 VeeamReplic3

Creating the Replication Job

We can now create the replication job using the above information.

#This command will create the backup job with no schedule defined.
Add-VBRViReplicaJob -Name "PS Replication Job" -Server $server -Entity $vm -ResourcePool $pool -Datastore $datastore -Suffix "_replicated"

010618 2121 VeeamReplic4

Now that we have the job created you can also see the job created in the Veeam Backup & Replication console.

010618 2121 VeeamReplic5

At this point or later we also have the option to add additional VMs to the replication job. The first one will add the additional VMs to the variable. The second is how you would update the job.

#If you then wanted to add additional VMs to the job then you could do by adding to this line of code 
$vm = Find-VBRViEntity -Server vc03.aperaturelabs.biz -Name "TPM04-DC-01", "TPM04-SQL-02" 

010618 2121 VeeamReplic6

#This command will then update the job with the newly added Virtual Machines. 
Set-VBRViReplicaJob -Job $job -Server $server -Entity $vm

See below the console view now with the newly added extra VM in the job,

010618 2121 VeeamReplic7

010618 2121 VeeamReplic8

If we had a large number of Virtual Machines to add to the job maybe with a similar naming convention such as the one I have used here, then we also have the ability to add against a wildcard to the job.

#This command will also allow you to add wildcards to your replication job Note that this will add all objects with this name. 
Find-VBRViEntity -Name TPM04-* | Add-VBRViJobObject -Job $job 

010618 2121 VeeamReplic9

We can then use the following command to give us the configuration for the job. As well as setting a variable for the job.

#This command will show the job configuration 
Get-VBRJob -name "PS Replication Job" 

010618 2121 VeeamReplic10

Setting the schedule

For those that have noticed we have the job created but no schedule so next up is creating a schedule for the requirements. The below command will set the schedule on the job.

#This command schedules the job represented by the $job variable to run every 1 hour
Set-VBRJobSchedule -Job $job -Periodicaly -FullPeriod 1 -PeriodicallyKind Hours

010618 2121 VeeamReplic11

As well as setting the schedule we need to enable the schedule on the job this can be done like this.

#This will enable the job schedule
Enable-VBRJobSchedule -Job $job 

010618 2121 VeeamReplic12

Navigating back into the job in the console then you will now see the schedule is now configured.

010618 2121 VeeamReplic13

Starting the Job

We have come this far so let’s continue and start our job from the PowerShell script as well.

#The following command will allow us to start the job 
Start-VBRJob -Job $job

010618 2121 VeeamReplic14

within the console

010618 2121 VeeamReplic15

As the job progresses we will see on the destination ESXi host our replicated virtual machine.

010618 2121 VeeamReplic16

When the job completes we get the following summary back in our PowerShell window.

010618 2121 VeeamReplic17

Obviously, we can get that same information and some more from the console.

010618 2121 VeeamReplic18

The complete script

#Add the Veeam PSSnapIn to get access to the Veeam Backup & Replication cmdlets library. Run the following command:
Add-PSSnapin VeeamPSSnapin
#Connects to Veeam backup server.
Connect-VBRServer -server "10.0.40.10"
#Returns hosts connected to Veeam Backup & Replication.
Get-VBRServer 
#This will show all objects 
Find-VBRViEntity
#This will show all resource pools available on the defined server either use variable or add in your ESXi host name/ip
Find-VBRViResourcePool -Server $server
#This will show available datastores on host 
Find-VBRViDatastore -Server "tpm03-131.aperaturelabs.biz"
#This will show availabel VMs 
Find-VBRViEntity -Server vc03.aperaturelabs.biz -name "TPM04-*"

#Set Destination ESXi Server 
$server = Get-VBRServer -Name "tpm03-131.aperaturelabs.biz"
#Define Source Virtual Machine(s)
$vm = Find-VBRViEntity -Server vc03.aperaturelabs.biz -Name "TPM04-DC-01"
#Define destination resource pool 
$pool = Find-VBRViResourcePool -Server $server -Name "TPM04-MC"
#Define destination datastore
$datastore = Find-VBRViDatastore -Server "tpm03-131.aperaturelabs.biz" -Name "SolidFire005_iSCSI"
 

#Now that you have defined your variables I would suggest running the following to ensure all have been populated.
$server
$vm
$pool
$datastore

#This command will create the backup job with no schedule defined.
Add-VBRViReplicaJob -Name "PS Replication Job" -Server $server -Entity $vm -ResourcePool $pool -Datastore $datastore -Suffix "_replicated"

#This command will show the job configuration 
Get-VBRJob -name "PS Replication Job"

#If you then wanted to add additional VMs to the job then you could do by adding to this line of code 
$vm = Find-VBRViEntity -Server vc03.aperaturelabs.biz -Name "TPM04-DC-01", "TPM04-SQL-02"

#Setting a variable to the newly created replication job
$job = Get-VBRJob -name "PS Replication Job"

#Confirming that the variable has populated
$job

#This command will then update the job with the newly added Virtual Machines. 
Set-VBRViReplicaJob -Job $job -Server $server -Entity $vm
#This command will also allow you to add wildcards to your replication job Note that this will add all objects with this name. 
Find-VBRViEntity -Name TPM04-* | Add-VBRViJobObject -Job $job

#This command schedules the job represented by the $job variable to run every 1 hour
Set-VBRJobSchedule -Job $job -Periodicaly -FullPeriod 1 -PeriodicallyKind Hours

#This will enable the job schedule
Enable-VBRJobSchedule -Job $job

#The following command will allow us to start the job 
Start-VBRJob -Job $job

 


Reference – https://helpcenter.veeam.com/docs/backup/powershell/create_replica_vmware.html?ver=95


]]>
https://vzilla.co.uk/vzilla-blog/veeam-replication-powershell/feed 0
Veeam Replication – Walkthrough https://vzilla.co.uk/vzilla-blog/veeam-replication-walkthrough https://vzilla.co.uk/vzilla-blog/veeam-replication-walkthrough#respond Wed, 10 Jan 2018 08:02:46 +0000 https://vzilla.co.uk/?p=689 veeam replication series data protection

Veeam Replication – Walkthrough

Now we have set the scene with our previous posts I wanted to share a complete walkthrough on Veeam Replication and show the simple steps it takes to send your virtual machines from Site A to Site B.

Veeam Replication Series

1VeeamReplication 101  2VeeamReplication workflowcomponents e1515696389784  3VeeamReplication transportmodes  

4VeeamReplication walkthrough 1  5VeeamReplication PowerShell  6VeeamReplication advft

7VeeamReplication wan  8VeeamReplication failover  9VeeamReplication surereplica

10VeeamReplication sandbox  12VeeamReplication cdp  11VeeamReplication storage

This post is aimed for those environments that have a Veeam Backup & Replication server installed and configured. It will also expect that you have deployed an additional Veeam Proxy on that secondary site.

When I mention secondary site, this could be considered either onsite or offsite. If it is the same onsite location then the same source proxy can be classed and used as the target proxy for data movement, however it must have access to the source and target environments.

It is also required that the backup management server can connect to both source and target hosts.

Once you have that deployed its then time to create your replication job. The license file I have is only for VMware but Microsoft Hyper-V is also an option.

Create Replication Job

010318 2201 VeeamReplic1

Job Name and Advanced Configuration

A wizard will then open and you can begin to configure how the replication job should look. On the first screen as well as the Job name and description there are some advanced features that we will cover in later posts.

010318 2201 VeeamReplic2

Add Objects

The following screen is probably the most important, this is where you will select the VMs you want to replicate, this can be done by either selecting the individual VM or you can use different construct options in the top right-hand corner, these could be based on cluster or host, datastore or the ability to use vSphere tags or Microsoft SCVMM tags.

010318 2201 VeeamReplic3010318 2201 VeeamReplic4010318 2201 VeeamReplic5010318 2201 VeeamReplic6

Another useful tip here is if you have a large environment and you want to pin point certain named VMs then you have the ability to search and filter at the bottom of the screen.

010318 2201 VeeamReplic7

Where are we going to get that data from? We have two options, we can choose to actually take directly from the production storage which is going to take the most recent state of the VM. Or we can choose to take from a backup location. We may cover this second option off in a later post.

On this same screen, we also have the ability to exclude certain objects from the replication job, this could be certain attached disks that are not required in a failure scenario. But for the sake of this walkthrough I am going to keep things very simple. It is worth noting that Veeam will already exclude VM log files and templates to reduce the size of the replicated data.

010318 2201 VeeamReplic8

You may have noticed some additional buttons here on this data source screen. The Up and Down function. This allows us to set an order. Mission critical VMs should be at the top because we want to make sure we process them first. (only relevant if you have added individual VMs)

010318 2201 VeeamReplic9

Destination Options

Now we must choose where we want our VMs to be replicated to, we choose our destination cluster or host, we can choose a resource pool or VM folder if we want. Finally, we choose the datastore we would like this replicated VM to be stored in.

010318 2201 VeeamReplic10

Job Settings

This first screen requires a backup repository that has been configured within the management server where the source side metadata can be stored, a replica suffix also should be specified this is how the VM will look within the secondary location i.e VMNAME_replica in this instance, and then restore points. How many restore points would you like or need on that secondary location.

010318 2201 VeeamReplic11

Under the advanced tab we have some additional settings that you may choose to configure depending on your environment and infrastructure.

Traffic settings – By default this is what the compression and deduplication configuration will look like, to understand more about these options then you can find more information here – https://helpcenter.veeam.com/docs/backup/vsphere/replica_advanced_traffic_vm.html?ver=95

010318 2201 VeeamReplic12

Notification settings – This screen allows us to set specific notifications for this specific job either via email or via SNMP or both. https://helpcenter.veeam.com/docs/backup/vsphere/replica_advanced_notify_vm.html?ver=95

vSphere settings – By default it is set that CBT will be enabled but this can be changed if required. You also have the option to enable VMware tools quiescence. https://helpcenter.veeam.com/docs/backup/vsphere/replica_advanced_vsphere_vm.html?ver=95

Integration settings – If adding over 100 VMs to a backup or replication job. This could cause a very high VM snapshot lifetime for the first VMs in the job list.

010318 2201 VeeamReplic13

Script settings – This allows you to run pre-or post-scripts before or after the job if you would like to run specific scripts within a VM then you have that option later on in the process under the guest processing wizard.

Data Transfer

Now to choose how the VM data is going to be transferred to the secondary site. The appropriate proxies at this stage should be selected and you should also determine if WAN Acceleration is used or not this is something we will again cover later on.

010318 2201 VeeamReplic14010318 2201 VeeamReplic15

Guest Processing

Depending on the workload it may require your VMs are application consistent. Options can be configued here by providing the appropriate requirements.

010318 2201 VeeamReplic16

In this instance, I have a backup job dealing with truncation of logs. I have configured the job to copy only the logs to the secondary site. There is also the ability to exclude OS files from the VM and to run specific scripts from within the VM.

010318 2201 VeeamReplic17

Scheduling

Lastly in the configuration wizard is when we want to schedule this job to run.

010318 2201 VeeamReplic18

Summary

Summary of the configuration and then we are good to go and run this job against that schedule we just set.

010318 2201 VeeamReplic19

Next up we will look at this same process through PowerShell.

]]>
https://vzilla.co.uk/vzilla-blog/veeam-replication-walkthrough/feed 0
Veeam Replication – Transport Modes https://vzilla.co.uk/vzilla-blog/veeam-replication-transport-modes https://vzilla.co.uk/vzilla-blog/veeam-replication-transport-modes#respond Tue, 09 Jan 2018 08:32:30 +0000 https://vzilla.co.uk/?p=650 veeam replication series data protection

Veeam Replication – Transport Modes

I decided to give this topic its own post. This is probably the most important consideration when it comes to replication and how best to utilise the environment. If you are using Veeam you will most likely know that there are multiple options when it comes to how Veeam will take the backup and send to the repository. This is the same with the replication engine.

Veeam Replication Series

1VeeamReplication 101  2VeeamReplication workflowcomponents e1515696389784  3VeeamReplication transportmodes  

4VeeamReplication walkthrough 1  5VeeamReplication PowerShell  6VeeamReplication advft

7VeeamReplication wan  8VeeamReplication failover  9VeeamReplication surereplica

10VeeamReplication sandbox  12VeeamReplication cdp  11VeeamReplication storage

In this post, I want to outline all available transport modes and how possibly each one will be best in the environment.

This post will only reflect VMware estates and not that of Hyper-V.

VMware

For all transport modes, Veeam leverages the VMware vStorage APIs for Data Protection (VADP)

There is no “Best” transport modes as such but being able to leverage Storage integration is going to give you the benefits of not having to hold a VMware snapshot open for so long. But this and all modes depend on the environment.

Each transport mode depends really on the type of datastore being used at source. Also, the backup proxy depending if that is physical or virtual.

Picture1 4

Before we continue I want to highlight one more thing on this diagram.

Associated to the transport modes are the following steps:

  1. The source Veeam Data Mover establishes a connection with the target Veeam Data Mover, and Veeam Data Mover.
  2. The source Veeam Data Mover reads the VM data from the read-only VM disk and copies it. During incremental job sessions, the source Veeam Data Mover uses CBT to retrieve only those data blocks that have changed since the previous job session. If CBT is not available, the source Veeam Data Mover interacts with the Veeam Data Mover on the backup repository to obtain replica metadata, and uses this metadata to detect blocks that have changed since the previous job session.
  3. The target Veeam Data Mover decompresses VM data and writes the result to the destination datastore.

These steps and diagram were taken from the Veeam user guide – https://helpcenter.veeam.com/docs/backup/vsphere/replication_chain.html?ver=95

Picture2 4

Network Mode

The beauty of network mode is that it can be used in any infrastructure regardless of storage presented. All the data is going to be retrieved via the vSphere environment over the LAN using the Network Block Device Protocol (NBD) or NBDSSL in vSphere 6.5 or above.

The impact on of this mode is that of low data transfers over the LAN this is why we would recommend where possible to move down this list to find a more performant and efficient transport mode.

Picture3 4

Virtual Appliance (Hot-Add)

This mode is not as efficient as the following Direct Storage modes but can be more efficient than Network mode that we just discussed. Ideal if you are looking to achieve a 100% virtualised footprint in your data centre.

The appliance (A windows VM) is deployed within the environment. It uses the VMware SCSI HotAdd capability that allows attaching devices to a VM while the VM is running. During replication, the disks of the required VM are attached to the Veeam Proxy, data is retrieved directly from the datastore.

Picture4 2

Direct Storage Access

This mode reads and writes data directly from and to the storage system where VM data or backups are located.

Direct SAN

If you are running your virtual machines hosted on VMFS SAN LUNs. Then this is the right transport mode for you, apart from if you have storage integration and even then, this is underlining the same process but it will leverage storage snapshots rather than the live LUN from the storage array.

Rather than step 7 in this instance transferring to a backup repository this will transfer to another proxy on a remote site and into the secondary remote datastore as a replica VM.

Picture5 2

 

Direct NFS

If you are running virtual machines hosted on an NFS export. This mode allows for the proxy to bypass the vSphere environment and reads directly from the NFS datastore or NFS storage array. VM data still travels over LAN but there is no load on the ESXi host.

Picture6 1

Storage Integration

If this can be achieved as your transport mode then this is the route to take. The ability to leverage the same benefits of Direct Storage access but now not from the live volume or LUN but to a storage snapshot, which is going to benefit from the fast removal of VMware snapshot as well as no contention when pulling data from the production LUN or volume. The snapshot will be presented to the Veeam Proxy and data retrieved.

Picture7

More on Storage integration will come later in the series.

I hope this has given you a good understanding of what each transport mode represents and also some use cases on how they can be used in certain environments.

More detail on all things Veeam and Replication can be found here – User Guide for VMware vSphere

 

 

]]>
https://vzilla.co.uk/vzilla-blog/veeam-replication-transport-modes/feed 0
Veeam Replication – The Workflow & Components https://vzilla.co.uk/vzilla-blog/veeam-replication-workflow-components https://vzilla.co.uk/vzilla-blog/veeam-replication-workflow-components#respond Mon, 08 Jan 2018 08:05:53 +0000 https://vzilla.co.uk/?p=642 veeam replication series data protection

The Workflow & Components

In the last post, I covered a 101 of replication in general and mentioned where Veeam has a play in A-Synchronous and will have Near Synchronous Replication in v10 in the form of CDP.

Veeam Replication Series

1VeeamReplication 101  2VeeamReplication workflowcomponents e1515696389784  3VeeamReplication transportmodes  

4VeeamReplication walkthrough 1  5VeeamReplication PowerShell  6VeeamReplication advft

7VeeamReplication wan  8VeeamReplication failover  9VeeamReplication surereplica

10VeeamReplication sandbox  12VeeamReplication cdp  11VeeamReplication storage

In this post, I wanted to walk through the steps taken when Veeam is used for A-Synchronous replication in your virtualised estate.

First, I want to touch on the components that are required for any sort of Veeam replication, these can double up for the backup elements also.

Veeam Backup Server

  • The “configuration and control center”
  • Coordinates tasks, controls job scheduling and resource allocation
  • Set up and manage backup infrastructure components
  • Windows-based physical or virtual machine on which Veeam Backup & Replication is installed

Veeam Proxy Server

  • Used to process jobs and deliver backup traffic
  • Takes data processing off the Veeam Backup server
  • Controls Deduplication and compression tasks on the data
  • A dedicated Windows server (physical or virtual)
  • You can deploy backup proxies both in the primary site and remote sites

Veeam Replication Architecture

Picture1 3

The Snapshot

During the first run of a replication job, Veeam Backup & Replication copies the VM running on the source host and creates its full replica on the target host. The replica is stored uncompressed, in a native hypervisor format.

All subsequent replication jobs are incremental. Veeam Backup & Replication copies only those data blocks that have changed since the last replication cycle.

These image level replicas are stored as restore points in the target destination as snapshots, there are some small differences between VMware and Microsoft Hyper-V, I will differentiate those below.

VMware

If you are leveraging Veeam replication on VMware then you will still something like this in your secondary location.

Picture2 3

VMware has a supported limit of 32 snapshots, minus the working snapshots that Veeam requires and a bit of overhead, this would allow for a 28 snapshots/restore points limitation.

Hyper-V

For those using Microsoft Hyper-V you will see something like this on the target destination.

Picture3 3

Microsoft Hyper-V has a supported limit of 50 snapshots, minus the working snapshots Veeam requires and a bit of overhead, this would allow for 47 snapshots/restore points.

The Process

VM data is moved block by block, with multiple processing cycles. The replication process is performed in the following way.

Picture4 1

  1. The Source Data Mover speaks with the Target Data Mover to start any data collection.
  2. The Source Data Mover then checks the VM and copies the data using the correct Veeam transport mode. This is related to how the Proxy has been configured to access the virtual infrastructure. During scheduled incremental job runs the Veeam data mover service will only take the changed blocks since the last replication job. As well as copying the data the source data mover will perform some additional tasks, it will consolidate and filter out zero data block and swaps files from the virtual disks.
  3. Those changed blocks are compressed and transferred from the source data mover to the target data mover.
  4. The target data move decompresses the data and writes the result to the destination datastore.

How do we get that first bit of data over?

When planning for offsite replication, we should consider available features to reduce the amount of replication traffic and streamline replica configuration.

Seeding

Replica Seeding allows us to send over the bulk of data via physical methods, this could be by placing the copy of data onto moveable hardware and shipping that to the target destination. USB Drive, Storage Array etc. Don’t underestimate the bandwidth of a man in a van. It will save a lot of time and bandwidth over those expensive links.

Picture5 1

WAN Accelerator

Another Veeam component, software only and it’s there to optimize VM data transfer, this is achieved by Global data caching and deduplication. The configuration for the WAN Accelerator is a One to many configuration, or can be used in pairs if you have an active pair of sites, A single WAN accelerator in the primary location, multiple WAN accelerators at branches or targets destinations.

There are some great best practices noted in the Veeam Best Practice guide around WAN Acceleration.

Picture6

I will be sure to cover off in more detail some of these components and highlight some best practices, sizing and deployment facts from the field.

 

]]>
https://vzilla.co.uk/vzilla-blog/veeam-replication-workflow-components/feed 0
Veeam Replication 101 https://vzilla.co.uk/vzilla-blog/veeam-replication-101 https://vzilla.co.uk/vzilla-blog/veeam-replication-101#respond Fri, 05 Jan 2018 08:57:59 +0000 https://vzilla.co.uk/?p=637 veeam replication series data protection

Replication 101

If I had access to some big data at Veeam I am confident that the numbers would state that most people that have bought and use Veeam Backup & Replication for the Backup and maybe secondly replication function, or maybe they don’t use the replication function at all.

I want to give some airtime to that replication side of the house, I would have thought it’s got a bit more attention over the last few months because of the CDP (Continuous Data Protection) feature that will be coming in a future version of Veeam Backup & Replication extending those replication capabilities to achieve a second to minute Recovery Point Objective (RPO) whilst still having the ability with the already existing replication engine to provide application consistent image level replicas at your secondary data centre or your service provider data centre.

On the 5th December was the virtual VeeamON event where I spoke very briefly about how the new CDP feature looks and feels but for this post I want to go back to what we already have available to us, and if this sways the decision for someone to pick it up and start achieving more within their business, then it’s worth the time.

Veeam Replication Series

1VeeamReplication 101  2VeeamReplication workflowcomponents e1515696389784  3VeeamReplication transportmodes  

4VeeamReplication walkthrough 1  5VeeamReplication PowerShell  6VeeamReplication advft

7VeeamReplication wan  8VeeamReplication failover  9VeeamReplication surereplica

10VeeamReplication sandbox  12VeeamReplication cdp  11VeeamReplication storage

Type of Replication

The first thing we need to understand is what is replication, if you are here I guess you already know, but replication is about moving data from one site or infrastructure to another. To go just a little deeper on that it’s about getting the same data blocks over to a different location so that if something was to happen at the first site then the copied or replicated blocks could then take over.

Next up is understanding what type of data replication is required by your business and possibly what is achievable for the infrastructure you have. Let get into the different types available.

Synchronous Replication

“we cannot afford any loss of data if we were to invoke a disaster recovery plan” Ok so if this what you have been told as a consultant as an administrator then this is the first and most powerful type of replication. Synchronous Replication, Synchronous replication ensures all data written in the source storage is also at the same time written in the target storage, and waits for acknowledgement from both storage arrays before completing the operation. A possible downside to this would be this requires matching storage at both the source and target destinations with fibre channel latencies to reduce the performance overhead of the link between the storage arrays. This option is always going to be at the storage layer due to the performance impact potential, you won’t find any virtual appliance offering this, if there is please let me know but I expect they will be offering near-synchronous replication which I will touch on later.

Picture1 2

Asynchronous Replication

“I need a copy of our systems in a secondary location, if I can achieve an RPO of minutes rather than hours that would be great but I have not got the money to buy all this cross site network links nor two expensive storage arrays”

This is where I have had most my conversations regarding replication, it’s the cheapest and you can still achieve a pretty good recovery point objective. Asynchronous replication does not write data to both source and target destinations at the same time. It will leverage snapshots (this could be virtualisation snapshots as well as storage snapshots) this snapshot acts as a point in time copy of the data, this data is then sent to the target destination based on a schedule. Now normally this would be based on hours or days from a scheduling point of view but this will depend on the environment, snapshots especially virtualisation (VMware) can impact performance if they are constantly being taken throughout the day and it depends on the source storage that’s got to be able to withstand this additional operation.

This type of replication is what Veeam uses today to transfer virtual machines from source to target allowing for minutes / hours’ recovery point objectives.

Picture2 2

Near-Synchronous Replication

“I have a handful of VMs that I would state are my platinum tier of workloads, A-Synchronous Replication will work for anything under platinum but if a disaster was to strike I need to ensure that these mission critical applications are up on that secondary site”

We now enter the world in between Synchronous and A-Synchronous and that’s clearly going to be called Near-Synchronous Replication. This is generally going to be a policy based engine and most likely software based rather than the hardware based mentions of synchronous replication. A policy is going to be always running unlike a job that will start and stop on a schedule. The policy will be constantly replicating the changed data blocks to the target destination. This will allow for a second to minute RPO. The key benefit here is that it doesn’t use snapshots. It also doesn’t need to wait for an acknowledgement from the target destination when sending the changes over.

The caveat here is that you will need some level hardware to achieve this, most importantly this would be the networking links between source and target. This is the category that the Veeam CDP feature would fall into and I will touch on later in this series on how this looks from an architecture point of view.

Picture3 2

Hopefully that’s given you a good overview of replication and the options available. I will be back soon with some more detail on the Veeam replication workflow, also how Veeam can leverage some of their alliances storage integrations to maybe get the best of both worlds.

]]>
https://vzilla.co.uk/vzilla-blog/veeam-replication-101/feed 0