Automating your Veeam lab

Building out a new lab or refreshing your lab is tedious and a job nobody really wants to do, I have long been a fan of infrastructure as code and automation and we have had various projects where we have done similar things and highlighted this in some of our community speaking sessions and hopefully helped people.

Now I am also not saying this is the right way but in my lab situation I wanted something I could easily automate the spinning up and down of my Veeam environment for testing, this really came about with the latest release of Veeam Backup & Replication v11 and with beta releases, rc releases and then rtm and GA.

My challenge is the share on our lab domain is shared and can only be authenticated by our very secure domain accounts. Well this is one of the challenges.

Desired Lab Day 0

A lot of my role is either showing off features and functionality in various settings, testing scenarios to help our customers or for learning to help better aid the first two parts of the role. Oh, and I must create content on the features and functionality too.

The purpose of the lab or my little corner of the lab is to enable me to do that, in the lab we have our vSphere environment, our shared storage solution providing our virtual machine storage and we also have a management cluster providing all of the joyous Active Directory, DNS and lot’s of other things that get maintained like they were our pets.

My lab on the other hand is not a pet friendly zone, it is more like a train station pre 2020 where virtual machines come and go on a daily sometimes hourly basis. Ok maybe I do have some specific test subjects that provide with source workloads for backup tasks, things like web servers or database servers but they are all deployed using terraform and there is nothing of value on them.

What I need from my lab though is the ability to spin up and down a set number of instances that make up the Veeam environment I am needing for a specific task. These machines where need be can be easily scaled up using the code later in this post. They are also all virtual machines which in a real-life scenario this is not going to be following best practices, you do not want to have your repository running on the same infrastructure as your “production” systems.

These are:

Up until the release of v11 or at least prior releases, I would still use terraform to deploy my machines from a template, but it would be on a one by one basis generally. Therefore, I thought I need to have a better more uniformed way of being able to automate the whole lab deployment.

The Veeam Install

Now in the past as I have mentioned we have spent a long time talking and showing you the power of Chef and Ansible for configuration management and the deployment of Veeam components, as well as chocolatey. All great solutions that work great out in the wild. I wanted to take a very simple approach and I wanted to leverage the Veeam unattended methods to deploy Veeam Backup & Replication, I wanted to incorporate that into the terraform script to deploy the above systems as well as more if need be.

The only server that really needs the bulk installation is the “brain” the Veeam Backup & Replication server. The others are just services that are all deployed from the Veeam Backup & Replication management server.

First of all here I will give a shout out to Joe Houghes, he has had a huge hand in the making the bulk of the smart stuff when it comes to the unattended scripts, you can find the latest ones on the VeeamHub page.

What you will need

You will need your vSphere environment, a Windows template, CentOS template and an Ubuntu Template. Alongside this project I have also been updating my packer builds and will try and write this up later. Then we need a network share where we can store our scripts and Veeam ISO and license files. I have two shares, I have one that is located on my management system where I run all my code from and this is completely open and free access to everyone in the lab if they navigate to the address of the machine. The other is an authenticated share which is where our ISOs are stored and other data, this is the reason I had to have two scripts for this task because of this authentication method, if you have open shares then you may be able to just use a single run_once command script with installation and then add the configuration script into this block also.

A massive shout out to the community, I knew I uploaded this with an older version of terraform so the code likely would not work with anything newer than 0.11 of Terraform but Gareth Edwards made took the effort to bring this up to date and also the push for me to now use 0.14 of Terraform

You can find Gareth on Twitter here and that update here

AutomatedVBRInstall.ps1

This script is what I will use within the terraform run_once_command_list found under the windows_options block, this will enable me to mount the other shared folders that require permissions and any other commands like changes to firewall, enable PowerShell remote etc. This script will also mount the Veeam Backup & Replication ISO and start our next script.

Install_Veeam.ps1

It is this command that will perform the installation of Veeam Backup & Replication and all of the components I wish to be installed. This is stored in that domain share and is made up of two other files, VeeamInstallFunctions.ps1 and VeeamConfigVariables.ps1, the latter file is the only one you need to modify with your environment variables.

VeeamConfiguration.ps1

Finally, once the Veeam Backup & Replication server and all the desired components are installed it is time for us to implement the Veeam Configurations, this is going to add those additional component servers, the proxies and the repository server, it will also create a small local cache repository for our NAS backup test jobs, a scale out backup repository that only consists of the performance tier although later could include object storage capacity tier and archive tier options. It will also create a VM backup job and NAS backup job.

The environment that I am using does not really allow for me to use one open to the world share, so I have had to work around that. I have also spent some time looking at running the above commands and scripts through a remote-exec function but so far this has been the best way to achieve what I wanted and needed.

You will find the above 3 scripts in this location here.

Infrastructure as Code

This can then be pulled together by using the terraform code found here

Timing

One of the main reasons for this project was to be able to quickly spin up an entire Veeam environment and not just parts of the environment although that is still super possible with this configuration, I can add additional blocks into my terraform code and it can deploy a number of new instances for me to use. But regarding time and this varies but it generally takes no longer than 25 minutes to complete in my environment.

Hopefully people find this useful, if need be, we can also put some more content together to walk through the configuration steps needed here. I am sure there are many improvements to be made here and I would also be interested in people taking this on and testing in their lab to see if this can be made portable with very few variable changes. As always this is a work in progress and there are lots of plans to increase certain infrastructure components but also to make the variable changes much more simple.