Building a Resilient Kubernetes Cluster with Talos, Ceph, and Veeam Kasten

More and more clusters have data appearing on them in the Kubernetes world. Either via a StatefulSet, Operator or at least closely tied to a managed database external to the cluster.

But in the cloud native world we have to consider the whole application which includes the data, be it inside or outside of the cluster.

Equally depending on the importance of this data (probably important if you pay for the privilege of having it managed) it’s going to need some care and attention when it comes to data management, protection against accidents, misconfigurations and the ever popular world of cyber threats.

Overview

In this post, yes it’s going to focus on my home lab but it’s also relevant to any Kubernetes cluster far and wide.

We are going to explain the current configuration of how I am running Talos Linux a distribution focused only on Kubernetes but also security. Only using the binaries it absolutely needs to run, making it a very secure and safe option over a full blown Linux distribution.

We will also talk about the Ceph environment and where we store those “mission-critical” data services. Finally we will touch on the ease of protecting these data services with Veeam Kasten for Kubernetes.

Talos

Talos Linux is a lightweight, container-optimised Linux distribution designed specifically for Kubernetes deployments. It is immutable, meaning its filesystem is read-only, enhancing security and consistency. Talos eliminates SSH, relying instead on a secure API for management, making it highly automated and ideal for cloud-native environments. With minimal surface area, it’s designed for maximum reliability and focuses on running Kubernetes efficiently.

Ceph

Ceph is an open-source, highly scalable storage platform that provides object, block, and file storage under a unified system. Rook-Ceph integrates Ceph with Kubernetes, enabling dynamic, cloud-native storage orchestration. Rook-Ceph automates the deployment, management, scaling, and recovery of Ceph clusters within Kubernetes environments. It abstracts complex Ceph operations, making it easier to use as a Kubernetes storage backend for persistent volumes, supporting block storage (RBD), shared filesystems (CephFS), and S3-compatible object storage. This integration ensures resilient, self-managing storage optimized for containerized workloads.

Veeam Kasten

Veeam provides the ability to natively protect your Kubernetes application and data. Natively becoming part of the Kubernetes API through aggregated APIs and Custom Resources, things can be managed via the API or via a UI.

Application consistency and mobility are two other areas where Veeam shines above the open source de facto offerings. The ability to orchestrate the protection of Postgres databases or MongoDB collections vs just grabbing the PVC and hoping for the best. Mobility wise, the ability to help migrate, restore, test your application and data in another Kubernetes cluster by way of transforming the application for the target cluster. This target cluster might be a completely different flavour of Kubernetes. You might be going from a cloud managed version to Talos for example.

Lots more to cover here but let’s get into how to get going. You will also notice that there is no Talos logo listed below, there are also many other distributions not included in the marketing slide… fundamentally though we focus on supporting upstream Kubernetes versions.

Installation

The deployment of Veeam Kasten be done by many marketplace operator hubs. Or via Helm. In this case with Talos we will use Helm to get our deployment of Veeam Kasten up and running.

As mentioned a pre requisite of this would be to have access to your cluster via kubectl and to have helm installed on your machine. You will also require some persistent storage.

helm repo add kasten https://charts.kasten.io/

Helm repo update

helm install k10 kasten/k10 --namespace=kasten-io --create-namespace

You can watch / follow the pod creation with the following command:

Watch kubectl get pods -n Kasten-io

We have used the basic installation, there are many options when it comes to helm chart values, you can find those out in the documentation.

kubectl --namespace kasten-io port-forward service/gateway 8080:80

The Veeam Kasten dashboard will be available at http://127.0.0.1:8080/k10/#/.

We now need a token to authenticate with the default settings on the helm chart. We can obtain this with the following command:

The token’s default expiration is 1 hour (3600 seconds) if no

flag is specified in the command.

kubectl --namespace kasten-io create token k10-k10

Copy that token and restart the port forward if need be. Paste the token into the web browser and you should see something that resembles the following, minus the amount of apps, policies, and data usage.

Protection

There can be a lot more to the initial deployment and different chart values depending on the cluster you are deploying, authentication options etc etc.

But let’s start with protecting a simple stateful app deployed in our Talos cluster.

First we will need a backup location profile somewhere off cluster to send our backups to if something bad was to happen at the cluster and storage layer.

We can also leverage snapshots for a real fast recovery option but snapshots alone are not going to cut it for a backup. Repeat after me…

Snapshots are not backups.

Something I and many others have been preaching for a very long time in the infrastructure and operations space. Anyway back to it…

Let’s start with showing you some of this important data in our database.

Within the dashboard we should define a location profile

Ok so we have somewhere now to send our backups to, we next need to create a policy to protect our previously installed “mission-critical” application and data.

Create a policy against a pre deployed application.

Now I can’t expect you to trust me at this point that when something bad happens to the app and it’s data this is going to just work. So let’s show you.

Cause a problem in the database

Uh oh! Things don’t seem very well.

Let’s use our backup to restore our application and data back to before the bad thing happened.

Steps to restore

What’s next

There is so much more to cover when it comes to protecting data in and out of Kubernetes but also the great thing about Kubernetes we can also run it anywhere which means we can do this with Talos as well, in my home lab I have a VMware vSphere cluster running my virtual machines, I also have as VMs a second Talos cluster that I would like to cover in the next post. This will highlight some of the additional integration when it comes to storage that we can achieve on this cluster as well as moving workloads between environments with Veeam Kasten.

I also want to get in Virtual Machines on Kubernetes and how these should be protected as well, this will be another follow up post.

We didn’t get into the application consistent side of protecting those databases either and with our mission critical app, everything was fine but if that database has lots of transactions hitting it you can be sure something would get lost in the process.