Object storage doesn’t need to be in the public cloud it can also be on-premises. But there is a strong use case for why people want to be putting their long-term retention backups into object storage.
On-Premises Object Storage Introduction
Since the release of Veeam Availability Suite 9.5 Update 4 and the introduction of the Cloud Tier feature, more can be read here on what the premise is here. I have been looking at other options and specifically around on-premises customers a lot of the verticals I am speaking to cannot for various reasons leverage the power of the big hyperscale public cloud providers, but they have a huge use case for this long term data retention or the ability to tier their backups from their primary backup storage to a more cost efficient media.
The first one I have been looking at is SwiftStack, this option allows for the customer to achieve the benefits of cloud storage but whilst all data resides on their hardware on their premises.
One interesting part from SwiftStack is that it can still offer the ability to expand and leverage those public cloud providers such as Amazon S3 and Google Cloud Storage. I thought this brought some compelling use cases to some of our customers.
SwiftStack Architecture 101
I wanted to briefly cover the architecture of SwiftStack what are all the components that make up a SwiftStack environment and as we mentioned before SwiftStack is like public cloud storage but running behind your firewall and it’s focused on capacity based high throughput workloads for unstructured data and what’s unique about SwiftStack is that it allows anyone to easily deploy operate and scale a large scale storage system just like the hyperscale public cloud providers
Let’s look at the architecture of this stack unlike traditional storage architectures SwiftStack is software! it’s based on open source and it runs on standard servers running Linux and these are really cost effective building blocks that you just network together via standard Ethernet and it enables the system to be built up over a period of time as you need to scale and the SwiftStack software is installed on each one of those servers and that creates a node and that node is a member of a cluster.
- Proxy – Applications interact with the proxy service to read and write objects. The proxy layer can be run on dedicated nodes to manage and scale performance independent of capacity.
- Storage – The storage service stores and protects all objects. This layer can be scaled independently for capacity and redundancy.
- Combined – This mode runs both proxy and storage services on the node. This is ideal for testing, small environments, or archive solutions.
The core engine in SwiftStack is the open source project OpenStack Swift. SwiftStack state and feel strongly that you should own your data and every function that touches it should be open and OpenStack Swift is the storage core much like a car engine, these components of Swift are extremely durable highly available very high throughput storage system and these components power some of the world’s largest storage cloud environments including some huge telco environments. This thriving community continues to push capabilities forward.
For example, like enabling multi-tenancy by integrating two authentication systems that you may have or by including load balancing right into the product or connecting and managing the physical infrastructure of the physical hardware and all these capabilities are tied together with what is called the SwiftStack controller
The SwiftStack controller is used to deploy, to scale, to manage, and operate a SwiftStack cluster.
The controller can either run as a service or it can be deployed on servers to run in your own environment right next to the rest of the servers and either way its abilities are the same. The controller can be deemed the Orchestrator of a distributed storage system.
Another interesting attribute worth mentioning is SwiftStack can also have multi region nodes and a SwiftStack cluster can be deployed across multiple data centres, this can protect against single site failures and it also allows your data to be replicated to where it could best be used.
For example, when the storage request is made the system will prioritise getting to the data in the most efficient manner (i.e. closest and fastest) it can, and the great thing is it allows nodes to exist anywhere across the globe even over high latency networks and the way.
This is achieved through a policy-based architecture of how and where data is protected and how it is stored as a matter of policy
For example, a policy could be how should this data be protected should it use whole replicas – for accessing it very efficiently or how many should there should there be, or should it use a space-efficient erasure coding strategy or oversubscribe the number of parity bits
Erasure Coding
“Instead of replicating whole objects multiple times across the cluster, an erasure coding algorithm can be used to encode and store fragments of an object across the cluster to reduce the physical footprint needed to protect the data. You can think of erasure coding like RAID5, but across different physical nodes in a cluster. Erasure coding can be used for both single-region and multi-region clusters.”
Also, as I mentioned in the opening part of the post, the ability to have your own cloud-based storage on-premises but also the ability to set a policy to replicate data to a public cloud and then SwiftStack executes based on these storage policies.
For example, when there’s a drive failure the system will automatically increase the durability up to that policies intent without you as an operator needing to do anything this makes it easy for to operate the system, but it also makes it more durable for the data.
SwiftStack includes object API access and file API access and what’s unique about SwiftStack is that the same data can be accessed over both file and object APIs with object either the OpenStack Swift or the s3p API can be used by applications that understand how to natively use object storage applications that cannot be modified to use an object.
Resource Downloads
The images I have used above have not been via my creative talents, I thought it was better to leave that to the experts. Locations for the images can be found in these very useful links. I just wanted to highlight in a short post the benefits and a 101 on how SwiftStack looks.
3-2-1 Cloud! – Erik Pounds, SwiftStack Blog
Veeam + SwiftStack Solution Brief
Veeam + SwiftStack How-To Guide
SwiftStack – Hardware Infrastructure Configuration
1 Comment