On December 9th, 2020, I had the pleasure of speaking with two great minds on the topic of simplifying storage in Kubernetes. Alex Chircop is the Founder and CEO at StorageOS (https://storageos.com), and Bruno Andrade is the Founder and CEO at Shipa (https://www.shipa.io). The full recording of this discussion is below. Here is a summary of what we discussed:
Simplify Deploying and Operating Stateful Applications within Kubernetes
All applications are stateful in some form, in the sense that all applications need to store state somewhere, whether it’s in a database or message queue, or just files or an object store for example. The reality is that as applications become portable, more and more organizations are looking to Kubernetes as the leading orchestrator. The challenge has always been, what do you need to do when it comes to storage? What is the key to simplifying storage in Kubernetes? A cloud native storage environment, such as StorageOS can provide developers that declarative and composable way of defining their storage and the attributes that their storage needs.
Everything has become very dynamic in the world of cloud compute, and traditional system storage concepts don’t necessarily apply. Newer concepts tend to bring new complexity from a developer perspective, and it is important minimize these complexities so that the development team is not burdened with managing an infrastructure. The benefits of having a declarative format that applies to every layer of the infrastructure effectively means that now the developer is in control of the attributes for their application, and it means that they can deploy their application in exactly the same way they always have.
Storage Challenges at Scale
There is a scale of the data plane, and that is more of a traditional “speeds and feeds” kind of discussion where we’re talking about IOPS and megabytes per second, etc. But then there’s also the speeds of the control plane and this is really important because in a Kubernetes environment, apart from the dynamic structure of the cluster, in terms of nodes and scaling, you also have the concept of volumes coming and going much more dynamically.
StorageOS provides a disaggregated consensus concept in the control plane where every single volume effectively has its own mini brain and can manage placement decisions and failover decisions independently, so you don’t get a serialization. And this allows lots of activity to happen at the same time. Alex shares a quick demo of this in the video.
“Portability is one of those things that’s still really, really difficult”
As people get more and more comfortable with Kubernetes, cloud native apps, and microservices, one tendency is that they’re scaling across multiple clusters. These multiple clusters can be on premises, or in the cloud, and in hybrid environments. Moving between environments, changing cluster versions, etc brings an entirely new set of complexity.
Shipa enables the concept of volume plans. Volume plans are directly attached to your storage class, and with a higher level integration into StorageOS, you can easily bind your storage to the application, regardless of the cluster you are using. Bruno shares a quick demo of this in the video.