Kubernetes Persistent Volumes with NetApp Trident
Kubernetes is an open source project that provides the ability to manage containers. It has a (now defacto standard) API for operating containers and applications built on them; it helps organizations manage the container lifecycle.
However, there are many things it does not do on its own, and one of the things it does not do is operate persistent storage. It provides access to storage but is itself not a storage system.
Indeed containers themselves can be created, deleted, and created again, as they are ephemeral, but often we need to store data that is not ephemeral and does not go away when a container does.
Storage as an Attachable Resource
Cloud native and other similar paradigms, such as “The 12-factor App”, suggest that persistent storage should be an attachable resource. Meaning systems and applications that need to store persistent data should be able to attach and detach persistent storage volumes on demand.
For example, when a container that has a persistent volume attached is deleted, any persistent volumes assigned to it are NOT removed, and instead becomes available for another container to claim. Kubernetes provides storage as an attachable resource through persistent volumes and claims.
NetApp provides a project called Trident that acts as an intermediary between Kubernetes and various NetApp storage platforms–a storage provisioner. Trident does this by using the Kubernetes persistent volume framework: Trident listens for events on the Kubernetes API to determine when to build requested persistent volumes dynamically. It provides the ability to use Kubernetes storage class system to provide multiple types of storage, be they NFS or iSCSI based, or on any of several attributes, such as encryption.
Trident makes persistent volumes available to Kubernetes applications in an entirely cloud-native way. Applications deployed into Kubernetes can request persistent volumes from NetApp storage systems through the Kubernetes API, and thus access many of the features that you would typically expect from an enterprise storage system.
Example of Creating a Persistent Volume
Let’s quickly show creating a persistent volume using the Solidfire backend.
However, instead of directly creating a persistent volume, we will alternatively request a claim, and behind the scenes Trident will create the volume on the correct storage class.
Here’s the YAML for creating a PVC on the SolidFire Storage class.
# cat k8s/pvcforsolidfire.yaml kind: PersistentVolumeClaim apiVersion: v1 metadata: name: persistent-volume-claim-solidfire spec: accessModes: - ReadWriteOnce resources: requests: storage: 1Gi storageClassName: storage-class-solidfire
Before creating the claim, lets show that there are no PVC or PV resources yet.
# kubectl get pvc No resources found. # kubectl get pv NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS REASON AGE trident 2Gi RWO Retain Bound trident/trident 123d
The only PV that exists is the one backing Trident’s etcd instance (more on that later). There are no other PVs.
Let’s create the PVC.
# kubectl create -f k8s/pvcforsolidfire.yaml persistentvolumeclaim "persistent-volume-claim-solidfire" created # kubectl get pv NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS REASON AGE default-persistent-volume-claim-solidfire-69eca 1Gi RWO Delete Bound default/persistent-volume-claim-solidfire storage-class-solidfire 1s trident 2Gi RWO Retain Bound trident/trident 123d
As can be seen above, we now have a PV called “default-persistent-…”. Without any manual intervention, Trident has gone ahead and created the PV for us on the correct storage system.
The volume is also shown in the SolidFire web interface.
While Trident helps us with persistent volumes in Kubernetes, it still has to be correctly deployed into Kubernetes. As well, every container host has to have appropriate access to storage devices. Automation, perhaps with Ansible or something similar, can help ensure this, but it is something that has to be done externally and is not handled by Trident.
Further, Trident should be bootstrapped into the cluster. Trident stores information about persistent volumes in its etcd instance or cluster. Persistent volumes should back this etcd instance but how do you get Trident access to persistent volumes when Trident is required to provide them? That’s where some bootstrapping magic comes into play.
Cloud native storage
To participate in cloud-native paradigms, we need attachable storage resources. Trident provides this to both Docker and Kubernetes and makes using persistent volumes on top of NetApp storage easy.