Kubernetes Persistent Volumes with NetApp Trident

Kubernetes is an open source project that provides the ability to manage containers. It has a (now defacto standard) API for operating containers and applications built on them; it helps organizations manage the container lifecycle.

However, there are many things it does not do on its own, and one of the things it does not do is operate persistent storage. It provides access to storage but is itself not a storage system.

Indeed containers themselves can be created, deleted, and created again, as they are ephemeral, but often we need to store data that is not ephemeral and does not go away when a container does.

 

Storage as an Attachable Resource

Cloud native and other similar paradigms, such as “The 12-factor App”, suggest that persistent storage should be an attachable resource. Meaning systems and applications that need to store persistent data should be able to attach and detach persistent storage volumes on demand.

For example, when a container that has a persistent volume attached is deleted, any persistent volumes assigned to it are NOT removed, and instead becomes available for another container to claim. Kubernetes provides storage as an attachable resource through persistent volumes and claims.

 

NetApp Trident

NetApp provides a project called Trident that acts as an intermediary between Kubernetes and various NetApp storage platforms–a storage provisioner. Trident does this by using the Kubernetes persistent volume framework: Trident listens for events on the Kubernetes API to determine when to build requested persistent volumes dynamically. It provides the ability to use Kubernetes storage class system to provide multiple types of storage, be they NFS or iSCSI based, or on any of several attributes, such as encryption.

Trident makes persistent volumes available to Kubernetes applications in an entirely cloud-native way. Applications deployed into Kubernetes can request persistent volumes from NetApp storage systems through the Kubernetes API, and thus access many of the features that you would typically expect from an enterprise storage system.

 

Example of Creating a Persistent Volume

Let’s quickly show creating a persistent volume using the Solidfire backend.

However, instead of directly creating a persistent volume, we will alternatively request a claim, and behind the scenes Trident will create the volume on the correct storage class.

Here’s the YAML for creating a PVC on the SolidFire Storage class.

# cat k8s/pvcforsolidfire.yaml
kind: PersistentVolumeClaim
apiVersion: v1
metadata:
  name: persistent-volume-claim-solidfire
spec:
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 1Gi
  storageClassName: storage-class-solidfire

Before creating the claim, lets show that there are no PVC or PV resources yet.

 

# kubectl get pvc
No resources found.
# kubectl get pv
NAME      CAPACITY   ACCESS MODES   RECLAIM POLICY   STATUS    CLAIM             STORAGECLASS   REASON    AGE
trident   2Gi        RWO            Retain           Bound     trident/trident                            123d

 

The only PV that exists is the one backing Trident’s etcd instance (more on that later). There are no other PVs.

Let’s create the PVC.

 

# kubectl create -f k8s/pvcforsolidfire.yaml
persistentvolumeclaim "persistent-volume-claim-solidfire" created
# kubectl get pv
NAME                                              CAPACITY   ACCESS MODES   RECLAIM POLICY   STATUS    CLAIM                                       STORAGECLASS              REASON    AGE
default-persistent-volume-claim-solidfire-69eca   1Gi        RWO            Delete           Bound     default/persistent-volume-claim-solidfire   storage-class-solidfire             1s
trident                                           2Gi        RWO            Retain           Bound     trident/trident                                                                 123d

As can be seen above, we now have a PV called “default-persistent-…”. Without any manual intervention, Trident has gone ahead and created the PV for us on the correct storage system.

The volume is also shown in the SolidFire web interface.

 

Operational Items

While Trident helps us with persistent volumes in Kubernetes, it still has to be correctly deployed into Kubernetes. As well, every container host has to have appropriate access to storage devices. Automation, perhaps with Ansible or something similar, can help ensure this, but it is something that has to be done externally and is not handled by Trident.

Further, Trident should be bootstrapped into the cluster. Trident stores information about persistent volumes in its etcd instance or cluster.  Persistent volumes should back this etcd instance but how do you get Trident access to persistent volumes when Trident is required to provide them? That’s where some bootstrapping magic comes into play.

 

Cloud native storage

To participate in cloud-native paradigms, we need attachable storage resources. Trident provides this to both Docker and Kubernetes and makes using persistent volumes on top of NetApp storage easy.