Back

Navigating OVHcloud File Storage with Manila CSI (RWX) on Kubernetes clusters (MKS)

Aurélie Vache16/07/202612 min read
File Storage,Kubernetes,Public Cloud
Navigating OVHcloud File Storage with Manila CSI (RWX) on Kubernetes clusters (MKS)

If you run stateful applications on Kubernetes, one common challenge is providing shared persistent storage that can be accessed by multiple workloads.

While Kubernetes Persistent Volumes usually rely on block storage with ReadWriteOnce access, some applications require shared filesystem access with ReadWriteMany (RWX) capabilities.

OVHcloud File Storage provides managed NFS shares that can be dynamically consumed by Kubernetes workloads through the Manila CSI driver.

In this blog post, we will see how to integrate OVHcloud File Storage with OVHcloud Managed Kubernetes Service (MKS) using Manila CSI and test RWX storage capabilities.

OVHcloud File Storage

OVHcloud File Storage managed NFS shares for cloud-native Kubernetes workloads.

OVHcloud Public Cloud File Storage is a fully managed shared file storage service designed for cloud-native workloads running on Public Cloud instances and Kubernetes clusters.

Built on OpenStack Manila, it provides shared NFSv3 volumes that can be mounted simultaneously by multiple clients, making it an ideal solution for applications requiring ReadWriteMany (RWX) access. Volumes can be provisioned from 150 GiB up to 10 TiB, with predictable, linear performance that scales with the allocated capacity.

Because the service is fully managed, you don't need to deploy or maintain your own NFS server. File Storage integrates with the OVHcloud platform through the Control Panel, API, CLI, Terraform provider, and Kubernetes via the Manila CSI driver, allowing shared volumes to be dynamically provisioned directly from your cluster.

Why use Manila CSI?

The Kubernetes Container Storage Interface (CSI) provides a standard mechanism for exposing external storage systems to Kubernetes.

Instead of manually creating NFS mounts and PersistentVolumes, the Manila CSI driver allows Kubernetes to dynamically create and manage file shares using Kubernetes resources such as StorageClasses (SC), PersistentVolumeClaims (PVC) and PersistentVolumes (PV).

Prerequisites

Before starting, you need:

  • An OVHcloud Public Cloud project
  • An OVHcloud Managed Kubernetes Service (MKS) cluster that is connected to a private network
  • Terraform CLI installed
  • kubectl CLI installed

Deploying Manila CSI step by step: let’s do it!

We already have a MKS cluster, in EU-WEST-PAR region, running inside a private network and a subnet. In this blog post we will:

  • create a Public Cloud user for the Manila CSI driver
  • install the CSI NFS driver
  • install the Manila CSI driver
  • deploy a Secret for Manila CSI that allows the Manila CSI driver to authenticate against OpenStack and manage Manila resources in your cluster
  • create a file shared network
  • deploy a ConfigMap to configure the Manila CSI driver
  • deploy the csi-manila-nfs StorageClass to enable the Manila CSI driver to dynamically create Manila shares and use them as Kubernetes volumes

We will use Terraform to deploy this architecture easily.

Create a provider.tf file and fill it with the information:

bash
terraform {
  required_providers {
    helm = {
      source = "hashicorp/helm"
    }

    kubectl = {
      source = "alekc/kubectl"
      version = "2.1.6"
    }

    ovh = {
      source = "ovh/ovh"
    }
  }
}

provider "helm" {
  kubernetes = {
    host                   = data.ovh_cloud_project_kube.mks_cluster.kubeconfig_attributes[0].host
    client_certificate     = base64decode(data.ovh_cloud_project_kube.mks_cluster.kubeconfig_attributes[0].client_certificate)
    client_key             = base64decode(data.ovh_cloud_project_kube.mks_cluster.kubeconfig_attributes[0].client_key)
    cluster_ca_certificate = base64decode(data.ovh_cloud_project_kube.mks_cluster.kubeconfig_attributes[0].cluster_ca_certificate)
  }
}

provider "kubectl" {
  host                   = data.ovh_cloud_project_kube.mks_cluster.kubeconfig_attributes[0].host
  client_certificate     = base64decode(data.ovh_cloud_project_kube.mks_cluster.kubeconfig_attributes[0].client_certificate)
  client_key             = base64decode(data.ovh_cloud_project_kube.mks_cluster.kubeconfig_attributes[0].client_key)
  cluster_ca_certificate = base64decode(data.ovh_cloud_project_kube.mks_cluster.kubeconfig_attributes[0].cluster_ca_certificate)
  load_config_file       = false
}

Set the environment variables, for the OVHcloud Terraform provider, with your credentials:

bash
# OVHcloud provider needed keys
export OVH_ENDPOINT="ovh-eu"
export OVH_APPLICATION_KEY="xxx"
export OVH_APPLICATION_SECRET="xxx"
export OVH_CONSUMER_KEY="xxx"
export OVH_CLOUD_PROJECT_SERVICE="xxx"

Create a variables.tf.template file and fill it with these information:

bash
variable "service_name" {
  default = "$OVH_CLOUD_PROJECT_SERVICE"
}

variable "mks_cluster_id" {
  default = "<your_mks_cluster_id>"
}

⚠️ In the file, replace the MKS ID with your existing MKS cluster ID information.

Replace the value of the OVH_CLOUD_PROJECT_SERVICE environment variable in the variables.tf file:

bash
envsubst < variables.tf.template > variables.tf

Create a nfs_share.tf file and fill it with these information:

markdown
data "ovh_cloud_project_kube" "mks_cluster" {
  service_name = var.service_name
  kube_id      = var.mks_cluster_id
}

data "ovh_cloud_network_private_vrack_subnet" "mks_cluster_subnet" {
  service_name = var.service_name
  network_id   = data.ovh_cloud_project_kube.mks_cluster.private_network_id
  id           = data.ovh_cloud_project_kube.mks_cluster.nodes_subnet_id
}

# CSI Manila

module "csi_manila" {
  source = "git::https://github.com/ovh/public-cloud-examples.git//containers-orchestration/managed-kubernetes/install-csi-manila/modules/ovhcloud/csi_manila?ref=v1.5.0"

  service_name       = var.service_name
  region             = data.ovh_cloud_project_kube.mks_cluster.region
  share_network_name = "${data.ovh_cloud_project_kube.mks_cluster.name}-share-network"
  network_id         = data.ovh_cloud_project_kube.mks_cluster.private_network_id
  subnet_id          = data.ovh_cloud_project_kube.mks_cluster.nodes_subnet_id
  subnet_cidr        = data.ovh_cloud_network_private_vrack_subnet.mks_cluster_subnet.cidr
}

output "manila-user" {
  value = module.csi_manila.manila-user
}

💡In this Terraform file we are using an existing csi_manila Terraform module hosted in the OVHcloud Public Cloud Examples GitHub repository.

The Terraform configuration is ready. Let’s init it:

bash
terraform init

The output should be like this:

bash
$ terraform init

Initializing the backend...
Initializing modules...
Downloading git::https://github.com/ovh/public-cloud-examples.git?ref=v1.5.0 for csi_manila...
- csi_manila in .terraform/modules/csi_manila/containers-orchestration/managed-kubernetes/install-csi-manila/modules/ovhcloud/csi_manila
Initializing provider plugins...
- Reusing previous version of hashicorp/helm from the dependency lock file
- Reusing previous version of alekc/kubectl from the dependency lock file
- Reusing previous version of ovh/ovh from the dependency lock file
- Installing hashicorp/helm v3.2.0...
- Installed hashicorp/helm v3.2.0 (signed by HashiCorp)
- Installing alekc/kubectl v2.1.6...
- Installed alekc/kubectl v2.1.6 (self-signed, key ID 772FB27A86DAFCE7)
- Installing ovh/ovh v2.16.1...
- Installed ovh/ovh v2.16.1 (signed by a HashiCorp partner, key ID F56D1A6CBDAAADA5)
Partner and community providers are signed by their developers.
If you'd like to know more about provider signing, you can read about it here:
https://developer.hashicorp.com/terraform/cli/plugins/signing

Terraform has been successfully initialized!

You may now begin working with Terraform. Try running "terraform plan" to see
any changes that are required for your infrastructure. All Terraform commands
should now work.

If you ever set or change modules or backend configuration for Terraform,
rerun this command to reinitialize your working directory. If you forget, other
commands will detect it and remind you to do so if necessary.

Apply it:

bash
terraform apply

The output should be like this:

bash
$ terraform apply

data.ovh_cloud_project_kube.mks_cluster: Reading...
data.ovh_cloud_project_kube.mks_cluster: Read complete after 1s [id=xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxx]
data.ovh_cloud_network_private_vrack_subnet.mks_cluster_subnet: Reading...
data.ovh_cloud_network_private_vrack_subnet.mks_cluster_subnet: Read complete after 1s [id=xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxx]

Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols:
  + create

Terraform will perform the following actions:

  # module.csi_manila.helm_release.csi-driver-nfs will be created
  + resource "helm_release" "csi-driver-nfs" {
      + atomic                     = false
      + chart                      = "csi-driver-nfs"
      + cleanup_on_fail            = false
      + create_namespace           = false
      + dependency_update          = false
      + disable_crd_hooks          = false
      + disable_openapi_validation = false
      + disable_webhooks           = false
      + force_update               = false
      + id                         = (known after apply)
      + lint                       = false
      + max_history                = 0
      + metadata                   = (known after apply)
      + name                       = "csi-driver-nfs"
      + namespace                  = "kube-system"
      + pass_credentials           = false
      + recreate_pods              = false
      + render_subchart_notes      = true
      + replace                    = false
      + repository                 = "https://raw.githubusercontent.com/kubernetes-csi/csi-driver-nfs/master/charts"
      + reset_values               = false
      + reuse_values               = false
      + set_wo                     = (write-only attribute)
      + skip_crds                  = false
      + status                     = "deployed"
      + take_ownership             = false
      + timeout                    = 300
      + upgrade_install            = false
      + verify                     = false
      + version                    = "4.13.4"
      + wait                       = true
      + wait_for_jobs              = false
    }

  # module.csi_manila.helm_release.openstack-manila-csi will be created
  + resource "helm_release" "openstack-manila-csi" {
      + atomic                     = false
      + chart                      = "openstack-manila-csi"
      + cleanup_on_fail            = false
      + create_namespace           = false
      + dependency_update          = false
      + disable_crd_hooks          = false
      + disable_openapi_validation = false
      + disable_webhooks           = false
      + force_update               = false
      + id                         = (known after apply)
      + lint                       = false
      + max_history                = 0
      + metadata                   = (known after apply)
      + name                       = "openstack-manila-csi"
      + namespace                  = "kube-system"
      + pass_credentials           = false
      + recreate_pods              = false
      + render_subchart_notes      = true
      + replace                    = false
      + repository                 = "https://kubernetes.github.io/cloud-provider-openstack"
      + reset_values               = false
      + reuse_values               = false
      + set_wo                     = (write-only attribute)
      + skip_crds                  = false
      + status                     = "deployed"
      + take_ownership             = false
      + timeout                    = 300
      + upgrade_install            = false
      + verify                     = false
      + version                    = "2.36.0"
      + wait                       = true
      + wait_for_jobs              = false
    }

  # module.csi_manila.kubectl_manifest.csi-manila-secrets will be created
  + resource "kubectl_manifest" "csi-manila-secrets" {
      + api_version             = (known after apply)
      + apply_only              = false
      + field_manager           = "kubectl"
      + force_conflicts         = false
      + force_new               = false
      + id                      = (known after apply)
      + kind                    = (known after apply)
      + live_manifest_incluster = (sensitive value)
      + live_uid                = (known after apply)
      + name                    = (known after apply)
      + namespace               = (known after apply)
      + server_side_apply       = false
      + uid                     = (known after apply)
      + validate_schema         = true
      + wait_for_rollout        = true
      + yaml_body               = (sensitive value)
      + yaml_body_parsed        = (known after apply)
      + yaml_incluster          = (sensitive value)
    }

  # module.csi_manila.kubectl_manifest.manila-runtime-configmap will be created
  + resource "kubectl_manifest" "manila-runtime-configmap" {
      + api_version             = "v1"
      + apply_only              = false
      + field_manager           = "kubectl"
      + force_conflicts         = false
      + force_new               = false
      + id                      = (known after apply)
      + kind                    = "ConfigMap"
      + live_manifest_incluster = (sensitive value)
      + live_uid                = (known after apply)
      + name                    = "manila-csi-runtimeconf-cm"
      + namespace               = "default"
      + server_side_apply       = false
      + uid                     = (known after apply)
      + validate_schema         = true
      + wait_for_rollout        = true
      + yaml_body               = (sensitive value)
      + yaml_body_parsed        = <<-EOT
            apiVersion: v1
            data:
              runtimeconfig.json: |
                {
                  "nfs": {
                    "matchExportLocationAddress": "10.1.0.0/16"
                  }
                }
            kind: ConfigMap
            metadata:
              annotations:
                meta.helm.sh/release-name: manila-csi
                meta.helm.sh/release-namespace: default
              labels:
                app.kubernetes.io/managed-by: Helm
              name: manila-csi-runtimeconf-cm
              namespace: default
        EOT
      + yaml_incluster          = (sensitive value)
    }

  # module.csi_manila.kubectl_manifest.storage-class will be created
  + resource "kubectl_manifest" "storage-class" {
      + api_version             = (known after apply)
      + apply_only              = false
      + field_manager           = "kubectl"
      + force_conflicts         = false
      + force_new               = false
      + id                      = (known after apply)
      + kind                    = (known after apply)
      + live_manifest_incluster = (sensitive value)
      + live_uid                = (known after apply)
      + name                    = (known after apply)
      + namespace               = (known after apply)
      + server_side_apply       = false
      + uid                     = (known after apply)
      + validate_schema         = true
      + wait_for_rollout        = true
      + yaml_body               = (sensitive value)
      + yaml_body_parsed        = (known after apply)
      + yaml_incluster          = (sensitive value)
    }

  # module.csi_manila.ovh_cloud_project_user.manila-user will be created
  + resource "ovh_cloud_project_user" "manila-user" {
      + creation_date = (known after apply)
      + description   = "User for the Manila CSI driver"
      + id            = (known after apply)
      + openstack_rc  = (known after apply)
      + password      = (sensitive value)
      + role_name     = "share_operator"
      + roles         = (known after apply)
      + service_name  = "xxxxxxxxxxxxxxxxxxxxx"
      + status        = (known after apply)
      + username      = (known after apply)
    }

  # module.csi_manila.ovh_cloud_storage_file_share_network.sharenetwork will be created
  + resource "ovh_cloud_storage_file_share_network" "sharenetwork" {
      + checksum        = (known after apply)
      + created_at      = (known after apply)
      + current_state   = (known after apply)
      + description     = (known after apply)
      + id              = (known after apply)
      + name            = "mks_standard_3az-share-network"
      + network_id      = "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxx"
      + region          = "EU-WEST-PAR"
      + resource_status = (known after apply)
      + service_name    = "xxxxxxxxxxxxxxxxxxxxx"
      + subnet_id       = "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxx"
      + updated_at      = (known after apply)
    }

Plan: 7 to add, 0 to change, 0 to destroy.

Changes to Outputs:
  + manila-user = (known after apply)

Do you want to perform these actions?
  Terraform will perform the actions described above.
  Only 'yes' will be accepted to approve.

  Enter a value: yes

module.csi_manila.ovh_cloud_storage_file_share_network.sharenetwork: Creating...
module.csi_manila.ovh_cloud_project_user.manila-user: Creating...
module.csi_manila.kubectl_manifest.manila-runtime-configmap: Creating...
module.csi_manila.kubectl_manifest.manila-runtime-configmap: Creation complete after 0s [id=/api/v1/namespaces/default/configmaps/manila-csi-runtimeconf-cm]
module.csi_manila.helm_release.csi-driver-nfs: Creating...
module.csi_manila.ovh_cloud_storage_file_share_network.sharenetwork: Still creating... [00m10s elapsed]
module.csi_manila.ovh_cloud_project_user.manila-user: Still creating... [00m10s elapsed]
module.csi_manila.ovh_cloud_storage_file_share_network.sharenetwork: Creation complete after 12s [id=xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxx]
module.csi_manila.kubectl_manifest.storage-class: Creating...
module.csi_manila.ovh_cloud_project_user.manila-user: Creation complete after 13s [id=718526]
module.csi_manila.kubectl_manifest.csi-manila-secrets: Creating...
module.csi_manila.kubectl_manifest.csi-manila-secrets: Creation complete after 1s [id=/api/v1/namespaces/default/secrets/csi-manila-secrets]
...

Let's create dynamically File Storage on Kubernetes

Now, in your Kubernetes cluster, create a pvc.yaml file witht this content:

yaml
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: nfs-share-fs-pvc
spec:
  accessModes:
    - ReadWriteMany
  resources:
    requests:
      storage: 150Gi
  storageClassName: csi-manila-nfs

Apply this Persistent Volume Claim (PVC), thanks to that Kubernetes users can request shared storage:

bash
kubectl apply -f pvc.yaml

Check the status of the PVC:

bash
kubectl get pvc nfs-share-fs-pvc

Wait until the status of the PVC changes to Bound.

You should have an output like this:

bash
$ kubectl get pvc nfs-share-fs-pvc

NAME               STATUS   VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS     VOLUMEATTRIBUTESCLASS   AGE
nfs-share-fs-pvc   Bound    pvc-af1349b7-eb51-43ac-b0f0-94d0e8139c57   150Gi      RWX            csi-manila-nfs   <unset>                 3h48m

Create a deploy.yaml file and fill it with this content:

yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-deployment
  labels:
    app: nginx
spec:
  replicas: 1
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      labels:
        app: nginx
    spec:
      volumes:
      - name: nfs-share-fs-pvc
        persistentVolumeClaim:
          claimName: nfs-share-fs-pvc
      containers:
      - name: nginx
        image: nginx
        ports:
          - containerPort: 80
            name: "http-server"
        volumeMounts:
          - mountPath: "/usr/share/nginx/html"
            name: nfs-share-fs-pvc

Apply this deployment with one pod that has a volume attached to nfs-share-fs-pvc:

bash
kubectl apply -f deploy.yaml

Verify that the pod is running:

bash
kubectl get pod

You should have an output like this:

bash
$ kubectl get pod
NAME                                READY   STATUS    RESTARTS   AGE
nginx-deployment-6cbf9b898c-svpzf   1/1     Running   0          6m

Verify that you can scale the deployment (multi-attach volume):

bash
kubectl scale deploy/nginx-deployment --replicas=2

Check another pod is running:

bash
kubectl get pod

You should have an output like this:

bash
$ kubectl get pod

NAME                                READY   STATUS    RESTARTS   AGE
nginx-deployment-6cbf9b898c-867bq   1/1     Running   0          3m
nginx-deployment-6cbf9b898c-svpzf   1/1     Running   0          9m

To verify RWX functionality, connect to one pod and create a file in the mounted directory (e.g., /usr/share/nginx/html).

bash
MY_POD=$(kubectl get po -o name | sed -n '1p')
echo $MY_POD

#Create a file in the pod number 1
kubectl exec $MY_POD -it -- touch /usr/share/nginx/html/index.html

Then connect to the second pod and confirm the file is visible:

bash
MY_POD_2=$(kubectl get po -o name | sed -n '2p')
echo $MY_POD_2

#Display it in the pod number two
kubectl exec $MY_POD_2 -it -- ls -alrt /usr/share/nginx/html/

You should have an output like this:

bash
$ kubectl exec $MY_POD_2 -it -- ls -alrt /usr/share/nginx/html/

total 24
drwxr-xr-x 3 root root  4096 Jul 14 01:22 ..
drwx------ 2 root root 16384 Jul 15 09:18 lost+found
drwxrwxrwx 3 root root  4096 Jul 15 09:32 .
-rw-r--r-- 1 root root     0 Jul 15 13:09 index.html

Now your Manila share exposed through NFS is functioning the way you want! 🎉

Conclusion

In this blog post we showed that using Manila CSI with OVHcloud File Storage makes it possible to provide Kubernetes applications with dynamically provisioned shared storage.

We recommend you also take a look at our Cloud Roadmap & Changelog for an overview of all the coming features for OVHcloud Public Cloud products.

Share on: