Indietro

Navigare in OVHcloud File Storage con Manila CSI (RWX) su cluster Kubernetes (MKS)

Aurélie Vache12 min. di lettura
,,
Navigare in OVHcloud File Storage con Manila CSI (RWX) su cluster Kubernetes (MKS)

Quando si eseguono applicazioni stateful su Kubernetes, una delle sfide più comuni consiste nel fornire uno storage persistente condiviso a cui possano accedere più workload.

Mentre i Persistent Volume di Kubernetes si basano solitamente su uno storage a blocchi con accesso ReadWriteOnce, alcune applicazioni richiedono filesystem condivisi con capacità ReadWriteMany (RWX).

OVHcloud File Storage offre share NFS gestite che possono essere utilizzate dinamicamente da applicazioni Kubernetes tramite il driver Manila CSI.

In questo articolo mostreremo come integrare OVHcloud File Storage con OVHcloud Managed Kubernetes Service (MKS) utilizzando Manila CSI e come testare le capacità dello storage RWX.

OVHcloud File Storage

OVHcloud File Storage gestisce gli share NFS per i carichi di lavoro Kubernetes cloud-native.

OVHcloud Public Cloud File Storage è un servizio di file storage condiviso totalmente gestito, pensato per carichi di lavoro cloud-native eseguiti su istanze Public Cloud e cluster Kubernetes.

Basata su OpenStack Manila, questa soluzione mette a disposizione volumi NFSv3 condivisi che possono essere montati simultaneamente da più client ed è quindi ideale per applicazioni che richiedono un accesso ReadWriteMany (RWX). I volumi possono essere forniti con una capacità da 150 GiB a 10 TiB, con prestazioni lineari prevedibili che aumentano con lo spazio allocato.

Poiché il servizio è completamente gestito, non è necessario installare o mantenere un proprio server NFS. File Storage si integra con la piattaforma OVHcloud tramite lo Spazio Cliente, l'API, la CLI e il provider Terraform, e con Kubernetes attraverso il driver Manila CSI, consentendo il provisioning dinamico dei volumi condivisi direttamente dal tuo cluster.

Perché utilizzare Manila CSI?

La Container Storage Interface (CSI) di Kubernetes fornisce un meccanismo standard per esporre sistemi di storage esterni a Kubernetes.

Invece di creare manualmente mount NFS e Persistent Volume, il driver Manila CSI permette a Kubernetes di generare e gestire dinamicamente i file share utilizzando risorse Kubernetes come StorageClasses (SC), PersistentVolumeClaims (PVC) e PersistentVolumes (PV).

Prerequisiti

Prima di iniziare, serve:

  • un progetto Public Cloud OVHcloud
  • un cluster Managed Kubernetes Service (MKS) di OVHcloud connesso a una rete privata
  • Terraform CLI installata
  • kubectl CLI installata

Deploy di Manila CSI passo dopo passo: iniziamo!

Disponiamo già di un cluster MKS nella Region EU-WEST-PAR, eseguito all'interno di una rete privata e una sottorete. In questo articolo vedremo come:

  • creare un utente Public Cloud per il driver Manila CSI
  • installare il driver CSI NFS
  • installare il driver Manila CSI
  • creare un Secret per Manila CSI che permetta al driver di autenticarsi in OpenStack e gestire le risorse Manila nel cluster
  • creare una rete condivisa di file
  • creare una ConfigMap per configurare il driver Manila CSI
  • installare csi-manila-nfs StorageClass per permettere al driver Manila CSI di creare dinamicamente share Manila e utilizzarle come volumi Kubernetes

Utilizzeremo Terraform per effettuare il deploy di questa architettura.

Crea un file provider.tf e inserisci le informazioni:

bash
terraform {
  required_providers {
    helm = {
      source = "hashicorp/helm"
    }

    kubectl = {
      source = "alekc/kubectl"
      version = "2.1.6"
    }

    ovh = {
      source = "ovh/ovh"
    }
  }
}

provider "helm" {
  kubernetes = {
    host                   = data.ovh_cloud_project_kube.mks_cluster.kubeconfig_attributes[0].host
    client_certificate     = base64decode(data.ovh_cloud_project_kube.mks_cluster.kubeconfig_attributes[0].client_certificate)
    client_key             = base64decode(data.ovh_cloud_project_kube.mks_cluster.kubeconfig_attributes[0].client_key)
    cluster_ca_certificate = base64decode(data.ovh_cloud_project_kube.mks_cluster.kubeconfig_attributes[0].cluster_ca_certificate)
  }
}

provider "kubectl" {
  host                   = data.ovh_cloud_project_kube.mks_cluster.kubeconfig_attributes[0].host
  client_certificate     = base64decode(data.ovh_cloud_project_kube.mks_cluster.kubeconfig_attributes[0].client_certificate)
  client_key             = base64decode(data.ovh_cloud_project_kube.mks_cluster.kubeconfig_attributes[0].client_key)
  cluster_ca_certificate = base64decode(data.ovh_cloud_project_kube.mks_cluster.kubeconfig_attributes[0].cluster_ca_certificate)
  load_config_file       = false
}

Configura le variabili d'ambiente per il provider OVHcloud Terraform con le tue credenziali:

bash
# OVHcloud provider needed keys
export OVH_ENDPOINT="ovh-eu"
export OVH_APPLICATION_KEY="xxx"
export OVH_APPLICATION_SECRET="xxx"
export OVH_CONSUMER_KEY="xxx"
export OVH_CLOUD_PROJECT_SERVICE="xxx"

Crea un file variables.tf.template e inserisci le seguenti informazioni:

bash
variable "service_name" {
  default = "$OVH_CLOUD_PROJECT_SERVICE"
}

variable "mks_cluster_id" {
  default = "<your_mks_cluster_id>"
}

⚠️ Nel file, sostituisci l'ID MKS con le informazioni del tuo ID cluster MKS esistente.

Sostituisci il valore della variabile d'ambiente OVH_CLOUD_PROJECT_SERVICE nel file variables.tf:

bash
envsubst < variables.tf.template > variables.tf

Crea un file nfs_share.tf e inserisci le seguenti informazioni:

markdown
data "ovh_cloud_project_kube" "mks_cluster" {
  service_name = var.service_name
  kube_id      = var.mks_cluster_id
}

data "ovh_cloud_network_private_vrack_subnet" "mks_cluster_subnet" {
  service_name = var.service_name
  network_id   = data.ovh_cloud_project_kube.mks_cluster.private_network_id
  id           = data.ovh_cloud_project_kube.mks_cluster.nodes_subnet_id
}

# CSI Manila

module "csi_manila" {
  source = "git::https://github.com/ovh/public-cloud-examples.git//containers-orchestration/managed-kubernetes/install-csi-manila/modules/ovhcloud/csi_manila?ref=v1.5.0"

  service_name       = var.service_name
  region             = data.ovh_cloud_project_kube.mks_cluster.region
  share_network_name = "${data.ovh_cloud_project_kube.mks_cluster.name}-share-network"
  network_id         = data.ovh_cloud_project_kube.mks_cluster.private_network_id
  subnet_id          = data.ovh_cloud_project_kube.mks_cluster.nodes_subnet_id
  subnet_cidr        = data.ovh_cloud_network_private_vrack_subnet.mks_cluster_subnet.cidr
}

output "manila-user" {
  value = module.csi_manila.manila-user
}

💡In questo file Terraform stiamo utilizzando un modulo Terraform csi_manila esistente ospitato nel repository GitHub OVHcloud Public Cloud Examples.

La configurazione Terraform è pronta. Eseguiamo l’init:

bash
terraform init

L'output restituito dovrebbe essere di questo tipo:

bash
$ terraform init

Initializing the backend...
Initializing modules...
Downloading git::https://github.com/ovh/public-cloud-examples.git?ref=v1.5.0 for csi_manila...
- csi_manila in .terraform/modules/csi_manila/containers-orchestration/managed-kubernetes/install-csi-manila/modules/ovhcloud/csi_manila
Initializing provider plugins...
- Reusing previous version of hashicorp/helm from the dependency lock file
- Reusing previous version of alekc/kubectl from the dependency lock file
- Reusing previous version of ovh/ovh from the dependency lock file
- Installing hashicorp/helm v3.2.0...
- Installed hashicorp/helm v3.2.0 (signed by HashiCorp)
- Installing alekc/kubectl v2.1.6...
- Installed alekc/kubectl v2.1.6 (self-signed, key ID 772FB27A86DAFCE7)
- Installing ovh/ovh v2.16.1...
- Installed ovh/ovh v2.16.1 (signed by a HashiCorp partner, key ID F56D1A6CBDAAADA5)
Partner and community providers are signed by their developers.
If you'd like to know more about provider signing, you can read about it here:
https://developer.hashicorp.com/terraform/cli/plugins/signing

Terraform has been successfully initialized!

You may now begin working with Terraform. Try running "terraform plan" to see
any changes that are required for your infrastructure. All Terraform commands
should now work.

If you ever set or change modules or backend configuration for Terraform,
rerun this command to reinitialize your working directory. If you forget, other
commands will detect it and remind you to do so if necessary.

Eseguiamo l’apply:

bash
terraform apply

L'output restituito dovrebbe essere di questo tipo:

bash
$ terraform apply

data.ovh_cloud_project_kube.mks_cluster: Reading...
data.ovh_cloud_project_kube.mks_cluster: Read complete after 1s [id=xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxx]
data.ovh_cloud_network_private_vrack_subnet.mks_cluster_subnet: Reading...
data.ovh_cloud_network_private_vrack_subnet.mks_cluster_subnet: Read complete after 1s [id=xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxx]

Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols:
  + create

Terraform will perform the following actions:

  # module.csi_manila.helm_release.csi-driver-nfs will be created
  + resource "helm_release" "csi-driver-nfs" {
      + atomic                     = false
      + chart                      = "csi-driver-nfs"
      + cleanup_on_fail            = false
      + create_namespace           = false
      + dependency_update          = false
      + disable_crd_hooks          = false
      + disable_openapi_validation = false
      + disable_webhooks           = false
      + force_update               = false
      + id                         = (known after apply)
      + lint                       = false
      + max_history                = 0
      + metadata                   = (known after apply)
      + name                       = "csi-driver-nfs"
      + namespace                  = "kube-system"
      + pass_credentials           = false
      + recreate_pods              = false
      + render_subchart_notes      = true
      + replace                    = false
      + repository                 = "https://raw.githubusercontent.com/kubernetes-csi/csi-driver-nfs/master/charts"
      + reset_values               = false
      + reuse_values               = false
      + set_wo                     = (write-only attribute)
      + skip_crds                  = false
      + status                     = "deployed"
      + take_ownership             = false
      + timeout                    = 300
      + upgrade_install            = false
      + verify                     = false
      + version                    = "4.13.4"
      + wait                       = true
      + wait_for_jobs              = false
    }

  # module.csi_manila.helm_release.openstack-manila-csi will be created
  + resource "helm_release" "openstack-manila-csi" {
      + atomic                     = false
      + chart                      = "openstack-manila-csi"
      + cleanup_on_fail            = false
      + create_namespace           = false
      + dependency_update          = false
      + disable_crd_hooks          = false
      + disable_openapi_validation = false
      + disable_webhooks           = false
      + force_update               = false
      + id                         = (known after apply)
      + lint                       = false
      + max_history                = 0
      + metadata                   = (known after apply)
      + name                       = "openstack-manila-csi"
      + namespace                  = "kube-system"
      + pass_credentials           = false
      + recreate_pods              = false
      + render_subchart_notes      = true
      + replace                    = false
      + repository                 = "https://kubernetes.github.io/cloud-provider-openstack"
      + reset_values               = false
      + reuse_values               = false
      + set_wo                     = (write-only attribute)
      + skip_crds                  = false
      + status                     = "deployed"
      + take_ownership             = false
      + timeout                    = 300
      + upgrade_install            = false
      + verify                     = false
      + version                    = "2.36.0"
      + wait                       = true
      + wait_for_jobs              = false
    }

  # module.csi_manila.kubectl_manifest.csi-manila-secrets will be created
  + resource "kubectl_manifest" "csi-manila-secrets" {
      + api_version             = (known after apply)
      + apply_only              = false
      + field_manager           = "kubectl"
      + force_conflicts         = false
      + force_new               = false
      + id                      = (known after apply)
      + kind                    = (known after apply)
      + live_manifest_incluster = (sensitive value)
      + live_uid                = (known after apply)
      + name                    = (known after apply)
      + namespace               = (known after apply)
      + server_side_apply       = false
      + uid                     = (known after apply)
      + validate_schema         = true
      + wait_for_rollout        = true
      + yaml_body               = (sensitive value)
      + yaml_body_parsed        = (known after apply)
      + yaml_incluster          = (sensitive value)
    }

  # module.csi_manila.kubectl_manifest.manila-runtime-configmap will be created
  + resource "kubectl_manifest" "manila-runtime-configmap" {
      + api_version             = "v1"
      + apply_only              = false
      + field_manager           = "kubectl"
      + force_conflicts         = false
      + force_new               = false
      + id                      = (known after apply)
      + kind                    = "ConfigMap"
      + live_manifest_incluster = (sensitive value)
      + live_uid                = (known after apply)
      + name                    = "manila-csi-runtimeconf-cm"
      + namespace               = "default"
      + server_side_apply       = false
      + uid                     = (known after apply)
      + validate_schema         = true
      + wait_for_rollout        = true
      + yaml_body               = (sensitive value)
      + yaml_body_parsed        = <<-EOT
            apiVersion: v1
            data:
              runtimeconfig.json: |
                {
                  "nfs": {
                    "matchExportLocationAddress": "10.1.0.0/16"
                  }
                }
            kind: ConfigMap
            metadata:
              annotations:
                meta.helm.sh/release-name: manila-csi
                meta.helm.sh/release-namespace: default
              labels:
                app.kubernetes.io/managed-by: Helm
              name: manila-csi-runtimeconf-cm
              namespace: default
        EOT
      + yaml_incluster          = (sensitive value)
    }

  # module.csi_manila.kubectl_manifest.storage-class will be created
  + resource "kubectl_manifest" "storage-class" {
      + api_version             = (known after apply)
      + apply_only              = false
      + field_manager           = "kubectl"
      + force_conflicts         = false
      + force_new               = false
      + id                      = (known after apply)
      + kind                    = (known after apply)
      + live_manifest_incluster = (sensitive value)
      + live_uid                = (known after apply)
      + name                    = (known after apply)
      + namespace               = (known after apply)
      + server_side_apply       = false
      + uid                     = (known after apply)
      + validate_schema         = true
      + wait_for_rollout        = true
      + yaml_body               = (sensitive value)
      + yaml_body_parsed        = (known after apply)
      + yaml_incluster          = (sensitive value)
    }

  # module.csi_manila.ovh_cloud_project_user.manila-user will be created
  + resource "ovh_cloud_project_user" "manila-user" {
      + creation_date = (known after apply)
      + description   = "User for the Manila CSI driver"
      + id            = (known after apply)
      + openstack_rc  = (known after apply)
      + password      = (sensitive value)
      + role_name     = "share_operator"
      + roles         = (known after apply)
      + service_name  = "xxxxxxxxxxxxxxxxxxxxx"
      + status        = (known after apply)
      + username      = (known after apply)
    }

  # module.csi_manila.ovh_cloud_storage_file_share_network.sharenetwork will be created
  + resource "ovh_cloud_storage_file_share_network" "sharenetwork" {
      + checksum        = (known after apply)
      + created_at      = (known after apply)
      + current_state   = (known after apply)
      + description     = (known after apply)
      + id              = (known after apply)
      + name            = "mks_standard_3az-share-network"
      + network_id      = "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxx"
      + region          = "EU-WEST-PAR"
      + resource_status = (known after apply)
      + service_name    = "xxxxxxxxxxxxxxxxxxxxx"
      + subnet_id       = "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxx"
      + updated_at      = (known after apply)
    }

Plan: 7 to add, 0 to change, 0 to destroy.

Changes to Outputs:
  + manila-user = (known after apply)

Do you want to perform these actions?
  Terraform will perform the actions described above.
  Only 'yes' will be accepted to approve.

  Enter a value: yes

module.csi_manila.ovh_cloud_storage_file_share_network.sharenetwork: Creating...
module.csi_manila.ovh_cloud_project_user.manila-user: Creating...
module.csi_manila.kubectl_manifest.manila-runtime-configmap: Creating...
module.csi_manila.kubectl_manifest.manila-runtime-configmap: Creation complete after 0s [id=/api/v1/namespaces/default/configmaps/manila-csi-runtimeconf-cm]
module.csi_manila.helm_release.csi-driver-nfs: Creating...
module.csi_manila.ovh_cloud_storage_file_share_network.sharenetwork: Still creating... [00m10s elapsed]
module.csi_manila.ovh_cloud_project_user.manila-user: Still creating... [00m10s elapsed]
module.csi_manila.ovh_cloud_storage_file_share_network.sharenetwork: Creation complete after 12s [id=xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxx]
module.csi_manila.kubectl_manifest.storage-class: Creating...
module.csi_manila.ovh_cloud_project_user.manila-user: Creation complete after 13s [id=718526]
module.csi_manila.kubectl_manifest.csi-manila-secrets: Creating...
module.csi_manila.kubectl_manifest.csi-manila-secrets: Creation complete after 1s [id=/api/v1/namespaces/default/secrets/csi-manila-secrets]
...

Creare dinamicamente File Storage su Kubernetes

Nel tuo cluster Kubernetes, crea un file pvc.yaml con questo contenuto:

yaml
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: nfs-share-fs-pvc
spec:
  accessModes:
    - ReadWriteMany
  resources:
    requests:
      storage: 150Gi
  storageClassName: csi-manila-nfs

Applica questo Persistent Volume Claim (PVC), che permette agli utenti Kubernetes di richiedere lo storage condiviso:

bash
kubectl apply -f pvc.yaml

Controlla lo stato del PVC:

bash
kubectl get pvc nfs-share-fs-pvc

Attendi che lo stato del PVC cambi in Bound.

L'output restituito dovrebbe essere di questo tipo:

bash
$ kubectl get pvc nfs-share-fs-pvc

NAME               STATUS   VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS     VOLUMEATTRIBUTESCLASS   AGE
nfs-share-fs-pvc   Bound    pvc-af1349b7-eb51-43ac-b0f0-94d0e8139c57   150Gi      RWX            csi-manila-nfs   <unset>                 3h48m

Crea un file deploy.yaml e inserisci il seguente contenuto:

yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-deployment
  labels:
    app: nginx
spec:
  replicas: 1
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      labels:
        app: nginx
    spec:
      volumes:
      - name: nfs-share-fs-pvc
        persistentVolumeClaim:
          claimName: nfs-share-fs-pvc
      containers:
      - name: nginx
        image: nginx
        ports:
          - containerPort: 80
            name: "http-server"
        volumeMounts:
          - mountPath: "/usr/share/nginx/html"
            name: nfs-share-fs-pvc

Applica questo deployment con un pod che ha un volume associato a nfs-share-fs-pvc:

bash
kubectl apply -f deploy.yaml

Verifica che il pod sia in esecuzione:

bash
kubectl get pod

L'output restituito dovrebbe essere di questo tipo:

bash
$ kubectl get pod
NAME                                READY   STATUS    RESTARTS   AGE
nginx-deployment-6cbf9b898c-svpzf   1/1     Running   0          6m

Verifica di poter scalare il deployment (volume multi-attach):

bash
kubectl scale deploy/nginx-deployment --replicas=2

Verifica che un altro pod sia in esecuzione:

bash
kubectl get pod

L'output restituito dovrebbe essere di questo tipo:

bash
$ kubectl get pod

NAME                                READY   STATUS    RESTARTS   AGE
nginx-deployment-6cbf9b898c-867bq   1/1     Running   0          3m
nginx-deployment-6cbf9b898c-svpzf   1/1     Running   0          9m

Per verificare la funzionalità RWX, connettiti a un pod e crea un file nella directory montata (per es., /usr/share/nginx/html).

bash
MY_POD=$(kubectl get po -o name | sed -n '1p')
echo $MY_POD

#Create a file in the pod number 1
kubectl exec $MY_POD -it -- touch /usr/share/nginx/html/index.html

Quindi connettiti al secondo pod e conferma che il file sia visibile:

bash
MY_POD_2=$(kubectl get po -o name | sed -n '2p')
echo $MY_POD_2

#Display it in the pod number two
kubectl exec $MY_POD_2 -it -- ls -alrt /usr/share/nginx/html/

L'output restituito dovrebbe essere di questo tipo:

bash
$ kubectl exec $MY_POD_2 -it -- ls -alrt /usr/share/nginx/html/

total 24
drwxr-xr-x 3 root root  4096 Jul 14 01:22 ..
drwx------ 2 root root 16384 Jul 15 09:18 lost+found
drwxrwxrwx 3 root root  4096 Jul 15 09:32 .
-rw-r--r-- 1 root root     0 Jul 15 13:09 index.html

La tua condivisione Manila esposta tramite NFS funziona! 🎉

Conclusioni

In questo articolo abbiamo mostrato che utilizzare Manila CSI con OVHcloud File Storage permette di fornire alle applicazioni Kubernetes uno storage condiviso con provisioning dinamico.

Consigliamo di dare un'occhiata anche alla nostra Cloud Roadmap & Changelog per conoscere tutte le funzionalità in arrivo sulle soluzioni Public Cloud di OVHcloud.

Condividi su::