K8S 实战（六）| 配置 NFS 动态卷提供持久化存储-六虎

本节中 K8S 运用 NFS 长途存储，为保管的 pod 提供了动态存储服务，pod 创立者无需关怀数据以何种方法存在哪里，只需求提出需求多大空间的请求即可。

总体流程是：

创立 NFS 服务器。
创立 Service Account。用来管控 NFS provisioner 在k8s集群中运转的权限。
创立 StorageClass。担任创立 PVC 并调用 NFS provisioner 进行预定的作业，并相关 PV 和 PVC。
创立 NFS provisioner。有两个功用,一个是在NFS同享目录下创立挂载点(volume),二是建立 PV 并将 PV 与 NFS 挂载点建立相关。

装备NFS服务器

server ip: 192.168.10.17

[root@work03 ~]# yum install nfs-utils rpcbind -y
[root@work03 ~]# systemctl start nfs
[root@work03 ~]# systemctl start rpcbind
[root@work03 ~]# systemctl enable nfs
[root@work03 ~]# systemctl enable rpcbind
[root@work03 ~]# mkdir -p /data/nfs/
[root@work03 ~]# chmod 777 /data/nfs/
[root@work03 ~]# cat /etc/exports
/data/nfs/    192.168.10.0/24(rw,sync,no_root_squash,no_all_squash)
[root@work03 ~]# exportfs -arv
exporting 192.168.10.0/24:/data/nfs
[root@work03 ~]# showmount -e localhost
Export list for localhost:
/data/nfs 192.168.10.0/24
参数:
sync：将数据同步写入内存缓冲区与磁盘中，功率低，但能够保证数据的一致性
async：将数据先保存在内存缓冲区中，必要时才写入磁盘

所有work节点装置 nfs-utils rpcbind

yum install nfs-utils rpcbind -y
systemctl start nfs
systemctl start rpcbind
systemctl enable nfs
systemctl enable rpcbind

创立动态卷提供者

创立RBAC授权

# wget https://raw.githubusercontent.com/kubernetes-incubator/external-storage/master/nfs-client/deploy/rbac.yaml
# kubectl apply -f rbac.yaml

创立 Storageclass

# cat class.yaml
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: managed-nfs-storage
provisioner: fuseim.pri/ifs # or choose another name, must match deployment's env PROVISIONER_NAME'
parameters:
  archiveOnDelete: "false"

创立nfs-client-provisioner主动装备程序，以便主动创立持久卷(PV)

主动创立的 PV 以 namespace−namespace−

{pvcName}-${pvName} 的命名格局创立在 NFS 上
当这个 PV 被回收后会以 archieved-namespace−namespace−

{pvcName}-${pvName} 的命名格局存在 NFS 服务器上

cat deployment.yaml

apiVersion: apps/v1
kind: Deployment
metadata:
name: nfs-client-provisioner
labels:
app: nfs-client-provisioner

replace with namespace where provisioner is deployed

namespace: default
spec:
replicas: 1
strategy:
type: Recreate
selector:
matchLabels:
app: nfs-client-provisioner
template:
metadata:
labels:
app: nfs-client-provisioner
spec:
serviceAccountName: nfs-client-provisioner
containers:
– name: nfs-client-provisioner
image: quay.io/external_storage/nfs-client-provisioner:latest
volumeMounts:
– name: nfs-client-root
mountPath: /persistentvolumes
env:
– name: PROVISIONER_NAME
value: fuseim.pri/ifs
– name: NFS_SERVER
value: 192.168.10.17
– name: NFS_PATH
value: /data/nfs
volumes:
– name: nfs-client-root
nfs:
server: 192.168.10.17
path: /data/nfs

创立一个有状况应用

# cat statefulset-nfs.yaml
apiVersion: v1
kind: Service
metadata:
  name: nginx
  labels:
    app: nginx
spec:
  ports:
  - port: 80
    name: web
  clusterIP: None
  selector:
    app: nginx
---
apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: nfs-web
spec:
  serviceName: "nginx"
  replicas: 3
  selector:
    matchLabels:
      app: nfs-web # has to match .spec.template.metadata.labels
  template:
    metadata:
      labels:
        app: nfs-web
    spec:
      terminationGracePeriodSeconds: 10
      containers:
      - name: nginx
        image: nginx:1.7.9
        ports:
        - containerPort: 80
          name: web
        volumeMounts:
        - name: www
          mountPath: /usr/share/nginx/html
  volumeClaimTemplates:
  - metadata:
      name: www
      annotations:
        volume.beta.kubernetes.io/storage-class: managed-nfs-storage
    spec:
      accessModes: [ "ReadWriteOnce" ]
      resources:
        requests:
          storage: 1Gi
[root@master01 ~]# kubectl apply -f statefulset-nfs.yaml

检查 Pod/PV/PVC

[root@master01 ~]# kubectl get pods
NAME                                      READY   STATUS    RESTARTS   AGE
nfs-client-provisioner-5f5fff65ff-2pmxh   1/1     Running   0          26m
nfs-web-0                                 1/1     Running   0          2m33s
nfs-web-1                                 1/1     Running   0          2m27s
nfs-web-2                                 1/1     Running   0          2m21s
[root@master01 ~]# kubectl get pvc
NAME            STATUS   VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS          AGE
www-nfs-web-0   Bound    pvc-62f4868f-c6f7-459e-a280-26010c3a5849   1Gi        RWO            managed-nfs-storage   2m35s
www-nfs-web-1   Bound    pvc-47b68872-35f2-4d3b-bc70-fc59d3bcdbf9   1Gi        RWO            managed-nfs-storage   2m29s
www-nfs-web-2   Bound    pvc-0af3ac53-56d9-4526-8c60-eb0ce3f281e0   1Gi        RWO            managed-nfs-storage   2m23s
[root@master01 ~]# kubectl get pv
NAME                                       CAPACITY   ACCESS MODES   RECLAIM POLICY   STATUS   CLAIM                   STORAGECLASS          REASON   AGE
pvc-0af3ac53-56d9-4526-8c60-eb0ce3f281e0   1Gi        RWO            Delete           Bound    default/www-nfs-web-2   managed-nfs-storage            2m25s
pvc-47b68872-35f2-4d3b-bc70-fc59d3bcdbf9   1Gi        RWO            Delete           Bound    default/www-nfs-web-1   managed-nfs-storage            2m31s
pvc-62f4868f-c6f7-459e-a280-26010c3a5849   1Gi        RWO            Delete           Bound    default/www-nfs-web-0   managed-nfs-storage            2m36s

检查 nfs server 目录中信息，同时各子目录中内容为空

[root@work03 ~]# ls -l /data/nfs/
total 12
default-www-nfs-web-0-pvc-62f4868f-c6f7-459e-a280-26010c3a5849
default-www-nfs-web-1-pvc-47b68872-35f2-4d3b-bc70-fc59d3bcdbf9
default-www-nfs-web-2-pvc-0af3ac53-56d9-4526-8c60-eb0ce3f281e0

破坏性测试

将每个 pod 中写入内容

[root@master01 ~]# for i in 0 1 2; do kubectl exec nfs-web-$i -- sh -c 'echo $(hostname) > /usr/share/nginx/html/index.html'; done

长途nfs各子目录中不再为空，出现了内容

[root@work03 ~]# ls /data/nfs/default-www-nfs-web-0-pvc-62f4868f-c6f7-459e-a280-26010c3a5849/            
index.html
[root@work03 ~]#

检查每个容器中内容，均为各自主机名

[root@master01 ~]# for i in 0 1 2; do kubectl exec -it nfs-web-$i -- cat /usr/share/nginx/html/index.html; done
nfs-web-0
nfs-web-1
nfs-web-2

删除对应 pod

[root@master01 ~]# kubectl get pod -l app=nfs-web
NAME        READY   STATUS    RESTARTS   AGE
nfs-web-0   1/1     Running   0          7m7s
nfs-web-1   1/1     Running   0          7m3s
nfs-web-2   1/1     Running   0          7m
[root@master01 ~]# kubectl delete pod -l app=nfs-web   
pod "nfs-web-0" deleted
pod "nfs-web-1" deleted
pod "nfs-web-2" deleted

能够看到又被主动创立了

[root@master01 ~]# kubectl get pod -l app=nfs-web
NAME        READY   STATUS    RESTARTS   AGE
nfs-web-0   1/1     Running   0          15s
nfs-web-1   1/1     Running   0          11s
nfs-web-2   1/1     Running   0          8s

再次检查每个pod中内容，能够看到文件内容没有变化

[root@master01 ~]# for i in 0 1 2; do kubectl exec -it nfs-web-$i -- cat /usr/share/nginx/html/index.html; done
nfs-web-0
nfs-web-1
nfs-web-2

总结

能够看到， statefulset 控制器经过固定的 pod 创立顺序能够保证 pod 之间的拓扑联系一向处于稳定不变的状况，经过 nfs-client-provisioner 主动创立和每个 pod 有固定对应联系的长途存储卷，保证 pod 重建后数据不会丢失

K8S 实战（六）| 配置 NFS 动态卷提供持久化存储

装备NFS服务器

创立动态卷提供者

创立RBAC授权

创立 Storageclass

创立nfs-client-provisioner主动装备程序，以便主动创立持久卷(PV)

cat deployment.yaml

replace with namespace where provisioner is deployed

创立一个有状况应用

破坏性测试

总结

相关文章

写给android开发的Linux 信号 – 上篇

Android Fragment 要你何用？2.0版本

CodeWhisperer 的正确使用

跳槽、换房、坚持输出，与你分享我匆忙的 2022～

作者信息