Configure a Pod to Use a PersistentVolume for Storage
how to configure a Pod to use a PersistentVolumeClaim for storage.
Here is a summary of the process:
-
A cluster administrator creates a PersistentVolume that is backed by physical storage. The administrator does not associate the volume with any Pod.
-
A cluster user creates a PersistentVolumeClaim, which gets automatically bound to a suitable PersistentVolume.
-
The user creates a Pod that uses the PersistentVolumeClaim as storage.
Create a PersistentVolume
Kubernetes supports hostPath for development and testing on a single-node cluster.
A hostPath PersistentVolume uses a file or directory on the Node to emulate network-attached storage.
In a production cluster, you would not use hostPath.
Instead a cluster administrator would provision a network resource like a Google Compute Engine persistent disk, an NFS share, or an Amazon Elastic Block Store volume.
Cluster administrators can also use StorageClasses to set up dynamic provisioning.
#create file #Open a shell to the Node in your cluster. #create a /mnt/data directory mkdir /mnt/data #In the /mnt/data directory,create an index.html file echo 'Hello from Kubernetes storage' > /mnt/data/index.html
kind: PersistentVolume apiVersion: v1 metadata: name: task-pv-volume labels: type: local spec: storageClassName: manual capacity: storage: 10Gi accessModes: - ReadWriteOnce hostPath: path: "/mnt/data"
configuration file for the hostPath PersistentVolume
The configuration file specifies that the volume is at /mnt/data
on the cluster’s Node.
The configuration also specifies a size of 10 gibibytes and an access mode of ReadWriteOnce
, which means the volume can be mounted as read-write by a single Node.
It defines the StorageClass name manual
for the PersistentVolume, which will be used to bind PersistentVolumeClaim requests to this PersistentVolume.
#get PersistentVolumnClaim kubectl get pv task-pv-volume
The output shows that the PersistentVolume has a STATUS
of Available
. This means it has not yet been bound to a PersistentVolumeClaim.
Create a PersistentVolumeClaim
Pods use PersistentVolumeClaims to request physical storage.
kind: PersistentVolumeClaim apiVersion: v1 metadata: name: task-pv-claim spec: storageClassName: manual accessModes: - ReadWriteOnce resources: requests: storage: 3Gi
configuration file for the PersistentVolumeClaim
create a PersistentVolumeClaim that requests a volume of at least three gibibytes that can provide read-write access for at least one Node.
After you create the PersistentVolumeClaim, the Kubernetes control plane looks for a PersistentVolume that satisfies the claim’s requirements.
If the control plane finds a suitable PersistentVolume with the same StorageClass, it binds the claim to the volume.
#Look again at the PersistentVolume: kubectl get pv task-pv-volume # a STATUS of Bound. #Look at the PersistentVolumeClaim: kubectl get pvc task-pv-claim # shows that the PersistentVolumeClaim is bound to your PersistentVolume, task-pv-volume.
Create a Pod
create a Pod that uses your PersistentVolumeClaim as a volume.
kind: Pod apiVersion: v1 metadata: name: task-pv-pod spec: volumes: - name: task-pv-storage persistentVolumeClaim: claimName: task-pv-claim containers: - name: task-pv-container image: nginx ports: - containerPort: 80 name: "http-server" volumeMounts: - mountPath: "/usr/share/nginx/html" name: task-pv-storage
the Pod’s configuration file specifies a PersistentVolumeClaim, but it does not specify a PersistentVolume.
From the Pod’s point of view, the claim is a volume.
#Verify that the Container in the Pod is running; kubectl get pod task-pv-pod #Get a shell to the Container running in your Pod: kubectl exec -it task-pv-pod -- /bin/bash #In your shell, verify that nginx is serving the index.html file from the hostPath volume: root@task-pv-pod:/# apt-get update root@task-pv-pod:/# apt-get install curl root@task-pv-pod:/# curl localhost #The output shows the text that you wrote to the index.html file on the hostPath volume: Hello from Kubernetes storage
Access control
Storage configured with a group ID (GID)
allows writing only by Pods using the same GID.
Mismatched or missing GIDs cause permission denied errors.
To reduce the need for coordination with users, an administrator can annotate a PersistentVolume with a GID.
Then the GID is automatically added to any Pod that uses the PersistentVolume.
kind: PersistentVolume apiVersion: v1 metadata: name: pv1 annotations: pv.beta.kubernetes.io/gid: "1234"
Use the pv.beta.kubernetes.io/gid
annotation
When a Pod consumes a PersistentVolume that has a GID annotation, the annotated GID is applied to all Containers in the Pod in the same way that GIDs specified in the Pod’s security context are.
Every GID, whether it originates from a PersistentVolume annotation or the Pod’s specification, is applied to the first process run in each Container.
Configure a Pod to Use a Projected Volume for Storage
how to use a projected
volume to mount several existing volume sources into the same directory.
Currently, secret
, configMap
, and downwardAPI
volumes can be projected.
Configure a projected volume for a pod
apiVersion: v1 kind: Pod metadata: name: test-projected-volume spec: containers: - name: test-projected-volume image: busybox args: - sleep - "86400" volumeMounts: - name: all-in-one mountPath: "/projected-volume" readOnly: true volumes: - name: all-in-one projected: sources: - secret: name: user - secret: name: pass
create username and password Secrets from local files.
create a Pod that runs one Container, using a projected
Volume to mount the Secrets into the same shared directory.
#Create the Secrets: # Create files containing the username and password: echo -n "admin" > ./username.txt echo -n "1f2d1e2e67df" > ./password.txt # Package these files into secrets: kubectl create secret generic user --from-file=./username.txt kubectl create secret generic pass --from-file=./password.txt #Create the Pod: kubectl create -f projected-volume.yaml #Verify that the Pod’s Container is running, and then watch for changes to the Pod: kubectl get --watch pod test-projected-volume #In another terminal, get a shell to the running Container: kubectl exec -it test-projected-volume -- /bin/sh #In your shell, verify that the projected-volume directory contains your projected sources: ls /projected-volume/
Configure a Security Context for a Pod or Container
A security context defines privilege and access control settings for a Pod or Container.
Security context settings include:
-
Discretionary Access Control: Permission to access an object, like a file, is based on user ID (UID) and group ID (GID).
-
Security Enhanced Linux (SELinux): Objects are assigned security labels.
-
Running as privileged or unprivileged.
-
Linux Capabilities: Give a process some privileges, but not all the privileges of the root user.
-
AppArmor: Use program profiles to restrict the capabilities of individual programs.
-
Seccomp: Filter a process’s system calls.
-
AllowPrivilegeEscalation: Controls whether a process can gain more privileges than its parent process. This bool directly controls whether the
no_new_privs
flag gets set on the container process. AllowPrivilegeEscalation is true always when the container is: 1) run as Privileged OR 2) hasCAP_SYS_ADMIN
.
Set the security context for a Pod
apiVersion: v1 kind: Pod metadata: name: security-context-demo spec: securityContext: runAsUser: 1000 fsGroup: 2000 volumes: - name: sec-ctx-vol emptyDir: {} containers: - name: sec-ctx-demo image: gcr.io/google-samples/node-hello:1.0 volumeMounts: - name: sec-ctx-vol mountPath: /data/demo securityContext: allowPrivilegeEscalation: false
To specify security settings for a Pod, include the securityContext
field in the Pod specification.
The securityContext
field is a PodSecurityContext object.
The security settings that you specify for a Pod apply to all Containers in the Pod.
a configuration file for a Pod that has a securityContext
and an emptyDir
volume
In the configuration file, the runAsUser
field specifies that for any Containers in the Pod, the first process runs with user ID 1000.
The fsGroup
field specifies that group ID 2000 is associated with all Containers in the Pod.
Group ID 2000 is also associated with the volume mounted at /data/demo
and with any files created in that volume.
kubectl exec -it security-context-demo -- sh #list the running processes: ps aux #The output shows that the processes are running as user 1000, which is the value of runAsUser: USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND 1000 1 0.0 0.0 4336 724 ? Ss 18:16 0:00 /bin/sh -c node server.js 1000 5 0.2 0.6 772124 22768 ? Sl 18:16 0:00 node server.js ... #navigate to /data, and list the one directory: cd /data ls -l # shows that the /data/demo directory has group ID 2000, which is the value of fsGroup drwxrwsrwx 2 root 2000 4096 Jun 6 20:08 demo #navigate to /data/demo, and create a file: cd demo echo hello > testfile #List the file in the /data/demo directory: ls -l #The output shows that testfile has group ID 2000, which is the value of fsGroup. -rw-r--r-- 1 1000 2000 6 Jun 6 20:08 testfile #Exit your shell: exit
Set the security context for a Container
apiVersion: v1 kind: Pod metadata: name: security-context-demo-2 spec: securityContext: runAsUser: 1000 containers: - name: sec-ctx-demo-2 image: gcr.io/google-samples/node-hello:1.0 securityContext: runAsUser: 2000 allowPrivilegeEscalation: false
To specify security settings for a Container, include the securityContext
field in the Container manifest. The securityContext
field is a SecurityContext object.
Security settings that you specify for a Container apply only to the individual Container, and they override settings made at the Pod level when there is overlap.
Container settings do not affect the Pod’s Volumes.
the configuration file for a Pod that has one Container. Both the Pod and the Container have a securityContext
field:
#Get a shell into the running Container: kubectl exec -it security-context-demo-2 -- sh #In your shell, list the running processes: ps aux #The output shows that the processes are running as user 2000. This is the value of runAsUser specified for the Container. It overrides the value 1000 that is specified for the Pod. USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND 2000 1 0.0 0.0 4336 764 ? Ss 20:36 0:00 /bin/sh -c node server.js 2000 8 0.1 0.5 772124 22604 ? Sl 20:36 0:00 node server.js ...
Set capabilities for a Container
With Linux capabilities, you can grant certain privileges to a process without granting all the privileges of the root user.
To add or remove Linux capabilities for a Container, include the capabilities
field in the securityContext
section of the Container manifest.
apiVersion: v1 kind: Pod metadata: name: security-context-demo-3 spec: containers: - name: sec-ctx-3 image: gcr.io/google-samples/node-hello:1.0
First, see what happens when you don’t include a capabilities
field.
Here is configuration file that does not add or remove any Container capabilities:
kubectl exec -it security-context-demo-3 -- sh ps aux #The output shows the process IDs (PIDs) for the Container: USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND root 1 0.0 0.0 4336 796 ? Ss 18:17 0:00 /bin/sh -c node server.js root 5 0.1 0.5 772124 22700 ? Sl 18:17 0:00 node server.js #In your shell, view the status for process 1: cd /proc/1 cat status #The output shows the capabilities bitmap for the process: ... CapPrm: 00000000a80425fb CapEff: 00000000a80425fb ... #Make a note of the capabilities bitmap, and then exit your shell: exit
Next, run a Container that is the same as the preceding container, except that it has additional capabilities set.
Here is the configuration file for a Pod that runs one Container. The configuration adds the CAP_NET_ADMIN
and CAP_SYS_TIME
capabilities:
apiVersion: v1 kind: Pod metadata: name: security-context-demo-4 spec: containers: - name: sec-ctx-4 image: gcr.io/google-samples/node-hello:1.0 securityContext: capabilities: add: ["NET_ADMIN", "SYS_TIME"]
kubectl exec -it security-context-demo-4 -- sh #In your shell, view the capabilities for process 1: cd /proc/1 cat status #The output shows capabilities bitmap for the process: ... CapPrm: 00000000aa0435fb CapEff: 00000000aa0435fb ... #Compare the capabilities of the two Containers: 00000000a80425fb 00000000aa0435fb
In the capability bitmap of the first container, bits 12 and 25 are clear.
In the second container, bits 12 and 25 are set. Bit 12 is CAP_NET_ADMIN
, and bit 25 is CAP_SYS_TIME
.
See capability.h for definitions of the capability constants.
Note:
Linux capability constants have the form CAP_XXX
.
But when you list capabilities in your Container manifest, you must omit the CAP_
portion of the constant.
For example, to add CAP_SYS_TIME
, include SYS_TIME
in your list of capabilities.
Assign SELinux labels to a Container
To assign SELinux labels to a Container, include the seLinuxOptions
field in the securityContext
section of your Pod or Container manifest.
The seLinuxOptions
field is an SELinuxOptions object.
Here’s an example that applies an SELinux level:
... securityContext: seLinuxOptions: level: "s0:c123,c456"
Note:
To assign SELinux labels, the SELinux security module must be loaded on the host operating system.
The security context for a Pod applies to the Pod’s Containers and also to the Pod’s Volumes when applicable.
Specifically fsGroup
and seLinuxOptions
are applied to Volumes as follows:
-
fsGroup
: Volumes that support ownership management are modified to be owned and writable by the GID specified infsGroup
. See the Ownership Management design document for more details. -
seLinuxOptions
: Volumes that support SELinux labeling are relabeled to be accessible by the label specified underseLinuxOptions
. Usually you only need to set thelevel
section. This sets the Multi-Category Security (MCS) label given to all Containers in the Pod as well as the Volumes.