Use the securityContext of a pod and of its containers to control the user, the Linux capabilities and the filesystem access of the processes, and understand the risks of the host namespaces.
The lab works on a cluster shared by the trainees: each one works in a namespace of their own. On OpenShift, the SCCs reject or modify most of these pods: compare with the SCC lab.
Prerequisites
# The shared kind cluster (on the CKS training server: kind export kubeconfig, then kubectx kind-kind)kubectx kind-kind
NS=security-context-$(whoami)kubectl create namespace $NS
kubectl config set-context --current --namespace=$NS
All the pods of the lab run sleep in an alpine:3.19 container.
Exercise 1: the default user
Run a pod pod-with-defaults without any security context. Which user runs the process?
Solution
kubectl run pod-with-defaults --image=alpine:3.19 --restart=Never -- sleep 999999kubectl wait --for=condition=Ready pod/pod-with-defaults
kubectl exec pod-with-defaults -- id
# uid=0(root) gid=0(root) groups=0(root),1(bin),2(daemon),...
Without a security context, the container runs with the user of the image, here root. It is the root user of the node, only restricted by the container isolation: a container escape gives root on the node.
Exercise 2: run as a given user
Run a pod pod-as-user-guest whose container runs as the user guest of the alpine image (uid 405). Then run a pod pod-run-as-non-root with runAsNonRoot: true and no user. What happens?
Solution
cat <<EOF | kubectl apply -f -
apiVersion: v1
kind: Pod
metadata:
name: pod-as-user-guest
spec:
containers:
- name: main
image: alpine:3.19
command: ["/bin/sleep", "999999"]
securityContext:
runAsUser: 405
---
apiVersion: v1
kind: Pod
metadata:
name: pod-run-as-non-root
spec:
containers:
- name: main
image: alpine:3.19
command: ["/bin/sleep", "999999"]
securityContext:
runAsNonRoot: true
EOFkubectl wait --for=condition=Ready pod/pod-as-user-guest
kubectl exec pod-as-user-guest -- id
# uid=405(guest) gid=100(users) groups=100(users)sleep 10kubectl get pod pod-run-as-non-root
# STATUS: CreateContainerConfigErrorkubectl describe pod pod-run-as-non-root | grep -i "non-root"# Error: container has runAsNonRoot and image will run as root
runAsNonRoot does not choose a user: the kubelet refuses to start the container if its user is root. Set runAsUser too, or use an image whose USER is not root.
Exercise 3: privileged container
Run a pod pod-privileged with privileged: true. Compare the devices visible in /dev with pod-with-defaults.
A privileged container sees all the devices of the node (disks, …) and has all the capabilities: it can mount the disk of the node and read or change any file. Only some system pods need it (CNI, CSI drivers).
Exercise 4: Linux capabilities
In pod-with-defaults, change the owner of /tmp to guest. Is it allowed?
Run a pod pod-drop-chown-capability which drops the CHOWN capability, and try again.
Even as root, a process can only do what its capabilities allow. The container runtime gives a default set (CHOWN, NET_BIND_SERVICE, KILL, …): the good practice is drop: ["ALL"], then add only what the application needs.
capabilities.add: ["SYS_TIME"] would allow the container to change the clock… of the node, shared by all its pods: do not try it on a shared cluster.
Exercise 5: read-only root filesystem
Run a pod pod-with-readonly-filesystem whose root filesystem is read-only, with an emptyDir volume mounted on /volume. Where can the container write?
An attacker cannot install tools or change the binaries of the image: the container is immutable at runtime (CKS domain “Monitoring, logging and runtime security”). The application writes only in the volumes.
Exercise 6: share a volume between users
Run a pod pod-with-shared-volume-fsgroup with two containers, first (uid 1111) and second (uid 2222), sharing an emptyDir volume, with fsGroup: 555 at the pod level. Which group owns the files that first writes in the volume, and in /tmp?
The volume belongs to the group fsGroup (555), and so do the files created in it: both containers can share them. Outside of the volume, the files belong to the primary group of the user (0, or runAsGroup if set).
Exercise 7: host namespaces
Look at these two pods, without running them. What can their containers see?
hostNetwork: the container uses the network interfaces of the node (ifconfig lists them all), it can listen on any port of the node and bypasses the NetworkPolicies.
hostPID and hostIPC: the container sees all the processes of the node (ps aux lists the kubelet, the other containers…), with their command lines, which may contain secrets.
These fields are forbidden by the baseline level of the Pod Security Standards: only some infrastructure pods (CNI, monitoring agents) need them.