Install the gVisor runtime (runsc) on the nodes, and declare it to containerd
Run a pod in a sandbox with a RuntimeClass, and see the difference with a normal pod
A normal container shares the kernel of the node: a kernel exploit from a container gives the node. gVisor runs the container on top of a kernel written in Go (the Sentry), which runs in user space and handles the system calls of the container: the kernel of the node only sees a few, filtered system calls of the Sentry.
In short: gVisor is a Linux kernel rewritten in Go, which runs as a plain process, and only asks the real kernel for basic services: memory, file descriptors, network packets. The pipes, the processes, the signals and the TCP/IP stack of the container live inside the Sentry: a kernel exploit such as Dirty Pipe (splice() writing into read-only files) never reaches the code of the node’s kernel.
The CKS curriculum lists “Understand and implement isolation techniques (multi-tenancy, sandboxed containers, etc.)” in the Minimize microservice vulnerabilities domain, and “Verify platform binaries before deploying” in the Cluster setup domain.
Prerequisites
The lab installs a runtime on the nodes: use a kind cluster of your own, never a shared one.
# On the CKS training server, use your own cluster ($CLUSTER_NAME),# elsewhere create a cluster for the labCLUSTER=${CLUSTER_NAME:-gvisor-$(whoami)}kind get clusters | grep -qx $CLUSTER || kind create cluster --name $CLUSTER
kind export kubeconfig --name $CLUSTER
mkdir -p $HOME/gvisor && cd $HOME/gvisor
Exercise 1: download and verify gVisor
Download the gVisor release 20261005 for x86_64, and its checksum, from https://storage.googleapis.com/gvisor/releases/release/20261005/x86_64/: the files are gvisor.tar.zstd and gvisor.tar.zstd.sha512. Verify the archive, then extract it.
In the exam, the same check applies to the Kubernetes binaries (kubelet, kubectl…): compare sha512sum <binary> with the .sha512 file published next to the release. A checksum downloaded from the same server as the binary only detects a corrupted download: a signature (cosign, GPG) also proves who published it.
Exercise 2: install runsc on the nodes
On each node of your cluster:
copy runsc, containerd-shim-runsc-v1 and the directory gvisor-bin in /usr/local/bin
declare the runtime runsc in /etc/containerd/config.toml, with runtime_type = "io.containerd.runsc.v1"
restart containerd
Tip
kind get nodes --name $CLUSTER lists the nodes: they are docker containers, use docker cp and docker exec. Look at the section of the runc runtime in /etc/containerd/config.toml to find the right syntax.
Solution
# In the exam: ssh to the control plane node, then run the command without docker execdocker exec $CLUSTER-control-plane grep -A2 "runtimes.runc\]" /etc/containerd/config.toml
for node in $(kind get nodes --name $CLUSTER); do docker cp runsc $node:/usr/local/bin/runsc
docker cp containerd-shim-runsc-v1 $node:/usr/local/bin/containerd-shim-runsc-v1
docker cp gvisor-bin $node:/usr/local/bin/
docker exec $node grep -q "runtimes.runsc\]" /etc/containerd/config.toml ||\
printf '%s\n''[plugins."io.containerd.grpc.v1.cri".containerd.runtimes.runsc]'\
' runtime_type = "io.containerd.runsc.v1"' | docker exec -i $node tee -a /etc/containerd/config.toml
docker exec $node systemctl restart containerd
donekubectl wait --for=condition=Ready node --all --timeout=120s
containerd finds the shim from the runtime type: io.containerd.runsc.v1 means the binary containerd-shim-runsc-v1 in the PATH. runsc needs the binaries of gvisor-bin next to it: without them, the pods fail with sidecar "gvisor_sentry" not usable.
Restarting containerd does not stop the running containers.
Exercise 3: a RuntimeClass and a sandboxed pod
Create a RuntimeClass named gvisor, whose handler is runsc. Then run two nginx pods: sandboxed, which uses the gvisor RuntimeClass, and normal, which does not. Compare the output of dmesg and uname -r in both pods.
Solution
cat <<EOF | kubectl apply -f -
apiVersion: node.k8s.io/v1
kind: RuntimeClass
metadata:
name: gvisor
handler: runsc
---
apiVersion: v1
kind: Pod
metadata:
name: sandboxed
spec:
runtimeClassName: gvisor
containers:
- name: nginx
image: nginx:1.27
EOFkubectl run normal --image=nginx:1.27
kubectl wait --for=condition=Ready pod/sandboxed pod/normal --timeout=120s
kubectl exec sandboxed -- dmesg | head -3
# [ 0.000000] Starting gVisor...kubectl exec normal -- dmesg || echo "EXPECTED ERROR"# dmesg: read kernel buffer failed: Operation not permittedkubectl exec sandboxed -- uname -r
# 4.19.0-gvisorkubectl exec normal -- uname -r
# the kernel version of the node
The handler of the RuntimeClass is the name of the runtime in the containerd configuration. The normal pod sees the kernel of the node; the sandboxed pod sees the Sentry, the kernel of gVisor.
Exercise 4: from the node
On the node of the pods, list the processes named nginx. Then delete the normal pod and list them again. Which processes run the sandboxed pod?
Solution
NODE=$(kubectl get pod sandboxed -o jsonpath='{.spec.nodeName}')# In the exam: ssh to this node, then run the command without docker execdocker exec $NODE sh -c 'ps -eo comm | grep -x nginx | wc -l'kubectl delete pod normal
sleep 5docker exec $NODE sh -c 'ps -eo comm | grep -x nginx | wc -l'# 0docker exec $NODE sh -c 'ps -eo comm | grep -E "runsc|gvisor" | sort | uniq -c'
The processes of a sandboxed container are not processes of the node: they run inside the Sentry (gvisor_sentry). A tool on the node which watches the system calls of the containers, like Falco with its default kernel driver, no longer sees them: Falco needs its gVisor integration for these pods.
Questions
What happens if a pod with runtimeClassName: gvisor is scheduled on a node without runsc? How do you avoid it?
What does gVisor cost?
Which other sandbox runtimes do you know?
Answers
The pod stays in ContainerCreating, with the event FailedCreatePodSandBox ... no runtime for "runsc" is configured. Label the nodes which have the runtime, and set scheduling.nodeSelector in the RuntimeClass: the pods of this class are only scheduled on these nodes.
Performance (the system calls, the file system and the network go through the Sentry) and compatibility: some system calls and some /proc or /sys files are not implemented. Use it for the untrusted workloads (code of the users, multi-tenancy), not for everything.
Kata Containers runs each pod in a small virtual machine, with its own Linux kernel: stronger isolation, but it needs hardware virtualization on the nodes. On the exam, the RuntimeClass and the handler name are given in the task.
Clean up
kubectl delete pod sandboxed normal --ignore-not-found
kubectl delete runtimeclass gvisor
cd $HOME && rm -rf $HOME/gvisor
# Delete the cluster only if it was created for the labif[ -z "${CLUSTER_NAME:-}"]; then kind delete cluster --name $CLUSTER; fi
On the CKS training server, runsc stays installed in the nodes of your cluster: it is harmless for the next labs.