Restrict the system calls of a container with seccomp (secure computing mode): a filter in the Linux kernel which allows, denies or logs each system call of a process.
Profile type
Where is the profile?
Use
Unconfined
None: no filter
The default of the kubelet, unless --seccomp-default is set
RuntimeDefault
Built in the container runtime (containerd, CRI-O)
A good default: denies about 50 dangerous system calls
Localhost
A JSON file on the node, under /var/lib/kubelet/seccomp/
A profile for one application
The CKS curriculum lists “Use kernel hardening tools such as AppArmor, seccomp” in the System hardening domain.
Prerequisites
The lab writes files on the nodes: use a kind cluster of your own, never a shared one.
# On the CKS training server, use your own cluster ($CLUSTER_NAME),# elsewhere create a cluster for the labCLUSTER=${CLUSTER_NAME:-seccomp-$(whoami)}kind get clusters | grep -qx $CLUSTER || kind create cluster --name $CLUSTER
kind export kubeconfig --name $CLUSTER
kubectl create namespace seccomp
kubectl config set-context --current --namespace=seccomp
Exercise 1: the default profile
Run two pods with the image busybox:1.36 and the command sleep infinity: default, without any security context, and runtime-default, with the RuntimeDefault seccomp profile. In each pod, read the Seccomp field of /proc/1/status, then run unshare -U true, which creates a user namespace.
Seccomp: 0 means no filter, 2 means a filter is active. Without a security context, the container runs Unconfined: it can call unshare, used by many container escapes. The RuntimeDefault profile denies it.
Set the profile at the pod level (spec.securityContext), it applies to all its containers; a container can override it in its own securityContext. The restricted level of the Pod Security Standards requires RuntimeDefault or Localhost.
Exercise 2: a custom profile
Write the profile no-mkdir.json, which allows all the system calls except mkdir and mkdirat, which return an error. Copy it on all the nodes, in /var/lib/kubelet/seccomp/profiles/. Then run a pod no-mkdir which uses it, and try to create a directory, then a file, in /tmp.
Tip
A profile is a JSON document with a defaultAction and a list of syscalls with their action: SCMP_ACT_ALLOW, SCMP_ACT_ERRNO, SCMP_ACT_LOG… The path in the pod is relative to /var/lib/kubelet/seccomp/.
Solution
for node in $(kind get nodes --name $CLUSTER); do# In the exam: ssh to the node, then run the command without docker exec docker exec $node mkdir -p /var/lib/kubelet/seccomp/profiles
cat <<EOF | docker exec -i $node tee /var/lib/kubelet/seccomp/profiles/no-mkdir.json
{
"defaultAction": "SCMP_ACT_ALLOW",
"syscalls": [
{
"names": ["mkdir", "mkdirat"],
"action": "SCMP_ACT_ERRNO"
}
]
}
EOFdonecat <<EOF | kubectl apply -f -
apiVersion: v1
kind: Pod
metadata:
name: no-mkdir
spec:
securityContext:
seccompProfile:
type: Localhost
localhostProfile: profiles/no-mkdir.json
containers:
- name: main
image: busybox:1.36
command: ["sleep", "infinity"]
EOFkubectl wait --for=condition=Ready pod/no-mkdir --timeout=120s
kubectl exec no-mkdir -- mkdir /tmp/new-dir || echo "EXPECTED ERROR"# mkdir: can't create directory '/tmp/new-dir': Operation not permittedkubectl exec no-mkdir -- touch /tmp/new-file && echo "allowed"# allowed
The kubelet reads the profile on the node where the pod runs: a missing file on one node makes the pods of this node fail, hence the copy on all the nodes. In the exam, the task often gives a profile to copy in /var/lib/kubelet/seccomp/ of a node, with scp or ssh.
A real profile does the opposite: defaultAction: SCMP_ACT_ERRNO and the list of the system calls the application needs. Writing it by hand is long: the Security Profiles Operator records the system calls of an application and generates the profile.
Exercise 3: a missing profile
Run a pod missing-profile whose Localhost profile is profiles/missing.json, a file which does not exist. What is the status of the pod? Where is the error?
Solution
cat <<EOF | kubectl apply -f -
apiVersion: v1
kind: Pod
metadata:
name: missing-profile
spec:
securityContext:
seccompProfile:
type: Localhost
localhostProfile: profiles/missing.json
containers:
- name: main
image: busybox:1.36
command: ["sleep", "infinity"]
EOFsleep 15kubectl get pod missing-profile
# NAME READY STATUS RESTARTS AGE# missing-profile 0/1 CreateContainerError 0 15skubectl get events --field-selector involvedObject.name=missing-profile,reason=Failed -o custom-columns=MESSAGE:.message
# Error: failed to create containerd container: cannot load seccomp profile# "/var/lib/kubelet/seccomp/profiles/missing.json": open ...: no such file or directory
The API server accepts the pod: the profile is only read by the container runtime, on the node. The error is in the events of the pod.
Questions
How do you apply RuntimeDefault to all the pods of a cluster, without changing their manifests?
What is the difference with AppArmor?
Answers
Start the kubelets with --seccomp-default (or seccompDefault: true in their configuration file): the pods without a profile get RuntimeDefault instead of Unconfined. Test the applications first: a few need a system call that RuntimeDefault denies.
Seccomp filters system calls, whatever their arguments: “this process cannot call mkdir”. AppArmor filters access to resources, by path: “this process cannot write in /etc”. Both are often used together.
Clean up
kubectl delete namespace seccomp
kubectl config set-context --current --namespace=default
for node in $(kind get nodes --name $CLUSTER); do# In the exam: ssh to the node, then run the command without docker exec docker exec $node rm -f /var/lib/kubelet/seccomp/profiles/no-mkdir.json
done# Delete the cluster only if it was created for the labif[ -z "${CLUSTER_NAME:-}"]; then kind delete cluster --name $CLUSTER; fi
The seccomp.sh script of the k8s-advanced repository runs more profiles, on a cluster it creates.