Confine a container with an AppArmor profile: a Linux security module which allows or denies the access of a process to files, capabilities and network, by path.
Profile mode
Effect
enforce
The forbidden accesses are denied and logged
complain
The forbidden accesses are allowed and logged: to write a profile
A profile is loaded in the kernel of each node with apparmor_parser, then a pod uses it with the appArmorProfile field of its securityContext (GA since Kubernetes 1.30, it replaces the container.apparmor.security.beta.kubernetes.io annotations).
The CKS curriculum lists “Use kernel hardening tools such as AppArmor, seccomp” in the System hardening domain.
Prerequisites
The lab loads profiles in the kernel of the nodes: use a kind cluster of your own, never a shared one. The kernel must support AppArmor (Ubuntu, Debian):
cat /sys/module/apparmor/parameters/enabled
# Y# On the CKS training server, use your own cluster ($CLUSTER_NAME),# elsewhere create a cluster for the labCLUSTER=${CLUSTER_NAME:-apparmor-$(whoami)}kind get clusters | grep -qx $CLUSTER || kind create cluster --name $CLUSTER
kind export kubeconfig --name $CLUSTER
kubectl create namespace apparmor
kubectl config set-context --current --namespace=apparmor
mkdir -p $HOME/apparmor && cd $HOME/apparmor
Prepare the kind nodes
A real node (Ubuntu, Debian) already has the AppArmor tools, and so do the nodes of the exam. The kind nodes are containers: install apparmor_parser in them, mount the AppArmor file system, and restart containerd and the kubelet, which check AppArmor at startup.
for node in $(kind get nodes --name $CLUSTER); do# In the exam: ssh to the node, then run the command without docker exec docker exec $node sh -c 'apt-get update -qq && DEBIAN_FRONTEND=noninteractive apt-get install -y -qq apparmor >/dev/null' docker exec $node sh -c 'mountpoint -q /sys/kernel/security || mount -t securityfs securityfs /sys/kernel/security' docker exec $node systemctl restart containerd kubelet
donekubectl wait --for=condition=Ready node --all --timeout=120s
Exercise 1: write and load a profile
Write the profile k8s-deny-write-$(whoami), which allows all the file accesses except the writes, and load it in enforce mode on all the nodes. Check that the kernel knows it.
Tip
A profile starts with profile <name> flags=(attach_disconnected) {, and its rules look like file, (allow all the files) or deny /** w, (deny writes everywhere). apparmor_parser -r <file> loads or replaces a profile; the loaded profiles are listed in /sys/kernel/security/apparmor/profiles.
Solution
PROFILE=k8s-deny-write-$(whoami)cat <<EOF > $PROFILE
#include <tunables/global>
profile $PROFILE flags=(attach_disconnected) {
#include <abstractions/base>
file,
# Deny all file writes
deny /** w,
}
EOFfor node in $(kind get nodes --name $CLUSTER); do docker cp $PROFILE $node:/etc/apparmor.d/$PROFILE
# In the exam: ssh to the node, then run the command without docker exec docker exec $node apparmor_parser -r /etc/apparmor.d/$PROFILE
donedocker exec $CLUSTER-control-plane grep $PROFILE /sys/kernel/security/apparmor/profiles
# k8s-deny-write-<user> (enforce)
On a real cluster, connect to each node (ssh) and run apparmor_parser there. The profile is in the kernel, not in Kubernetes: a node rebooted or added later does not have it, unless the profile is in /etc/apparmor.d/ (loaded at boot) or installed by a DaemonSet.
The kind nodes share the kernel of the server: that is why the profile name ends with your account.
Exercise 2: a confined pod
Run a pod deny-write (image busybox:1.36, command sleep infinity) confined by your profile. Which profile does its process run under? Try to write a file in /tmp, then to read /etc/hostname.
localhostProfile is the name of the profile in the kernel, not a file path (unlike seccomp). The field can also be set for one container, in its own securityContext.
Older clusters, and older exam tasks, use an annotation per container instead of the field:
Run a pod not-loaded like deny-write, with the profile k8s-not-loaded, which does not exist. What is its status? Where is the error?
Solution
cat <<EOF | kubectl apply -f -
apiVersion: v1
kind: Pod
metadata:
name: not-loaded
spec:
securityContext:
appArmorProfile:
type: Localhost
localhostProfile: k8s-not-loaded
containers:
- name: main
image: busybox:1.36
command: ["sleep", "infinity"]
EOFsleep 15kubectl get pod not-loaded
# NAME READY STATUS RESTARTS AGE# not-loaded 0/1 CreateContainerError 0 15skubectl get events --field-selector involvedObject.name=not-loaded,reason=Failed -o custom-columns=MESSAGE:.message
# Error: failed to get container spec opts: failed to generate apparmor spec opts: apparmor profile not found k8s-not-loaded
As for seccomp, the API server accepts the pod: the container runtime of the node checks the profile. Load the profile on all the nodes, or schedule the pods only on the nodes which have it (nodeSelector).
Questions
Which AppArmor profile does a container get without appArmorProfile?
How do you write the profile of an application?
Answers
RuntimeDefault: the default profile of the container runtime (cri-containerd.apparmor.d for containerd), if AppArmor is enabled on the node. It denies some sensitive accesses (writes in /proc, /sys, mount…). Unconfined disables AppArmor: the baseline level of the Pod Security Standards forbids it.
Load a first profile in complain mode (apparmor_parser -C), run the application, and read the logged accesses (journalctl -k | grep apparmor, or aa-logprof, which proposes the rules). Then load it in enforce mode. The Security Profiles Operator automates this on Kubernetes.
Clean up
kubectl delete namespace apparmor
kubectl config set-context --current --namespace=default
for node in $(kind get nodes --name $CLUSTER); do# The kind nodes share one kernel: the profile is already unloaded after the first node# In the exam: ssh to the node, then run the command without docker exec docker exec $node apparmor_parser -R /etc/apparmor.d/$PROFILE 2>/dev/null || true
docker exec $node rm /etc/apparmor.d/$PROFILE
donecd $HOME && rm -rf $HOME/apparmor
# Delete the cluster only if it was created for the labif[ -z "${CLUSTER_NAME:-}"]; then kind delete cluster --name $CLUSTER; fi