PodSecurity is enabled by default since Kubernetes 1.23 (GA in 1.25), it replaced the PodSecurityPolicies.
Exercise 2: enforce the restricted level
Enforce the restricted level on your namespace, and audit on restricted. Then try to deploy the privileged pod.
Solution
kubectl label --overwrite namespace $NS \
pod-security.kubernetes.io/enforce=restricted \
pod-security.kubernetes.io/audit=restricted
kubectl apply -f $HOME/tmp/ubuntu-privileged.yaml || echo "EXPECTED ERROR"# Error from server (Forbidden): error when creating ".../ubuntu-privileged.yaml": pods "ubuntu-privileged" is forbidden:# violates PodSecurity "restricted:latest": privileged (container "ubuntu" must not set securityContext.privileged=true),# allowPrivilegeEscalation != false (...), unrestricted capabilities (...), runAsNonRoot != true (...), seccompProfile (...)
The error lists all the fields to change. kubectl apply --dry-run=server runs the admission controllers without creating the pod: a good way to test a manifest against a namespace.
Exercise 3: privileged level, with warnings
Enforce the privileged level, with warnings and audit on the baseline level. Deploy the privileged pod again.
Solution
kubectl label --overwrite namespace $NS \
pod-security.kubernetes.io/enforce=privileged \
pod-security.kubernetes.io/warn=baseline \
pod-security.kubernetes.io/audit=baseline
kubectl apply -f $HOME/tmp/ubuntu-privileged.yaml
# Warning: would violate PodSecurity "baseline:latest": privileged (container "ubuntu" must not set securityContext.privileged=true)# pod/ubuntu-privileged created
The pod is created, but kubectl displays a warning. The audit mode adds the same information as an annotation to the event of the audit log: it is only visible if the API server has an audit policy, see the audit logs lab.
warn and audit are the way to roll out a level: first see which workloads would be rejected, then enforce.
Exercise 4: before tightening the namespace
The privileged pod is running. Check which running pods would violate the restricted level, without changing the namespace.
The running pods are never evicted by a new label: only the new pods are checked. A Deployment whose pods violate the level keeps running, but cannot re-create its pods any more.
Exercise 5: a restricted ubuntu pod
Enforce the restricted level, with warnings and audit on restricted. Then update the ubuntu pod specification so that it runs in the namespace, with the NET_BIND_SERVICE capability enabled. Use the Pod Security Standards page.
Tip
The ubuntu image runs as root: choose a non-root uid, for example 65534 (the nobody user of the image).
restricted requires: no privileged container, allowPrivilegeEscalation: false, a non-root user, a RuntimeDefault or Localhost seccomp profile, and drop: ["ALL"]. NET_BIND_SERVICE is the only capability it allows to add back.
Exercise 6: play with the warn label
Enforce baseline and warn on restricted. Then create a copy of the restricted pod, named ubuntu-chown, which also adds the CHOWN capability. Is it created? What does the warning say? And with enforce=restricted?
Solution
kubectl label --overwrite namespace $NS \
pod-security.kubernetes.io/enforce=baseline \
pod-security.kubernetes.io/warn=restricted
sed -e 's/ubuntu-restricted/ubuntu-chown/' -e 's/add: \["NET_BIND_SERVICE"\]/add: ["NET_BIND_SERVICE", "CHOWN"]/'\
$HOME/tmp/ubuntu-restricted.yaml > $HOME/tmp/ubuntu-chown.yaml
kubectl apply -f $HOME/tmp/ubuntu-chown.yaml
# Warning: would violate PodSecurity "restricted:latest": unrestricted capabilities (container "ubuntu" must not include "CHOWN" in securityContext.capabilities.add)# pod/ubuntu-chown createdkubectl delete pod ubuntu-chown
kubectl label --overwrite namespace $NS pod-security.kubernetes.io/enforce=restricted
kubectl apply -f $HOME/tmp/ubuntu-chown.yaml || echo "EXPECTED ERROR"# Error from server (Forbidden): ... violates PodSecurity "restricted:latest": unrestricted capabilities ...
CHOWN is in the default capabilities of the container runtimes: baseline accepts it, restricted does not. The same message is a warning with warn, an error with enforce.
Questions
Who can bypass the Pod Security Admission of a namespace?
How does OpenShift do the same?
Answers
Anybody allowed to change the labels of the namespace: protect the update and patch verbs on namespaces with RBAC. The API server can also exempt users, runtime classes or namespaces (AdmissionConfiguration), and set a cluster-wide default level.
OpenShift uses the Security Context Constraints (SCC): they validate the pods and mutate them (random uid of the namespace range, for example). OpenShift also syncs the Pod Security Admission labels of the namespaces from the SCCs of their service accounts. See the SCC lab.