In this lab, you run kube-bench as Kubernetes Jobs, on the control plane and on a worker, and read its report. The next lab fixes some of the failed checks.
The CKS curriculum lists “Use CIS benchmark to review the security configuration of Kubernetes components” in the Cluster setup domain.
Prerequisites
The Jobs only read the files of the nodes: the lab works on the shared cluster, in a namespace of your own.
# The shared kind cluster (on the CKS training server: kind export kubeconfig, then kubectx kind-kind)kubectx kind-kind
NS=kube-bench-$(whoami)kubectl create namespace $NS
kubectl config set-context --current --namespace=$NS
mkdir -p $HOME/kube-bench && cd $HOME/kube-bench
Exercise 1: scan the control plane
kube-bench must run on the node it checks, with the PID namespace of the node (to see the processes of the control plane) and the configuration directories of the node mounted read-only. Aqua Security publishes a job.yaml.
Write a Job kube-bench-master, which runs kube-bench run --targets master (image docker.io/aquasec/kube-bench:v0.14.1) on the control plane node. Run it, and display the summary of the report.
Tip
The control plane node has the label node-role.kubernetes.io/control-plane and a taint with the same key: use a nodeSelector and a toleration.
hostPID: true lets kube-bench see the command lines of the control plane processes (kube-apiserver --profiling=...): it checks their flags there, and the files of the manifests in /etc/kubernetes.
Exercise 2: read the report
List the failed checks. For each one, the report gives a remediation: display the one of check 1.4.1. Which failed checks are covered by other labs of the training?
Solution
grep "^\[FAIL\]" master.txt
# [FAIL] 1.1.12 Ensure that the etcd data directory ownership is set to etcd:etcd (Automated)# [FAIL] 1.2.5 Ensure that the --kubelet-certificate-authority argument is set as appropriate (Automated)# [FAIL] 1.2.15 Ensure that the --profiling argument is set to false (Automated)# [FAIL] 1.2.16 Ensure that the --audit-log-path argument is set (Automated)# [FAIL] 1.2.17 Ensure that the --audit-log-maxage argument is set to 30 or as appropriate (Automated)# [FAIL] 1.2.18 Ensure that the --audit-log-maxbackup argument is set to 10 or as appropriate (Automated)# [FAIL] 1.2.19 Ensure that the --audit-log-maxsize argument is set to 100 or as appropriate (Automated)# [FAIL] 1.2.30 Ensure that the --service-account-extend-token-expiration parameter is set to false (Automated)# [FAIL] 1.3.2 Ensure that the --profiling argument is set to false (Automated)# [FAIL] 1.4.1 Ensure that the --profiling argument is set to false (Automated)grep -A4 "^1.4.1 " master.txt
# 1.4.1 Edit the Scheduler pod specification file /etc/kubernetes/manifests/kube-scheduler.yaml file# on the control plane node and set the below parameter.# --profiling=false
1.2.16 to 1.2.19: the audit log of the API server, see the audit logs lab.
1.2.30: the service account tokens valid for one year, see step 3 of the kubectl proxy lab.
1.2.15, 1.3.2, 1.4.1: the profiling endpoints of the API server, the controller manager and the scheduler, see the next lab.
The WARN checks are manual: for example 1.2.27 and 1.2.28, the encryption of the Secrets at rest, see the etcd encryption lab.
A benchmark is a list of recommendations, not of bugs: some checks do not apply to every cluster (1.1.12: kind runs etcd without an etcd user). Fix what applies, and document the others.
Exercise 3: scan a worker node
Write a Job kube-bench-node, which runs kube-bench run --targets node. Which components does it check?
Without nodeSelector, the scheduler puts the Job on any node, a worker on a cluster with workers. The node target checks the kubelet (its configuration file and flags) and kube-proxy: the components of every node.