Kubernetes Admission Control Integration

Integrate build.security's PDP as Kubernetes admission controller

Introduction

In the following guide we will focus on integrating build.security's PDP as a validating admission controller. Using the PDP will allow you to enjoy the control plane's rich capabilities on top of OPA.

What is a Kubernetes admission controller?

In a nutshell, Kubernetes admission controllers are plugins that govern and enforce how the cluster is used. They can be thought of as a gatekeeper that intercept (authenticated) API requests and may change the request object or deny the request altogether. The admission control process has two phases: the mutating phase is executed first, followed by the validating phase. Consequently, admission controllers can act as mutating or validating controllers or as a combination of both.

source: https://kubernetes.io/blog/2019/03/21/a-guide-to-kubernetes-admission-controllers/

Prerequisites

This tutorial requires Kubernetes 1.13 or later. To run the tutorial locally ensure you start a cluster with Kubernetes version 1.13+, we recommend using minikube or KIND.

Start minikube:

minikube start

To use admission control rules that validate Kubernetes resources during create, update, and delete operations, you must enable the ValidatingAdmissionWebhook when the Kubernetes API server is started. The ValidatingAdmissionWebhook admission controller is included in the recommended set of admission controllers to enable

Steps

1. Create a PDP using the control plane

If you are not familiar with how to create a PDP follow this guide, then generate an api key and secret.

2. Create a Kubernetes policy and publish it to a PDP

Create a new policy in the control plane. Make sure you pick Kubernetes integration.

Notice that the default policy comes with a set of rules that are all set as Monitored. By default we set the policy to allow all requests, so at first we won't enforce any of the rules.

Now let's publish the new policy so it would be served to the PDP once it is deployed.

Publish the changes so they'll take effect

When PDP is deployed on top of Kubernetes, policies are automatically loaded whenever you publish a policy using the control plane.

kubectl create namespace buildsecurity

Configure kubectl to use this namespace:

kubectl config set-context pdp-tutorial --user minikube --cluster minikube --namespace buildsecurity
kubectl config use-context pdp-tutorial

4. Deploy PDP on top of Kubernetes

Communication between Kubernetes and PDP must be secured using TLS. To configure TLS, use openssl to create a certificate authority (CA) and certificate/key pair for PDP:

openssl genrsa -out ca.key 2048
openssl req -x509 -new -nodes -key ca.key -days 100000 -out ca.crt -subj "/CN=admission_ca"

Generate the TLS key and certificate for PDP:

cat >server.conf <<EOF
[req]
req_extensions = v3_req
distinguished_name = req_distinguished_name
prompt = no
[req_distinguished_name]
CN = pdp.buildsecurity.svc
[ v3_req ]
basicConstraints = CA:FALSE
keyUsage = nonRepudiation, digitalSignature, keyEncipherment
extendedKeyUsage = clientAuth, serverAuth
subjectAltName = @alt_names
[alt_names]
DNS.1 = pdp.buildsecurity.svc
EOF
openssl genrsa -out server.key 2048
openssl req -new -key server.key -out server.csr -config server.conf
openssl x509 -req -in server.csr -CA ca.crt -CAkey ca.key -CAcreateserial -out server.crt -days 100000 -extensions v3_req -extfile server.conf

Note: the Common Name value and Subject Alternative Name you give to openssl MUST match the name of the PDP service created below.

Create a Secret to store the TLS credentials for PDP:

kubectl create secret tls pdp-server --cert=server.crt --key=server.key

Next, use the file below to deploy PDP as an admission controller.

Pay attention to fill your own PDP api key and api secret (Lines 44 & 46 admission-controller.yaml)

admission-controller.yaml:

kind: Service
apiVersion: v1
metadata:
name: pdp
namespace: buildsecurity
spec:
selector:
app: pdp
ports:
- name: https
protocol: TCP
port: 443
targetPort: 8181
---
apiVersion: apps/v1
kind: Deployment
metadata:
labels:
app: pdp
namespace: buildsecurity
name: pdp
spec:
replicas: 1
selector:
matchLabels:
app: pdp
template:
metadata:
labels:
app: pdp
name: pdp
spec:
containers:
- name: pdp
image: buildsecurity/pdp:latest
imagePullPolicy: Always
args:
- "--tls-cert-file=/certs/tls.crt"
- "--tls-private-key-file=/certs/tls.key"
env:
- name: API_KEY
value: "YOUR API KEY HERE"
- name: API_SECRET
value: "YOUR API SECRET HERE"
- name: CONTROL_PLANE_ADDR
value: "https://api.build.security/v1/api/pdp"
volumeMounts:
- readOnly: true
mountPath: /certs
name: pdp-server
readinessProbe:
httpGet:
path: /health?plugins&bundle
scheme: HTTPS
port: 8181
initialDelaySeconds: 10
periodSeconds: 5
livenessProbe:
httpGet:
path: /health
scheme: HTTPS
port: 8181
initialDelaySeconds: 10
periodSeconds: 5
volumes:
- name: pdp-server
secret:
secretName: pdp-server

And run the following:

kubectl apply -f admission-controller.yaml

You can now check that the PDP is online in the control plane.

In this example we set our replicas to 4

5. Register PDP as an admission controller:

Generate the manifest that will be used to register PDP as an admission controller. This webhook will ignore any namespace with the label build.security/webhook=ignore.

cat > webhook-configuration.yaml <<EOF
kind: ValidatingWebhookConfiguration
apiVersion: admissionregistration.k8s.io/v1
metadata:
name: pdp-validating-webhook
webhooks:
- name: validating-webhook.build.security
namespaceSelector:
matchExpressions:
- key: build.security/webhook
operator: NotIn
values:
- ignore
rules:
- operations: ["CREATE", "UPDATE"]
apiGroups: ["*"]
apiVersions: ["*"]
resources: ["*"]
sideEffects: None
failurePolicy: Ignore # Change to Fail - to strict on webhook failure and fail the action
clientConfig:
caBundle: $(cat ca.crt | base64 | tr -d '\n')
service:
namespace: buildsecurity
name: pdp
path: /v0/data-ex/kubernetes/admission/response
admissionReviewVersions: ["v1", "v1beta1"]
timeoutSeconds: 5
EOF

The generated configuration file includes a base64 encoded representation of the CA certificate so that TLS connections can be established between the Kubernetes API server and PDP.

Next label kube-system and the buildsecurity namespace so that PDP does not control the resources in those namespaces.

kubectl label ns kube-system build.security/webhook=ignore
kubectl label ns buildsecurity build.security/webhook=ignore

Finally, register the PDP as an admission controller:

kubectl apply -f webhook-configuration.yaml

You can follow the PDP logs to see the webhook requests being issued by the Kubernetes API server:

# ctrl-c to exit
kubectl logs -l app=pdp -c pdp -f

6. Check your control plane

Go to Decision Logs to see the requests being made and being sent to the control plane properly:

7. Exercise the policy

nginx.yaml

apiVersion: apps/v1
kind: Deployment
metadata:
name: nginx-deployment
spec:
selector:
matchLabels:
app: nginx
replicas: 1
template:
metadata:
labels:
app: nginx
spec:
containers:
- name: nginx
image: nginx:latest
resources:
requests:
memory: 64Mi
cpu: 250m
ports:
- containerPort: 80

Now lets create the nginx and see the results:

kubectl create -f nginx.yaml -n default

Use Decision Logs filters to find the relevant events to nginx:

Here you can see that if the policy was enforcing, the request would have been denied

Now let's change some of the rules that could have denied the request and publish our changes

Changing K.SEC 06 to Active
Publish the changes so they'll take effect

Now, let’s try and recreate nginx. Remember that we expect this operation to fail since the relevant rule is now active

kubectl delete -f nginx.yaml -n default
kubectl create -f nginx.yaml -n default
# Error from server (nginx in the Deployment nginx-deployment is not using a read only root filesystem): error when creating "nginx.yaml": admission webhook "validating-webhook.build.security" denied the request: nginx in the Deployment nginx-deployment is not using a read only root filesystem

And in the Decision Logs you would see the following: