Kubernetes Admission Control Integration
Integrate build.security's PDP as Kubernetes admission controller

Introduction

In the following guide we will focus on integrating build.security's PDP as a validating admission controller. Using the PDP will allow you to enjoy the control plane's rich capabilities on top of OPA.
What is a Kubernetes admission controller?
In a nutshell, Kubernetes admission controllers are plugins that govern and enforce how the cluster is used. They can be thought of as a gatekeeper that intercept (authenticated) API requests and may change the request object or deny the request altogether. The admission control process has two phases: the mutating phase is executed first, followed by the validating phase. Consequently, admission controllers can act as mutating or validating controllers or as a combination of both.

Prerequisites

This tutorial requires Kubernetes 1.13 or later. To run the tutorial locally ensure you start a cluster with Kubernetes version 1.13+, we recommend using minikube or KIND.
Start minikube:
1
minikube start
Copied!
To use admission control rules that validate Kubernetes resources during create, update, and delete operations, you must enable the ValidatingAdmissionWebhook when the Kubernetes API server is started. The ValidatingAdmissionWebhook admission controller is included in the recommended set of admission controllers to enable

Steps

1. Create a PDP using the control plane

If you are not familiar with how to create a PDP follow this guide, then generate an api key and secret.

2. Create a Kubernetes policy and publish it to a PDP

Create a new policy in the control plane. Make sure you pick Kubernetes integration.
Notice that the default policy comes with a set of rules that are all set as Monitored. By default we set the policy to allow all requests, so at first we won't enforce any of the rules.
Now let's publish the new policy so it would be served to the PDP once it is deployed.
Publish the changes so they'll take effect
When PDP is deployed on top of Kubernetes, policies are automatically loaded whenever you publish a policy using the control plane.
1
kubectl create namespace buildsecurity
Copied!
Configure kubectl to use this namespace:
1
kubectl config set-context pdp-tutorial --user minikube --cluster minikube --namespace buildsecurity
2
kubectl config use-context pdp-tutorial
Copied!

4. Deploy PDP on top of Kubernetes

Communication between Kubernetes and PDP must be secured using TLS. To configure TLS, use openssl to create a certificate authority (CA) and certificate/key pair for PDP:
1
openssl genrsa -out ca.key 2048
2
openssl req -x509 -new -nodes -key ca.key -days 100000 -out ca.crt -subj "/CN=admission_ca"
Copied!
Generate the TLS key and certificate for PDP:
1
cat >server.conf <<EOF
2
[req]
3
req_extensions = v3_req
4
distinguished_name = req_distinguished_name
5
prompt = no
6
[req_distinguished_name]
7
CN = pdp.buildsecurity.svc
8
[ v3_req ]
9
basicConstraints = CA:FALSE
10
keyUsage = nonRepudiation, digitalSignature, keyEncipherment
11
extendedKeyUsage = clientAuth, serverAuth
12
subjectAltName = @alt_names
13
[alt_names]
14
DNS.1 = pdp.buildsecurity.svc
15
EOF
Copied!
1
openssl genrsa -out server.key 2048
2
openssl req -new -key server.key -out server.csr -config server.conf
3
openssl x509 -req -in server.csr -CA ca.crt -CAkey ca.key -CAcreateserial -out server.crt -days 100000 -extensions v3_req -extfile server.conf
Copied!
Note: the Common Name value and Subject Alternative Name you give to openssl MUST match the name of the PDP service created below.
Create a Secret to store the TLS credentials for PDP:
1
kubectl create secret tls pdp-server --cert=server.crt --key=server.key
Copied!
Next, use the file below to deploy PDP as an admission controller.
Pay attention to fill your own PDP api key and api secret (Lines 44 & 46 admission-controller.yaml)
admission-controller.yaml:
1
kind: Service
2
apiVersion: v1
3
metadata:
4
name: pdp
5
namespace: buildsecurity
6
spec:
7
selector:
8
app: pdp
9
ports:
10
- name: https
11
protocol: TCP
12
port: 443
13
targetPort: 8181
14
15
---
16
17
apiVersion: apps/v1
18
kind: Deployment
19
metadata:
20
labels:
21
app: pdp
22
namespace: buildsecurity
23
name: pdp
24
spec:
25
replicas: 1
26
selector:
27
matchLabels:
28
app: pdp
29
template:
30
metadata:
31
labels:
32
app: pdp
33
name: pdp
34
spec:
35
containers:
36
- name: pdp
37
image: buildsecurity/pdp:latest
38
imagePullPolicy: Always
39
args:
40
- "--tls-cert-file=/certs/tls.crt"
41
- "--tls-private-key-file=/certs/tls.key"
42
env:
43
- name: API_KEY
44
value: "YOUR API KEY HERE"
45
- name: API_SECRET
46
value: "YOUR API SECRET HERE"
47
- name: CONTROL_PLANE_ADDR
48
value: "https://api.build.security/v1/api/pdp"
49
volumeMounts:
50
- readOnly: true
51
mountPath: /certs
52
name: pdp-server
53
readinessProbe:
54
httpGet:
55
path: /health?plugins&bundle
56
scheme: HTTPS
57
port: 8181
58
initialDelaySeconds: 10
59
periodSeconds: 5
60
livenessProbe:
61
httpGet:
62
path: /health
63
scheme: HTTPS
64
port: 8181
65
initialDelaySeconds: 10
66
periodSeconds: 5
67
volumes:
68
- name: pdp-server
69
secret:
70
secretName: pdp-server
Copied!
And run the following:
1
kubectl apply -f admission-controller.yaml
Copied!
You can now check that the PDP is online in the control plane.
In this example we set our replicas to 4

5. Register PDP as an admission controller:

Generate the manifest that will be used to register PDP as an admission controller. This webhook will ignore any namespace with the label build.security/webhook=ignore.
1
cat > webhook-configuration.yaml <<EOF
2
kind: ValidatingWebhookConfiguration
3
apiVersion: admissionregistration.k8s.io/v1
4
metadata:
5
name: pdp-validating-webhook
6
webhooks:
7
- name: validating-webhook.build.security
8
namespaceSelector:
9
matchExpressions:
10
- key: build.security/webhook
11
operator: NotIn
12
values:
13
- ignore
14
rules:
15
- operations: ["CREATE", "UPDATE"]
16
apiGroups: ["*"]
17
apiVersions: ["*"]
18
resources: ["*"]
19
sideEffects: None
20
failurePolicy: Ignore # Change to Fail - to strict on webhook failure and fail the action
21
clientConfig:
22
caBundle: $(cat ca.crt | base64 | tr -d '\n')
23
service:
24
namespace: buildsecurity
25
name: pdp
26
path: /v0/data-ex/kubernetes/admission/response
27
admissionReviewVersions: ["v1", "v1beta1"]
28
timeoutSeconds: 5
29
EOF
Copied!
The generated configuration file includes a base64 encoded representation of the CA certificate so that TLS connections can be established between the Kubernetes API server and PDP.
Next label kube-system and the buildsecurity namespace so that PDP does not control the resources in those namespaces.
1
kubectl label ns kube-system build.security/webhook=ignore
2
kubectl label ns buildsecurity build.security/webhook=ignore
Copied!
Finally, register the PDP as an admission controller:
1
kubectl apply -f webhook-configuration.yaml
Copied!
You can follow the PDP logs to see the webhook requests being issued by the Kubernetes API server:
1
# ctrl-c to exit
2
kubectl logs -l app=pdp -c pdp -f
Copied!

6. Check your control plane

Go to Decision Logs to see the requests being made and being sent to the control plane properly:

7. Exercise the policy

nginx.yaml
1
apiVersion: apps/v1
2
kind: Deployment
3
metadata:
4
name: nginx-deployment
5
spec:
6
selector:
7
matchLabels:
8
app: nginx
9
replicas: 1
10
template:
11
metadata:
12
labels:
13
app: nginx
14
spec:
15
containers:
16
- name: nginx
17
image: nginx:latest
18
resources:
19
requests:
20
memory: 64Mi
21
cpu: 250m
22
ports:
23
- containerPort: 80
Copied!
Now lets create the nginx and see the results:
1
kubectl create -f nginx.yaml -n default
Copied!
Use Decision Logs filters to find the relevant events to nginx:
Here you can see that if the policy was enforcing, the request would have been denied
Now let's change some of the rules that could have denied the request and publish our changes
Changing K.SEC 06 to Active
Publish the changes so they'll take effect
Now, let’s try and recreate nginx. Remember that we expect this operation to fail since the relevant rule is now active
1
kubectl delete -f nginx.yaml -n default
2
kubectl create -f nginx.yaml -n default
3
4
# Error from server (nginx in the Deployment nginx-deployment is not using a read only root filesystem): error when creating "nginx.yaml": admission webhook "validating-webhook.build.security" denied the request: nginx in the Deployment nginx-deployment is not using a read only root filesystem
Copied!
And in the Decision Logs you would see the following:
Last modified 5mo ago