building homelab cluster part 9
Table of Content
building homelab cluster part 9¶
I am setting up IAM service using Keycloak, and also PostgreSQL Operator by Zalando while doing so in this part.
installation¶
https://www.keycloak.org/guides#operator
There is a section on operator for keycloak, so that's what I will be using.
kubernetes operator¶
And before going any further, I need to check what is an operator in kubernetes. An operator already appeared in this series when I installed metallb. I used helm chart to install metallb, and the chart was about setting up operator and metallb components.
https://www.cncf.io/blog/2022/06/15/kubernetes-operators-what-are-they-some-examples/
Kubernetes has an ace up its sleeve that makes it even more useful, powerful, and flexible. That ace is called an Operator. Operators exist because Kubernetes was designed from the beginning for automation, which is built directly into the very heart of the software. In fact, Kubernetes allows users to automate the deployment and execution of workloads, and also the way that Kubernetes does these things
https://kubernetes.io/docs/concepts/extend-kubernetes/operator/
Operators are software extensions to Kubernetes that make use of custom resources to manage applications and their components. Operators follow Kubernetes principles, notably the control loop.
installing keycloak operator¶
Below two lines are what is described as the manual installation method for the crds, so what I will do is to download these files in my infrastructure crds directory.
kubectl apply -f https://raw.githubusercontent.com/keycloak/keycloak-k8s-resources/24.0.1/kubernetes/keycloaks.k8s.keycloak.org-v1.yml
kubectl apply -f https://raw.githubusercontent.com/keycloak/keycloak-k8s-resources/24.0.1/kubernetes/keycloakrealmimports.k8s.keycloak.org-v1.yml
# find the latest tags
curl -s https://api.github.com/repos/keycloak/keycloak-k8s-resources/tags | awk 'NR==50{exit} /name/ {print $2}' | cut -d\" -f2
# download two files and place them in the crds directory
cd ./infrastructure/homelab/controllers/crds
curl -L https://raw.githubusercontent.com/keycloak/keycloak-k8s-resources/24.0.1/kubernetes/keycloaks.k8s.keycloak.org-v1.yml -o keycloaks-v1-24.0.1.yaml
curl -L https://raw.githubusercontent.com/keycloak/keycloak-k8s-resources/24.0.1/kubernetes/keycloakrealmimports.k8s.keycloak.org-v1.yml -o keycloakrealmimports-v1-24.0.1.yaml
Update k8s kustomization.yaml to include the two keycloak crds.
And then here is the another line to deploy the operator, which I will also download the source and place it in the gitops repository to let flux install it.
# manual deployment of the operator, which I won't execute
kubectl apply -f https://raw.githubusercontent.com/keycloak/keycloak-k8s-resources/24.0.1/kubernetes/kubernetes.yml
# instead, I will download and edit it
cd ./infrastructure/homelab/controllers
curl -L https://raw.githubusercontent.com/keycloak/keycloak-k8s-resources/24.0.1/kubernetes/kubernetes.yml -o keycloak-operator.yaml
I added .metadata.namespace=keycloak to the namespaced resources in the operator manifest file, for role, rolebinding, svc, deployment, and so on.
I skip the content but I also prepare the keycloak namespace at ./clusters/homelab/namespace/keycloak.yaml.
Here is the result.
# kubectl api-resources | grep -i keycloak
keycloakrealmimports k8s.keycloak.org/v2alpha1 true KeycloakRealmImport
keycloaks kc k8s.keycloak.org/v2alpha1 true Keycloak
# kubectl -n keycloak get all
NAME READY STATUS RESTARTS AGE
pod/keycloak-operator-585774ccf9-fp9nl 1/1 Running 0 2m25s
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/keycloak-operator ClusterIP 10.103.115.115 <none> 80/TCP 2m25s
NAME READY UP-TO-DATE AVAILABLE AGE
deployment.apps/keycloak-operator 1/1 1 1 2m25s
NAME DESIRED CURRENT READY AGE
replicaset.apps/keycloak-operator-585774ccf9 1 1 1 2m25s
preparing for keycloak deployment¶
https://www.keycloak.org/operator/basic-deployment
There are things to be prepared before I can actually deploy keycloak through the operator, and they are database, hostname, and TLS certificate/key pair.
database¶
I am going to prepare PostgreSQL operator and the database deployed through the operator for keycloak.
postgres operator installation¶
https://github.com/zalando/postgres-operator/
# add helm charts
helm repo add postgres-operator https://opensource.zalando.com/postgres-operator/charts/postgres-operator
helm repo add postgres-operator-ui https://opensource.zalando.com/postgres-operator/charts/postgres-operator-ui
# check available versions
helm search repo -l postgres-operator
# get values files
cd ./infrastructure/homelab/controllers/default-values
helm show values postgres-operator/postgres-operator --version=1.11.0 > postgres-operator-values.yaml
helm show values postgres-operator-ui/postgres-operator-ui --version=1.11.0 > postgres-operator-ui-values.yaml
I am using the default setup so here is my script to generate flux helmrepo and helmrelease for postgres-operator, without specifying values file.
#!/bin/bash
# add flux helmrepo to the manifest
flux create source helm postgres-operator \
--url=https://opensource.zalando.com/postgres-operator/charts/postgres-operator \
--interval=1h0m0s \
--export >postgres-operator.yaml
# add flux helm release to the manifest including the customized values.yaml file
flux create helmrelease postgres-operator \
--interval=10m \
--target-namespace=postgres \
--source=HelmRepository/postgres-operator \
--chart=postgres-operator \
--chart-version=1.11.0 \
--export >>postgres-operator.yaml
I create postgres namespace manifest file, flux helmrepo and hr manifest file without custom values file, update infra-controllers kustomization, and here is the result.
$ kubectl api-resources | grep -i zalan
operatorconfigurations opconfig acid.zalan.do/v1 true OperatorConfiguration
postgresqls pg acid.zalan.do/v1 true postgresql
postgresteams pgteam acid.zalan.do/v1 true PostgresTeam
$ kubectl -n postgres get all
NAME READY STATUS RESTARTS AGE
pod/postgres-postgres-operator-5b57879674-mm7qn 1/1 Running 0 70s
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/postgres-postgres-operator ClusterIP 10.108.104.173 <none> 8080/TCP 70s
NAME READY UP-TO-DATE AVAILABLE AGE
deployment.apps/postgres-postgres-operator 1/1 1 1 70s
NAME DESIRED CURRENT READY AGE
replicaset.apps/postgres-postgres-operator-5b57879674 1 1 1 70s
NAME IMAGE CLUSTER-LABEL SERVICE-ACCOUNT MIN-INSTANCES AGE
operatorconfiguration.acid.zalan.do/postgres-postgres-operator ghcr.io/zalando/spilo-16:3.2-p2 cluster-name postgres-pod -1 70s
postgres deployment¶
The postgres database deployment can be done by preparing the custom postgres manifest.
See the examples on the official repository:
https://github.com/zalando/postgres-operator/blob/master/manifests/minimal-postgres-manifest.yaml
https://github.com/zalando/postgres-operator/blob/master/manifests/complete-postgres-manifest.yaml
Now the database I will first deploy will be for keycloak. This manifest below will deploy postgresdb version 16 with 2 instances in keycloak namespace, creating the database named keycloak and owner user kc. The persistence volume will be from directpv-min-io storage class with size of 5Gi, and the nodeSelector will be used with the usual label.
---
apiVersion: "acid.zalan.do/v1"
kind: postgresql
metadata:
name: keycloak-db
namespace: keycloak
spec:
teamId: "acid"
volume:
size: 5Gi
storageClass: directpv-min-io
numberOfInstances: 2
users:
kc: # database owner
- superuser
- createdb
databases: # {database name}:{database owner}
keycloak: kc
postgresql:
version: "16"
resources:
requests:
cpu: 10m
memory: 100Mi
limits:
cpu: 500m
memory: 500Mi
nodeAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
nodeSelectorTerms:
- matchExpressions:
- key: app.kubernetes.io/part-of
operator: In
values:
- directpv
Update ./infrastructure/homelab/configs/kustomization.yaml to include this keycloak database manifest, and the result is shown as below.
NAME READY STATUS RESTARTS AGE
pod/keycloak-db-0 1/1 Running 0 106s
pod/keycloak-db-1 1/1 Running 0 103s
pod/keycloak-operator-585774ccf9-fp9nl 1/1 Running 0 115m
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/keycloak-db ClusterIP 10.109.189.251 <none> 5432/TCP 106s
service/keycloak-db-config ClusterIP None <none> <none> 99s
service/keycloak-db-repl ClusterIP 10.103.31.108 <none> 5432/TCP 106s
service/keycloak-operator ClusterIP 10.103.115.115 <none> 80/TCP 115m
NAME READY UP-TO-DATE AVAILABLE AGE
deployment.apps/keycloak-operator 1/1 1 1 115m
NAME DESIRED CURRENT READY AGE
replicaset.apps/keycloak-operator-585774ccf9 1 1 1 115m
NAME READY AGE
statefulset.apps/keycloak-db 2/2 106s
NAME TEAM VERSION PODS VOLUME CPU-REQUEST MEMORY-REQUEST AGE STATUS
postgresql.acid.zalan.do/keycloak-db acid 16 2 5Gi 10m 100Mi 106s Running
The operator also generates secrets for postgres including the kc user specified. Password can be retrieved like this.
kubectl -n keycloak get secret kc.keycloak-db.credentials.postgresql.acid.zalan.do -o 'jsonpath={.data.password}' | base64 -d
hostname and TLS cert/key¶
I will use certmanager to generate TLS certificate for keycloak.
I can use the gateway like I did for everything else so far, such as s3.
# kind: Gateway
# .spec.listeners[]
spec:
listeners:
# omit existing listeners
- name: https-keycloak
hostname: kc.blink-1x52.net
port: 443
protocol: HTTPS
allowedRoutes:
namespaces:
from: Selector
selector:
matchLabels:
gateway-available: yes
tls:
mode: Terminate
certificateRefs:
- name: tls-keycloak-20240318
namespace: gateway
kind: Secret
And I also add HTTPRoute to access it. The service name specified in the backendRefs can be confirmed after deploying keycloak, the actual step I took was to create gateway, deployed keycloak through the operator, and then added this HTTPRoute manifest.
---
apiVersion: gateway.networking.k8s.io/v1beta1
kind: HTTPRoute
metadata:
name: kc
namespace: keycloak
spec:
parentRefs:
- name: gateway
sectionName: https-keycloak
namespace: gateway
hostnames:
- "kc.blink-1x52.net"
rules:
- matches:
- path:
type: PathPrefix
value: /
backendRefs:
- name: kc-service
port: 8080
And (finally) the deployment of keycloak itself through the keycloak operator by this manifest.
The credentials for the database was created by postgres operator so I will just refer to that.
The kubernetes gateway implementation I'm using does not have support for the TLSRoute which is required for TLS passthrough, so I will just enable plain http, leave tslSecret empty, but set the https access URL for adminUrl value.
---
apiVersion: k8s.keycloak.org/v2alpha1
kind: Keycloak
metadata:
name: kc
namespace: keycloak
spec:
instances: 1
db:
vendor: postgres
host: keycloak-db
usernameSecret:
name: kc.keycloak-db.credentials.postgresql.acid.zalan.do
key: username
passwordSecret:
name: kc.keycloak-db.credentials.postgresql.acid.zalan.do
key: password
http:
# tlsSecret: example-tls-secret
httpEnabled: true
hostname:
hostname: kc.blink-1x52.net
adminUrl: https://kc.blink-1x52.net
strict: false
proxy:
headers: xforwarded # double check your reverse proxy sets and overwrites the X-Forwarded-* headers
admin credential¶
The keycloak deployed through operator creates a secret named "*-initial-admin". Refer to its data for the admin user initial password.
setting up IAM service¶
https://www.keycloak.org/docs/latest/server_admin/index.html