Skip to content

building homelab cluster part 9

building homelab cluster part 9

I am setting up IAM service using Keycloak, and also PostgreSQL Operator by Zalando while doing so in this part.


There is a section on operator for keycloak, so that's what I will be using.

kubernetes operator

And before going any further, I need to check what is an operator in kubernetes. An operator already appeared in this series when I installed metallb. I used helm chart to install metallb, and the chart was about setting up operator and metallb components.

Kubernetes has an ace up its sleeve that makes it even more useful, powerful, and flexible. That ace is called an Operator. Operators exist because Kubernetes was designed from the beginning for automation, which is built directly into the very heart of the software. In fact, Kubernetes allows users to automate the deployment and execution of workloads, and also the way that Kubernetes does these things

Operators are software extensions to Kubernetes that make use of custom resources to manage applications and their components. Operators follow Kubernetes principles, notably the control loop.

installing keycloak operator

Below two lines are what is described as the manual installation method for the crds, so what I will do is to download these files in my infrastructure crds directory.

kubectl apply -f
kubectl apply -f
# find the latest tags
curl -s | awk 'NR==50{exit} /name/ {print $2}' | cut -d\" -f2

# download two files and place them in the crds directory
cd ./infrastructure/homelab/controllers/crds
curl -L -o keycloaks-v1-24.0.1.yaml
curl -L -o keycloakrealmimports-v1-24.0.1.yaml

Update k8s kustomization.yaml to include the two keycloak crds.

And then here is the another line to deploy the operator, which I will also download the source and place it in the gitops repository to let flux install it.

# manual deployment of the operator, which I won't execute
kubectl apply -f

# instead, I will download and edit it
cd ./infrastructure/homelab/controllers
curl -L -o keycloak-operator.yaml

I added .metadata.namespace=keycloak to the namespaced resources in the operator manifest file, for role, rolebinding, svc, deployment, and so on.

I skip the content but I also prepare the keycloak namespace at ./clusters/homelab/namespace/keycloak.yaml.

Here is the result.

# kubectl api-resources | grep -i keycloak
keycloakrealmimports                                                                    true         KeycloakRealmImport
keycloaks                         kc                                                    true         Keycloak

# kubectl -n keycloak get all
NAME                                     READY   STATUS    RESTARTS   AGE
pod/keycloak-operator-585774ccf9-fp9nl   1/1     Running   0          2m25s

NAME                        TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)   AGE
service/keycloak-operator   ClusterIP   <none>        80/TCP    2m25s

NAME                                READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/keycloak-operator   1/1     1            1           2m25s

NAME                                           DESIRED   CURRENT   READY   AGE
replicaset.apps/keycloak-operator-585774ccf9   1         1         1       2m25s

preparing for keycloak deployment

There are things to be prepared before I can actually deploy keycloak through the operator, and they are database, hostname, and TLS certificate/key pair.


I am going to prepare PostgreSQL operator and the database deployed through the operator for keycloak.

postgres operator installation

# add helm charts
helm repo add postgres-operator
helm repo add postgres-operator-ui

# check available versions
helm search repo -l postgres-operator

# get values files
cd ./infrastructure/homelab/controllers/default-values
helm show values postgres-operator/postgres-operator --version=1.11.0 > postgres-operator-values.yaml
helm show values postgres-operator-ui/postgres-operator-ui --version=1.11.0 > postgres-operator-ui-values.yaml

I am using the default setup so here is my script to generate flux helmrepo and helmrelease for postgres-operator, without specifying values file.


# add flux helmrepo to the manifest
flux create source helm postgres-operator \
  --url= \
    --interval=1h0m0s \
    --export >postgres-operator.yaml

# add flux helm release to the manifest including the customized values.yaml file
flux create helmrelease postgres-operator \
    --interval=10m \
    --target-namespace=postgres \
    --source=HelmRepository/postgres-operator \
    --chart=postgres-operator \
    --chart-version=1.11.0 \
    --export >>postgres-operator.yaml

I create postgres namespace manifest file, flux helmrepo and hr manifest file without custom values file, update infra-controllers kustomization, and here is the result.

$ kubectl api-resources | grep -i zalan
operatorconfigurations            opconfig                                                       true         OperatorConfiguration
postgresqls                       pg                                                             true         postgresql
postgresteams                     pgteam                                                         true         PostgresTeam

$ kubectl -n postgres get all
NAME                                              READY   STATUS    RESTARTS   AGE
pod/postgres-postgres-operator-5b57879674-mm7qn   1/1     Running   0          70s

NAME                                 TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)    AGE
service/postgres-postgres-operator   ClusterIP   <none>        8080/TCP   70s

NAME                                         READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/postgres-postgres-operator   1/1     1            1           70s

NAME                                                    DESIRED   CURRENT   READY   AGE
replicaset.apps/postgres-postgres-operator-5b57879674   1         1         1       70s

NAME                                                             IMAGE                             CLUSTER-LABEL   SERVICE-ACCOUNT   MIN-INSTANCES   AGE   cluster-name    postgres-pod      -1              70s

postgres deployment

The postgres database deployment can be done by preparing the custom postgres manifest.

See the examples on the official repository:

Now the database I will first deploy will be for keycloak. This manifest below will deploy postgresdb version 16 with 2 instances in keycloak namespace, creating the database named keycloak and owner user kc. The persistence volume will be from directpv-min-io storage class with size of 5Gi, and the nodeSelector will be used with the usual label.

apiVersion: ""
kind: postgresql
  name: keycloak-db
  namespace: keycloak
  teamId: "acid"
    size: 5Gi
    storageClass: directpv-min-io
  numberOfInstances: 2
    kc: # database owner
      - superuser
      - createdb
  databases: # {database name}:{database owner}
    keycloak: kc
    version: "16"
      cpu: 10m
      memory: 100Mi
      cpu: 500m
      memory: 500Mi
        - matchExpressions:
            - key:
              operator: In
                - directpv

Update ./infrastructure/homelab/configs/kustomization.yaml to include this keycloak database manifest, and the result is shown as below.

kubectl -n keycloak get all
NAME                                     READY   STATUS    RESTARTS   AGE
pod/keycloak-db-0                        1/1     Running   0          106s
pod/keycloak-db-1                        1/1     Running   0          103s
pod/keycloak-operator-585774ccf9-fp9nl   1/1     Running   0          115m

NAME                         TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)    AGE
service/keycloak-db          ClusterIP   <none>        5432/TCP   106s
service/keycloak-db-config   ClusterIP   None             <none>        <none>     99s
service/keycloak-db-repl     ClusterIP    <none>        5432/TCP   106s
service/keycloak-operator    ClusterIP   <none>        80/TCP     115m

NAME                                READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/keycloak-operator   1/1     1            1           115m

NAME                                           DESIRED   CURRENT   READY   AGE
replicaset.apps/keycloak-operator-585774ccf9   1         1         1       115m

NAME                           READY   AGE
statefulset.apps/keycloak-db   2/2     106s

NAME                                   TEAM   VERSION   PODS   VOLUME   CPU-REQUEST   MEMORY-REQUEST   AGE    STATUS   acid   16        2      5Gi      10m           100Mi            106s   Running

The operator also generates secrets for postgres including the kc user specified. Password can be retrieved like this.

kubectl -n keycloak get secret -o 'jsonpath={.data.password}' | base64 -d

hostname and TLS cert/key

I will use certmanager to generate TLS certificate for keycloak.

I can use the gateway like I did for everything else so far, such as s3.

# kind: Gateway
# .spec.listeners[]
    # omit existing listeners
    - name: https-keycloak
      port: 443
      protocol: HTTPS
          from: Selector
              gateway-available: yes
        mode: Terminate
          - name: tls-keycloak-20240318
            namespace: gateway
            kind: Secret

And I also add HTTPRoute to access it. The service name specified in the backendRefs can be confirmed after deploying keycloak, the actual step I took was to create gateway, deployed keycloak through the operator, and then added this HTTPRoute manifest.

kind: HTTPRoute
  name: kc
  namespace: keycloak
    - name: gateway
      sectionName: https-keycloak
      namespace: gateway
    - ""
    - matches:
        - path:
            type: PathPrefix
            value: /
        - name: kc-service
          port: 8080

And (finally) the deployment of keycloak itself through the keycloak operator by this manifest.

The credentials for the database was created by postgres operator so I will just refer to that.

The kubernetes gateway implementation I'm using does not have support for the TLSRoute which is required for TLS passthrough, so I will just enable plain http, leave tslSecret empty, but set the https access URL for adminUrl value.

kind: Keycloak
  name: kc
  namespace: keycloak
  instances: 1
    vendor: postgres
    host: keycloak-db
      key: username
      key: password
    # tlsSecret: example-tls-secret
    httpEnabled: true
    strict: false
    headers: xforwarded # double check your reverse proxy sets and overwrites the X-Forwarded-* headers

admin credential

The keycloak deployed through operator creates a secret named "*-initial-admin". Refer to its data for the admin user initial password.

kubectl get secret -n keycloak kc-initial-admin -o jsonpath='{.data.password}' | base64 --decode

setting up IAM service