building homelab cluster part 10

#bytebase

Table of Content

building homelab cluster part 10¶

I will setup bytebase in this part which enables the GitOps workflow for databases.

[x] create a postgres database for bytebase using postgres operator
[x] create a secret containing postgres dsn for bytebase to use
[x] deploy bytebase on kubernetes cluster using helm
[x] setup gateway and httproute for the bytebase GUI access
[x] setup gitops integration on bytebase using self-managed gitlab

bytebase¶

https://github.com/bytebase/bytebase

Bytebase is a Database CI/CD solution for the Developers and DBAs

helm chart¶

https://github.com/bytebase/bytebase/tree/main/helm-charts/bytebase

As usual, I will get the chart, confirm version, and download the values file.

# add helm repo
helm repo add bytebase-repo https://bytebase.github.io/bytebase

# confirm the version, and it's 1.1.0 as of March 2024
helm search repo -l bytebase
helm show chart bytebase-repo/bytebase

# get the values file
helm show values bytebase-repo/bytebase --version=1.1.0 > bytebase-values.yaml

values file¶

https://www.bytebase.com/docs/administration/production-setup/

There are few things listed in the production setup guide, and I will ensure they are set in the values file.

enable https and websocket
configure external URL
store metadata in external postgres database

The https access will be ready through the gateway using NGINX Gateway Fabric.

The external URL can be set at bytebase.option.external-url in the values file.

As for the external postgres database, I can pass the URL or DSN using the secret. I of course need the database first, so here we go.

postgres database for bytebase¶

I will use the postgres operator setup in the previous part.

First of all, the namespace is going to be "bytebase" with gateway access.

./clusters/homelab/namespace/bytebase.yaml

---
apiVersion: v1
kind: Namespace
metadata:
  name: bytebase
  labels:
    service: bytebase
    type: infrastructure
    gateway-available: yes

Next, I will create a postgres database for bytebase. I will copy-paste the keycloak database manifest and prepare something like this. And do not forget to add this to the infra-config ks.

./infrastructure/homelab/configs/bytebase-db.yaml

---
apiVersion: "acid.zalan.do/v1"
kind: postgresql
metadata:
  name: bytebase-db
  namespace: bytebase
spec:
  teamId: "acid"
  volume:
    size: 2Gi
    storageClass: directpv-min-io
  numberOfInstances: 2
  users:
    bb: # database owner
      - superuser
      - createdb
  databases: # {database name}:{database owner}
    bytebase: bb
  postgresql:
    version: "16"
  resources:
    requests:
      cpu: 10m
      memory: 100Mi
    limits:
      cpu: 500m
      memory: 500Mi
  nodeAffinity:
    requiredDuringSchedulingIgnoredDuringExecution:
      nodeSelectorTerms:
        - matchExpressions:
            - key: app.kubernetes.io/part-of
              operator: In
              values:
                - directpv

This will spin up the database and create secret storing credentials for the user "bb". This command will display the password string.

kubectl -n bytebase get secret bb.bytebase-db.credentials.postgresql.acid.zalan.do -o jsonpath='{.data.password}' | base64 -d

https://www.bytebase.com/docs/get-started/install/external-postgres/#pg-connection-string

Now prepare the DSN secret.

postgresql://<>:<>@<>:<>/<>

postgres dsn for bytebase

# on sops repo
cd clusters/homelab
mkdir bytebase
cd bytebase
kubectl create secret generic bytebase-db \
  --from-literal=dsn='postgresql://bb:{PASSWORD_HERE}@bytebase-db/bytebase' \
  -n bytebase \
  --dry-run=client \
  -o yaml > bytebase-db.yaml

sops -i --encrypt bytebase-db.yaml

# git commit and push

deploying bytebase¶

I will prepare the same usual script to generate flux helmrepo and helmrelease manifest for bytebase.

./infrastructure/homelab/controllers/bytebase.sh

# add flux helmrepo to the manifest
flux create source helm bytebase-repo \
    --url=https://bytebase.github.io/bytebase \
    --interval=1h0m0s \
    --export >bytebase.yaml

# add flux helm release to the manifest including the customized values.yaml file
flux create helmrelease bytebase \
    --interval=10m \
    --target-namespace=bytebase \
    --source=HelmRepository/bytebase-repo \
    --chart=bytebase \
    --chart-version=1.1.0 \
    --values=./bytebase-values.yaml \
    --export >>bytebase.yaml

enable https access through gateway¶

I will copy-paste the keycloak-routes.yaml created in the previous part, and modify a little bit and prepare something like this as the HTTPRoute for bytebase GUI access.

./infrastructure/homelab/configs/bytebase-routes.yaml

---
apiVersion: gateway.networking.k8s.io/v1beta1
kind: HTTPRoute
metadata:
  name: bytebase
  namespace: bytebase
spec:
  parentRefs:
    - name: gateway
      sectionName: https-bytebase
      namespace: gateway
  hostnames:
    - "bytebase.blink-1x52.net"
  rules:
    - matches:
        - path:
            type: PathPrefix
            value: /
      backendRefs:
        - name: bytebase-entrypoint
          port: 80

And of course I am going to add the gateway listener named "https-bytebase". The gateway manifest file is fairly long now, so below is just the listener section added.

./infrastructure/homelab/configs/gateway.yaml

- name: https-bytebase
  hostname: bytebase.blink-1x52.net
  port: 443
  protocol: HTTPS
  allowedRoutes:
    namespaces:
      from: Selector
      selector:
        matchLabels:
          gateway-available: yes
  tls:
    mode: Terminate
    certificateRefs:
      - name: tls-bytebase-20240324
        namespace: gateway
        kind: Secret

Add bytebase-routes.yaml to the infra-config ks and all is set. I can access https://bytebase.blink-1x52.net, and it will ask to register the admin account.

database gitops, vcs integration¶

https://www.bytebase.com/docs/vcs-integration/overview/

I can use bytebase to change database using GitOps methodology, and to do that I need to setup a project and associated database on bytebase to manage through GitOps.

Here is what I am going to do to setup the GitOps on bytebase:

create a new namespace "dns-report-cloudflare"
create a new repository on my self-managed GitLab to be the VCS of this project
create a new postgres database for this namespace for a cronjob to insert record of metric data obtained using Cloudflare API for DNS service
register the database on bytebase
create a new project for the database on bytebase
setup GitOps integration with the GitLab repo for the bytebase project
(GitOps DEMO) create a table using bytebase database GitOps
prepare necessary credentials for a cloudflare-dns-report script to run as encrypted secret on sops repository
prepare CronJob manifest to run the script periodically

create namespace "dns-report-cloudflare"¶

./clusters/homelab/namespace/dns-report-cloudflare

---
apiVersion: v1
kind: Namespace
metadata:
  name: dns-report-cloudflare
  labels:
    service: dns-report-cloudflare
    type: application

create a new repository "dns-report-cloudflare"¶

I will create a separate repository to store manifests, and create flux gitrepo and kustomization to watch and reconcile the cluster using the manifests on the repository.

[x] create a new blank project on my self-managed GitLab
[x] create a read-only deploy token
[x] create a secret from the deploy token username and password and place it and encrypt it on SOPS repository
[x] place flux gitrepo and kustomization manifest in ./apps/base/dns-report-cloudflare/repo.yaml and include it in ./apps/homelab/kustomization.yaml

deploy token secret¶

kubectl create secret generic deploy-token-dns-report-cloudflare \
  --from-literal=username="DEPLOY_TOKEN_NAME" \
  --from-literal=password="DEPLOY_TOKEN_PASSWORD" \
  --dry-run=client \
  --namespace=flux-system \
  -o yaml > {./clusters/homelab/flux-system/deploy-token-dns-report-cloudflare.yaml on SOPS repo}

flux gitrepo and kustomization for the new repository¶

./apps/base/dns-report-cloudflare/repo.yaml

---
apiVersion: source.toolkit.fluxcd.io/v1
kind: GitRepository
metadata:
  name: dns-report-cloudflare
  namespace: flux-system
spec:
  interval: 1m0s
  ref:
    branch: main
  secretRef:
    name: deploy-token-dns-report-cloudflare
  url: https://cp.blink-1x52.net/network/dns-report-cloudflare.git
---
apiVersion: kustomize.toolkit.fluxcd.io/v1
kind: Kustomization
metadata:
  name: dns-report-cloudflare
  namespace: flux-system
spec:
  interval: 1m0s
  path: ./deploy/homelab
  prune: true
  sourceRef:
    kind: GitRepository
    name: dns-report-cloudflare
  targetNamespace: dns-report-cloudflare

create database¶

Add this manifest and include it on infra-config ks. Note that there is no hyphen used in the database name.

./infrastructure/homelab/configs/app-db-dns-report-cloudflare.yaml

---
apiVersion: "acid.zalan.do/v1"
kind: postgresql
metadata:
  name: db
  namespace: dns-report-cloudflare
spec:
  teamId: "acid"
  volume:
    size: 5Gi
    storageClass: directpv-min-io
  numberOfInstances: 2
  users:
    dns-report-cloudflare-user:
      - superuser
      - createdb
  databases:
    dnsreportcloudflare: dns-report-cloudflare-user
  postgresql:
    version: "16"
  resources:
    requests:
      cpu: 10m
      memory: 100Mi
    limits:
      cpu: 500m
      memory: 500Mi
  nodeAffinity:
    requiredDuringSchedulingIgnoredDuringExecution:
      nodeSelectorTerms:
        - matchExpressions:
            - key: app.kubernetes.io/part-of
              operator: In
              values:
                - directpv

register the database on bytebase¶

Navigate to instances and click "add instance" to add the database created.

select PostgreSQL as the database type
give it an instance name - "dns-report-cloudflare"
select environment - "Prod"
host or socket - db.dns-report-cloudflare:5432
username: dns-report-cloudflare-user
password: kubectl -n dns-report-cloudflare get secret dns-report-cloudflare-user.db.credentials.postgresql.acid.zalan.do -o jsonpath='{.data.password}' | base64 -d
database - dnsreportcloudflare, note that there is no hyphen when creating the db and the same name goes here
ssl and ssh connections - select None

Test the connection and add the instance.

create a bytebase project¶

Navigate to projects and click "new project" to create a new bytebase project.

give it a name - dns-report-cloudflare
key - anything
mode - standard

add database to the bytebase project¶

Navigate to Database > Databases menu on left pane and click "transfer in DB" to add the unassigned database just created.

setup gitops workflow¶

Navigate to Integration > GitOps to start GitOps workflow configuration.

url - https://cp.blink-1x52.net
display name - anything
follow the guide to fill the rest ... https://www.bytebase.com/docs/vcs-integration/self-host-gitlab/
- create a new app on self-managed gitlab as admin
- set the app id and secret provided by gitlab on bytebase
- confirm and add

Once the GitOps integration is configured, go to the project > Integration > GitOps menu again to select the configured GitOps integration to choose the repository and branch to use to manage the bytebase project (and its database).

select repository - the new gitlab repo created for this
branch - main by default
base directory - bytebase by default
schema change type - imperative, DDL, by default
file path template - {{ENV_ID}}/{{DB_NAME}}##{{VERSION}}##{{TYPE}}##{{DESCRIPTION}}.sql by default

add DDL SQL to create a table¶

Add this file on "dns-report-cloudflare" repository and bytebase will pick this up, create an issue, apply necessary changes to the database, and resolve/close the issue on the project.

./bytebase/prod/dnsreportcloudflare##2024032701##ddl##create-dnstable.sql

CREATE TABLE dnsreport (
  hashed_id TEXT PRIMARY KEY,
  time TIMESTAMP WITH TIME ZONE,  -- iso8601, 2000-01-23T01:23:45Z
  dname TEXT,
  metric INTEGER
);

deleting bytebase¶

The bytebase helm added using flux can be deleted by removing the corresponding files/ks from the gitops repository. One thing to watch out is that the postgres database separately created for bytebase to use must be deleted separately, otherwise the bytebase release will continue to remain and run.