Self-host Bytebase

Latest release version: 3.1.0

Self-host is for business with security/compliance requirements. Bytebase offers an air-gapped deployment with a single Go binary.

Prerequisites#

Check System Requirements.
Check External PostgreSQL Prerequisites if you configure it.
If you use gateway such as Nginx, enable WebSocket for SQL Editor autocomplete to work.

Docker#

Estimated time: 5 minutes.

Docker version must be at least 20.10.24 or pass --security-opt seccomp=unconfined to docker run, otherwise, you will get pthread_create failed error.
If accessing from China, pull image registry.cn-shanghai.aliyuncs.com/bytebase/bytebase:3.1.0.

Installation#

docker run --rm --init \
  --name bytebase \
  --publish 8080:8080 --pull always \
  --volume ~/.bytebase/data:/var/opt/bytebase \
  bytebase/bytebase:3.1.0

Once you see the Bytebase logo, you can access the console at http://localhost:8080.

██████╗ ██╗   ██╗████████╗███████╗██████╗  █████╗ ███████╗███████╗
██╔══██╗╚██╗ ██╔╝╚══██╔══╝██╔════╝██╔══██╗██╔══██╗██╔════╝██╔════╝
██████╔╝ ╚████╔╝    ██║   █████╗  ██████╔╝███████║███████╗█████╗
██╔══██╗  ╚██╔╝     ██║   ██╔══╝  ██╔══██╗██╔══██║╚════██║██╔══╝
██████╔╝   ██║      ██║   ███████╗██████╔╝██║  ██║███████║███████╗
╚═════╝    ╚═╝      ╚═╝   ╚══════╝╚═════╝ ╚═╝  ╚═╝╚══════╝╚══════╝

Version 3.1.0 has started on port 8080 🚀

Configuration#

Use external PostgreSQL to store metadata#

By default, Bytebase will use an embedded PostgreSQL database to store metadata. For production usage, it is recommended to use an external PostgreSQL database instead. Check Configure External PostgreSQL for details.

Customize startup options#

If you need more control over the server configuration, check other Server Startup Options.

Allow external access via External URL#

Check Configure External URL for details.

Enable WebSocket for SQL Editor#

SQL Editor autocomplete requires WebSocket. If you access Bytebase via a gateway, you need to enable WebSocket there. Here is an sample NGINX configuration (including the optional HTTPS mentioned below):

http {
    map $http_upgrade $connection_upgrade {
      default upgrade;
      '' close;
    }

    server {
        listen       80;
        listen  [::]:80;
        # Listen HTTPS
        listen       443 ssl;
        listen  [::]:443 ssl;
        server_name  bytebase.example.com;

        # SSL cert and key
        ssl_certificate /path/to/certificate/file;
        ssl_certificate_key /path/to/private/key/file;

       location ~ ^/(v1:adminExecute|lsp) {
            proxy_pass http://bytebase.example.com;
            proxy_http_version 1.1;
            # Enables WebSocket which is required for SQL Editor autocomplete
            proxy_set_header Upgrade $http_upgrade;
            proxy_set_header Connection $connection_upgrade;
        }

        location / {
            proxy_pass http://bytebase.example.com;
        }

        proxy_read_timeout 3600;
        proxy_send_timeout 3600;
    }
}

Enable HTTPS#

Bytebase does not support enabling HTTPS in server configuration. We suggest use NGINX or Caddy as a reverse proxy in front of Bytebase to enable HTTPS.

Troubleshooting#

bind: address already in use#

If you see bind: address already in use error, it means the port 8080 is already in use on your host. You need to either stop the existing process using the port or configure Bytebase to use a different port via --publish <<YOUR_PORT>>:8080.

Connect database instance on the same host#

If you run Bytebase inside Docker on Linux, then you need to supply the additional --network host flags in docker run command. This allows Bytebase to connect to database instance running on the same host with localhost.
If you run Bytebase inside Docker Desktop on Mac , then you need to use host.docker.internal to connect to database instance running on the same host.

Manifest not found#

There may be a few reasons the manifest file is not found:

The docker image only supports linux/amd64 and linux/arm64 arch. If it doesn't match your OS arch, you may supply --platform linux/amd64 as a best effort.
Your Docker version is too old and doesn't support manifest list. Please install the latest Docker version.

Unable to start using Colima#

Due to the vm mechanism of Colima, try to use the --mount option when starting colima as shown below:

mkdir ~/volumes
colima start --mount ~/volumes:w
docker run --init \
  --name bytebase \
  --restart always \
  --publish 80:8080 \
  --volume ~/.bytebase/data:/var/opt/bytebase bytebase/bytebase:3.1.0 \
  --data /var/opt/bytebase \
  --external-url http://bytebase.example.com \
  --port 8080

Kubernetes#

Estimated time: 15 minutes.

Deploy to Kubernetes#

Make sure to set the replicas to 1, otherwise, it may cause data race issues.

Here is a sample Kubernetes YAML file bb.yaml describing the minimal components and configuration required to run Bytebase in Kubernetes.

apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: bytebase
  namespace: default
spec:
  # To prevent data races, only request one replica.
  replicas: 1
  selector:
    matchLabels:
      app: bytebase
  template:
    metadata:
      labels:
        app: bytebase
    spec:
      containers:
        - name: bytebase
          image: bytebase/bytebase:3.1.0
          imagePullPolicy: Always
          env:
            - name: PG_URL
              value: 'postgresql://<<user>>:<<secret>>@<<host>>:<<port>>/<<dbname>>'
          args:
            [
              '--data',
              '/var/opt/bytebase',
              '--external-url',
              'http://bytebase.example.com',
              '--port',
              '8080',
            ]
          ports:
            - containerPort: 8080
          livenessProbe:
            httpGet:
              path: /healthz
              port: 8080
            initialDelaySeconds: 300
            periodSeconds: 300
            timeoutSeconds: 10
---
apiVersion: v1
kind: Service
metadata:
  name: bytebase-entrypoint
  namespace: default
spec:
  # Optional
  type: ClusterIP
  selector:
    app: bytebase
  ports:
    - protocol: TCP
      port: 80
      targetPort: 8080

Start Bytebase with the following command:
```
kubectl apply -f bb.yaml
```
Make sure everything worked by listing your deployments:
```
kubectl get statefulsets
```
Do the same check for your services:
```
kubectl get services
```
Open a browser and visit http://localhost, you should see Bytebase.

Upgrade#

When a new Bytebase release is published, you can change the image version in the yaml file

containers:
  - name: bytebase
    image: bytebase/bytebase:3.1.0

Use Helm Chart#

Production Setup External URL#

For production setup, you should configure a proper External URL.

Installing the Chart#

helm -n <YOUR_NAMESPACE> \
--set "bytebase.option.port"={PORT} \
--set "bytebase.option.external-url"={EXTERNAL_URL} \
--set "bytebase.option.externalPg.url"={PGDSN} \
--set "bytebase.version"={VERSION} \
install <RELEASE_NAME> bytebase-repo/bytebase

For example:

helm -n bytebase \
--set "bytebase.option.port"=443 \
--set "bytebase.option.external-url"="http://bytebase.example.com" \
--set "bytebase.option.externalPg.url"="postgresql://user:secret@foo.ap-east-1.rds.amazonaws.com/postgres" \
--set "bytebase.version"=1.7.0 \
install bytebase-release bytebase-repo/bytebase

Uninstalling the Chart#

helm delete --namespace <YOUR_NAMESPACE> <RELEASE_NAME>

Upgrade Bytebase Version/Configuration#

Use helm upgrade command to upgrade the bytebase version or configuration.

helm -n <YOUR_NAMESPACE> \
--set "bytebase.option.port"={NEW_PORT} \
--set "bytebase.option.external-url"={NEW_EXTERNAL_URL} \
--set "bytebase.option.externalPg.url"={NEW_PGDSN} \
--set "bytebase.version"={NEW_VERSION} \
upgrade bytebase-release bytebase-repo/bytebase

Deploy with Ingress#

We use Ingress-Nginx Controller as ingress controller. You need to config Ingress-Nginx Controller according to your environment.

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations:
    nginx.ingress.kubernetes.io/ssl-redirect: 'true'
    # https://kubernetes.github.io/ingress-nginx/user-guide/miscellaneous/#websockets
    nginx.ingress.kubernetes.io/proxy-read-timeout: '3600'
    nginx.ingress.kubernetes.io/proxy-send-timeout: '3600'
  name: bytebase-ingress
  namespace: default
spec:
  ingressClassName: nginx
  rules:
    - host: example.com
      http:
        paths:
          - backend:
              service:
                name: bytebase-entrypoint
                port:
                  number: 80
            path: /
            pathType: ImplementationSpecific
  tls:
    - hosts:
        - example.com
      secretName: tls-secret

Note If you use ingress, make sure to use https to access bytebase;

External PostgreSQL#

Instead of specify PostgreSQL connection string in helm or Kubernetes yaml file, we allows users to using Kubernetes secrets resources.

Kubernetes#

Using the following yaml section to replace the spec.templates.spec.containers.env section:

env:
  - name: PG_URL
    valueFrom:
      secretKeyRef:
        name: secret_name
        key: secrete_key

Helm#

Using --set bytebase.option.existingPgURLSecret and --set bytebase.option.existingPgURLSecretKey to specify the secret key and secret name instead of --set "bytebase.option.external-url"={NEW_EXTERNAL_URL}. See more details in Bytebase - Artifact Hub.

Persistent Volume#

We don't recommend this. However, if you do not configure External PostgreSQL, then to persist data, you need to use the Persistent Volumes in the cluster. Each cloud provider has its own solution.

For Amazon Elastic Kubernetes Service(EKS)#

In AWS EKS, you can use the Amazon EBS CSI driver for persistent volumes. Follow the managing EBS CSI to add it as an Amazon EKS add-on.

Here is a simple example to use an EBS volume:

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: bytebase-ebs-claim
spec:
  accessModes:
    - ReadWriteOnce
  storageClassName: bytebase-resize-sc
  resources:
    requests:
      storage: 4Gi
---
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: bytebase-resize-sc
provisioner: ebs.csi.aws.com
allowVolumeExpansion: true

Note Also need to update the statefulset spec of bytebase to replace the emptyDir volume with persistentVolumeClaim:

volumes:
  - name: bytebase-volume
    persistentVolumeClaim:
      claimName: bytebase-ebs-claim

For Google Kubernetes Engine(GKE)#

Please follow the Persistent volumes and dynamic provisioning.

Build from Source#

Due to the uncontrolled nature of building from source, we only provide technical assistance to the Enterprise customers.

Prerequisites

Install pnpm, Bytebase requires Node.js >=17.0.
Install Go, Bytebase requires Go >= 1.16
It's recommended to run Bytebase application as non-root user for security reason. If you don't have other non-root users on the system, you can follow the following steps to create one, e.g. user bytebase.
```
groupadd bytebase && useradd -g bb bytebase
```
```
sudo su bytebase
```

Download source code from GitHub, then go to the source root directory

If you want to build from a specific release such as 3.1.0, then switch to that release tag.

git checkout tags/3.1.0

Build the source

scripts/build_bytebase.sh [<<out_directory>>]

If out_directoryis not specified, the default directory is ./bytebase-build

Suppose you run scripts/build_bytebase.sh foo After build completes, run:

foo/bytebase --port 8080

(check Server Startup Options for other startup options)

You should see something like this in the console:

██████╗ ██╗   ██╗████████╗███████╗██████╗  █████╗ ███████╗███████╗
██╔══██╗╚██╗ ██╔╝╚══██╔══╝██╔════╝██╔══██╗██╔══██╗██╔════╝██╔════╝
██████╔╝ ╚████╔╝    ██║   █████╗  ██████╔╝███████║███████╗█████╗
██╔══██╗  ╚██╔╝     ██║   ██╔══╝  ██╔══██╗██╔══██║╚════██║██╔══╝
██████╔╝   ██║      ██║   ███████╗██████╔╝██║  ██║███████║███████╗
╚═════╝    ╚═╝      ╚═╝   ╚══════╝╚═════╝ ╚═╝  ╚═╝╚══════╝╚══════╝

Version 3.1.0 has started on port 8080 🚀

Troubleshoot

error: too many open files#

Change the open file limit:

ulimit -n 10240

Self-host Bytebase

Prerequisites#

Docker#

Installation#

Configuration#

Use external PostgreSQL to store metadata#

Customize startup options#

Allow external access via External URL#

Enable WebSocket for SQL Editor#

Enable HTTPS#

Troubleshooting#

bind: address already in use#

Connect database instance on the same host#

Manifest not found#

Unable to start using Colima#

Kubernetes#

Deploy to Kubernetes#

Upgrade#

Use Helm Chart#

Production Setup External URL#

Installing the Chart#

Uninstalling the Chart#

Upgrade Bytebase Version/Configuration#

Deploy with Ingress#

External PostgreSQL#

Kubernetes#

Helm#

Persistent Volume#

For Amazon Elastic Kubernetes Service(EKS)#

For Google Kubernetes Engine(GKE)#

Build from Source#

error: too many open files#

Deploy to PaaS#

Subscribe to Newsletter