Kubernetes Deployment

TinyMQ v2.9.5 introduces native support for Kubernetes orchestration. Because TinyMQ handles its own masterless P2P clustering, deploying it to K8s is straightforward using a StatefulSet.

Quick Start (kind)

You can test the cluster locally using kind.

# 1. Create a local cluster
kind create cluster --name tinymq-test

# 2. Build the image locally
docker build -t tinymq:dev .
kind load docker-image tinymq:dev --name tinymq-test

# 3. Create the shared cluster secret
kubectl create secret generic tinymq-secrets --from-literal=cluster-secret=super_secret_key

# 4. Apply the manifest
kubectl apply -f k8s/tinymq-cluster.yaml

# 5. Watch the cluster form and elect a leader
kubectl logs -l app=tinymq -f --prefix --max-log-requests 3

The Official Manifest

The official manifest (k8s/tinymq-cluster.yaml) deploys a 3-node StatefulSet with a Headless Service for peer discovery and a Load-Balanced Service for client access.

Critical Design Decisions

Deploying a stateful database to Kubernetes requires specific configuration. Here are the design decisions baked into the manifest:

1. publishNotReadyAddresses: true

In the Headless Service, we set publishNotReadyAddresses: true. Without this, tinymq-1 cannot resolve tinymq-0.tinymq-headless during the rolling startup because the pods are not marked "Ready" yet. This would cause the first election to fail indefinitely.

2. TINYMQ_CLUSTER_SELF via Downward API

By default, TCP servers bind to 0.0.0.0. If a node announces 0.0.0.0 as its address to peers, they will fail to connect. We use the Downward API to inject the pod's specific DNS name into TINYMQ_CLUSTER_SELF.

env:
  - name: POD_NAME
    valueFrom:
      fieldRef:
        fieldPath: metadata.name
  - name: TINYMQ_CLUSTER_SELF
    value: "$(POD_NAME).tinymq-headless:7901"

3. Increased Replication Timeout

DNS resolution inside Kubernetes pods starting up can be slow. The default TINYMQ_CLUSTER_REPLICATE_TIMEOUT of 500ms is insufficient. The manifest overrides this to 2s.

4. Port Isolation

Port 7901 (cluster communication) is only exposed on the Headless Service. The main balanced tinymq-service only exposes HTTP (7800) and MQTT (1883), protecting the cluster protocol from external clients.

Troubleshooting

Symptom	Command	Likely cause
Pods stuck in Pending	kubectl describe pod tinymq-0	Missing default StorageClass / PVC provisioner
CrashLoopBackOff	kubectl logs tinymq-0 --previous	Bad env var or missing tinymq-secrets
Election loop	kubectl logs -l app=tinymq | grep -E "Election|VOTE|ONLINE"	DNS not resolving; check publishNotReadyAddresses
SEC-ALERT: Invalid HMAC	kubectl logs tinymq-0 | grep SEC-ALERT	Secret mismatch between pods
TCP blocked	kubectl exec -it tinymq-0 -- nc -zv tinymq-1.tinymq-headless 7901	NetworkPolicy blocking port 7901