mirror of
https://github.com/zitadel/zitadel.git
synced 2026-08-19 01:14:48 -05:00
Mridang Agarwalla
db8f475a02
# Which Problems Are Solved The login v2 container has no way to verify upstream Zitadel API connectivity before accepting traffic. The existing `/healthy` endpoint always returns 200 regardless of whether the backend is reachable. This means Kubernetes routes user traffic to pods that cannot actually serve login requests, causing silent failures where the root layout swallows gRPC errors and renders pages with default settings. # How the Problems Are Solved A new `/ui/v2/login/ready` endpoint creates a gRPC client using the same auth and transport path as real requests, calls `SettingsService.getGeneralSettings()`, and returns `200 OK` on success or `503 Service unavailable` on failure. This verifies both network connectivity and authentication credentials to the upstream Zitadel API. Kubernetes supports readiness probes natively, so this endpoint can be configured as a readiness probe to gate traffic until the login container can connect to Zitadel. Docker Compose does not have a readiness probe concept, so the compose healthcheck and `healthcheck.mjs` default remain unchanged and continue using `/healthy`. The Docker HEALTHCHECK in the Dockerfile has been updated to use `/ready` so that the image-level health status reflects actual upstream connectivity. This will also allow us to simplify the Helm chart by dropping the `wait4x` init containers that currently poll the Zitadel API before starting the login pod, since the readiness probe makes them redundant. # Additional Changes The `/ready` endpoint has been added to the OTEL HTTP instrumentation ignore list to avoid noisy probe traces. Unit tests, a wiring spec integration test, and an OTEL span filtering test have been added for the new endpoint. The `# TODO: Check healthy, not ready` comment has been removed from the Dockerfile. The `/healthy` endpoint is kept as-is for liveness probes. # Additional Context The Helm chart changes to use `/ready` for readiness and startup probes and to remove the `wait-for-zitadel` init container will be done in a follow-up PR in the `zitadel-charts` repository.
ZITADEL Docker Compose — Developer Reference
User-facing documentation: zitadel.com/docs/self-hosting/deploy/compose
This README is the contributor/developer reference — architecture decisions, file conventions, and routing logic.
Architecture
┌─────────────────────────┐
Browser ──────►│ Traefik (proxy) │
│ Port 80 / 443 │
└───┬──────────┬──────────┘
│ │
┌──────────▼──┐ ┌───▼──────────┐
│ zitadel-api │ │ zitadel-login │
│ Go :8080 │ │ Next.js :3000 │
└──────┬───────┘ └──────────────┘
│
┌──────▼───────┐
│ PostgreSQL │
└──────────────┘
Optional services via profiles: redis (cache), otel-collector (observability).
File Conventions
| File | Role | Notes |
|---|---|---|
docker-compose.yml |
Base stack — all modes start from this | Must work standalone with .env.example |
docker-compose.mode-letsencrypt.yml |
TLS overlay: ACME HTTP challenge | Declares its own letsencrypt volume |
docker-compose.mode-external-tls.yml |
TLS overlay: upstream LB terminates TLS | Enables forwarded headers |
docker-compose.mode-local-tls.yml |
TLS overlay: self-signed certs | Mounts ./certs/ and traefik-local-tls.yml |
docker-compose.prodlike.yml |
Init/setup/start split | Uses YAML anchors for shared DB env |
docker-compose.test.yml |
CI smoke test overlay | Overrides images to :local tags |
.env.example |
User-facing config template | Copy to .env before first run |
.env.test |
CI-only config | Used by NX targets: test-run, test-e2e, test-full, stop |
otel-collector-config.yaml |
OTEL Collector pipeline config | Logs traces to stdout; configure OTEL_BACKEND_ENDPOINT to forward to a backend |
traefik-local-tls.yml |
Traefik dynamic config for local certs | Referenced by local-tls overlay |
project.json |
NX project definition | Targets: test-config, test-run, test-e2e, test, test-full, stop |
AGENTS.md |
AI agent instructions for this directory |
Routing Rules
Traefik routes all traffic for ${ZITADEL_DOMAIN} via Docker labels:
| Priority | Rule | Target | Middleware |
|---|---|---|---|
| 400 | Path(/) |
zitadel-login |
replacepath=/ui/v2/login/ |
| 250 | PathPrefix(/ui/v2/login) |
zitadel-login |
— |
| 200 | PathPrefix(/api) |
zitadel-api |
stripprefix=/api |
| 100 | Everything else (OIDC, SAML, gRPC, gRPC-web, API v2 REST, ...) | zitadel-api (h2c) |
— |
No dedicated gRPC router is needed: Traefik's h2c backend scheme forwards gRPC and gRPC-web transparently. API v2 is served as REST/JSON via the gRPC-gateway at /v2/... paths.
Both web (HTTP) and websecure (HTTPS) entrypoints have identical router sets.
Why this routing model
/apialias exists for DX — tools can usehttps://auth.example.com/api/...- Canonical paths (e.g.,
/.well-known/openid-configuration,/oauth/v2/...) must remain at root for OIDC/SAML protocol compliance - gRPC, gRPC-web, and REST all share the catch-all router — no separate gRPC router is needed because the
h2cbackend scheme makes Traefik forward all protocols transparently over HTTP/2
External Settings Invariant
ZITADEL_EXTERNALDOMAIN, ZITADEL_EXTERNALPORT, and ZITADEL_EXTERNALSECURE must match the public URL that users see. If they don't, ZITADEL returns "Instance not found" errors. This is the single most common deployment issue.
Testing
Local NX targets for testing the compose stack:
| Target | What it does | Requires Docker? |
|---|---|---|
test-config |
Validates all overlay combinations parse with docker compose config |
No (just the CLI) |
test-run |
Builds local images (@zitadel/api:pack + @zitadel/login:pack), starts the stack with docker compose up --wait |
Yes |
test-e2e |
Runs the full Playwright suite (wiring.spec.ts + smoke.spec.ts) against localhost:8888 through Traefik: per-service wiring checks (login, console, OIDC, SAML, API v1 REST, gRPC h2c, gRPC-web, API v2 REST HTTP/1.1 + HTTP/2) and the browser login flow |
Yes (stack must be running) |
test |
Lightweight — delegates to test-config only. Safe for nx affected |
No |
test-full |
Full pipeline: test-config → test-run → Playwright wiring + browser tests → teardown |
Yes |
stop |
Tears down the zitadel-compose-test stack and removes volumes |
Yes |
Rejected Alternatives
| Alternative | Why rejected |
|---|---|
Strict /api-only rewrite |
Breaks canonical protocol paths (OIDC, SAML) |
/grpc path-prefix routing |
gRPC clients/tools don't use path prefixes |
Dedicated gRPC router (HeaderRegexp(Content-Type, ^application/grpc.*)) |
Redundant: h2c backend handles gRPC, gRPC-web, and Connect-RPC natively; the catch-all covers all three |
| Single container (API + Login) | Not aligned with v4 architecture; Login is a separate Next.js process |
| Merged TLS configs | Each TLS mode must remain independently composable |
network_mode: service: for Login |
Fragile, port conflicts, doesn't work with Traefik routing |