Changelog

Servers of their own for bigger apps: a dedicated Postgres or Redis per app, a whole stack on one reserved node, and dedicated agents on k3s. The k3s runtime ran its first live cluster and got its own dashboard page.

Dedicated databases and servers

• Dedicated Postgres: services add --dedicated --node <n> <app> postgres runs the app's own Postgres 17 on a node you pick, with its own data volume. It listens only on the node's tailnet address and only over TLS (per-instance CA, hostssl + SCRAM, plaintext refused). Nightly dumps, WAL-G point-in-time recovery, restore, swap and the restore drills cover it like the shared cluster; its WAL archive sits beside the shared one, never inside it.
• Runtime role: add --runtime-role <name> and Girder creates a least-privilege login (no superuser, no RLS bypass, 40 connections) and keeps its password. The owner role can grant to it. Web and worker attach with services add --from <owner> --role runtime and never see the owner password. The CA is injected as DATABASE_CA_CERT_B64 for clients that verify the server.
• Dedicated Redis: services add --dedicated --node <n> <app> redis moves an app's Redis off the shared data node. services rebind-redis keeps dedicated instances on the pinned image too.
• A server per stack: apps that share a stack (web, worker, migrate) can all claim the same reserved dedicatedNode. nodes create --reserved creates such a server in one step.
• Dashboard: an app's services card adds dedicated… and attach from app…, labels where each service lives, and the provision form has a reserved box.

k3s runtime

• Dedicated agents: an app on the cluster can claim a k3s agent. Girder labels and taints it for that stack, pins the app's pods there and moves other apps' pods off. Releasing the claim returns the agent and unpins the app. The autoscaler never removes a claimed agent.
• k3s cluster page in the dashboard: turn the cluster on or off, the pod egress filter, kubeconfig, servers and agents with readiness and dedication, and every app on the cluster with pods ready/desired.
• First live cluster: the control node now installs a verified kubectl itself, and pods authenticate to Girder's image registry. Pods may reach the tailnet endpoints in their own apps' env, so a dedicated Postgres or Redis works from k3s.

Security and DNS

• Pinned images: PgBouncer 1.26.1 (fixes a SCRAM authentication bypass), Traefik 3.7.16, Redis 7.4.11 and Go 1.26.10. Floating tags were never re-pulled, so these are now exact versions.
• Fresh base images: every app build pulls its base image again, falling back to the cached copy if the registry is unreachable.
• appHeaders: apps config <app> appHeaders true lets an app send its own HSTS and Referrer-Policy instead of the platform's.
• DNS: dns set creates MX and CAA records as well, from the CLI, MCP and dashboard.

Faster pages far from your server, dedicated-node apps such as a mail server, tighter network isolation between apps, and a long list of reliability and security fixes.

Speed

• Edge HTML cache (opt-in): apps config <app> edgeCache true creates one Girder-owned Cloudflare Cache Rule for the app's proxied production hosts. Only requests without cookies are cached, and the edge TTL follows the origin's headers, so private, no-store and Set-Cookie responses are never cached. Every deploy and rollback purges the app's hosts. Rules you created yourself are never touched.
• Postgres headroom: the shared Postgres allows 200 connections and runs its newest patch release. Connection pressure now sends an alert as well as showing on the dashboard.

Dedicated-node apps

• Stateful, host-network apps (the reference case is a KumoMTA mail server): dedicatedNode reserves a node for one app, network=host gives it the node's own IPs, volumes keeps named volumes across deploys, and publicPorts opens exactly those ports in ufw and in the node's own Hetzner firewall. These apps deploy stop-then-start, so two copies never run at once. If the new copy fails, the old one is restored.

Security

• Per-app tailnet egress: an app container can reach only the tailnet endpoints in its own env (its database, Redis, S3, or a URL you set on it). It cannot reach other apps' services or other devices on your tailnet. Grant a service to every container with config set tailnetEgressAllow <ip>:<port>. Optional for k3s pods: config set clusterTailnetEgress true.
• Rate limits split by origin: traffic through Cloudflare is limited by the visitor's real IP, and direct traffic by the TCP peer. Proxied hosts return 404 when hit directly.
• TLS: every Girder Traefik drops the CBC and SHA-1 cipher suites.
• Traefik admin API is private (read through docker exec, no published port). Toolchain upgraded to Go 1.26.9 and x/crypto v0.56.0.

Reliability

• Self-heal: an app that stays down is restarted again at 5, 15 and 45 minutes. Failed alert deliveries are retried in the background, and a single fast transport error no longer marks a host down.
• Backups: restic runs and pg_dump lock waits have time limits, the restore drill no longer fails on other apps' grants, and an alert fires when the nightly backup has not run for 26 hours.
• Nodes: exec'd processes are reaped (--init), SSH logins to nodes cost less, owned nodes left stranded on their public IP are repaired, and imported Hetzner servers can be powered and resized from the dashboard again.
• Disk: the nightly prune removes dangling images on every host, plus images left behind by deleted apps (registry gc too).
• Fleet cost includes the primary IPv4 and shows the VAT-inclusive total, imported Hetzner servers included.
• Audit log survives entries zeroed by a crash and very long lines. Rollbacks are serialized with deploys of the same app. Releases refuse to republish a version that already exists.

A reliability and observability pass, triggered by "sites sometimes feel slow". We measured first (origin uptime was already 99.94–99.99%), then fixed what the numbers showed. Every CLI command now has a dashboard control.

Observability

• Per-app latency: p50 and p95 response time plus 5xx counts on the stats page and through the app_stats MCP tool. Download routers are excluded so large files do not skew the numbers.
• Docker network headroom per node, with a readiness check that warns before a node runs out.
• Uptime summaries come from in-memory day totals, which takes 0.08s instead of parsing 75 MB per request. The history file is no longer rewritten in place, which had shown readers a fake gap.

Zero-blip deploys

• A deploy waits until Traefik actually routes to the new container before retiring the old one. A recreated Traefik joins app networks before the swap.
• ECDSA certificates and a 15-minute idle timeout on Traefik. A guard catches blue/green router-label conflicts.
• Docker address pools moved off the provider gateway range. Preview teardown frees its network.
• An app confirmed down for 5 probes gets one automatic restart (for apps that lose the race with the database at boot).
• Node DNS moved off flaky provider resolvers.

Dashboard parity and providers

• Every operator command in the dashboard: tokens, DNS, cluster, watchdog, registry gc, license, service migrate, Garage join, Redis rebind and more, through one background-ops runner with a live log.
• Hetzner Robot (dedicated and auction servers): nodes robot list, nodes import --robot <number> <name>, reset, power and cancel. The server number is checked against the host's IP before anything happens.
• DigitalOcean: nodes create --provider digitalocean with create, destroy, power and resize.
• All data services bind privately. services rebind-redis moves an existing wildcard-bound Redis without data loss.

Security audit

• Token guessing is throttled per IP. One app cannot claim another app's or the platform's hosts, and DNS records Girder did not create are never repointed.
• Build-arg values are kept out of build logs and argv. Staging scrub SQL runs as the staging role, plain SQL only. App names cannot contain --, and the health-check URL is shell-quoted.

Bring your own servers, scoped credentials, and proof that a fleet can move to another provider.

• Bring your own server: nodes import <name> root@<ip> --public-ip <ipv4> --host-key SHA256:… (or connect an existing server in the dashboard) onboards any Ubuntu 24.04 machine, amd64 or arm64. It preflights OS, arch, root and capacity, joins Tailscale before locking ufw, and registers the node only once it is ready. It never formats disks or orders servers. Retrying after a failure is safe.
• Scoped API tokens: girder tokens create --role owner|admin|developer|viewer --apps a,b. Tokens are revocable, and access checks are enforced per request.
• Provider identity: each node records which provider and resource it is. Destroy and reset only apply to servers Girder created, and a failed or ambiguous create never deletes a server Girder does not own.
• Cross-provider recovery: the control plane can be restored onto another provider from backups (documented and drilled).
• Faster sites: compression for app responses at the edge. The dashboard serves revalidated, compressed assets.
• Dashboard redesign: panels load independently, stale responses are dropped, and status reads are bounded and coalesced. Private data-service ports stay private.

An optional Kubernetes (k3s) runtime next to Docker, templates, edge intrusion prevention, native alerts, and a production readiness score.

k3s cluster runtime (opt-in per app)

• Girder-managed k3s: nodes create --k3s-server / --k3s-agent brings up a cluster. cluster status reports health (degraded below 3 servers). cluster kubeconfig prints a kubeconfig whose API server is reachable on the tailnet only.
• Per-app opt-in: apps config <app> nodes cluster deploys the app as a Kubernetes Deployment behind Girder's Traefik (a DaemonSet on server nodes). Pods are rescheduled when a node is lost. Docker stays the default runtime, and both runtimes are supported permanently.
• Autoscaling: minReplicas, maxReplicas and targetCPU drive a native HPA. Logs, exec, restart, staging and rollback work the same as on Docker.

Operate

• Templates: a catalog of one-click scaffolds (static, Node, Go, Python, worker, plus Postgres and Redis combinations), with a gallery in the dashboard and apps templates in the CLI.
• CrowdSec on Traefik access logs, with a firewall bouncer. The dashboard security page shows doctor checks, auth-failure watch, unmanaged containers and high-risk audit entries.
• Alerts: native Slack, Discord and ntfy webhooks with no middleman, set up from a Config card.
• Production readiness checklist: a pass/warn/fail score for SPOFs (offsite backup, watchdog, notify, placement), available in the dashboard, doctor and MCP.
• Backup replicas: nodes create --backup installs garage-backup and wires restic automatically. A nightly restic copy goes to resticReplicaRepository. Dead replicas are pruned from multi-A DNS (the last IP is never removed).
• Observability charts and a log console with filter, since and tail. The Config UI controls how many rollback images are kept, with a prune-now button. abort_deploy MCP tool.

Staging environments, deploy visibility, scale hardening, Docker resource caps, Polar license plumbing, and website polish. Grouped by theme.

Staging environments

• Built-in staging per app: push a staging branch (default staging) to a durable host staging--<app>.<appsDomain>, optional own node, promote image to production without copying the staging database.
• Isolated data by default: staging Postgres (<app>_stg), Redis, and S3 bucket; stack workers share the owner’s staging DB. Opt-in share of prod Postgres is discouraged.
• Seed / scrub / S3 mirror: copy prod DB into staging (optional scrub SQL); mirror production bucket into the staging bucket; seed-on-enable option.
• Idle sleep, auto domains, stack promote: sleep after N hours idle; auto staging.<custom-domain> when prod has customs; enable/promote whole stacks from the dashboard or CLI.
• Backups skip staging and preview DBs so coverage does not flag disposable environments.

Scale and control plane

• buildNode: config set buildNode <node> runs docker build on a registered node (tar-streamed context, push to internal registry, scan on control). Empty keeps builds on control.
• Deploy queue: default two workers (different apps in parallel; one job per app); same-app+branch supersedes older queued jobs. pipelineWorkers config (1–4). GET /api/queue, overview card, status like queued (2/3).
• Resource caps: per-app memoryMB, pidsLimit, and cpus (docker --cpus); clearable; dashboard memory presets and multi-select replica placement.
• Backup window metrics: each run writes last-run.json (total duration, per-DB dump time/size, restic/mirror). Dashboard Backup → last window; GET /api/backup/metrics. Dumps run up to two at a time to shorten the window.
• Redis defaults: new Redis containers get a 256MB memory cap with allkeys-lru eviction so one app cannot fill the data node.
• Postgres pressure: connections vs max_connections (warn 70%, critical 90%), by database. Backup page card and overview alerts; GET /api/postgres/pressure.
• Control load budgets: ~10s app-list cache for the dashboard, ~15s probe cache, max 16 concurrent HTTP probes, remote docker stats every few minutes. Status includes last probe pass duration and buildNode.
• One-click update: overview banner when a newer release exists; Update now runs POST /api/update (download, checksum, swap, delayed restart). CLI girder update uses the same path.

Deploy pipeline

• Stage clocks: per-step elapsed time on the Deploys pipeline; live duration while running.
• Cancel queued / redeploy / plan: drop waiting jobs; re-queue a sha with current env; dry-run plan before apply.
• SSE progress + outputs: live stage stream; machine-readable url/image after finish for agents and CI.
• Preview TTL + destroy: previewsTTLHours, lease auto-destroy, dashboard/API teardown for previews; sleep staging from the card.
• Timeline + expiry radar: Activity merges deploys with audit; Status lists cert, license, and preview expiries.
• Deploy freeze: block production deploys for go-live (staging/previews optional); force override on redeploy when needed.
• Agent tools: MCP deploy_plan, redeploy, deploy_progress, deploy_outputs, timeline, expiry_radar, freeze helpers.
• Deploy feed: sidebar deploys page with expandable cards, day groups, filters, build logs, rollback. Runner-style progress during git push build as well as queue/deploy.

Support, product, site

• Support bundle: girder support-bundle or Config → download support bundle. Redacted .tar.gz (no secrets).
• Garage RF upgrade: status + guided RF=2 after a peer is joined.
• License editions + dashboard card: edition on signed licenses; Config → license status; PE-1 authz foundation (multi-user not wired).
• License commerce plumbing: services/license-bridge for Polar order.paid webhooks → signed license email (Resend). Site buy CTAs ready when checkout URLs are set. Ops: docs/LICENSE-CHECKOUT.md.
• Agency playbook and backupSkip for demo/throwaway apps.
• Website: AI-builder homepage and demo, SEO/perf pass, waitlist modal, changelog in header.

• Multi-node app replicas: opt-in nodes: [data-1, data-2] deploys the same image to every listed node, multi-A DNS to healthy IPs, partial success if one node fails. CLI/dashboard config key nodes.
• Preview branches: previews: true puts non-deploy branches at <slug>--<app>.<appsDomain>; branch delete tears them down.
• Build args & deploy hooks: per-app buildArgs for docker build --build-arg; deployHook runs in the new container after health, before traffic flips (failure rolls back).
• App templates: girder apps create --template static|nextjs|worker (dashboard create form too).
• Movable registry: internal Docker registry can leave the control node via registryNode / girder registry migrate (and dashboard move).
• Weekly auto restore + PITR drills on the backup page; disk auto-prune at 85%; deploy queue position in the deploying state.
• Dashboard config page: write-only secret fields (token rotation and R2 flip without SSH).
• Reliability: remote apps no longer flap red on transient SSH/inspect failures; post-health step failures no longer mark a live deploy failed; registry health reflects the actual registry node.

• Licensing model: a valid license runs forever; the expiry date is an update entitlement window. girder update refuses versions released after the window (with a renewal message). Each publish writes a RELEASED marker; unmarked versions are grandfathered.
• girder update --to <version> for pin/downgrade; girder update --list lists available versions.
• Dashboard status: transient check failures show amber unknown instead of flip-flopping red; confirmed outages only after sustained failures.
• Dashboard assets served with Cache-Control: no-cache so hard-refresh is no longer required after upgrades.

• Visitor analytics: a stats page shows requests, unique visitors (Cloudflare-aware real IPs), sessions, and average session duration per app, computed from Traefik access logs, no cookies, no scripts on your sites.
• No more false down alerts: outages now require 3 consecutive failed probes before paging or turning an app red. Transient blips show as amber "flaky" instead.
• The Girder logo lands on the admin: sidebar brand, login card, and favicon.
• Releases now publish at dl.rungirder.com. The installer and self-update live on the Girder domain.
• Rollback now ensures the node's ingress exists first. Rolling back while a node's Traefik is down no longer blackholes the app.
• girder update --force re-fetches even when the version matches, for repairing a broken same-version release.
• Skills renamed to girder-deploy and girder-install; dashboard wording, SECURITY.md, and docs fully rebranded.

• The platform is now Girder: Go module, binaries (girder, girder-mcp), CLI, dashboard, installer, and docs rebranded end to end.
• New installs land in /opt/girder with girder.service; existing installs keep running untouched. Runtime identifiers are unchanged.
• The MCP server registers as girder. Agents deploy and manage the platform under the new name.

• Deploy apps from any git branch, not just main.
• Control-plane watchdog: get paged when the platform itself goes down.
• The restore drill now verifies the offsite object mirror too.
• Deleting an app removes its DNS records along with its containers and services.
• Point-in-time recovery drill runs from the dashboard; build logs refresh live.
• Provision and manage nodes through the MCP server.
• Weekly automatic node patching (opt-in).
• HSTS and transport security headers on every app by default.
• Cloudflare proxying per custom domain (CDN and DDoS protection).
• Cost visibility: live node prices on provision, rescale, and disk growth.
• Backup coverage check flags any database without a fresh dump.
• Uptime probes now cover custom domains, and a global domains page checks every host on demand.

• Initial control plane: git-push deploys with streaming build logs, zero-downtime rollouts, automatic rollback, and the web dashboard.
• Multi-node: provision Hetzner servers from the CLI or dashboard, per-node ingress and DNS, and a dedicated data node for shared Postgres, Redis, and S3.
• Move an app's database between nodes without losing data; deploy history with click-to-rollback.
• Tamper-evident audit log of every change; Trivy scans on every built image.
• Signed customer licenses, enforced at daemon start in release builds.
• Nightly backups with restore drills, per-database dumps, and WAL-G point-in-time recovery.
• Runtime logs over the API, per-node disk usage, and node power controls.