Changelog
All notable changes to RunR are documented in this file.
The format is based on Keep a Changelog. Version numbers are the Helm chart versions; a release whose application images did not change says so.
Unreleased#
0.7.5 - 2026-11-15#
Upgrade notes#
- No migration, no new Helm value. The region Job moves the region agent and the event gateway (
agent,queue-consumer) to 0.7.5 as in every release; both images and the supply-chain image 1.1.2 (from 1.1.1) build on alpine 3.22. A region that pulls through a mirror of its own needsagent:0.7.5,queue-consumer:0.7.5andsupply-chain:1.1.2there first (0.7.4 → 0.7.5 in upgrade.md). - kube-prometheus-stack 93.3.0 does not reach an installed region by itself:
php artisan cluster:patch-observability [REGION-ID]moves it when it suits the region (the Prometheus Operator, Prometheus and node-exporter restart once). No CRD step is needed. - Your own monitoring objects: RunR needs no CRD step. If you create Prometheus Operator objects of your own (ServiceMonitors, PrometheusRules, AlertmanagerConfigs) with fields that v0.95.2 adds, apply the chart's CRDs on the region's cluster first;
--force-conflictsis needed because Helm created the installed CRDs and owns their fields:helm show crds kube-prometheus-stack --repo https://prometheus-community.github.io/helm-charts --version 93.3.0 | kubectl apply --server-side --force-conflicts -f -. - The region Traefik moves to v3.7.18 by itself (
region:reconcile, steptraefik-chart). A region whose nodes pull through a mirror of their own needsghcr.io/brf-tech/runr/reqs/traefik:v3.7.18there first; withknative.traefikImageset, the region keeps that image on the new chart. - Traefik v3.7.18 fixes five upstream advisories (four high, among them a bypass of path-based authorization through path-parameter dot-segments, GHSA-cp28-5662-543f, on entry points with
sanitizePath, which RunR's regions enable) and changes what they touched: a request for/a/..;x/breaches the function as/b, and BasicAuth refuses passwords over 1024 bytes (Traefik's migration notes). - CLI and Terraform provider 0.7.5 are the first since 0.7.3 (0.7.4, a dependency release, shipped neither): the code of 0.7.3, built with the CLI's and the provider's Go modules below. Their commands, flags, resources and data sources stay those of 0.7.3; the CLI's help writes a boolean flag as
--debug[=true|false]. - Back to 0.7.4 is a
helm rollback; nothing runs on the database. The region Traefik stays on chart 42.4.0 (0.7.4'sregion:reconcilemoves no Traefik to an older chart and warns that it stays), and a region moved to kube-prometheus-stack 93.3.0 stays on it.
Changed#
- CLI Go modules moved (minor and patch updates, nothing a release image ships): github.com/spf13/cobra v1.8.1 → v1.10.2, github.com/spf13/pflag v1.0.5 → v1.1.0, golang.org/x/sys v0.48.0 → v0.50.0, golang.org/x/term v0.32.0 → v0.48.0.
- Sentry tunnel Go modules moved (minor and patch updates, nothing a release image ships): github.com/cespare/xxhash/v2 v2.2.0 → v2.3.0, github.com/dgryski/go-rendezvous v0.0.0-20200823014737-9f7001d12a5f → none, github.com/redis/go-redis/v9 v9.7.3 → v9.23.0, go.uber.org/atomic none → v1.12.0, golang.org/x/sys none → v0.48.0.
- Terraform provider Go modules moved (minor and patch updates, nothing a release image ships): github.com/ProtonMail/go-crypto v1.4.1 → v1.5.2, github.com/hashicorp/go-plugin v1.7.0 → v1.8.0, github.com/hashicorp/hc-install v0.9.4 → v0.10.0, golang.org/x/sys v0.48.0 → v0.50.0.
- CI tool pins moved (minor and patch updates, nothing a release image ships): helm v4.3.1 → v4.3.2, terraform 1.13.3 → 1.17.2, govulncheck v1.1.4 → v1.9.0.
- The dependency test bot attempts every update by itself. A new major, a release outside a manifest's constraint or exact pin (the constraint moves mechanically:
^7→^8,==X→==Y, a Go module's new major path with its imports, an Actions pin's new major) and a chart's new major are tested on the weekly run like any other version; a Traefik chart with a new app version gets its image copied into the reqs mirror (ghcr.io/brf-tech/runr/reqs/traefik) by the job Reqs mirror before its suites. What the bot cannot move mechanically, or what fails its suites, gets the labelnext update required(instead ofneeds attention), with the failing suite and its log, and a line in the weekly summary's own section Next update required; every weekly run tries it again, and once it passes it ships, loses the label and closes. Holds (ci/deps/holds.txt) are never attempted and are listed there with their reasons. The major versions issues are closed with a pointer to the summary. - A release of the bot says when a major moved: its upgrade notes,
docs/upgrade.mdand the GitHub release say A new major version moves: …, and the dependency's line says what an installed region does (the step that moves its chart, and for a chart whose CRDs changed the command that applies them). A dry run copies nothing into the reqs mirror: its suites take the upstream image under the mirror's name. - Dockerfile base images moved (minor and patch updates): ghcr.io/brf-tech/runr/reqs/alpine 3.21 → 3.22, ghcr.io/brf-tech/runr/reqs/alpine 3.20 → 3.22, supply-chain image 1.1.1 → 1.1.2 (its own version: release-images.yml builds it). The images build on the new bases; the Spin shim's stays on 3.21 (#80).
- Hugging Face weights example packages moved (outside the constraint
==5.22.2, an exact pin moved by hand; nothing a release image ships): transformers 5.22.2 → 5.23.0 inexamples/ai/hf-python/weights. 5.23.0 changes nothing the example uses (AutoTokenizerandAutoModelloading a mounted snapshot offline); its handler answers on exec-pytorch's torch 2.14.1 (CPU). - Region Traefik chart 42.4.0 (from 42.3.1; Traefik v3.7.17 → v3.7.18). New regions install it;
region:reconcilemoves the Traefik of an existing region to it (steptraefik-chart), one pod at a time. Its image comes from the reqs mirror (ghcr.io/brf-tech/runr/reqs/traefik:v3.7.18); withknative.traefikImageset, the region keeps that image. Only the Traefik Hub CRDs change (an optionalcaSecretNameinAPIAuthandAPIPortalAuth), which RunR does not use: an installed region needs no CRD step. - kube-prometheus-stack 93.3.0 (from 92.22.0, a new major version; Prometheus Operator v0.94.2 → v0.95.2). Regions installed from now on run it; an installed region keeps its chart until
php artisan cluster:patch-observability, a change back to RunR's own monitoring or a new Serving install moves it. Prometheus (v3.16.0), kube-state-metrics and node-exporter keep their versions; the operator's ClusterRole follows upstream's of v0.95 (alertmanagerconfigs/statusadded,deleteon pods gone). On that move the operator, Prometheus (its config reloader) and node-exporter restart once. The operator's new CRDs only add optional fields and validations of fields RunR does not set: an installed region keeps its CRDs (Helm does not upgrade them) and needs no CRD step.
0.7.4 - 2026-11-14#
Upgrade notes#
- A dependency release of 0.7.3: upgrading needs no step. There is no database migration, the images keep their names and nothing has to be set; only the 5 dependencies below change. The CLI, the Terraform provider and runr.sh stay those of 0.7.3. See docs/upgrade.md.
Changed#
- API Composer packages moved (minor and patch updates): doctrine/inflector 2.1.0 → 2.1.1, dragonmantank/cron-expression v3.6.0 → v3.6.1, firebase/php-jwt v7.0.3 → v7.2.1, guzzlehttp/guzzle 7.15.5 → 7.16.1, guzzlehttp/promises 2.5.3 → 2.6.0, guzzlehttp/psr7 2.13.1 → 2.13.3, laminas/laminas-diactoros 3.8.0 → 3.9.0, laravel/framework v12.70.0 → v12.72.0 and 56 more. The release's images carry them.
- Gateway-auth Go modules moved (minor and patch updates): github.com/dgryski/go-rendezvous v0.0.0-20200823014737-9f7001d12a5f → none, github.com/redis/go-redis/v9 v9.7.3 → v9.23.0, go.uber.org/atomic none → v1.12.0, golang.org/x/sys none → v0.48.0, gateway-auth image 1.1.0 → 1.1.1 (its own version: release-images.yml builds it). The release's images carry them.
- Zot chart 0.1.134 (from 0.1.130; Zot v2.1.23, as before). Registries set up from now on run it; an installed Zot keeps its chart and version until
php artisan registry:upgrade-zot <region>, which restarts the registry. - Image tool pins moved (minor and patch updates): helm v4.3.1 → v4.3.2, scalar 1.75.1 → 1.78.1, syft 1.18.1 → 1.57.1, oras 1.2.2 → 1.3.6, supply-chain image 1.1.0 → 1.1.1 (its own version: release-images.yml builds it). The release's images carry them.
- Console npm packages moved (minor and patch updates): @eslint/config-array 0.23.5 → 0.23.6, @intlify/core-base 11.4.19 → 11.4.20, @intlify/devtools-types 11.4.19 → 11.4.20, @intlify/message-compiler 11.4.19 → 11.4.20, @intlify/shared 11.4.19 → 11.4.20, @isaacs/nonrecursive-glob-match none → 1.1.0, @lucide/vue 1.72.0 → 1.73.0, chokidar 5.0.0 → 6.0.0 and 9 more. The release's images carry them.
0.7.3 - 2026-11-14#
Upgrade notes#
- Nothing an installation runs changes: upgrading from 0.7.2 changes the version only. No migration, no new value; the images of 0.7.2 are built again at 0.7.3 (the API's with one line less that did nothing: the tenancy ticket list no longer calls
array_values()on a list, and its response is the same), and the region Job moves the agent and the event gateway to 0.7.3 as in every release (0.7.2 → 0.7.3 in upgrade.md). - Back to 0.7.2 is a
helm rollback; nothing runs on the database.
Added#
- runr.sh and the README point to the live demo. demo.runr.sh, a read-only RunR that resets every night, gets a Live demo button in runr.sh's hero (and its mobile menu, English and Turkish) with the sign-in under it (
demo@runr.sh/runr-demo), and a badge and a line at the top of README.md and README.tr.md. Until now nothing linked to it.
Changed#
- GitHub Actions pins moved (minor and patch updates, nothing a release image ships): docker/login-action v4.6.0 → v4.8.0.
Fixed#
- The dependency test bot tests its green bumps together even when one of the run's bumps went red. The combined-tree jobs had no status function in their
if, so GitHub skipped them after any red bump of the first phase. The bot then read the missing results as a failure: 0.7.2 took only the first green bump of each path, and two bumps that passed (#78 zot 0.1.134, #30 go-cli) were labelled "needs attention". A combined job that leaves no result now blames no bump: the first goes, the others wait for the next run with no label.
0.7.2 - 2026-11-14#
Upgrade notes#
- A dependency release of 0.7.1: upgrading needs no step. There is no database migration, the images keep their names and nothing has to be set; only the dependency below changes. The CLI, the Terraform provider and runr.sh stay those of 0.7.1. See docs/upgrade.md.
Changed#
- Region agent Go modules moved (minor and patch updates): github.com/emicklei/go-restful/v3 v3.11.0 → v3.13.0, github.com/fxamacker/cbor/v2 v2.7.0 → v2.9.1, github.com/go-logr/logr v1.4.2 → v1.4.3, github.com/go-openapi/jsonpointer v0.21.0 → v1.0.0, github.com/go-openapi/jsonreference v0.20.2 → v1.0.0, github.com/go-openapi/swag v0.23.0 → v0.27.1, github.com/go-openapi/swag/cmdutils none → v0.27.1, github.com/go-openapi/swag/conv none → v0.27.1 and 38 more. The release's images carry them.
0.7.1 - 2026-11-14#
Upgrade notes#
- Upgrading from 0.7.0 needs no migration and no new value, but the scheduler's new default has to reach the release. The images of 0.7.0 are built again at 0.7.1, and the scheduler pod is replaced once.
helm upgrade --reuse-valuesrenders with the defaults of the chart the release runs, so it would keep the scheduler at64Mi/256Mi: add--set scheduler.resources.requests.memory=160Mi --set scheduler.resources.limits.memory=512Mi, or upgrade with--reset-then-reuse-valuesinstead (0.7.0 → 0.7.1 in upgrade.md). - Back to 0.7.0 is a
helm rollback; nothing runs on the database.
Changed#
- The dependency bots report in one weekly summary. Instead of an issue per finding, the daily release check and the dependency test bot keep one issue per week,
deps: week <ISO week>, with one row per dependency and its state (new,testing,green,green → shipped in vX.Y.Z,red,needs attention,held (<reason>),closed); a quiet week opens none, and the rows still open carry over to the next one. A security finding is a row flaggedsecurity: <advisory>and is still tested the same day. A separate issue only where a person has to do something: a bump that went red, a test run that needs a person, and one issue per group for its major versions (closed when its list is empty); a major held with a written reason is aheldrow instead. The findings filed as issues of their own before are folded into the first summary. See docs/development.md.
Fixed#
- The scheduler no longer runs out of memory. Since 0.7.0 the run at the top of the hour can take longer than a minute, and then it overlaps the next minute's run.
schedule:work, eachschedule:runand every artisan command they start hold about 70 MB each. With those overlapsrunr-schedulerwent past its 256Mi limit and was OOMKilled several times a day: 5 times in its first 31 hours on both test regions. The default is now requests50m/160Miand limits256m/512Mi(scheduler.resources). A release that setsscheduler.resourcesitself keeps its own values.
0.7.0 - 2026-11-13#
Upgrade notes#
- Upgrading from 0.6.13 needs nothing set. Two migrations, central and per organization:
knative_regions.kubernetes_version(a new column) and the WebAssembly pin below. The region Job records each region's Kubernetes version and moves the region agent and the event gateway to 0.7.0, gateway-auth to 1.1.0 and, on a region with WebAssembly, the Spin shim to v0.27.2-runr.1 (one node at a time, no container runtime restart); a region that pulls through a mirror of its own needs these images there first (0.6.13 → 0.7.0 in upgrade.md). - WebAssembly: the migration pins the existing
wasm-rustandwasm-jsprojects without a runtime version to 4.1, so the new default 4.2 does not move them to an SDK their code was not written for; nothing has to be done (docs/upgrade.md). - Helm 4: RunR's region releases stay client-side and upgrade as before. With your own Helm 4, do not pass
--server-side=trueto the upgrade of an existing release (Helm 4 in upgrade.md). - kubectl: a
SANDBOX_KUBECTLset throughextraEnvstill names one kubectl for every region; take it out to let RunR run the kubectl of each region's Kubernetes minor (kubectl per region). - A patch feed of your own signed with cosign 3 needs
--use-signing-config=false --new-bundle-format=false --tlog-upload=false(docs/security.md). - Back to 0.6.13 is a
helm rollback; the column and the 4.1 pins stay. Set a project moved to 4.2 back to 4.1 first: 0.6.13 has no 4.2 (Back to 0.6.13).
Added#
- WebAssembly runtimes 4.2, next to 4.1, the default of new projects.
wasm-rustandwasm-jshave a second version with the next SDK generation: Rust on the Spin Rust SDK 8 (#[http_service] async fn,wasm32-wasip2, WASIp3), JavaScript bundled with esbuild 0.28. New projects get 4.2 unless they pick 4.1 (the runtime picker,runtime_version); the starter code is the picked version's. 4.1 is unchanged, and the upgrade pins the existing projects without a version to it (see the upgrade notes). Code of one generation does not build on the other: a project moves to 4.2 with its code ported, byPUT /projects/{id}{"runtime_version": "4.2"}(docs/wasm.md);examples/wasmis 4.2's code. Both versions run on the same Spin shim (Spin 4.2.6); the images areghcr.io/brf-tech/runr/exec-wasm-{rust,js}:4.2of BRF-Tech/runr-runtimes.wasm-gostays on 4.1 (TinyGo, Spin Go SDK v2): the Spin Go SDK v4 needs WASIp3 support in a Go release. - A runtime version can bring its own starter and build files (
versions[].template,versions[].build.filesin the runtime manifest: docs/runtimes.md), for an SDK generation side by side with the one before it.GET /runtimes/{id}/templatetakes?version=, the console's starter preview shows the picked version's, andrunr runtimes template --runtime-version/runr init --runtime-versionfetch it. A project moves to another version of its runtime withruntime_versioninPUT /projects/{id}. - A new API key works at once on every gateway-auth pod. A pod that meets a key or a route it does not know asks RunR again before it answers (the on-demand sync) instead of answering 401 (or 503
route_unknown) until its next fetch, up toSYNC_SECONDSlater: a key used right after its creation no longer gets 401 from one pod and 200 from another. One such fetch runs at a time per pod, at most one every 2 seconds (apiProduct.onDemandSyncSeconds,0turns it off), so a flood of invalid keys costs RunR little; a revoked key still stops withinSYNC_SECONDS(docs/api-product.md). It comes with gateway-auth1.1.0, which a region gets from the upgrade'sregion:reconcile.
Changed#
- The API runs the kubectl of each region's Kubernetes minor. kubectl is supported within one minor of the API server it talks to, and RunR accepts clusters from Kubernetes 1.30 on: the API image carries kubectl 1.31.14, 1.33.13, 1.35.10 and 1.37.2 (
/usr/local/lib/runr/kubectl/<minor>/;kubectlon the PATH is 1.37), and the console's terminal and the sandbox commands that go throughkubectl execrun the one of the region's minor, else the one below it, else the one above. A region records its cluster's version (knative_regions.kubernetes_version, a new column) when its kubeconfig is checked and when it moves to another cluster; theregion:reconcileof the upgrade records it for the existing regions and every run reads it again. A cluster newer than 1.38 takes kubectl 1.37 and the API logs a warning once;SANDBOX_KUBECTL(now unset by default) still names one binary for every cluster (docs/regions.md). The chart's bootstrap Job reads and patchesrunr-adminwithrunr-kubectl, which picks the same way by the install cluster's version (GET /version). The dependency bot moves each kubectl within its minor and reports a Kubernetes minor newer than the newest one's. - Helm 4. The API image runs Helm 4.3.1 (0.6.x: 3.16.0) for the releases RunR installs on the region clusters, and CI checks the chart with it. RunR applies and waits for those releases as Helm 3 did (
--server-side=false,--wait=legacy,--rollback-on-failurefor--atomic): an existing region's releases upgrade as before, a region added now gets them the same way, and nothing has to be done. The chart installs and upgrades with Helm 3 or Helm 4; with Helm 4 an upgrade of a release installed with Helm 3 stays client-side, and a new install is server-side (docs/upgrade.md). - The API reference (
/api/docs,/api/docs.json) is built with Scramble 0.13: the same 687 operations, now in an OpenAPI 3.1.2 document. The success responses of 304 operations say more (the items of their lists, the fields of their objects, constant values; 50 of them read onlystringorobjectbefore), the GitHub, GitLab and SAML callbacks answer 302, and the traces' endpoints have their success responses. The API takes Laravel Tinker 3 and laravel-modules 12 (the modules' configuration has the new generator keys; the modules are unchanged). - The Python MCP starter runs on the MCP Python SDK 2.
server.pyof a new Python MCP server (Projects → New MCP server → Python,mcp_server_create) usesMCPServer, FastMCP's name in SDK 2, and itsrequirements.txtasks formcp>=2.3,<3; the tools, the gateway signature check and the caller's identity work as before, and 2025-era MCP clients are still served. A server made from the earlier starter keeps its own files and SDK 1; to move it, compare it withexamples/mcp/python. The TypeScript starter takes Zod 4 (zod ^4.6.7with@modelcontextprotocol/sdk ^1.23.0);server.tsis unchanged. - The console's dependencies: Vite 8, Vitest 5, jsdom 31, vue-router 5, Pinia 4, Sentry 11, xterm.js 6 and the Lucide icons from
@lucide/vue1 (lucide-vue-nextended deprecated at 1.0.0), with the minor and patch updates of its other packages. The console works as before: Alt+arrow in the project and sandbox shell still moves by word, a dev stack withVITE_SENTRY_DSNsends Sentry what it sent before, and seven icons have Lucide's redrawn shapes (the clock calendar, the check and help circles, the credit card, the key, the bolt). Building the console needs Node.js 22.12 or newer, its unit tests 22.22.2, 24.15 or 26. - The SDKs' development dependencies: the Sandbox SDK (Node) builds with TypeScript 7 and
@types/node26, the Workflow SDK (Node) also tests on Vitest 5 (development needs Node.js 22.12 or newer); the published packages still run on Node.js 18 and their output is the same. The Python SDKs accept websockets up to 17 (Sandbox), test on pytest up to 9 and build with setuptools up to 84 (Workflow, whose license is now the SPDX expressionMIT). - The Python SDKs require Python 3.11 or newer (
runr-sandboxandrunr-workflows:requires-python >=3.11; Python 3.9 and 3.10 are past their end of life); their classifiers name 3.11 to 3.15. Both pass their tests on Python 3.11, 3.13, 3.14 and 3.15, and on CI's 3.12. - The PyTorch weights example (
examples/ai/hf-python/weights) pins transformers 5.22.2. - CI: Node.js 26 (the current LTS line) in every workflow, Kind v0.33.0, cloud-provider-kind v0.12.0, kubeconform v0.8.0 and crane v0.22.1, each with its checksum. The Kind suites (the weekly e2e and the dependency bot's) run the newest Kubernetes only: Kind v0.33.0's node image of Kubernetes 1.37.0, from the reqs mirror, with kubectl 1.37.2; the chart is checked with kubeconform against Kubernetes 1.30, the oldest RunR supports, and 1.37.
- Go 1.27. Every module (the region agent, the queue consumer, gateway-auth, image-upload, the CLI, the Terraform provider, the dev stack's Sentry tunnel and the Q interpreter example) declares
go 1.27.4, CI runs Go 1.27.4 and the images build onreqs/golang:1.27-alpine: go1.27.4 also carries the standard library's security fixes of go1.26.11 (cgo, the go command, the compiler, the linker,net/http). gateway-auth is1.1.0and image-upload1.2.0now, so the release builds both again. - The examples' images build on Node 24 and Python 3.15: the MCP starters' Dockerfiles (
reqs/node:24-slim,reqs/python:3.15-slim) and the GPU CaaS example (reqs/python:3.15-slim). Themcp-nodeandmcp-pythonruntime templates take them too (reqs/node:24-slim,reqs/python:3.15-slim; the Python starter'smcpanduvicorninstall on 3.15 from wheels). A runtime registered from a template earlier keeps its manifest:PUT /api/runtimes/{id}with the template'sruntime_manifest(GET /api/mcp-servers/templates) moves it. - The supply-chain image runs Trivy 0.76 and cosign 3 (
supply-chain:1.1.0; Trivy 0.76.2, cosign 3.1.4). Scans, SBOMs and the scanner DB mirror work as before (the same report schema,trivy-db:2andtrivy-java-db:1). cosign 3 signs into a Sigstore bundle with the public instance's transparency log by default; RunR signs with--use-signing-config=false --new-bundle-format=false --tlog-upload=false, so signatures and attestations stay the.sigand.atttags that policy-controller, the API andcosign verify --key … --insecure-ignore-tlog=trueread. A patch feed of your own signed with cosign 3 needs the same flags (docs/security.md). - The Spin shim is containerd-shim-spin v0.27.2 (
containerd-shim-spin:v0.27.2-runr.1, Spin runtime 4.2.6), built as before with the runwasi fix that gives a Wasm pod's Linux containers their own environment. A region with WebAssembly gets it from the region Job of the upgrade (region:reconcile): the wasm-installer replaces the binary on its nodes one at a time, without a container runtime restart (docs/wasm.md).
Fixed#
- The timed warmer and the kept-warm previous revision on Knative 1.21 and later. Since Knative Serving 1.21 the Revision's autoscaling annotations are copied onto its PodAutoscaler, and a value set on the PodAutoscaler alone is put back at once: the warmer's count and the blue-green
previousrevision'smin-scale: 1did not hold there. On a region with Knative 1.21 or later RunR now sets them on the Revision (no new revision, no restart); before 1.21 it sets them on the PodAutoscaler as before (docs/scaling.md). region:reconcileheals a registry Pending on a dedicated worker on newer Kubernetes. The scheduler of newer Kubernetes (1.35 and 1.37 tested) does not name the taint any more ("had untolerated taint(s)"), and thededicated-worker-tolerationstep did not find the Pending pod of a component installed before its node became a dedicated worker. It reads both messages now.- A region created with the setup wizard records its Kubernetes version (
POST /registry-setup/start), asPOST /knative-regionsdoes, sokubectl exectakes the region's minor from the start rather than after the firstregion:reconcile. - The admission policy's install no longer fails while policy-controller's webhook starts. Right after the install, its Deployment can be available before its Service has a ready endpoint, and the API server refused the ClusterImagePolicies with "failed calling webhook …: connection refused" (the region's admission status
failed). RunR now waits for a ready endpoint of the webhook (up to 2 minutes) and tries a write the webhook could not be called for again, for up to 2 minutes; a settings change andregion:reconciledo the same (docs/supply-chain.md).
0.6.13 - 2026-11-12#
Upgrade notes#
- A dependency release of 0.6.12: upgrading needs no step. There is no database migration, the images keep their names and nothing has to be set; only the dependency below changes. The CLI, the Terraform provider and runr.sh stay those of 0.6.12. See docs/upgrade.md.
Changed#
- Image-upload Go modules moved (minor and patch updates): github.com/klauspost/compress v1.19.2 → v1.20.2, image-upload image 1.1.1 → 1.1.2 (its own version: release-images.yml builds it). The release's images carry them.
0.6.12 - 2026-11-12#
Upgrade notes#
- Nothing an installation runs changes: upgrading from 0.6.11 changes the version only. No migration, no new value; the images of 0.6.11 are built again at 0.6.12 and the region Job moves the agent and the event gateway to 0.6.12 as in every release (0.6.11 → 0.6.12 in upgrade.md).
- Back to 0.6.11 is a
helm rollback; nothing runs on the database.
Fixed#
- The GitHub Action
brf-tech/runr/actions/deployruns where actions must be pinned to a commit. Its CLI cache step usedactions/cache@v4, so an organization that requires every action pinned to a full commit SHA refused the deploy action even when it was pinned itself. The step usesactions/cacheby its commit now (v6.1.0, the same inputs and outputs).
0.6.11 - 2026-11-12#
Upgrade notes#
- A dependency release of 0.6.10: upgrading needs no step. There is no database migration, the images keep their names and nothing has to be set; only the dependency below changes. The CLI, the Terraform provider and runr.sh stay those of 0.6.10. See docs/upgrade.md.
Changed#
- Queue consumer Go modules moved (minor and patch updates): github.com/dgryski/go-rendezvous v0.0.0-20200823014737-9f7001d12a5f → none, github.com/jackc/pglogrepl v0.0.0-20261003132456-662581bb6bb6 → v0.0.0-20261111174325-d50336b09c47, github.com/jackc/pgx/v5 v5.11.0 → v5.13.0, github.com/klauspost/compress v1.20.0 → v1.20.2, github.com/nats-io/nats.go v1.54.0 → v1.55.0, github.com/nats-io/nkeys v0.4.16 → v0.4.17, github.com/redis/go-redis/v9 v9.7.3 → v9.23.0, go.uber.org/atomic none → v1.12.0. The release's images carry them.
0.6.10 - 2026-11-12#
Upgrade notes#
- A security release of the internal network: upgrade from 0.6.0 to 0.6.9. On those releases a project in access mode
allowlistcould lose the IP allow list of its public host, which then answered every client, and a removed, unbound or deleted caller could stay admitted to a project or a model service, until the next deploy (0.6.9 → 0.6.10 in upgrade.md). No migration, no new value; the images keep their names. - The region Job of the upgrade puts lost callers right (
region:reconcile, stepinternal-network: a model service's callers and a project's allow list where the region's differ from the database's). It does not bring back a deleted allowlist route: after the upgrade, check that every project in access modeallowlisthas its IngressRoutesrunr-access-<service>and-tls, and redeploy one that has not (After the upgrade). - The same Job writes the model volume's Knative features (persistent volume claims) into each region's
config-featuresonce (region:reconcile, stepknative-serving-config); nothing restarts (What 0.6.10 changes). - Back to 0.6.9 is a
helm rollback(Back to 0.6.9); nothing runs on the database.
Fixed#
A model service's Knative Service is no longer refused for the model volume. The pod spec features of the model volume (a read-write PVC) were turned on by a region's first model service deploy, and Knative's webhook went on refusing it ("Persistent volume claim support is disabled") for tens of seconds while config-features already carried them, and at times again later: the deploy failed, or a write to the deployed service (its callers on the internal network) was refused. They are on every region from its install now, and every
region:reconcileenables them again (stepknative-serving-config; on a GPU region the GPU features too). A refusal for a feature enables them again before the write is tried again, a CR that could not be read no longer hasconfig-featurespatched behind the Knative Operator, and a refused allow list of a service on the internal network no longer fails the region's internal-network change (test-internal-network'sin-cross-orgfailed so).A project bound to a model service is admitted by it, also when the region did not take the binding's write. The internal network's authz reads a model service's callers from its Knative Service, where a binding added or removed was written once: a refusal or an error of the region's API server, or a failed read before it (taken for "not deployed"), lost it without a word. The bound project was refused (403) until the model service's next deploy, and an unbound one kept being admitted. The write is now tried again until the region takes it (for about twelve minutes; then
region:reconcile, whoseinternal-networkstep now also writes a model service's callers when they are not its bindings). A binding made while the model service deploys is written after the deploy's apply as well.test-internal-network'sin-modelfailed so on GitHub-hosted runners: in-api was still refused six minutes after its binding.A deleted project leaves the model services it was bound to, and a renamed one is its new name there. Its bindings went with its row without a word to the region: the model service kept admitting its old name, so a later project of that name in the organization was admitted without a binding, and a renamed bound project was refused (403) under its new name. Both reach the model service at once now, tried again like a binding.
A change of a project's allow list or access mode that the region did not take is tried again. It was saved and answered with a
warning, and waited for the project's next deploy: meanwhile a removed caller, or one deleted with its project (a later project of its name too), was still admitted. The change is now tried again for about twelve minutes, andregion:reconcilewrites a project's allow list when its Knative Service has another.A failed read of a project's Knative Service no longer removes its routes. A change of its network took the failed read for a project without a Knative Service and deleted its
runr.internalroute and, in access modeallowlist, the IP allow-list route of its public host, which then answered every client until the next deploy. Only a missing Knative Service removes them now; anything else is a change the region did not take (above).
Security#
- The dependency bot's secrets are secrets of the GitHub environment
release, which onlymaindeploys to. Its jobsbumpandreleaseare the only ones in it, so a workflow pushed to another branch with a stolen account cannot read the origin's deploy key, the GitHub App's key or the bot's signing key. A fork that runs the bot creates the environment and sets the secrets there (docs/development.md). The dependency audits' workflow isdependency-audit.yml("Dependency audit"), no longeraudit.yml.
0.6.9 - 2026-11-12#
Upgrade notes#
- Two migrations add one column (
2026_11_11_100001_add_capacity_released_at_to_projects, and on a multi-tenant installation0111_add_capacity_released_at_to_projectsin every tenant schema): the nullableprojects.capacity_released_at, set by a resume until a deploy admits the project again. Existing projects keep what they have (0.6.8 → 0.6.9 in upgrade.md). - The projects that are stopped at the upgrade give their capacity back at once in a pay-as-you-go region (every region while billing is off): they count neither in the region's budgets nor in their organization's quota, and their resume has to fit again. A fixed region keeps their reservation and lends their share (Stopped projects give their capacity back).
- A region with many stopped projects can alarm right after the upgrade: the capacity alarm now also watches what the region would need with every stopped project back (
data.basis: "with_stopped"), checked every minute. Look at Region → Workers → Capacity first (The capacity alarm with stopped projects). - Multi-tenant: the capacity alarm of an organization's own region goes to its owners and admins; the platform administrators no longer get its mail. Every capacity alarm mail honours the recipient's
resource_thresholdmail preference. - The region registry's
ghcr.ioretries (maxRetries: 5,retryDelay: 2s) reach a registry set up by an earlier release with the next changeregion:reconcilemakes to its config.json, or withregistry:upgrade-zot; the upgrade does not restart Zot for them. - Back to 0.6.8 is a
helm rollbackwithout rolling the migrations back (Back to 0.6.8). 0.6.8 ignores the column and counts stopped projects again. After upgrading to 0.6.9 again, deploy once each project that was resumed on 0.6.9 and deployed on 0.6.8: until then it counts as released.
Added#
- The dependency test bot releases what passes. Each new version of a region dependency that the daily release check filed an issue for is now moved and tested on the last release, not on the development branch, and the versions whose suites pass become one patch release per run: the chart's version, the docs' pins, its changelog section and upgrade notes with every version's line, a signed tag, the images that release-images.yml builds (checked against the registry without credentials, digest by digest), the GitHub release and the chart on helm.brf.sh; the development branch takes the release as a merge, so nothing unfinished ships and the next release carries it. The green versions of a run are made again together on the last release and, two or more, tested together; one that does not go with the others needs a person, and the ones after it wait for the next run, while the ones before it are released. A red suite, a bump that is not mechanical or a release step that fails stops with a comment on the issue that says what was done, and a red version never holds up the green ones; a release that stopped goes on with the workflow's
resume, anddry_runmakes and checks everything without pushing. A dependency release ships the chart and the images: the CLI, the Terraform provider and runr.sh stay those of the last full release. A suite whose script ends 0 while its results count failed tests is red. See docs/development.md. - The dependency test bot follows every package RunR ships or builds with. Besides the region charts and Knative, the daily check now reads the API's Composer packages, the npm packages of the console and the SDKs, the Go modules of the region images, the CLI, the Terraform provider and the dev tunnel, the GitHub Actions pins and the Dockerfiles' base images. The minor and patch updates of one group are one issue and one bump; a new major, an exact pin or a new toolchain is an issue of its own that a person decides, and a security advisory makes the group's issue urgent (tested the same day). Packages pinned on purpose stay where they are, with their reason (
ci/deps/holds.txt). A bump that changes what a release ships (an image, the chart, the API's code) goes into the run's patch release as before; a gateway-auth, image-upload, supply-chain or Spin shim image that changes gets a patch version of its own. The ones that ship nothing (test tooling, the CLI's and the provider's modules, the Actions pins) land on the development branch together, in one push per run, and come with the next release. A bump whose lockfile does not come out the same twice, or that would move a constraint, goes to a person. It also follows the tool pins of the images and the workflows (helm, kubectl, Scalar, syft, cosign, oras, kind, cloud-provider-kind, kubeconform, crane, actionlint, gitleaks, Terraform, govulncheck) and moves their published checksums with them (an image's in the run's patch release, a workflow's on the development branch), files new lines of the toolchains and every release of the patched Spin shim for a person, and follows the Python SDKs and the examples (npm, Go, PyPI, Cargo, Dockerfiles). The bot's commits and tags now carry the identity of RunR's public history and aRunR-Deps-Bottrailer, and CI skips their pushes. See docs/development.md. - The console's checks are shared workflows. Its lint, unit tests and build (
ui-tests.yml) and its dev-stack suite (ui-stack-suite.yml) are reusable workflows that CI, the weekly E2E run and the dependency test bot call. Their result comes from Vitest's and Playwright's JSON reports, so CI also fails when the in-app tutorial's spec did not run. CI now tests the Python SDKs and the Node Sandbox SDK, and the bot tests the Python SDKs. The API checks count PHPUnit's failures from its summary line, so a green run is no longer read as failed. - A region shows, and alarms on, what it would need with its stopped projects back. Region → Workers → Capacity shows next to the budgets what the region would need if every stopped project were resumed at the same time: what they add per resource, how full the fuller budget would be, by how much it would be exceeded and the capacity that keeps it under the alarm's level, with the largest stopped projects (
GET /api/knative-regions/{id}/capacity→with_stopped). When this figure crosses the capacity alarm's levels (60 % warning, 80 % critical), the people who get the capacity alarm today are told to add nodes — by how much the region would be exceeded, how much to add, and which stopped projects need how much in total — in the app, by mail and through theresource_thresholdalert rules (data.basis: "with_stopped", its own hourly cooldown). The card's ? starts a tour of it. On a multi-tenant installation the card shows what the alarm measures: a platform region adds up every tenant's and the platform's own projects. See docs/architecture.md.
Changed#
- A stopped project gives its capacity back in a pay-as-you-go region. Its reservation — its own workload and its environments — no longer counts in the region's budgets, nor in its organization's quota, so others can use the room. A resume admits it again: when the region (or the quota) has no room, the redeploy is refused with
capacity_exceeded(quota_exceeded) and the project is resumed without a deployment, holding nothing until a deploy of it fits; a git push or console deploy of it is refused the same way (422). The project page shows a banner while it waits: why (capacity_exceeded,quota_exceeded,fixed_allocation_exceeded), what each resource lacks and what to do — the project resource's newcapacity_waiting. A region whose billing is off counts as pay as you go. See docs/billing.md. - The capacity alarm of an organization's own region goes to the organization. Its members who manage its regions (owners and admins) see it in the app and get the mail; the platform administrators no longer get its mail. Every capacity alarm mail honours the recipient's
resource_thresholdmail preference, and a platform region's alarm also counts the platform's own projects. - A fixed region's allocations cover the projects that run. A stopped project keeps its reservation and its allocation stays billed, but its share is lent to its organization's other projects in the region: they may run beyond their own allocation up to what the stopped projects lend and no other project uses yet; a running project's own allocation is always its own. The two cannot run at the same time: the stopped project's resume has to fit its allocation minus what the others still use of the lent shares, else it is refused with
fixed_allocation_exceeded, like a deploy. See docs/billing.md. - The Kind e2e plans its waves with the stopped projects back, so a case that stops a project and resumes it later finds its room free.
Fixed#
- The Kind e2e fits a GitHub-hosted runner again. It runs its cases in waves the region runs at once and deletes each wave's projects once they are verified: since 0.6.8's runtimes it had kept every case's project, filled the region's FaaS overcommit budget and refused the later creates. A shell wake-up that fails names what holds the pod.
- The UI e2e of the dev stack runs on GitHub-hosted runners. The new
vue-initandvendor-initservices createvue/node_modulesandapi/vendor, where the containers' package volumes are mounted, owned like the checkout: Docker made them root's, sonpm cion the runner (andnpm ciorcomposer installon a Linux host) failed with EACCES. - A model service starts on a region whose Knative features were just turned on. When Knative's webhook refuses the model service's Knative Service for a pod-spec feature it has not seen yet (persistent volume claims, a field a feature flag gates), the deploy applies it again up to six times within about 40 seconds and logs each try; any other refusal still fails at once.
- A build no longer fails on the first failed pull through the region registry. Function and runtime image builds try a failed registry operation (a
FROMmanifest, layer unpacking, the push) up to five more times with backoff instead of ending on one 503 or 404 from Zot (unable to complete operation after 0 attempts), and the registry's ghcr.io upstream retries a failed fetch (maxRetries: 5,retryDelay: 2s; an installed registry takes it with its next config change). - A GitHub 504 no longer fails a region's setup. The charts a region installs from GitHub releases (Zot's, and the prometheus-community and grafana charts) and Knative Serving's CRDs are downloaded again when GitHub answers 5xx or 429 or the connection breaks, four tries in all, 2, 4 and 8 s apart, then once through the GitHub API's asset endpoint; a chart that comes that way is installed only with the digest its repository's index gives it. Before, one "504 Gateway Timeout" failed the whole registry setup ("Zot Helm install failed").
0.6.8 - 2026-11-08#
Upgrade notes#
- A migration adds runtime sources (
2026_11_07_100001_create_runtime_sources): the tableruntime_sources;runtimes.source_id,source_path,orphaned_at;executor_images.deprecated_at,sunset_at;runtime_patches.source_idandruntime_patch_feeds.source_id. It only adds columns and a table; existing runtimes, versions, patches and projects are unchanged. See docs/runtimes.md. - The built-in runtimes now come from RunR's signed catalog
ghcr.io/brf-tech/runr/runtime-catalog:stable(published by BRF-Tech/runr-runtimes, signed with the patch feed's key), read at the API's start and every 15 minutes. Until it has been read once the catalog the API image ships is in use, as before; an air-gapped installation sets HelmruntimeCatalog.offline: true(RUNR_RUNTIME_CATALOG_REF="") and keeps the bundled catalog. The API needs to reach ghcr.io — orruntimeCatalog.viaRegion(default:runtimePatches.viaRegion) — like the patch feed already does. EXECUTOR_RUNTIMESis now only the first default of the platform setting Built-in runtimes (Build & Run → Runtimes): its default changed from the list of eleven built-ins to every runtime of the catalog, so a runtime the catalog gains is offered without a release. An installation that setsEXECUTOR_RUNTIMESkeeps its list until a platform administrator saves the setting.- Projects without a runtime version are refused once their runtime's default version ends (end of life or sunset), like pinned projects already were; running deployments stay untouched. Before, a project on no version deployed a default that was past its end of life.
- The API image adds
openssh-clientfor Git sources with an SSH deploy key. - The console certificate is checked (0.6.7 → 0.6.8 in upgrade.md). When cert-manager issues it, the Role
runr-api-rolegainsgeton the console's Certificate andliston the CertificateRequests of the release namespace;runr-configgainsCONSOLE_TLS_MODE,CONSOLE_TLS_ISSUERandCONSOLE_TLS_CHECK. A marketplace that merges the package's default values into an upgrade can bringingress.tls.mode: letsencryptto a release that never set it: without the ClusterIssuerletsencrypt-prodthe console stays on the ingress controller's default certificate, which the new warning reports. Setingress.tls.mode: none(or the mode you use) in the release's own values. - Single-tenant: the Owners now get the Patches page, the platform banners and the platform administrators' notifications through the new permission
platform.manage. Nothing is migrated: built-in roles come from the catalog. The Admin role does not hold it; a custom role can (0.6.7 → 0.6.8 in upgrade.md). - Back to 0.6.7 is a
helm rollbackwithout rolling the migration back (Back to 0.6.7). 0.6.7 runs on the new table and columns and ignores them. Projects on Java, .NET, Ruby and Bun keep running and deploying there: 0.6.8 keeps every built-in runtime's manifest on its row, which 0.6.7 reads for the runtimes its own catalog lacks. A source's runtime that discovers its versions loses them on 0.6.7, and the images of a private source cannot be pulled there. After upgrading to 0.6.8 again, runphp artisan runtime-sources:sync --forceonce where sources are connected.
Added#
- A dependency test bot. Every week (and by hand for one issue), each new version of a region dependency that the daily release check filed an issue for (the Traefik, kube-prometheus-stack, loki-stack, Zot and kata-deploy charts, a Knative minor) is moved in the development branch with every test and doc that names the old version, tested with its suites (PHPUnit, the Kind e2e that installs it, and the move of a region installed by the last release), and committed only when they pass; the issue is then closed with
deps: passed. A failed suite, or a version that is no mechanical bump (a new major, changed CRDs, values the chart no longer takes, a new image of the reqs mirror), labels the issueneeds attentionwith what failed, and nothing is committed (development.md). - Runtime sources (docs/runtimes.md): platform administrators (for everyone) and organizations (for themselves, while custom runtimes are on) connect signed OCI catalogs and Git repositories of
runtimes/<id>/runtime.yaml, public or private (HTTPS token or SSH deploy key; registry credentials for a private catalog and private images). Each source is read periodically and on Sync now; every manifest is validated strictly, a bad one is reported with its file and JSON path and does not stop the others; the source records its status, revision, last read and problems; its runtimes are enabled or disabled one by one. Test connection reads a source before it is saved. - RunR's own catalog is a source: the built-in runtimes, a new language of BRF-Tech/runr-runtimes offered without a RunR release;
builtin.jsonstays as the offline fallback and the Runtimes page says which is in use. The Built-in runtimes setting picks the offered built-ins. - Runtime ids stay one namespace with explicit rules: built-in ids are RunR's, the first registration wins, a clash is a sync problem naming the holder (never another organization), orphans are adopted by the same owner.
- Private runtime images: function pods, Kaniko function and runtime image builds, WebAssembly toolchain Jobs, sandboxes, the node pre-pull and version discovery use the source's registry credentials (a RunR-managed
kubernetes.io/dockerconfigjsonSecret, Kaniko's docker config,crictl --creds). Credentials are encrypted at rest, write-only and masked everywhere. - Lifecycle for every runtime (docs/runtimes.md): manifest fields
deprecated(a date ortrue),sunset,replacementandnoticefor a runtime,deprecatedandsunsetfor a version,versionDiscovery.lifecyclefor discovered versions. A version ends at the earlier of its end of life and its sunset — the same reminders (13, 4, 1 weeks), picker removal and save / deploy refusal as an end of life; deprecations are announced once. A runtime that leaves its source is orphaned: no new projects, its projects keep running and deploying for 90 days (RUNR_RUNTIME_ORPHAN_GRACE_DAYS), then its sunset. - Patches of a source's own runtimes: from its signed catalog or its own signed patch feed, verified with the source's key — levels, deadlines, notifications, automatic migration and the Patches page like the built-ins. RunR's feed patches only the built-ins.
- Console: Build & Run → Runtimes → Sources (list, add / edit with credentials, test connection, sync now, problems, the source's runtimes with enable / disable, its patches), the same for organizations; the Built-in runtimes card; lifecycle badges (end of life, deprecated, sunset, orphaned) on runtimes, versions, the runtime picker and projects; a tour of the Runtimes page.
- API
/api/runtime-sources(list, connect, show, change, remove, sync, test) and/api/runtime-catalog; CLIrunr runtimes sources …andrunr runtimes catalog; MCP toolsruntime_sources_list,runtime_source_connect,runtime_source_sync,runtime_source_remove; Terraform resourcerunr_runtime_source; Helm valuesruntimeCatalog.*. - docs/executor-development.md: building an executor for any language, testing, publishing a repository or a signed catalog, private sources, signing, lifecycle and patches — with the sample repository BRF-Tech/runr-sample-custom-executor (Lua in persistent mode, compiled Zig in http mode).
- e2e suite
runtime-sources(runtime/ci.sh test-runtime-sources): RunR's published catalog, a signed OCI catalog, the sample repository served as a Git source in the cluster, a private image, a sunset refusing deploys. - Java, .NET (C#), Ruby and TypeScript on Bun as built-in runtimes. Four runtimes join the catalog (
java,dotnet,ruby,bun), each with its handler server, starter code, dependency file and the three triggers (http, schedule, queue), under runc, gVisor and Kata. Their images are published by BRF-Tech/runr-runtimes (exec-java,exec-dotnet,exec-ruby,exec-bun), run as a user other than root, and get patch tags and the patch feed like the others (runtimes).- Java: Eclipse Temurin, the LTS lines (21.0 and 25.0).
public static Object handler(Map<String, Object> event, Map<String, Object> context)inMain.java; every save is built with javac, and a FaaS project'spom.xmlbrings its jars through Maven. - .NET (C#): .NET 10.0.
public static object Handler(JsonNode? input, IDictionary<string, object?> context)inFunction.cs; every save is built with the SDK,Function.csprojnames the NuGet packages, and the function runs as a self-contained application on Microsoft's chiseled runtime-deps image. - Ruby: 3.3, 3.4 and 4.0.
def handler(event, context), orhandler(event:, context:)as on AWS Lambda; a FaaS project'sGemfileis installed with Bundler. - TypeScript (Bun): Bun 1.4, the newest release (Bun supports only its latest).
export async function handler(event, context)inmain.ts, JavaScript too; a FaaS project'spackage.jsonis installed withbun install.
- Java: Eclipse Temurin, the LTS lines (21.0 and 25.0).
- The code interpreter runs Java and C# (
java: Java's source launcher,csharp: .NET's file-based apps,dotnet run), and TypeScript and JavaScript with Bun in a Bun sandbox; a sandbox of the new runtimes defaults to its language (Sandbox API). - Console and CLI: the runtime picker, icons, editor highlighting, upload types and the code interpreter's language list know the four;
runr initandrunr devscaffold and run them (Main.java,Function.cs,main.rb,main.ts). - End-to-end: the main suite runs
xaas-<runtime>-<version>-http/schedule/queueandfaas-<runtime>-<version>-httpfor the four (the FaaS cases with Jedis, StackExchange.Redis, the redis gem and ioredis), and their gVisor sandbox cases. - A warning when the console certificate cannot be issued (install.md). With
ingress.tls.mode: letsencrypt(or aclusterIssuerof the older keys) the API reads the console's cert-manager Certificate and its newest CertificateRequest at most every five minutes; platform administrators (single-tenant: the holders ofplatform.manage) see a banner across the console while it cannot be issued — a missing ClusterIssuer, a failed, denied or invalid request, a certificate still pending after ten minutes, or a renewal that fails (with the current certificate's expiry). It says why, whether the console serves the ingress controller's default certificate, and what to do: create the ClusterIssuer, setingress.tls.clusterIssuerto an existing one, or usecustomornone.GET /api/admin/console-tlsreturns the state for every mode (ok,pending,failed,not_applicable,unknown). The ClusterIssuer itself is not read (it would need a cluster-wide permission): a missing one shows in the CertificateRequest. Helmingress.tls.certificateCheck: falseturns the check and its permissions off.
Changed#
- The node pre-pull pulls the image a function runs on, not every version image. For a runtime with a build that is the build's
runImage: .NET's chiseleddotnet-runtime-depsimage instead of its 1.3 GB SDK image, Go's Alpine instead of the Go toolchain images; Java (runImage: "{image}") and the runtimes without a build keep their version image. No function runs on an SDK, and Kaniko fetches it through the region registry, whichregion:reconcilepre-warms with it; a private run image of a runtime source is pulled with the source's credentials (docs/runtimes.md). A Go or .NET sandbox runs on the toolchain or SDK, so a node pulls it on its first such sandbox, as for any image it does not have; a warm pool keeps such sandboxes started (docs/sandbox-api.md). - Sandbox commands run without a kubectl process. The API opens the API server's exec stream itself (a WebSocket per command,
v5.channel.k8s.io): exec (JSON and SSE), the code interpreter, file transfers, background processes, kill and pause, and the exec WebSocket without a terminal. Exit codes, output caps, timeouts, kills, error messages and the activity records stay as they were. Terminal sessions ("tty": true, the console terminal) keepkubectl exec -tfor now. kubectl also takes over where the API's client cannot work: a region kubeconfig with an exec plugin, an auth provider, credentials or a CA in files, a proxy or impersonation, and an API server that refuses the WebSocket upgrade (one withoutv5.channel.k8s.iois remembered for 10 minutes); a command goes to kubectl only before it started.SANDBOX_EXEC_CLIENT=kubectl(viaextraEnv) brings back a kubectl process for every command. The region kubeconfig's CA, client certificate and key are kept in 0600 temp files of the API process, as the API's other cluster calls already do.- The native client takes a bearer
token(a ServiceAccount's too) andclient-certificate-data/client-key-data, alone or together. Every other credential form (an exec plugin, an auth provider, a token, certificate or CA in a file, basic auth, impersonation, aproxy-url) goes to kubectl, and so does a server that anHTTPS_PROXY/HTTP_PROXYof the API's environment covers andNO_PROXYdoes not exclude: kubectl and the API's other cluster calls go through that proxy. Each reason is logged once per process (sandbox exec: kubectl instead of the native client: …), also a command that went to kubectl because the server refused it. - e2e: the sandbox-api suite runs one command through the native client with a ServiceAccount bearer-token kubeconfig of the Kind cluster (
sbx-exec-bearer-token): refused without RBAC, run with a Role ofpods/exec.
- The native client takes a bearer
- Region Traefik chart 42.3.1 (from 42.3.0). New regions install it;
region:reconcilemoves the Traefik of an existing region to it (steptraefik-chart), one pod at a time. Traefik stays v3.7.17. The chart'straefik.ioCRDs gain optional fields that RunR does not set, so an installed region needs no CRD step (0.6.7 → 0.6.8 in upgrade.md). - kube-prometheus-stack 92.22.0 (from 92.21.2; Prometheus Operator v0.94.2, as before). Regions installed from now on run it; an installed region keeps its chart until
php artisan cluster:patch-observability, a change back to RunR's own monitoring or a new Serving install moves it. On that move node-exporter restarts once with a stricter security context, and nothing else restarts (upgrade.md).
Fixed#
php artisan runtimes:synckeeps the versions of a source's runtime that discovers them. Such a manifest (versionDiscovery) lists no versions, and the command took that for every version retired: the runtime's projects could not deploy until the nextexecutor:sync. It now leaves them toexecutor:sync, which reads them from the runtime's registry.The region Traefik gets its probe timing and Service types in keys the chart reads.
- RunR gave the Traefik chart keys it never read.
- The probes' timing went under
deployment. Traefik ran the chart's defaults instead: a readiness failure threshold of 1 and timeouts of 2 s. - Each Service's type went beside its
spec. tlsOptions.default.preferServerCipherSuiteswas set, which no Traefik v3 reads.
- The probes' timing went under
- Chart 42.3.1's schema refuses the
typekeys. They now sit in the Services'spec, the probes' timing in the chart's root keys, and the TLS key is gone. - A new region's Traefik now runs RunR's probes: readiness after 10 s with two successes, timeouts of 5 s, liveness after 20 s with five failures. Its Services render as before, except one line:
traefik-internalstatestype: ClusterIP, the type Kubernetes already gave it. - An installed region gets the same keys when
region:reconcilemoves its Traefik to chart 42.3.1 (steptraefik-chart, under Changed). That move rolls the pods anyway, and the probe timing comes with it.
- RunR gave the Traefik chart keys it never read.
The install notes of
ingress.tls.mode: letsencryptsay the ClusterIssuer has to exist. They said cert-manager issues the certificate even in a cluster without that issuer, where it never is; they now name the issuer, how to check it (kubectl get clusterissuer,kubectl get certificate -n <namespace>) and the other choices.Symfony 7.4.23 for the API. The update covers
http-foundation,http-kernel,routingandyaml, plus theerror-handler,var-dumperand contracts releases they need. It clears the seven advisories published on 2026-11-04. They cover:- HEAD requests that bypassed method-restricted access rules;
- path rules bypassed by a path appended to the script name;
- host placeholders that matched the start of the path;
- client-chosen hosts in generated URLs and in inline fragments;
- slow parse times in the YAML parser;
- Symfony's
private_rangesshortcut trusting IPv4-mapped, 6to4, Teredo and NAT64 addresses. RunR'sTRUSTED_PROXIES=private_rangesalready uses its own list, as before.
composer auditreports none.A single-tenant installation reaches its own Patches page, platform banners and notices.
- The single-tenant edition has no
users.is_admin, so what was for platform administrators reached nobody there:- the Patches page (
/api/admin/runtime-patches) andGET /api/admin/console-tls; - the banners of a newer RunR release, of a patch feed that is off or refused, and of a console certificate cert-manager cannot issue;
- the notifications of a critical patch's platform count, a refused feed, a region reconcile that did not finish and the bot protection's alarms.
- the Patches page (
- The new single-tenant permission
platform.manage(Infrastructure) stands in for the flag. The Owner role holds it; the Admin role does not; a custom role can include it (docs/permissions.md). - The Horizon dashboard follows the same permission. Owners keep their access; its link still needs a session or an API token with full access.
- An API token with a selection of permissions reaches those routes when it carries
platform.manage. - Multi-tenant installs are unchanged: only platform administrators, and their scoped tokens stay refused. The
@platform:billing.*routes stay as they are: tenant bills, archived bills of deleted tenants, the per-tenant queue consumers of a platform region and the 0.5.7 records (archived by a Tenancy migration) exist only in the multi-tenant edition.
- The single-tenant edition has no
RunR's catalog no longer reads as an authentication failure while it is not published. While
ghcr.io/brf-tech/runr/runtime-catalogdid not exist or was not public, the default source showedauth_failed: ghcr.io answers an anonymous read of a missing or private package with 401 / 403. A source read without credentials that the registry or Git host refuses, or that answers 404 / "not found", now has the statusnot_found, shown as Not found or not public (Bulunamadı ya da herkese açık değil), with a hint on RunR's catalog that the bundled catalog stays in use.auth_failedis left to a source whose own credentials were refused; registry credentials for another host than the catalog's do not count. The API, the CLI (runr.RuntimeSourceNotFound), the Terraform provider and the MCP tool list the new status (runtimes.md).The gVisor installer retries its download. A node fetched gVisor's release tarball and its checksum with one
wget: a node whose DNS was not back yet (CoreDNS restarting) failed the install. Both downloads now take five tries, 5, 10, 20 and 40 s apart, each failed try in the installer pod's log; the WebAssembly installer downloads its shim tarballs the same way.The sandbox-api e2e no longer passes its gVisor cases on runc. The setup restarted CoreDNS and slept 5 s; the nodes' gVisor download failed 16 s later, and the runc RuntimeClass the e2e stands in with ran the
gvisorclass. The setup now waits until CoreDNS is Ready again, and the suite fails when gVisor was not installed (sbx-gvisor-install, with the install's error) or when itsgvisorsandbox does not see gVisor's kernel (sbx-gvisor-runsc: the RuntimeClass handlerrunsc,uname -r,dmesg).ci.sh test-helm-upgradekeeps etcd in memory withKIND_ETCD_IN_MEMORY=1, as the other Kind suites do; its Kind configuration had no etcd patch.test-global-projects' second cluster gets it as well, and every suite takes the patch from one helper.
0.6.7 - 2026-11-06#
Upgrade notes#
- Nothing needs a step. There is no database migration, Kata is not reinstalled, and running sandboxes keep running: their next command runs the new way. See docs/upgrade.md.
- A sandbox command on gVisor takes about 100 ms more on the single-node test VM: it writes through pipes inside the sandbox, so that a large write no longer loses its end. On Dragonball, input goes in acknowledged frames of 48 KiB (a 10 MiB upload took 6.6 s on the same VM). See docs/upgrade.md.
- A custom sandbox image (
sandbox.allow_custom_images) now needscatfor/execwithstdin,mkfifofor the exec WebSocket without a terminal, and on Dragonballheadorddfor any input. coreutils and busybox have them all. - kube-prometheus-stack 92.21.2 does not reach an installed region by itself:
php artisan cluster:patch-observabilitymoves it and restarts Prometheus once. See docs/upgrade.md.
Changed#
- A sandbox command starts one process less. Every command ran
mkdir -p /tmp/.runr/execbefore it wrote its PID file; the sandbox pod now creates that directory (and/tmp/.runr/code) when it starts, and a command writes its PID file with the shell alone. On the single-node test VM (nested KVM, Kata 4.3.1) a command took 22–37 ms less at the median on Cloud Hypervisor, QEMU, Dragonball and gVisor (kubectl exec of the command's wrapper, n=60 per class; Firecracker unchanged). The code interpreter and the files of a create or of a claimed warm pod start no mkdir when their directories exist. Sandboxes started by 0.6.6 or earlier create the directory on their first command. - kube-prometheus-stack 92.21.2 (Prometheus v3.16.0, kube-webhook-certgen 1.11.1; the Prometheus Operator stays at v0.94.2). Regions installed from now on run it; an installed region keeps its chart until
php artisan cluster:patch-observability, a change back to RunR's own monitoring or a new Serving install moves it. The move restarts Prometheus and its operator once and needs no step of its own: the operator's CRDs are the same. The chart no longer puts its version in the labels of the operator's pod, so a later chart upgrade restarts the operator only when the operator changes (upgrade.md).
Fixed#
- On Kata, a sandbox command's output no longer stops when its input ends. With Kata Containers 4.3.1 (RunR 0.6.6), the Go runtime behind Cloud Hypervisor, QEMU and Firecracker stops passing a command's stdout once the exec's stdin is closed:
/runwith more than 32 KiB of output timed out with 32 KiB,/execwithstdinlost what the command wrote after its input ended (and more than 32 KiB of it hung until the timeout), and the exec WebSocket lost the output afterstdin_close. RunR no longer closes an exec's stdin while a command runs: the input of/execand the code of/runare written into the sandbox by an exec of their own and read from there (an input whose command never started is removed after an hour), uploads and the env file of a claimed warm pod take exactly their bytes, and the exec WebSocket passes its input on in frames that end in-band, with the exit code from a marker the command's wrapper writes; a kill reaches the command through its PID file as before. Dragonball (runtime-rs) and gVisor were not affected. A custom image now needscatfor/execwithstdinandmkfifofor the exec WebSocket without a terminal (upgrade.md). - On Dragonball, input over 64 KiB no longer ends at 64 KiB. Kata 4.3.1's Rust runtime with the Dragonball VMM ends an exec's stdin once 64 KiB of it wait unread in the guest, and the command still exits 0: an upload over 64 KiB was stored short (100000 bytes written, 65536 read back), and
/execwithstdin, a code run's code and the files of a create over 64 KiB were cut. Input to a Dragonball sandbox now goes in acknowledged frames of 48 KiB, the next once the sandbox read the last (on the single-node test VM a 10 MiB upload took 6.6 s). On every class, input that arrives short now fails with a message instead of being stored, and an archive is checked before it is extracted: tar takes an archive cut between two files without a word. - On gVisor, a large write of a command no longer loses its end. gVisor returns a short write when a program writes more than 64 KiB at once to an exec's stdout or stderr, and programs that do not write the rest lose it: Python's
print("y" * 100000)came back with 65537 bytes (RunR's Python runtime runs unbuffered). A user command on gVisor now writes through pipes inside the sandbox, whosecatwrites everything, with its exit code, kill and timeout as before; this costs about 100 ms per command on the single-node test VM. Kata and runc were not affected. - kubectl's "command terminated with exit code N" no longer ends a command's stderr when that stderr did not end in a newline and the trailer came in the same piece of output.
0.6.6 - 2026-11-06#
Upgrade notes#
- Kata Containers moves to 4.3.1, and running Kata pods stop.
region:reconcilere-runs kata-deploy on every region with Kata: the container runtime restarts twice on each node, one node at a time. Kata pods on a node stop while it is done: a project's pod starts again by itself on 4.3.1 (about 6 minutes in a test, most of it the image pull, which can be done first), and a Sandbox API sandbox ends. Plan it for a quiet hour. See docs/upgrade.md. - The region Traefik moves to chart 42.3.0 (Traefik v3.7.17).
region:reconcileupgrades existing regions: the Traefik pods roll once without an interruption, and a failed upgrade rolls itself back. A region whose nodes pull through a mirror of their own needsghcr.io/brf-tech/runr/reqs/traefik:v3.7.17there first, andknative.traefikImageset to it. See docs/upgrade.md. - A request header with
_,.or another such character in its name no longer reaches a project. The region Traefik deletes it on every entry point: a client that sendsX_Request_Idorapi_keygets its normal answer, but the project does not get that header, and nothing is logged. Rename such headers to dashes before the upgrade. gRPC metadata keys count too. See docs/upgrade.md. knative.traefikAliasHeadersStrategy: keepis a stopgap for an app that cannot rename its headers in time. The default isdelete.keepreopens the hole of the entry under Security: an underscore or dot alias of the identity headers can reach workloads on servers that fold header names. The value is install-wide, for every region;region:reconcilemoves the regions either way and warns on every run while they keep such headers.bootstrap.regionReconcileSecondsis 4800 by default, for the time the Traefik chart's move can take. An upgrade with--reuse-valueskeeps 3600 unless you set it: no step is cut, and what the deadline leaves in the worst case is done by the next attempt.
Security#
- A caller can no longer forge what RunR vouches for under another spelling. Traefik's ForwardAuth replaces
X-Runr-Callerand the gateway'sX-Runr-Consumerin their dash form only, and a backend that folds header names into variables readsX_Runr_CallerorX.Runr.Calleras the same header: a caller could pass itself off as another project on the internal network, or as another consumer on a gateway route, towards a project served by PHP's built-in server or a similar CGI or WSGI server (a test with a ForwardAuth showed it for PHP's built-in server and Python'swsgiref, not for FrankenPHP or RunR's built-in runtimes). Every entry point now deletes such headers before routing (global.aliasHeadersStrategy: delete, GHSA-rf44-j88r-hh8c, GHSA-x677-9fxg-v5c5), on new regions and on existing ones as they move to chart 42.3.0;region:reconcilewarns about a region that cannot (security.md).
Changed#
- Region Traefik chart 42.3.0. New regions install the chart 42.3.0 with Traefik v3.7.17 instead of 40.3.0 with v3.7.4, and need Kubernetes 1.30, RunR's minimum already. The new reconcile step
traefik-chartmoves existing regions: chart 41 renamed the logging keys and its schema refuses the old ones, so--reuse-valuescannot cross it, and the step rewrites the release's own values instead. Rendered with RunR's values the two charts give the same container arguments, RBAC and Services. A Traefik of a chart before 40 and a cluster below Kubernetes 1.30 are left alone and reported. - Kata Containers 4.3.1. RunR installs kata-deploy 4.3.1 instead of 3.28.0. clh, qemu and fc stay on Kata's Go runtime (deprecated upstream since 4.0, still shipped and supported), Dragonball stays on runtime-rs. The
kata-installerDaemonSet follows the upstream chart's daemonset mode: kata-deploy no longer enters the host, so the pod mounts/opt/kata, systemd's private socket and the other paths it reaches, not the node's root. A Dragonball guest stays 1 vCPU and 256 MiB plus the pod's limits: runtime-rs now adds the limits tooverhead_vcpus/overhead_memory(32 MiB by default), which RunR's drop-in sets. runtime-rs sizes the vCPUs itself, so the pod annotations RunR wrote for that are gone, and workloads that still carry them keep working (kata-hypervisors.md). - Knative 1.22, 1.23 and 1.24. A new install takes Knative 1.23 on Kubernetes 1.34 and 1.24 on 1.35 or newer (1.22 and 1.23 need Kubernetes 1.34, 1.24 needs 1.35, as in Knative's release schedule); a cluster up to 1.33 gets what it got. The Serving step,
GET /api/knative-regions/{id}/compatible-versionsandrunr regions knative versionsfollow, and an installed region stays on its version until it is upgraded. A running Serving is always offered its next minor, so a region left behind climbs one minor at a time (regions.md). - kube-prometheus-stack 92.18.0 (Prometheus Operator v0.94.2, Prometheus v3.15.1, security rebuilds of the two). Regions installed from now on run it; an installed region keeps its chart until
php artisan cluster:patch-observability, a change back to RunR's own monitoring or a new Serving install moves it. Its upgrade needs no step of its own: the operator's CRDs are the same, and the chart's pre-upgrade Job recreates the admission webhook's Secret askubernetes.io/tlswith the same certificates (regions.md). - Zot 2.1.23 (chart 0.1.130) for registries set up from now on: fixes of storage, garbage collection and on-demand sync. From Zot 2.1.22 an on-demand sync that fails answers 503, and an image in Docker's manifest formats syncs only with
http.compat: [docker2s2]. RunR's config covers both: Docker Hub, which answers 401 for a repository it does not know, is no longer asked for RunR's own namespaces (runr-builds/,runr-runtimes/,runr-uploads/), so a build's first push gets its 404, andhttp.compatis set. An installed Zot keeps its version untilphp artisan registry:upgrade-zot <region-id>moves it: an operator's step, because it restarts the registry, soregion:reconciledoes not. The release keeps its values and the config.json that runs, with these two settings added; Helm rolls a failed upgrade back, an upgrade a killed process left pending is rolled back first, and--dry-runshows what would change (regions.md). - The Kind e2e runs Kubernetes 1.34 and 1.35 (
gh workflow run e2e.yml -f suite=kind -f k8s=1.35.0), the versions that take Knative 1.23 and 1.24:e2e.ymlmoves to Kind v0.31.0, the first release with their node images, and the reqs mirror carries them. A run on a host of your own needs Kind v0.31.0 or later (development.md).
Fixed#
- A Kata reinstall no longer promises that running pods keep running.
region:reconcile, the install log and the console's warning said so, but kata-deploy's cleanup removes the Kata shim, and containerd restarting without it drops the node's Kata tasks. They now say what happens: Kata pods on the node stop, a project's pod starts again when the node is done, and a sandbox ends. A first install is not affected. - The Knative versions on offer are the ones that install. The Serving step offered five minors, but a Knative Operator holds four, its own and the three before it, and its chart does not follow the minor chosen: the oldest on the list failed when the operator got it, with "the manifests of the target version are not available to this release". The list now holds what the operator that runs can install, or the one an install is about to pick, so an older operator also no longer offers minors newer than its own. A Serving install without a version (
runr regions knative install REGION serving) takes the newest of that list instead of the newest the cluster supports, andPOST /knative-regions/{id}/install-servingrefuses aversionoutside it with a 422. - RunR applies the Knative Serving CRDs it downloads. Every document of
serving-crds.yamlopens with a licence comment, and the parser skipped a part that opens with a comment: the operator install and the Serving upgrade applied no CRD at all, on every version, and said nothing. The operator's own apply covered for it, later than the install expects: the region's Traefik started before its Knative CRD existed. All twelve are now applied, also the emptystatus: {}of each, which a plain PHP array would have sent as[]and the API server refused. An uninstall takes back all of them, found by theirapp.kubernetes.io/name=knative-servinglabel: Uninstall Operator removed six anduninstall-knativenone.uninstall-knativekeeps them while Knative Services are left on the cluster, and Uninstall Serving keeps them (regions.md). - An internal registry on S3 runs. Zot refused the config RunR gave it on S3 (dedupe on with no remote database, no download directory for its sync), so such a registry never started; with those two fixed it would still have gone to AWS, because the address was written under a key the S3 driver does not read. RunR now turns dedupe off, downloads to the pod's
emptyDir, and passes the endpoint asregionendpointwith path-style addressing. No region needs a migration: none could have run (regions.md). - The daily chart check reports only a version newer than RunR's.
ci/check-helm-releases.shreported the latest version of every chart without looking at the version RunR pins, so a chart already at its pin got an issue (deps: loki-stack 2.10.3 released). It now reads each chart's pin from its constant in the code and files nothing for a chart at its pin; a constant it cannot read fails the check (development.md). - The notification bell shows the unread count from the start. The console asked for the count only in the fallback poll, every 2 minutes while the WebSocket was down: with the socket up the badge stayed empty after every page load and counted only the notifications that arrived live (an installation with 533 unread showed none). The header now asks once when the console loads and again after a sign-in or an organization switch, with the socket up or down; the socket and the panel's reads and deletes keep the count from there, and the 2-minute poll still runs while the socket is down. After a sign-out and another member's sign-in the badge shows that member's count, never the previous one's. "Mark all as read" also shows when every notification the panel lists is read but older ones are not.
- The console's language: the first of the browser's languages it has. Without a choice of the language switch the console looked only at the browser's first language: a browser set to German, then Turkish, opened in English. It now takes the first of
navigator.languagesthat is Turkish or English, and English when none is; the switch's stored choice still wins, and no installation setting changes the order. The sign-in page, the console after the sign-in, the page'slangand the API'sAccept-Languagefollow it alike. (A headless browser started without a locale reports the system's language: on a Turkish system it gets Turkish.) GET /api/versionshows a build's label as it is. EveryAPP_VERSIONthat did not start withvgotmain-in front: the public demo'sdemo-0.6.5readmain-demo-0.6.5, and themain-<sha>ofruntime/ci.shreadmain-main-<sha>. Only a bare commit id (7 to 40 hex digits, the default ofcompose/deploy.sh) gets the prefix now; a tag,devand any other label stay as they are (versioning.md).- No client events on the WebSocket.
config/reverb.phpleftaccept_client_events_fromunset, and Reverb then takes client events (client-*messages, Echo'swhisper) from any connection, also one that has not subscribed to the channel. The console sends and listens for none, so it ignored them; Reverb now refuses them (none, envREVERB_APP_ACCEPT_CLIENT_EVENTS_FROM), and on a login several people share, such as a public demo, no visitor reaches the others' consoles (security.md). - The demo data seeded twice is seeded once, and its list starts with the first sample. A second
runr:install --demo(DemoSeeder) on the same database added eleven more demo projects; it now keeps the ones the demo region has. The project list (newest first) showed the samples backwards, starting with the Node.js 18 function: it now reads in their order, so the first card the tours open is the Python HTTP function with its rollback history. The demo data stays English and every project keeps its static history (four versions, a deployment each, no build and no region call); tests now check every text the seeder writes and the versions' code for Turkish, the first project's deployments and a second seeding of a fresh database, as the demo's nightly reset does. - The console's index.html is never cached, and a missing build file answers 404. The UI image's Caddyfile (and the Compose gateway's) set the cache headers before
try_filesrewrote the path:/,/loginand every other console route got index.html withoutCache-Control: no-store, so a browser could keep it across an upgrade, and a chunk of the previous build that a console opened before the upgrade asked for got index.html under the build files' one-yearimmutableheader. The headers are now decided on the rewritten path: index.html for every route and/branding-defaults.jsnever cached, the hashed files under/assets/for a year (also their pre-compressed.br/.gz), a missing file under/assets/404 withno-store. A console whose page chunk answers 404 loads again once (at most once a minute), now with the new build.ci/check-caddyfiles.sh(CI) runs both Caddyfiles in Caddy and checks these answers.
0.6.5 - 2026-11-04#
Upgrade notes#
One new database migration, notification_user_states (each member's read and deleted state of the shared notifications), run by the pre-upgrade migration Job in the central schema and, with multi-tenancy, in every organization's schema (tenant migration 0110). It only adds a table, is safe while 0.6.4 still serves requests, and a rollback to 0.6.4 ignores it; a shared notification read or deleted before the upgrade stays so for everyone. One new Helm value, notifications.retentionDays (default 90): values of 0.6.4, --reuse-values included, keep working and render it as 90 (0.6.4 → 0.6.5 in upgrade.md). 0.6.5 adds the daily task notification-retention (03:45). Its first run after the upgrade deletes every in-app notification older than 90 days, read or not, with the members' read state, and the notifications of accounts deleted earlier. To keep them longer, set notifications.retentionDays (RUNR_NOTIFICATION_RETENTION_DAYS) before upgrading; 0 keeps them. Deleting an account removes its notifications in every organization. Regions that pull through a mirror need ghcr.io/brf-tech/runr/image-upload:1.1.1 there, next to the agent and queue-consumer images of 0.6.5: the uploads and organization transfers started after the upgrade use it. An installation that sets images.uploadImage or orgTransfer.transferImage keeps the image named there. Gateway-auth, supply-chain and the Spin shim keep their versions. Revoking the other sessions keeps the API tokens; every user lists and revokes their own API tokens and MCP clients, creating one still needs tokens.manage ("Create API tokens"). GET /api/branding adds features.auth_providers and GET /api/tokens adds mcp_client. The API reference (/api/docs, /api/docs.json) serves a document built with the API image: it lists the endpoints of every module, also of those turned off, names its server relative to the host (/api), and six operationIds changed (workflowRun.index, workflowRun.indexForWorkflow, workflow.app, modelService.invoke, billingSettings.show, billingSettings.update): regenerate an SDK made from it. The UI images carry pre-compressed copies of their files (150 MB for linux/amd64, 124 MB in 0.6.4). A UI image built from source builds on reqs/node:22-alpine.
Security#
- Dependency updates for published advisories (the Audit workflow had failed since 0.6.3). The console's code editor sanitizes with DOMPurify 3.4.17 (GHSA-cg2g-xh64-v4q7, critical; GHSA-mrv2-3pqr-j5jr, GHSA-p98j-92pf-mc4p, GHSA-6688-9rhm-gjv2, GHSA-c6gr-qvv4-5v3f): monaco-editor 0.57.0 ships its own copy, DOMPurify 3.4.15, which the build now replaces with the dompurify package (
vue/build/monacoDompurifyPlugin.js; the build fails if a newer Monaco moves its copy). The console takes vue-i18n 11.4.18 (GHSA-83rm-6c8r-g728, GHSA-8rc2-428q-7ww3: no fixed 9.x or 10.x exists; the UI image builds on Node.js 22, which vue-i18n 11 requires) and follow-redirects 1.16.2 (GHSA-mrwp-f3q8-3g9p, used by axios). The API takes symfony/mime 7.4.22 (CVE-2026-45070, e-mail header injection through MIME parameter names) with symfony/polyfill-mbstring 1.43.3. Build tooling: Vite 6.4.5 (GHSA-8g5g-mxqh-grgj) and Vitest 4.1.12 (GHSA-qxg6-89mg-rqv8, GHSA-xp26-3454-mjgp).npm audit --omit=devreports nothing; the dev tooling keeps two moderate advisories of minimatch and glob that eslint 10 and js-beautify (of @vue/test-utils) do not take the fixed majors of yet.
Fixed#
Notifications no longer pile up forever. Nothing ever deleted an in-app notification: the shared ones, the read ones and the ones users had deleted (a soft delete) stayed in the database for good, with every member's read state of the shared ones. The new daily task
notification-retention(03:45) deletes the notifications created more than 90 days ago — read or not, deleted or not, with their members' state — in the central schema and in every organization's schema, in chunks of 1,000 rows. The retention is the new settingRUNR_NOTIFICATION_RETENTION_DAYS(Helm valuenotifications.retentionDays, default90,0keeps every notification; helm-values.md). The first run after the upgrade deletes every notification older than the retention.A deleted account's notifications go with it. Deleting an account, by its user or by an administrator, left the user's notifications and their read and deleted state of the shared ones behind; it now deletes them in the central schema and, in the multi-tenant edition, in every organization the account belonged to. Removing a member from an organization deletes their notifications there. The daily
notification-retentiontask also removes what deletions before this release left behind, whatever the retention.Revoking the other sessions no longer ends the user's API tokens. Account settings → Active Sessions listed every token of the user, API tokens included, and "Revoke All Other Sessions" (
POST /api/account/sessions/revoke-all) deleted them all but the current one: signing out the other browsers also broke the user's CI, CLI (runr-cli@<host>) and MCP client tokens. The session routes now list and revoke sign-in sessions only (the tokens namedapiandinvitation-accept, which API tokens cannot take since 0.5.5): revoking one by an API token's id answers 404, revoking all others ends every other sign-in session and keeps the API tokens. API tokens are listed and deleted under Administration → API Tokens only;DELETE /api/tokens/{id}answers 404 for a sign-in session's id. What the other paths end is unchanged and now tested: signing out ends the token of the request, a password change the other sign-in sessions, disabling the account every token but the current one, deleting it every token; two-factor changes and a frozen account's refused requests end none. Disabling the account now also signs out the other browsers at once (session.revoked), as a password change does, and Active Sessions no longer lists a session that ended (past its expiry or the session lifetime, as the sign-in check counts it) until the daily cleanup removed it. Signing out a cookie session (no stored token) answered 500; it now ends the web session. Listing and revoking one's own API tokens and connected MCP clients no longer needs a role permission (route rule@own:tokens.manage: a limited API token must still carrytokens.manage); creating a token still needstokens.manage. Administration → API Tokens is shown to every user, without the create form for a role that lacks it; in an organization of the multi-tenant edition such a member lists their own tokens only. Thetokens.managelabel reads "Create API tokens". The tokens of connected MCP clients stay on that list, now labelled "MCP client" (mcp_clientinGET /api/tokens). No data migration.One member's Mark all as read or Delete all no longer changes the other members' notifications. A shared notification (one without a user: a resource threshold, the result of a job nobody started) had one read state and one deletion for everyone, so when a member marked it read, deleted it, or used Mark all as read or Delete all, it was read or gone for every member of the organization. Each member now reads and deletes shared notifications for themselves (the new table
notification_user_states, in the central schema and in every organization's schema): the list, the unread badge and/api/notifications/unread-countare per member. A user's own notifications keep their state as before. A shared notification someone read or deleted before the upgrade stays read or deleted for everyone.The API reference opens again.
/api/docs.jsonand the reference page/api/docsgenerated the OpenAPI document on each request, which takes longer than a request may run: on every install they ended in a 500 at the 30 s limit, or never loaded. The API image now builds the document once (php artisan runr:api-docs, a stage ofapi/Dockerfile.prod) and both serve that file,/api/docs.jsonwith an ETag andCache-Control: public, max-age=300. It documents the endpoints of every module (Tenancy, AuthProvider, Ticket, Billing), also of those an install has turned off, and names its server relative to the host (/api). The page also loaded a viewer script the image never had: the image now carries the Scalar viewer (1.75.1, checked against its SHA-256), without its AI chat, MCP generator and telemetry. A checkout without the built file (development) still generates the document per request (development.md).Every operation of the API reference is in one group with a proper name, has a one-line summary and an operationId of its own (Projects > Deployments, Regions > Install, Sandboxes > Files, …). An operation whose controller names its group was also in a second one named after the controller class (Images and Image, Bindings and Binding), most others were in such a class name (AdminSupportTicket, GitWebhook), and the Sandbox API's operations were in pieces of their controllers' descriptions ("TTL", "delete", "activity log (docs/sandbox-api.md)."). The billing module's group also took the core's Settings endpoints. Most operations had no summary, one that restated their method and path, or a paragraph; each now says in one line what it does ("List the project's workflows"), the rest is its description. The two lists of workflow runs shared the operationId
workflows.workflowRun.index_0, which OpenAPI requires to be unique. These operationIds changed (SDK generators name their methods after them):workflowRun.indexandworkflowRun.indexForWorkflow,workflow.app(wasworkflows.workflow),modelService.invoke(wasai.modelService),billingSettings.showandbillingSettings.update(weresettings.showandmodules.billing.settings.update). The webhooks, Git callbacks, invitations and public status and developer pages no longer show a required token. The image build and CI fail when an operation lacks a summary, an operationId of its own or a group (php artisan runr:api-docs --check).The console's files are no longer compressed on each request. The UI image's Caddy compressed every response as it sent it: the 4.9 MB main bundle took 4.4 s with zstd on demo.runr.sh. The build now puts a Brotli (quality 11) and a gzip (-9) copy next to every text file above 1 KiB, and Caddy sends those as they are (
file_server { precompressed br gzip }), Brotli first: the main bundle goes out as 1.0 MB. A client that takes only zstd gets it compressed on the fly as before (every browser that takes zstd takes Brotli). Cache-Control and the other headers are unchanged; what has no copy (small files,branding-defaults.js, which the container writes at start) is still compressed on the fly, now gzip first. Both UI images (ui,ui-mt) are built this way, their build stage on the build host's platform.The transfer agent could report a failed pull without its reason, or a pull without its result (image-upload 1.1.1). It wrote its
.failedand.donemarkers by creating the file, then writing it, and its own watcher could read one in between, empty. Markers are now renamed into place; an empty.failedof a tool script (a shell redirection creates the file before the reason is in it) counts only when it is still empty at the next look. Regions that pull through a mirror needghcr.io/brf-tech/runr/image-upload:1.1.1.The sign-in page no longer asks for SSO providers without the AuthProvider module, which left a 404 of
GET /api/auth-providers/discoverin the browser console on every visit. It asks only when the console has the module's screens and the API reports the module on (features.auth_providersofGET /api/branding, new).The CAPTCHA widget is light on the sign-in and sign-up cards, which are light in both console themes; elsewhere (the invitation page) it follows the console's theme instead of the browser's, and is drawn again when the theme changes.
The console no longer shows message keys while it starts. The messages of the modules every image has (branding and onboarding, bot protection, images, sandboxes, …) came in only after the optional modules had loaded, so in the multi-tenant image the onboarding wizard of a new organization showed its keys (
whiteLabel.onboarding.title, …) for a moment. They are now in before the first render.
0.6.4 - 2026-11-03#
Upgrade notes#
One new database migration, captcha_settings (the bot protection's settings), run by the pre-upgrade migration Job. Bot protection stays off until a provider is set: the new Helm values captcha.* are optional (captcha.provider: none), and values of 0.6.3, --reuse-values included, keep working (0.6.3 → 0.6.4 in upgrade.md). A sign-up CAPTCHA keeps guarding the sign-up form only: one saved in Administration → Signup moves to Bot protection with the forms set to register, and signup.captcha.* keep applying while captcha.provider is none; the captcha_* fields of /api/admin/signup-settings are gone. A wrong configuration (a key missing, a secret the provider refuses or that cannot be decrypted) no longer refuses every sign-up: the forms get the local check and the administrators an alarm. With secret.create: true (the default) the API, UI, Horizon, scheduler and Reverb pods get the annotation checksum/secret-values: the first upgrade rolls them once, with the new images, and later a helm upgrade that changes only a secret value rolls them too. The region agent and the queue consumer follow the release (0.6.4); gateway-auth, image-upload, supply-chain and the Spin shim keep their versions.
Added#
- Bot protection on every anonymous form. A CAPTCHA guards the sign-in (also an organization's sign-in page and the two-factor step after it), the demo sign-in (the sign-in of a shared account of
RUNR_FROZEN_ACCOUNTS), the sign-up and the public forms (sign-up e-mail resend, invitation accept). Cloudflare Turnstile first, Google reCAPTCHA (v2 checkbox) and hCaptcha as options; off by default. While the provider cannot be reached (network error, timeout, 5xx, a 2xx answer that is not its JSON or namesinternal-error; one retry), the API answers 409 with a local proof-of-work challenge, the console solves it in a Web Worker with a progress bar and sends the form again — a refused token still answers 422, and so does any HTTP 4xx of the provider (429 included, with or without JSON), so a flood never opens the fallback; a local answer (base64 JSON only) is accepted only for 120 s after the provider failed. Every piece of that state is per scope — the platform's keys, or one organization's own keys: the window, the spent tokens and local answers, the configuration alarm, and the HMAC key of the local challenges, so an answer of one scope never passes in another and one scope's trouble never opens the other's window. A wrong configuration counts as an outage, never as a refusal for everyone: no site key, no secret, a stored secret that cannot be decrypted or keys the provider refuses (onlyinvalid-input-secret,missing-input-secret— Cloudflare answers them with HTTP 400 — and hCaptcha's key pair codes;bad-requeststays a refusal) hands out the local check and sends who can fix it — the platform administrators, or the organization'ssettings.manageholders for its own keys — a "configuration is wrong" notification (once an hour per problem); with a key missing the console skips the widget and solves the local check at once (misconfiguredinGET /api/captcha/config), and the settings page shows the problem under the keys. When only the widget cannot load (an ad blocker) while the API reaches the provider, the console says so and asks to turn the blocker off. Every guarded form is throttled, and a token is bounded before it goes out (Turnstile 2048 characters, reCAPTCHA 8192, hCaptcha 32768; base64url and dots only), else 422 without a call; the challenge endpoint refuses a form its scope does not guard (422). Settings: an organization's own provider and keys (multi-tenant, for its white-label host names; they guard only its own surfaces — its members' sign-ins and its invitations; a sign-in naming it is checked with its keys whoever signs in, and for anyone but a member it then fails like a wrong password, so its keys never let anyone else in and nothing before the password tells who is a member) over the platform's (Administration → Access & security → Bot protection, with the forms, the fallback and the timeout) over the Helm valuescaptcha.*(CAPTCHA_PROVIDER,CAPTCHA_SITE_KEY,CAPTCHA_SECRET,CAPTCHA_FORMS, schema-checked) over off; the secret is write-only and encrypted.GET /api/captcha/config,POST /api/captcha/local/challenge,GET/PUT /api/admin/captcha,GET/PUT /api/tenant/captcha;captcha_fallback_totalin the Prometheus format atGET /api/admin/captcha/metrics, and more than one local check in 10 minutes sends the platform administrators an in-app notification (once per 10 minutes). The in-app tutorial tours the new page (security.md).
Changed#
- The sign-up CAPTCHA is the bot protection's
registerform. It no longer fails closed: a provider without a secret or out of reach hands out the local check instead of refusing every sign-up. The CAPTCHA saved in Administration → Signup moves to Administration → Bot protection with the forms set toregister, so an upgraded install guards the same form as before; the Helm valuessignup.captcha.*keep guarding the sign-up form whilecaptcha.providerisnone. The CAPTCHA section of Administration → Signup and thecaptcha_*fields of/api/admin/signup-settingsare gone.
Fixed#
- A secret value changed with
helm upgradeapplies at once. The pods read the chart's Secret as env when they start, and only a ConfigMap change rolled them: a newcaptcha.secret, password,billing.webhookSecretorsecret.appKeywaited for the next restart. The pod annotationchecksum/secret-values(salted with the storedAPP_KEY) rolls them now (helm-values.md).
0.6.3 - 2026-11-03#
Upgrade notes#
No new database migrations. New Helm values, optional and empty by default: branding.colors.dark, branding.colors.light and branding.defaultTheme (the white-label themes) and auth.frozenAccounts (shared sign-ins, RUNR_FROZEN_ACCOUNTS; empty changes nothing); values of 0.6.2, --reuse-values included, keep working (0.6.2 → 0.6.3 in upgrade.md). The console's navy theme is gone: a browser that has it stored opens in Dark (the "Ocean" preset brings the navy look back as a white-label theme). The region agent and the queue consumer follow the release (0.6.3); gateway-auth, image-upload, supply-chain and the Spin shim keep their versions.
Added#
- White-label themes. The branding sets the tones of the dark and the light theme and the default theme, besides the brand scale and accent:
dark_color/light_color(one base colour, the page background; the other tones are derived from it, text keeping its contrast) or the full scaledark_scale/light_scale(--gray-50…--gray-950), anddefault_theme(dark|light, the theme of users who never picked one; a user's own pick always wins). Helm valuesbranding.colors.dark,branding.colors.lightandbranding.defaultTheme(schema-checked, envBRANDING_DARK_COLOR/_SCALE,BRANDING_LIGHT_COLOR/_SCALE,BRANDING_DEFAULT_THEME), the API (GET /api/branding,/api/admin/branding,/api/tenant/branding: stored, served, validated, per tenant where tenant branding is allowed), the CLI and the Terraform resourcerunr_branding(dark_color,light_color,default_theme). Settings → Branding has a Themes section with each theme's base colour, its 11 tones, a preview with the contrast of its text and "Default" back to the Helm value; the in-app tutorial tours it. The cached branding's tones and default theme are applied before the first paint, so a re-branded console does not flash RunR's palette. The "Ocean" preset brings the former navy dark theme back as a white-label theme (white-label.md). - runr.sh in Turkish at
/tr/. The site build (ci/runr-sh/build.py) writes the landing twice fromlanding/index.html:/in English and/tr/in Turkish, both static, with the Turkish texts of thedata-trattributes in place (title, description and social tags included) and the FAQ'sFAQPageJSON-LD in each page's language. Each page is its own canonical;hreflangpointsenandx-defaultat/andtrat/tr/. On/, a Turkish browser without a stored choice, a stored Turkish choice or an old?lang=trlink goes to/tr/before the page loads; crawlers and other browsers stay. The EN/TR button is a link to the other page and stores the choice;/tr/never switches to English. The sitemap is generated: both landing pages with their alternates, then every other page, with the build date aslastmod. - Comparison pages.
docs/compare/sets RunR next to Google Cloud Run, AWS Lambda, Vercel, Knative and OpenFaaS, one short page each: a table of differences, when to choose which and the RunR pages to read next, every statement about the other platform linked to its own documentation. They are published atrunr.sh/compare/<name>/(the index at/compare/) and linked from the landing's comparison section, the site footer and the docs index. - Click counting on runr.sh. The self-hosted Umami, whose script the web server adds to every page, counts the landing's calls to action (
quick-start-click,docs-click,download-click,github-click,install-guide-click,contribute-click,enterprise-open,compare-click,lang-switch), copies from the install snippets (install-snippet-copy), accepted release-news sign-ups and enterprise requests (release-news-signup,enterprise-contact-sent), the GitHub and download links of the documentation pages (docs-github-click,docs-download-click) and the archive links of/dl/(dl-file-click). - Frozen accounts for shared logins. A login several people share, such as a public demo account whose password is published, can be listed in
RUNR_FROZEN_ACCOUNTS(comma-separated e-mail addresses, case-insensitive; Helm valueauth.frozenAccounts, a list or a string, empty by default). It signs in and out and uses what its role grants, but its own account routes refuse every change with 403 andfrozen: true, in the caller's language: profile, avatar, password, two-factor authentication, sessions, notification preferences, API tokens, the tokens of MCP clients (revoke, consent), linked SSO identities, disabling and deleting the account, and the members' routes (user, role, group, 2FA reset, tenant membership, platform admin) when they name the account itself. No visitor can lock the others out; reads stay, and so do another administrator's actions on the account under Members and a frozen administrator's on other members.GET /api/accountandGET /api/auth/mecarryfrozen; Account settings then says it is a shared account and disables its change controls, and its tour starts with that note (security.md).
Changed#
- Documentation pages carry social and breadcrumb metadata. Every page of the docs template has Open Graph and Twitter card tags with the landing's social image, and a
BreadcrumbList(Documentation or Comparisons → page). The overview and runtime-images pages get titles that name them, and a page's description comes from its introduction (a paragraph that only leads into a list or table loses its last sentence), else from its line in the docs index, cut at a sentence end. - The landing is light on phones. Below 769 px, with a touch screen or with reduced motion, the hero shows a still of the 3D scene (
landing/assets/hero-static.webp, 10 KB, andhero-static-wide.webp, rendered byci/runr-sh/render-hero.js), and the page never downloads three.js, the 3D hero or the canvas background; a loader adds those two scripts only past that media query, so the desktop keeps them unchanged. Turnstile loads when a form gets focus or input or comes into view, and a submit waits for its token (or reports that the bot check could not load). The console screenshots are<picture>elements with WebP at 720 and 1440 px before the PNG;vue/shots/optimize.py, the harness step after every render, writes the WebP files (Pillow, the smaller of lossless and quality 80) and lists them inmanifest.json. The built pages carry no translation attributes (/is 135 KB instead of 228 KB), the reveal of the hero text runs before the page's other scripts, and below-the-fold screenshots decode asynchronously. Lighthouse 12.8.2 on a local build, median of three runs: mobile performance 50 → 99 on/and 47 → 99 on the Turkish page (largest contentful paint 6.6 → 1.8 s, total blocking time 919 → 6 ms, page weight 1,067 → 151 KB); desktop 81 → 83 on/(1,103 → 342 KB); the documentation pages stay at 100. - The landing's comparison table follows the vendors' documentation of 2026-11-03. OpenFaaS CE is under its EULA (commercial use: one installation per company for up to 60 days); Vercel Functions run container images (beta); AWS Lambda has durable functions and multi-Region endpoints for Web Functions; Vercel's durable workflows are Vercel Workflows. The new sources are listed.
- The README is a product page. It opens with the logo, the one-line positioning, badges (release, license, CI, Helm chart, Terraform Registry, Go module) and a hero screenshot, then six feature cards, the quick start first (Helm in three commands, the single-host install, the CLI), "Why RunR", a short comparison for people coming from Knative, Knative Functions, OpenFaaS, AWS Lambda or Vercel, a feature tour in seven parts with a screenshot and the documenting page each (
landing/shots/<name>-dark.png, the console's dark theme), the measured cold starts, the architecture, the repository layout and the project sections. The heading anchors other pages link to (#quick-start,#kubernetes-helm) are unchanged.README.tr.mdis the same page in Turkish. The runr.sh build (ci/runr-sh/build.py) now rewrites the images of a published page as it rewrites its links: a file underlanding/is served from the site root, any other repository image from GitHub raw at the release tag; the summary counts them asimages_rewritten. - runr.sh is English by default and shows the console. The landing's static HTML is English (what search engines and link previews read); the Turkish page is
/tr/(see Added). New on the page: a console screenshot under the hero and a tour of the 0.6 features (deploy strategies, environments and promotion, global projects, triggers and the event gateway, durable workflows, supply chain and secret stores, plus deployments, logs and metrics, roles and the region wizard), scenes in the project-type and AI sections, all fromlanding/shots/<scene>-dark.png(1440x900, the console's dark theme; a scene that is not published yet hides its frame); the comparison now sets RunR next to bare Knative, Knative Functions, OpenFaaS, AWS Lambda and Vercel, compiled from their documentation with the sources and a non-affiliation note;helm repo updatein the Helm card; a segmented team-size control in the enterprise form instead of the native select; the Three.js hero is downloaded and started only on devices that will run it (cores, memory, data saver, WebGL); the social images are rendered fromlanding/assets/og-image.src.html(og-image.png1200x630,github-social-preview.png1280x640, withci/runr-sh/render-social.js) and replaceog-image.svg; the tagline under the wordmark is anh2that names the platform, and the FAQ is also published asFAQPageJSON-LD. The site's palette is neutral black with the brand orange and red as its only accents: the backgrounds, surfaces, borders and text use the neutral grays (R = G = B;#0a0a0a,#171717,#262626,#404040,#525252,#737373,#a3a3a3,#e5e5e5), also in the glass layers, the documentation pages, the social image source, the 3D hero's fog and the README badges; the former slate (navy-tinted) palette, the cool grays (#030712,#111827, …), the blue and purple accents and the third-party blues of the diagrams are gone, and the console scenes are shown in the dark theme only.ci/runr-sh/verify.shfails on a blue-tinted colour in the built site (ci/runr-sh/check-colors.py). - One RunR logo everywhere. The console's header and sign-in page show the runr.sh wordmark ("Run" with the gradient "R" and the lowercase "serverless" tagline) instead of a text wordmark with an uppercase tagline; the console's favicon and the repository's
logo/files are the runr.sh ones. - The console has two themes: Dark and Light. The navy theme is removed. Dark is the pure gray palette that was called "Full Dark", and it is also the default before the app boots. A browser that still has the navy theme stored opens in Dark.
- The console is in RunR's colours. Its blue, sky, indigo, violet, purple, cyan and teal accents (category badges such as XaaS, "2–12 instances", the gVisor label, info chips, links, highlights, spinners, charts) take the brand scale, so they follow a white-label brand too; categories differ by the brand's style instead of by hue (XaaS solid, FaaS tinted, CaaS outlined; Kata solid, gVisor tinted, runc outlined; likewise trigger kinds, strategies, roles, run states and token groups). Status keeps its colours: green for success, red for errors (critical solid red, high a red tint), and warnings move from amber and orange, which read as the RunR orange, to yellow. The light theme is warm (whites and gray-browns from the RunR orange hue, the same contrast); the dark theme is neutral black (R = G = B:
#0a0a0apage, Tailwind's neutral surfaces; it was Tailwind's blue-tinted cool gray). The login, sign-up and verification pages, the shell terminal, the code editor's cursor, selection and focus and the log chart take the theme tones and the brand instead of slate hexes and vs-dark's blue. In e-mails the links (Laravel's theme gave them its own colour) take the branding's primary shade 700 and the primary button and wordmark its shade 600 (white on 500 read 2.8:1), the tenant's when a mail is sent for one. - GitHub Actions is the repository's only CI.
.gitlab-ci.ymlis gone; its jobs run in.github/workflows/:ci.ymlon every pull request and push tomain(gitleaks, the reqs mirror, the Dockerfiles and build scripts, the Helm chart with kubeconform, Pint, PHPUnit with the coverage floor and Larastan, the UI lint, unit tests, build and mocked e2e, Go, the Node SDK),audit.yml(npm and Composer audits onmain, daily and on pull requests that touch a lockfile),e2e.yml(the Kind suiteruntime/ci.sh testand the Playwright suite against the dev stack, weekly and by hand, on standard GitHub-hosted runners) andupstream-releases.yml(new Knative minors and Helm chart versions, daily, filed as issues). Every action is pinned to a commit SHA and every downloaded tool checked against its SHA256 (development.md).
Fixed#
- The API reference answers again. Since 0.6.0,
/api/docs,/api/docs.jsonandphp artisan scramble:export(which writes the CLI'stestdata/openapi.json) failed with a 500. The cause was one service method that declared the abstractModelas its return type, which the OpenAPI generator cannot inspect. It now declares the concrete models, and the CLI's copy of the document is regenerated, with the theme fields and the gateways'can_manage. - The project header stays below the console header. It stuck at the top of the window, so when the page scrolled it slid under the console header and its title was cut off. It now sticks at the console header's height, which the header measures (
--app-header-height). Narrower than 1024 px it scrolls with the page (it wraps to twice its height there), its actions sit below the title instead of past the screen's edge, and the project Access form is one column on a phone. - Every project tab can be reached. The tab row ran past the window: at 1440 px the tabs after Traces (Replay, Uptime, Security, Access) were neither visible nor reachable. The tabs stay in one row that scrolls sideways, with a fade and an arrow at each end that hides tabs; the active tab is scrolled into view when the page opens on it and when it changes, Tab reaches every tab, and a tour step on a tab out of view scrolls the row to it first.
- Badges read on the light theme. Badges, chips and notices tinted sky, green, teal or rose, amber backgrounds and borders, violet-300 and blue-400 text kept their dark-theme tones on the light theme: light text on a dark tint (the restored config badge, the rollout chips of the deployments, the sandbox Paused status, the supply chain Low severity, among others). Every tint the console uses now swaps to a light background with dark text (4.5:1 at least, checked by a unit test), and a hover colour (a delete icon's red) shows only while hovered instead of always. The approvals count, the routine runtime patch banner and the scale buttons' hover use tones that read on both themes.
- The tours fit a read-only role. For a user who can only look (the built-in Viewer, a custom read-only role such as the public demo's), the project tour highlighted an empty box as the project's Actions, the platform tour went on into the Add region wizard, which such a user cannot open, and the users tour opened the Add user form. The Actions step now shows only while the role has an action on the project, the region wizard follows the platform tour only with
regions.manageand the users tour opens the form only withmembers.invite; without them the steps drop out like any step whose control is hidden. - The console follows a read-only role. The public demo runs with a custom role that can only look (
projects.view,git.pull,env.view, the*.viewpermissions,logs.view,metrics.view,traces.view); the console showed it controls the API refused with 403, and asked for data it may not read. Now: the project card's edit pencil needsprojects.updateorenv.edit; the platform settings (base domain, registration, GPU demo, custom runtimes, cross-organization grants) show read-only withoutsettings.manage, without Save and without the demo data's Load / Remove; the Models tab needsmodels.view(a refused load says so instead of "No models bound") and binding needsmodels.deployandprojects.update; the sandbox settings ask for the organizations' quotas only withsandboxes.manage; the Git tab gives a role withoutgit.pushthe clone command and the API tokens link only withtokens.manage; the log panel sets a gone deployment's status only for a role that may deploy. A sweep of the console against the route permissions found more of the same, now gated: the Sandboxes page (create, extend, kill needsandboxes.create; terminal, pause / resume and the files, processes and run tabssandboxes.exec), API gateways (new gateway, routes, consumers, plans, domain and developer page follow the gateway's newcan_manageandprojects.update), Settings → AI agents (connecting a client needstokens.manage; the consent page shows a no-access card on a 403), Support (the page needstickets.view, New ticket and repliestickets.create, a list the API refuses no longer looks empty), an organization's Users page with tenancy (the member permissions of the single-tenant page), a synced region's demo button (regions.manage), and for other roles the Metrics tab and the rightsizing card (metrics.view) and a deployment's logs (logs.view). The tours follow: the rightsizing card's and the rollout panel's buttons are steps of their own that drop without the permission, the support tour opens the new-ticket form and an organization's users tour the add form only for a role that sees their buttons, and the texts of the platform settings and the deploy strategy button describe the setting instead of asking a role that cannot to change it. - A first visit opens in the browser's language. A new browser opened the console in Turkish whatever its language. Without a stored choice the console now starts in Turkish for a Turkish browser and in English for every other one (also the API's
Accept-Languageand the page'slang); the language switch's choice still wins. - A first visit opens in the installation's branding. Before
GET /api/brandingwas cached, a first visit showed RunR's dark theme until the API answered. The UI container now writes the platform's branding defaults (Helmbranding.colors.*andbranding.defaultTheme, the samerunr-configkeys as the API's) to/branding-defaults.jsat start, checked values only, and the pre-paint script reads it: the user's theme pick first, then the cached branding, then these defaults, then RunR's. The pre-paint is nowsrc/modules/white-label/prePaint.js, inlined intoindex.htmlby the build, so it derives a base colour's tones with the console's own code (white-label.md). - Demo data in English, with a deployment history. The demo projects' descriptions were partly Turkish, and only the three container projects had a deployment (a code project has no version until its code is saved, and the seeder deployed before saving). The eleven demo projects (the install's demo data and an organization's alike) now have English descriptions, four versions each and a history over the last twelve days: every version deployed in turn, two failed deployments, a rollback to the version before (with its config) and a canary rollout (10 / 50 / 100 %, promoted), so the project page and its tour have deployments to show.
- The public status page in the organization's colours. It showed the organization's brand colour on RunR's dark theme: the API left out the theme tones and the default theme. The page now gets them and applies the organization's branding while it shows (a visitor's own theme pick wins), with the logo of the theme shown.
- One palette for one brand colour. The API derived a single primary colour's 300–900 scale with other mix weights than the console (300: 40 % white against 45 %, 600: 18 % black against 12 % …), so mails, the status page and the console showed two palettes. The API now derives the console's scale exactly; tests on both sides pin the same values.
- Neutral mail tones. The mails' text, borders and panels used Laravel's zinc tones (
#18181b,#52525b,#a1a1aa,#e4e4e7), a slight blue cast next to the neutral palette. They are the neutral tones of the same shades now (R = G = B); the brand colours and the success / error buttons keep their hue. - Placeholders read as placeholders. Input placeholders (gray-400, 12:1) read almost like typed values (14.5:1); the log filter's and the login card's were nearly invisible (1.9:1, 1.4:1). They carry formats and examples, and WCAG 1.4.3 sets no lower bar for placeholder or hint text, so every placeholder now has 4.5:1 or more and stays clearly weaker than a typed value (5.7:1 dark, 4.9:1 light, 5.3:1 on the login card).
0.6.2 - 2026-10-26#
Upgrade notes#
No new database migrations and no new Helm values; values of 0.6.1, --reuse-values included, keep working (0.6.1 → 0.6.2 in upgrade.md). The region agent and the queue consumer follow the release (0.6.2); gateway-auth, image-upload, supply-chain and the Spin shim keep their versions.
Changed#
- The Terraform provider is on the Terraform Registry as
brf-tech/runr. Withsource = "brf-tech/runr",terraform initdownloads the archive from the provider's GitHub release and checks it against the release'sSHA256SUMS, signed with the RunR release key (key ID2588846B241BC095). runr.sh keeps serving the latest release, the same signed archives, underrunr.sh/brf-tech/runr, for OpenTofu and for configurations that use that address. Moving a configuration over is optional: changesource, runterraform state replace-provider runr.sh/brf-tech/runr registry.terraform.io/brf-tech/runronce, thenterraform init(docs/terraform.md). The provider's debug address andmake install(PLUGIN_HOST, defaultregistry.terraform.io) follow. - The release images are built and pushed by the repository's own GitHub Actions workflow.
.github/workflows/release-images.ymlruns on the release tagv<version>with the repository'sGITHUB_TOKEN:api,uiandui:<version>-mt, and the region images ofruntime/ci.sh release-imagesthe registry does not have yet. linux/amd64 and linux/arm64 build on runners of their own (no emulation), are pushed by digest and composed into multi-arch tags;latestmoves only to the highest release tag; the last job pulls every tag without credentials. Every image labels RunR's own OCI metadata in place of its base image's (org.opencontainers.image.source=https://github.com/BRF-Tech/runr, title, description, version, revision, created, licenses) and every index carries it as annotations, so each package on ghcr.io is linked to the repository. The run's artifactrunr-<version>-tarsholds the amd64 image tars ofapi,uiandui-mt(the pushed images, byte for byte), the chart andSHA256SUMS. The GitLab pipeline builds and pushes no image any more; its test jobs stay (development.md).ci/check-dockerfiles.shfails on an image that does not label the source.
Fixed#
- The MCP server's documentation resources are in the API image again. The released API image had no
resources/docs(the release builds did not copydocs/into it, 0.6.1 included), sorunr://docs/{page}found no page unlessMCP_DOCS_PATHor adocs/next to the API was set up. The release workflow copiesdocs/*.mdinto the image, ascompose/build.shandci.sh build-deploydo (mcp.md).
0.6.1 - 2026-10-24#
Upgrade notes#
No new database migrations and no new Helm values. With tenancy, the migration Job runs the Billing and Ticket tenant migrations that 0.6.0 skipped while the module was off. Horizon works a new deploys queue with a worker of its own. Before a rollback to 0.6.0, let the running deploys finish: the 0.6.0 → 0.6.1 section of upgrade.md says how.
Fixed#
- The console's tutorial follows the 0.6.0 console. Its steps point at the current controls again (a step whose target was gone dropped out without a word; a unit test now checks every target and text) and show the create wizard's global projects, New MCP server, rightsizing, environments, rollouts, deploy strategies, the timed warmer and the settings pages' tabs; the edit modal gets a tutorial button.
- The console says when it cannot reach the server. A request that got no answer (network down, DNS, a server certificate the browser rejected, CORS, a dropped connection) showed only "An error occurred."; it now says the console could not reach the server and to check the connection, and a timeout says the server did not answer in time. HTTP errors still show the API's message, cancelled requests stay silent. Deleting a deployment and saving the audit log retention show the API's message instead of "An error occurred." too.
- Tour steps on buttons that appear on hover show the button. The deployment table's log, rollback and delete steps highlighted an invisible button; the step's element is shown while the step is active.
- A tenant gets the tables of the Billing and Ticket modules while they are switched off. The entrypoint disables a module that is off, so its tenant migrations were never registered: a tenant created on 0.6.0 with billing off had no
billing_*tables, and older tenants missed 0.6.0'sbilling_cost_limitsand queue consumer charging, against docs/billing.md ("the billing tables exist … also while the module is off"). Tenant schemas now take every module's tenant migrations, enabled or not. - A redeploy is active within seconds again. 0.6.0 added calls to the region's API server on the way from
POST /deployto an active deployment, one after the other and each on a new TCP and TLS connection: the replaced deployment's teardown (queued ahead of the deploy on Horizon's one idle worker), the secret values and the internal-network routes. Against a real API server a redeploy took 10-12 s instead of 4-5 s. The apply and the readiness checks now run on adeploysqueue with a Horizon worker of their own (one more idle process inrunr-horizon), calls to an API server reuse their connections, a new deployment is checked every second for its first 10 s, and the response cache no longer keeps a deployment "pending" after its activation (a GET during it was cached under the new version with the old row until the project changed again: 11-22 s in Kind). Kind with 100 ms added to the API server: 7.0 s on 0.6.0 (6.1 s on 0.5.10) → 3.4 s, median; without it 3.5-3.7 s → 2.5 s.
0.6.0 - 2026-10-23#
Upgrade notes#
The 0.5.10 → 0.6.0 section of upgrade.md lists what to decide before the upgrade, the order of operations and the new Helm values.
- Every region enforces the internal network after the upgrade. A call between two projects needs an entry on the callee's allow list. Enforced on upgrade; switch a region to audit right after if needed (
runr regions set REGION --set internal_auth=audit): both are new and cannot be set before the upgrade (upgrade.md). - The regions are brought to the release by a Job of their own, after
helm upgradereturned.region:reconcileinstalls cert-manager on every region cluster that runs none (knative.certManager.install: falsekeeps it out) and rolls the region Traefik three times: about 5 minutes per region cluster, more than the bootstrap hook's two minutes (it was cut off and loggedregion reconcile skippedwhile the hook succeeded). The hook now does the database steps only, sohelm upgradetakes about a minute as before; the Jobrunr-region-reconcile-<revision>(no hook: helm does not wait for it) runsagent:redeploy --missing,region:reconcileandsecrets:move-to-regionwith up to three attempts, each starting steps for an hour (bootstrap.regionReconcileSeconds) and continuing where the last stopped. Regions it left behind show in the console's region status, the region API (reconcile),/api/version(region_reconcile) and a notification of the platform administrators (upgrade.md). - The region agent and the event gateway run at the installed version, not
:latest.ghcr.io/brf-tech/runr/agent(also the internal network's authz) andqueue-consumer(queue consumers, the event gateway, topic publishers) default to the API's own version (APP_VERSION, set by the chart from itsappVersion: a 0.6.0 install pulls the 0.6.0 images,helm rollbacktakes them back), andregion:reconcile(stepimage-registry) moves the regions' agents, queue consumers and topic publishers from:latestto it: each rolls once. The manual restart of the region agent after the upgrade is gone. A region that pulls through a mirror of your own needsagent:0.6.0andqueue-consumer:0.6.0there. The upload receiver defaults toimage-upload:1.1.0, the transfer agent's image (the receiver is the same). A release publishes the region images withruntime/ci.sh release-images --push(upgrade.md). - Upgraded tenants' Member role gets the new Developer permissions.
environments.manage,environments.promote,network.manage,secret_stores.viewandtopics.manageare part of the built-in Developer role, a fresh tenant's default for members; the tenant migration0108adds them to the custom role Member that the upgrade to 0.5.0 created, where that role is still as the upgrade left it. A Member role the organization changed is left alone (upgrade.md). - Schedule triggers get a time zone; the ones that exist keep UTC. A new schedule without a zone runs in the organization's time zone (
Europe/Istanbulunless set). NameUTCwhere a new schedule must stay in UTC (upgrade.md). - Global projects fail over on their own. A global project's primary that fails its health check for 180 s (
globalDns.failover.afterSeconds) moves to the next healthy region with its triggers and data; set the project's failover tomanualto keep it where it is (upgrade.md).region:reconcilerestarts each region's PowerDNS server once. - Project secret values move out of the database — and 0.5.10 stays installable until you finalize. Values written before this release stay encrypted in
project_secretsand keep working; the region Job of the upgrade moves them into their regions after the rollout (secrets:move-to-region, idempotent and resumable; run it by hand with--dry-runto check). Until the upgrade is finalized the database keeps an encrypted copy of every value — the moved ones and every value set afterwards — so that the 0.5.10 chart can be put back on the release (helm rollback, orhelm upgradewith the 0.5.10 chart where only one package version is kept; never uninstall) without its deploys writing empty values; secret store references wait until then (409 upgrade_not_finalized). Once this release is confirmed, finalize:helm upgrade … --reuse-values --set projectSecrets.finalize=true(orphp artisan secrets:move-to-region --finalize); 0.5.10 must not be installed again afterwards. KeepAPP_KEYuntil then. Fresh installations are finalized from the start.projectSecrets.storage: databasepostpones the change for new values (upgrade.md).
Added#
- Projects call each other on the region's internal network (internal-network.md). Every project has an internal name
<project>.<organization>and answers at<project>.<organization>.runr.internal; its workloads getRUNR_INTERNAL_TOKEN, which a call sends inX-Runr-Internal-Token.- Projects are isolated from each other, also inside one organization: a project admits itself, RunR and the projects on its allow list. Another organization's project needs a grant on that list, which a platform administrator can switch off for the whole install.
- The region agent in authz mode (
runr-internal-auth, two pods per Serving namespace) checks every call on the region Traefik's private entry point and hands the callee the verified caller inX-Runr-Caller. A caller is the project whose pod the call comes from (known by its address) or the project its token names. - Every region enforces it, also an existing one after the upgrade (
region:reconcile, stepinternal-network): a call between two projects needs an entry on the callee's allow list. The region settinginternal_auth: auditadmits every call and logs what the allow lists would refuse, for an operator who wants to look first. - RunR's own traffic keeps working across the upgrade without a redeploy: queue consumers, functions calling the model services they are bound to (no header needed), the region demo (its frontend is granted on its backend); the web shell and the model playground sign a short-lived token.
- Access modes: a project is public (the default), public from an IP allow list (its public host, custom domains and gateway route answer the listed addresses only) or internal only (no public URL; reached on the internal network).
- The project's Network tab,
runr network, the API (/projects/{project}/network), the MCP toolsnetwork_get,network_set,network_caller_addandnetwork_caller_remove, and thenetworksection ofrunr.yamlset both; permissionnetwork.manage. The region settinginternal_auth(enforce / audit) and the platform settinginternal_cross_tenant_grantsare in the console's region and platform settings. runr regions set REGION --set internal_auth=enforce|auditsets the region's mode from the CLI (the API's two values; an unset region enforces). A CLI test keeps the keys ofrunr regions setequal to the fields and values of the API's settings and monitoring requests.runtime/ci.sh test-internal-network: the Kind e2e suite of the internal network, with the agent and queue consumer built from the checkout, and a small model service (the AI suite's tiny CPU model on hf-serve) bound to one of its projects: the bound project's calls pass with and without a token and another project's are refused — before and afterregion:reconcilemoves a region of an earlier release, while a pod keeps calling the model through the switch (RUNR_E2E_IN_MODEL=0leaves the model out).
- Runtime patches: every project shows why a patch of its runtime came, and decides what to do with it (Runtime patches). The runtime images get immutable patch tags (
exec-php:8.3.36-r1; the floatingexec-php:8.3points to the newest, the last five per minor version are kept), and one signed feed tells every RunR installation about them: level (routine,security,critical,zero-day), reason in English and Turkish, CVEs, link, upstream release and, for critical and zero-day patches, a deadline.- Workloads run a patch by digest: Knative Services and CronJobs of code-mounted versions the project's pin or the newest patch, recorded on the deployment; versions built into an image are built
FROMthe patch by digest; sandboxes and warm pools the newest patch. - Projects have a patch state —
up_to_date,patch_available(with the most urgent level),pinned,overdue— shown next to the end-of-life badges, in the API (runtime_patchof a project),project_getandrunr projects get. - Apply redeploys the live version onto the newest patch; pin fixes a project on one of the last five patches (and redeploys onto it); console dialog with level, reason, CVEs, link and deadline,
POST /api/projects/{project}/runtime-patches/apply,PUT|DELETE …/runtime-patches/pin, the MCP toolsruntime_patches_get,runtime_patch_applyandruntime_patch_pin, andrunr projects patches,runr projects patch apply|pin|unpin,runr runtimes patches. - Deadlines (details): at a critical or zero-day patch's deadline RunR redeploys every project still behind onto the newest patch — a pin gives way — and rolls one back to the digest it ran before (pinned there) when the new revision does not become ready, with a critical notification. Platform administrators switch this off or extend a patch's deadline.
- Notifications by level (details): security patches in-app to the owner and last deployer of each project behind and the organization's owners and admins; critical ones also to the platform administrators and by mail; zero-day ones to everyone whatever their preferences, on the organization's alert channels too, with a console banner until no project runs behind. Each goes out once per patch and user (notification preference
runtime_patch). - End-of-life reminders 13, 4 and 1 week(s) before a runtime version's end of life to the owners of its projects and the organization's owners and admins (in-app; by mail a week ahead) — there were none before (End of life).
- Settings → Patches for platform administrators (The Patches page): the feed's status, last check and signature, Check now, the policy, every patch with the projects behind it per organization, Remind, Force now, Extend deadline; a banner announces a newer RunR release the feed names ("RunR 0.5.11 available (zero-day): …").
- The runtime patch feed is verified before it is used (Runtime patch feed): an OCI artifact (
ghcr.io/brf-tech/runr/runtime-patches:feed, media typeapplication/vnd.runr.patch-feed.v1+json), signed with cosign and checked against the configured public key in the API. RunR checks its digest every five minutes; a feed with a wrong signature or schema is ignored, the last good one stays and platform administrators are told once. Installations without a way out read it through a region's registry.- New Helm values
runtimePatches.feed,runtimePatches.publicKey(by default the key BRF-Tech/runr-runtimes signs the feed with, so the feed works out of the box; empty: the feed is off and platform administrators see a warning),runtimePatches.viaRegionandruntimePatches.autoMigrate(runtimePatches). - Audit events
runtime_patch.apply,.pin,.unpin,.feed.check,.policy.update,.remind,.force,.deadline.update, andruntime_patch.migrate/runtime_patch.rollbackfor what RunR does at a deadline.
- New Helm values
- Workloads run a patch by digest: Knative Services and CronJobs of code-mounted versions the project's pin or the newest patch, recorded on the deployment; versions built into an image are built
- Timed warmer: containers kept warm on a weekly schedule (Scaling). In the scale settings, Advanced: timed warmer takes up to 10 rows of a cron expression and a container count. Each row means "from this moment on, at least N containers" until the next row fires, for example 3 containers on weekdays from 09:00 and none from 18:00.
- The console explains each row in plain language (English or Turkish), shows a weekly timeline of the warm containers and, in pay-as-you-go regions, an estimate of the next 30 days in vCPU-hours, GB-hours and cost.
- The crons run in the organization's new time zone setting (Settings,
timezoneof/api/settings; default Europe/Istanbul). - While a schedule is set it replaces
min_scale;max_scalestill applies and no row may keep more containers warm. - The value is set on the Knative PodAutoscaler of the live revision every minute: no new revision, no restart, no deploy needed for a changed schedule. Every deploy starts the new revision with the value of the moment.
- Capacity, organization quota and GPU admission count the schedule's peak. Metering counts the containers the warmer actually kept warm.
- Also in
runr.yaml(scale.schedule), the CLI (runr scale --warm "<cron>=<count>",--no-warm), the MCP project tools and the Terraform provider (runr_project.scale_schedule). POST /api/scale-schedule/previewevaluates a schedule before it is saved.- New e2e suite
runtime/ci.sh test-warmer(test:e2e --suite=warmer).
- A monthly cost limit for pay-as-you-go projects (Billing module, docs). The project's owner sets one amount in the billing currency on the project's Billing tab. Under it the tab shows this month so far and a month-end forecast (so far ÷ elapsed days × days in the month). Fixed-price projects show nothing.
- At 80 % the owner is notified. At 100 % the owner chose either another notification or a stop of the project. Each goes out once per month and limit.
- A project the limit stopped is resumed by its owner on the Billing tab once the limit no longer stops it: raised, removed, switched to notify, or a new month began. With the limit still set, the first check of a new month resumes it by itself and redeploys the version that ran (event
project.resumedwithsystem: "cost_limit"), unless the new month already reached a stop limit (New month). - The limit is checked every 15 minutes (
billing-cost-limits) and when it is saved, so a stop limit below this month's cost stops the project right away (the console warns before saving). - New webhook events
project.cost_limit.warningandproject.cost_limit.reached;project.stoppedcarriessystem: "cost_limit". A project'sstoppedstate sayssystem: "cost_limit"and names no user. - API:
GET/PUT/DELETE /api/projects/{project}/billing/cost-limitandPOST …/cost-limit/resume. MCP:project_cost_limit_get,project_cost_limit_set. - It is a guardrail the owner sets. RunR still never stops a project because a bill is unpaid.
- Billing tools for AI agents (Billing module, MCP tools). While billing is on, the RunR MCP server has tools for the billing reads and settings an agent needs, each calling the billing API with its permissions:
- bills and their line items, split into the project's own usage, its preview environments and its queue consumer:
billing_bills_list(the organization's, the platform's to the tenant, the platform's to its tenants),billing_bill_get; a project's Billing tab:project_billing_get; billing_usage_get(metered usage per project and month),billing_prices_getandbilling_price_set(a price version that changes only the prices given);- queue consumer charging at the region and organization level:
billing_regions_list,billing_queue_consumers_set; per tenant for platform administrators:billing_region_organizations,billing_region_organization_set; - fixed allocations:
project_billing_quote,project_fixed_request(request or withdraw),billing_fixed_requests_list,billing_fixed_request_decide(approve or reject). - A tool's permissions may now name platform administrators (
@platform:billing.view, the rule of the API's platform routes); the console's tool catalog shows them so.
- bills and their line items, split into the project's own usage, its preview environments and its queue consumer:
- Load tests of HTTP projects (load-test.md). Project → Load test,
runr loadtest, the MCP toolsload_test_run/load_test_getandPOST /api/projects/{project}/load-testsrun k6 inside the project's region: 10, 50 or 100 concurrent users send a GET, or a POST with a small body, to a path of the project's own URL for 30 s, 1 min or 5 min. The results are p50 / p95 / p99 latency, the error rate and the requests per second, with the pods the project scaled to and its cold starts in the test window; the last five runs are shown side by side, and the console follows a running test live.- The target is only the project's function host or one of its active custom domains, resolved to the region's own ingress: the test measures the function, not the internet. One test per project at a time, a daily limit per organization (
observability.loadTest.dailyLimit, 20). - The k6 Job runs in the region's
runr-loadtestnamespace (created on the first test): restricted pods without a service account token or secrets, CPU and memory limits, a pod quota, egress only to the region's Traefik, removed after the test. - Test traffic is metered and billed like any other traffic of the project.
- Running and stopping a test needs
projects.deploy, reading the resultsmetrics.view. - New image
ghcr.io/brf-tech/runr/reqs/k6:1.8.2(the reqs mirror ofgrafana/k6); new Helm valuesobservability.loadTest.*. - The load-test namespace goes with the region's Knative Serving (Load tests). Uninstalling Serving (or all of Knative) left
runr-loadtestwith its policies, quota and running k6 Jobs on the cluster. It is now removed with Serving (only when RunR created it), the region's running tests end as failed with the reason, and the next test after a new install applies the namespace again. The end of a tenant's share of a platform region removes that tenant's k6 Jobs and their ConfigMaps from the shared namespace and ends its tests there.
- The target is only the project's function host or one of its active custom domains, resolved to the region's own ingress: the test measures the function, not the internet. One test per project at a time, a daily limit per organization (
- Deploy strategies: blue-green and canary rollouts per function (deploy strategies). Direct deploys stay the default; an HTTP or workflow function can switch on a strategy with its options.
- Blue-green: the new revision starts next to the one that serves with 0 % of the traffic and a test URL of its own (
green-<slug>.<function domain>). A promotion moves all traffic at once — from the console, the API, the MCP tooldeployment_promoteorrunr rollout promote— or by itself once the revision is ready and an optional health path answered 2xx a number of times in a row. The previous revision stays warm for a configurable window (15 minutes by default,previous-<slug>.<function domain>), so a rollback is instant; then it is released. An abort drops the new revision and the traffic stays. - Canary: the new revision gets its share in steps (10 % → 50 % → 100 % by default, every 5 minutes) and is promoted after the last one. When the new revision's error rate or p95 latency exceeds its threshold during a step, the rollout rolls back by itself. The numbers are the revision's own (Canary thresholds): its span metrics while tracing is on (the collector's span metrics carry the dimension
runr.revision;region:reconcileupdates a collector installed before), else Knative's request metrics of the revision where the region's Prometheus collects them; without either the canary progresses on time only. The rollout'smetricssay which (source,revision). Promote now and abort work at any time. - Settings in the project's console settings,
GET/PUT /api/projects/{project}/deploy-strategy,runr strategy, anddeploy:inrunr.yaml(CLI and git push); a change creates no version. One deploy can override it (strategy,runr deploy --strategy direct). - The rollout endpoints (
…/rollout,…/rollout/promote|abort|rollback), the live rollout panel of the console (traffic split, steps, health checks, metrics against the thresholds, test URLs, history) updated over thedeployment.rolloutWebSocket topic, the MCP toolsdeployment_promoteanddeployment_abort,deployment_statuswith the rollout,runr rollout status|promote|abort|rollback, and audit events for the settings, the actions and what a rollout does by itself. - Queue and schedule functions, previews and the deploys of a region move stay direct. While a rollout runs both revisions run and are metered. The revisions a rollout holds keep their images through registry retention and are never collected by Knative's revision GC.
- New Kind suite
runtime/ci.sh test-deploy-strategies(test:e2e --suite=deploy-strategies).
- Blue-green: the new revision starts next to the one that serves with 0 % of the traffic and a test URL of its own (
- Rightsizing recommendations. RunR turns what a running project's pods really used into a size recommendation: "requests 512 MB but used at most 140 MB in the last 14 days → 256 MB is enough", with what it saves per month when billing is on, applied with one click (Rightsizing).
- Every night the
rightsizingjob reads the day's peaks of every running project from its region's Prometheus — the highest and the 95th percentile working-set memory and CPU of one pod, OOM kills, throttled CPU periods, with the size that ran — and keeps them per project and day (resource_peaks). The first night after the upgrade reads back the last 14 days. - A project's last 14 days are compared with its size: memory from the highest working set × 1.3, CPU from the 95th percentile × 1.3, rounded up to the next size step; a larger size after an OOM kill, a memory peak near the limit or CPU throttling at the current size. Projects with GPUs and projects with too little data (fewer than 3 days, a scale-to-zero function with fewer than 20 sessions) get none. Thresholds, margins and steps are configurable (
RIGHTSIZING_*). - Applying puts the size into the project's config through the checks of a manual edit (region capacity, a fixed allocation) as a new version; the next deploy runs it. A recommendation can be dismissed (until the recommended size changes) or snoozed. Both are audited.
- With the Billing module a recommendation shows what it saves per month: an always-on workload its reservation at the hourly prices, a fixed project the allocation it could lower; a scale-to-zero function is billed by use, so its smaller size frees capacity, not money (Billing). Without billing the resources are shown.
- Console: a card on the project's page and /rightsizing, the organization's list sorted by saving, linked from the projects page. API:
GET /api/rightsizing,GET /api/projects/{id}/rightsizing,POST …/rightsizing/apply,POST …/rightsizing/dismiss. CLI:runr projects rightsizing [NAME]|apply|dismiss. MCP:rightsizing_list,rightsizing_apply.php artisan rightsizing:runruns it on demand.
- Every night the
- A project has a list of triggers (triggers.md): any combination of HTTP, schedules, queues, a webhook inbox, topics and platform events, each with its own name, settings, write-only secrets and an enabled switch.
- The first enabled trigger is the primary one:
config.trigger_type/schedule/queue_streammirror it, so clients of the single-trigger API, the Terraform provider and runr.yaml'strigger:keep working; setting them changes the primary trigger. GET|POST|PUT /api/projects/{project}/triggers,PATCH|DELETE …/triggers/{trigger},GET /api/trigger-types; the project resource carriestriggers. runr.yaml takes atriggers:list. Console: the trigger editor of a project. CLI:runr triggers. MCP:triggers_list,trigger_add,trigger_update,trigger_remove.- Every existing project gets its trigger from its config with the upgrade (tenant migration
0077_create_trigger_tables). - Webhook inbox: a stable public URL per trigger (
/api/hooks/<context>/<key>) with signature presets for Stripe, GitHub, Slack, a generic HMAC (header, algorithm, hex / base64, prefix) or none; a delivery id the inbox already stored is not delivered again. The sender gets 200 once the request is stored; the function gets it afterwards, retried. Stored events stay for the trigger's retention (7 days by default) and can be replayed from the console's inbox viewer, the API,runr triggers replayand MCP (webhook_events_list,webhook_event_replay). The inbox takesPOSTandPUT; the API document (/api/docs) has an operation for each. - Topics: fan-out topics of the organization on a region's Redis — every subscribed function gets each event (one consumer group per subscribing trigger). Functions publish with
RUNR_TOPICS_URL/RUNR_TOPICS_TOKENand thepublishhelper of the workflow SDKs; others withPOST /api/topics/{topic}/publish,runr topics publishor the MCP tooltopic_publish. New permissiontopics.manage(Developer and above) to create and delete topics. A topic'screated_byis the creator's id as a string in every answer — the201of the create was a number — in the API, its document, the CLI and the MCP tools. - Broker and external sources:
kafka(consumer groups),nats(JetStream durable consumers or core NATS),rabbitmq(manual acks),mqtt(QoS 1 / 2 with a persistent session),cdc(Postgres logical replication, in commit order),s3(object created / updated / deleted on any S3 API) andemail(IMAP) triggers. The event gateway reads them and acknowledges an event only once it is delivered or dead-lettered; credentials are write-only trigger secrets. The console renders their settings from the type's fields (GET /api/trigger-types). - Platform events:
deployment.finished,deployment.failed,alert.firedandalert.resolvedreach functions with aplatform_eventtrigger as CloudEvents (dev.runr.<event>);runtime.patch_availableandcost.limit_reachedare defined for the features that raise them. - Audit log: triggers, topics and the internal network.
project.trigger.create,.update,.replace,.delete,project.trigger.event.replay,project.trigger.events.replay,topic.create,.update,.delete,.publish,project.network.update,project.network.caller.addand.remove— names and counts, never secret values or event bodies (Security). - Schedule triggers run in a time zone (triggers.md):
config.timezone(an IANA zone), else the organization's time zone, the timed warmer's, which the schedule then follows — a change of the organization's zone reaches the deployed CronJobs (spec.timeZone) and gateway schedules at once. The trigger API showseffective_timezoneandtimezone_source; the console's trigger editor,runr triggers ls, runr.yaml (timezone:) and the MCP trigger tools take and show the zone. A tick's data carriestimezoneandlocal_time. The project API's single-trigger fields takeconfig.schedule_timezone(""= the organization's) and return it withconfig.schedule_effective_timezone; Terraform'srunr_projecthastrigger.timezoneand the computedtrigger.effective_timezone(data.runr_project:schedule_timezone,schedule_effective_timezone), the MCP toolsproject_create/project_updateschedule_timezone. Backward-compatible zone names (Europe/Kiev) are taken where the API's PHP knows them — RunR's API image does; the API's tests take such a name from the PHP that runs them, so they pass on a PHP built against a system tzdata without the legacy names (Ubuntu 24.04 withouttzdata-legacy) too. - A removed cdc trigger's replication slot is dropped (triggers.md): when a cdc trigger is removed, renamed or pointed at another slot or database, or its project or an environment is deleted, RunR drops its slots and the publication it created from the database once no deployed event gateway reads them — a one-shot pod of the gateway image (
GATEWAY_MODE=cdc-cleanup) in the region, as the gateway reaches the database. The API answer, the CLI and the console say what is dropped; a notification reports the outcome, with the SQL to run by hand when the database cannot be reached. The gateway marks the publications it creates, so one that existed before is never dropped. - runr.yaml
triggers:is checked by the CLI with the API's own rules:php artisan triggers:schemaderives a schema of the trigger entries from the trigger types (api/resources/triggers/triggers.schema.json) and the CLI embeds it (required keys, types, ranges, patterns, cron, time zones,delivery). What the API only ignores is now a warning, not an error:trigger:next totriggers:, asecretskey (not sent), keys a type does not have;enabled: 1andtrigger.schedule: "@daily"are accepted as by the API. runtime/ci.sh test-trigger-sourcesruns every broker source on Kind — Kafka, NATS JetStream, RabbitMQ, MQTT (no longer opt-in), Postgres CDC with the slot cleanup, IMAP, next to S3 — and a schedule time zone, with brokers from the reqs mirror.- The topic publisher counts in the region capacity on every cluster.
runr-topicsis a platform pod like gateway-auth; it now tolerates dedicated workers (and prefers the other nodes), so it is scheduled — and counted — also where every node is a dedicated worker, where it stayed Pending.
- The first enabled trigger is the primary one:
- Secret stores: references without credentials. A project secret variable can refer to a value in the organization's HashiCorp Vault or OpenBao (Kubernetes or JWT auth), AWS Secrets Manager, GCP Secret Manager or Azure Key Vault (workload identity federation): store, path and key. External Secrets Operator — an optional region feature installed from the region's page or
POST /api/knative-regions/{id}/external-secrets, with RunR's mirror of its image, kept at the pinned chart byregion:reconcileand installed again by a region migration — reads it in the region as the service accountrunr-ss-<store id>and fills the Secret<slug>-ext-envthe workload reads. The store never gets a credential from RunR: it trusts the region cluster's service account issuer, and a store has no setting for a token, password or key. The store's trust page shows per region the issuer, its keys (JWKS and PEM), the service account, namespace, subject and audience, and Verify applies and checks the store in a region (secret stores).- Secret store API, console, CLI and MCP.
/api/secret-storeswithtrustandverify(permissionssecret_stores.view— Developer has it — andsecret_stores.manage);PUT /api/projects/{project}/secrets/{name}takes{"reference": {…}}; Settings → Secret stores and references in the project's secret variables;runr env ref NAME STORE:PATH[#KEY],runr env sync,runr secret-stores ls|trust; the MCP toolssecret_ref_setandsecret_stores_list, andenv_listshows where each secret lives. - Sync status of references.
secrets:sync-statusreads every two minutes what External Secrets Operator reports —synced,failedwith the store's message,pending— onto each reference and onto the stores per region; Sync now (POST …/secrets/refresh,runr env sync) asks for a read right away.
- Secret store API, console, CLI and MCP.
- OpenBao in the region. A region can run an OpenBao that RunR installs and operates — the region's Secrets tab,
POST /api/knative-regions/{id}/openbao,runr regions openbao— next to External Secrets Operator, which it needs: one StatefulSet inrunr-openbaowith an unsealer beside the server, file storage on a volume, TLS issued by RunR, the image from RunR's mirror (ghcr.io/brf-tech/runr/reqs/openbao), kept at the pinned image and a fresh certificate byregion:reconcile, installed again by a region migration. While it runs, new secret values of the region's projects go into it (sourceandstored_inopenbao;PATCH …/openbaowithdefault_for_values: falseturns that off): each project reads its own values through a service account, an OpenBao role and policy of its own, its SecretStore and the ExternalSecret<slug>-bao-env(OpenBao in the region). The Secrets tab offers OpenBao's install as soon as the operator installed there is ready, without a reload.- The region's OpenBao keeps no credential in RunR. Its unseal key goes into the region's Secret
runr-openbao-unsealonly and its TLS key pair intorunr-openbao-tls; the root token is revoked once the first install configured it, and RunR logs in through the API server with a ten-minute token of the service accountrunr-controlthat it never stores. The Postgres role OpenBao issues dynamic users with gets a password RunR hands over once and OpenBao rotates at once: afterwards only OpenBao knows it. OpenBao's audit log goes to the container's output.
- The region's OpenBao keeps no credential in RunR. Its unseal key goes into the region's Secret
- Dynamic Postgres credentials. A Postgres attachment with
"credentials": "dynamic"gets a user of its own from the region's OpenBao instead of the binding's user and password: createdIN ROLEof the binding's role, living a week and renewed every day in the region (<prefix>PGUSER,<prefix>PGPASSWORD,<prefix>DATABASE_URLfrom<slug>-pgdyn-<…>); a renewal rolls the workload onto the new user, detaching or switching back revokes the users (RUNR_OPENBAO_DB_TTL,_MAX_TTL,_REFRESH;runr bindings attach --credentials dynamic|static) (bindings). - Environments and promotion (Environments). A project runs in several environments: its own workload —
production— and any others you add (dev,staging, …), each with its own Knative Service<slug>-<name>and URL<slug>-<name>.<function domain>.- Configuration layering: the project's configuration with the environment's overrides (env vars merged by name, a
nullremoves one; CPU, memory, scale, timeout, schedule, queue stream); secret variables of its own over the project's (inherit_secretsoff: only its own); bindings and custom domains attached to the environment only — staging's pods cannot reach production's data (binding NetworkPolicies admit them byrunr/environment-id). - An environment can run in another region than its project, within the organization's data residency. A built version (dependencies, a compile step, WebAssembly) whose image that region's registry does not hold is built there from its code, FROM the same runtime base: the deploy, promotion or push waits for the build while the environment's running deployment keeps serving, and every later deploy of the version into the region runs it at once (
version_region_images, central migration2026_10_22_100001, tenant migration0084). - How many environments a project may have is decided by the quotas: each one reserves its size in its region's capacity and the organization quota from its creation on. Preview environments stay separate.
- A git branch maps to an environment (
develop→staging): a push deploys it; itsrunr.yamldoes not change the project's configuration. - Promotion moves the version an environment runs to the next one — the same version and image, never built again (in another region: built into its registry) — with the target's configuration, after a "what will change" preview (code version and config differences, env var names only). Promotions into production wait for someone else with the new permission
environments.approve(the project can turn the approval off);environments.manageandenvironments.promoteare part of the Developer role. Approvals inbox:GET /environment-promotions, the console's Approvals page (live),runr approvals. - Approvers hear of a promotion that waits for them, the requester of its approval, rejection, deployment or failure: in-app and by mail through the new Promotions notification preference, and as the platform events
environment.promotion.pending_approval,.approved,.rejected,.deployed,.failedforplatform_eventtriggers. - Gateway routes and private database connections per environment: a route leads to the project's own workload or to one environment (
environmentonPOST /gateways/{gateway}/routes;POST /gatewaysgives you a gateway in another region), a Skupper connection to a managed PostgreSQL database belongs to one environment and reaches only its pods (central migration2026_10_22_120001, tenant migration0086). - Durable workflows per environment: every environment is a workflow app of its own — its definitions, its runs against its workload, its own signing key, its cron triggers and queue bridge — and events belong to an environment by its name (
environmentonPOST /api/events; central migration2026_10_22_110001, tenant migration0085). The workflow API, the MCP workflow tools andrunr workflows --envtake it; the console's Workflows tab follows the environment switcher. - Deploy strategies per environment (Deploy strategies): each environment rolls out with the project's blue-green / canary settings unless it sets its own strategy or options (kept in its overrides,
deploy_strategy). Deploys, promotions into it and rollbacks follow the environment's settings; its rollouts run on its own workload — test URL on its host, traffic, canary thresholds on its own traces, the kept-warm previous revision — and are promoted, aborted and rolled back there, next to production's.GET|PUT /projects/{project}/deploy-strategytakeenvironment(andreset),GET /projects/{project}/rollout?environment=; rollouts carryenvironment(central migration2026_10_22_100002, tenant migration0088:deployment_rollouts.environment_id). Console: the strategy on each environment's card and in the Deployments tab of the selected environment, the environment's rollout panel; CLIrunr strategy show|set|reset --env,runr rollout … --env; MCPdeployment_status,deployment_promote,deployment_aborttakeenvironment. - The timed warmer per environment (Scaling): an environment runs the project's schedule unless it has its own (
overrides.scale_schedule;[]keeps no containers warm there), on its own workload and every revision that takes traffic there (a rollout of the environment: both), in the organization's time zone; what was applied is recorded on the environment (warm_scale; central migration2026_10_22_100003, tenant migration0089) and a schedule turned off there gives the workload itsmin_scaleback. Environments showscale_schedule,scale_schedule_inheritedandwarm_scale. Console: the environment's settings → Timed warmer and its card; CLIrunr environments create|update --warm … | --no-warm | --project-warm; MCPenvironment_createtakesscale_scheduleand the newenvironment_updatechanges an environment's settings and overrides. - Saving an environment's overrides without its variables keeps them for a caller who cannot see them (no
env.view), and sending them back unchanged needs noenv.edit; the console's environment form keeps the overrides it does not show (schedule, queue stream, deploy strategy). - Console: environment switcher, environment settings, promote dialog, approvals inbox. CLI:
runr environments,runr promote,runr approvals,runr deploy --env,runr bindings attach|detach|attachments --env,runr domains … --env,runr gateway routes add … --env,runr private-connections … --env,runr workflows … --env. MCP:environments_list,environment_create,environment_delete,promote,promotion_approve;deployandrollbacktake anenvironment. - Metering records the environment of every usage record (
environment_id, and its name: central migration2026_10_22_130001, tenant migration0087), and bills split a project's lines per environment ("vCPU hours (staging, eu-1, HTTP)", same totals;GET /billing/usagetoo); fixed-region allocations count every environment in the region; a project stop stops every environment and a resume deploys each one again. - A region move deploys the environments that run on the region again on the new cluster — also of projects in other regions — like its projects (built into the new registry first when needed; refused or failed ones are not moved), and its report lists them; a rollback deploys them again on the restored cluster and removes what the move ran for them from the cluster it left. A region detached from an organization takes its environments down with its projects. Kind e2e suite:
runtime/ci.sh test-environments. - CLI: gateway routes and private database connections.
runr gateway create REGIONgives you your gateway in a region,runr gateway routes add GATEWAY PATH -p PROJECT [--env],runr gateway routes ls|rmadd, list and remove routes;runr private-connections ls|add|status|rm(alsorunr pconn) manage a project's Skupper connections to a managed PostgreSQL database, the password from stdin or a hidden prompt (CLI).
- Configuration layering: the project's configuration with the environment's overrides (env vars merged by name, a
- Regions set up cert-manager for the certificates of custom domains. Custom domains in TLS mode
cert-managerno longer depend on a cert-manager and a ClusterIssuer the operator installed by hand (cert-manager).- The Serving install installs
jetstack/cert-managerv1.21.2 (namespacecert-manager, images from the reqs mirrorghcr.io/brf-tech/runr/reqs/cert-manager-*) where the cluster runs none, and reuses a cert-manager the cluster already runs — never a second one, never removed by RunR.region:reconciledoes the same for existing regions and brings RunR's release to this release's chart. Helmknative.certManager.install: falsestops RunR from installing it. - RunR creates the default ClusterIssuer unless the region setting
cert_manager_cluster_issuer(orRUNR_CERT_MANAGER_CLUSTER_ISSUER) names one:runr-acme(Let's Encrypt, HTTP-01 through the region's Traefik) where the region is reachable from the internet and an ACME e-mail is set — the region settingcert_manager_acme_emailor Helmknative.certManager.acmeEmail, never an address RunR picks —, elserunr-ca, a region CA whose certificate the console, the CLI and the API download.cert_manager_issuer_mode(auto,acme,ca) chooses. - New domains start in TLS mode
cert-managerwhere the region issues certificates. Where it cannot yet, the console, the CLI and the API say why (installing, failed, not installed, no issuer) and what to do, instead of the raw "no ClusterIssuer" error. - Console: General → cert-manager (status, issuer, the reason of RunR's choice, Download CA, the issuer mode and ACME e-mail, Install / Remove); the Domains tab shows what new domains get and offers the region CA. API:
GET/POST/DELETE /api/knative-regions/{id}/cert-manager,…/cert-manager/ca-certificate,GET /api/projects/{id}/domains/ca-certificate,meta.cert_managerof the domain list. CLI:runr regions cert-manager [install|ca|rm],runr domains ca, the newrunr regions setkeys. MCP:cert_manager_status,cert_manager_install. - Removing cert-manager from a region is refused while custom domains use it; RunR's release and issuers go only when nothing else on the cluster uses cert-manager.
- The Serving install installs
- Global projects: one project in several regions behind one host name (docs/global-projects.md). A new project type runs in 2 to 8 regions as
<slug>.g.<base domain>.- Modes: HA (the primary region while its copy is healthy, the others when it is not) or proximity (the closest healthy region, by GeoIP).
- GeoIP for clusters without internet access: each region's server has a GeoIP source — the platform URL, an operator-provided URL (an internal mirror, a file in the region's object store), a file the operator copies onto the server's volume, or none — and the console shows whether the server loaded it, from where, or why not.
- Global DNS: an operator enables it per platform region. RunR then runs a small PowerDNS Authoritative server in that region (
reqs/pdns-auth:5.0.8, a LoadBalancer on UDP and TCP 53) and writes the same zone to every server through its HTTP API every minute, repairing drift. The zone is delegated once with the NS and glue records that Settings → Platform → Global DNS lists. A region without a public port 53 cannot take part. - Health checks: the project records are PowerDNS LUA records whose
ifurlup()checks of every region's copy end at the region agent's new global-health listener. It answers from the Knative Service's readiness, so scaled-to-zero copies stay at zero. - Deploys go to the primary region first. When the deployment is ready, its version rolls out to the other regions: built again into each region's own registry when it has a build image (WebAssembly modules too, from the same runtime base by digest; one build per version and region, shared with the project's environments there), an uploaded CaaS image copied into each region's registry, then applied there and polled until ready.
- Triggers: any trigger list. HTTP and workflow triggers answer in every region; schedules, queues, topics, webhooks, platform events and broker sources run in the primary only (the other regions hold the CronJob suspended and the event gateway at zero) and move with the primary.
- Failover: a switchover (
PUT /globalwith a new primary) or a failover (POST /global/failover, or automatic after 180 s of a failed primary — per projectautomaticormanual) moves the primary, its triggers, its data and the DNS answer; the old primary is fenced and rejoins as a secondary once its cluster answers. - Gateways, GPUs, custom domains: gateway routes from a gateway of any of the project's regions; GPUs admitted per region (quota counted in every region); custom domains served and certified by every region (DNS-01 through the global zone after one CNAME of the domain's
_acme-challenge). - TLS: cert-manager issues the wildcard
*.<zone>in every region with DNS-01 through RFC 2136 updates signed with RunR's TSIG key. This is RunR's first use of DNS-01. - Data across regions: a dedicated Postgres binding becomes a CloudNativePG distributed topology — a replica cluster in every region, streaming from the primary;
DATABASE_URL_ROreads the region's replica,DATABASE_URLwrites to the primary; a switchover promotes with the demotion token (nothing lost), a failover at once. KV, managed object storage and shared Postgres stay in their region and are reached from the others through the binding's data link (a forwarder in each region, mutual TLS by SNI to the binding region's Traefik). External bindings work as before. A region whose data is not reachable yet stays out of the DNS answers. Private database connections, model services and dynamic credentials are refused for global projects, with the reason in the docs. - Billing and capacity: usage is recorded and billed per region, and every copy counts in its region's capacity.
- API, CLI (
runr deploy --global-regions,global:in runr.yaml,runr projects globalwithset --failoverandfailover,runr regions global-dnswithgeoip,runr global-dns), MCP tools (project_global_get,project_global_update,global_dns_status), console (project type, Regions tab with failover and the data across regions, Global DNS with the GeoIP source in the region and platform settings, the custom domains' regions), Helm valuesglobalDns.*(failover.afterSeconds,dataLink.*), aregion:reconcilestep, and the Kind suiteci.sh test-global-projectson two clusters.
- API product: a gateway's API can be offered to others (docs/api-product.md).
- Every gateway route is
public,keyorjwt. Key and JWT routes are checked by the region's new gateway-auth service (ghcr.io/brf-tech/runr/gateway-auth, Traefik forwardAuth), which RunR installs with the first such route. The gateway had no authentication before. - Consumers of a gateway or of the organization, with an external customer number for the invoicing system, a JWT subject and a status (suspended consumers get 403).
- API keys are shown once and stored as a SHA-256 hash with a visible prefix. A key can be limited to some routes and methods, can expire, and can be rotated with an overlap or revoked; a revocation reaches the region within about 10 seconds. The function sees
X-Runr-Consumer, never the key. - JWT routes verify tokens with the issuer's JWKS (RS, PS and ES algorithms), issuer and audience, and map a claim to a consumer.
- Plans: requests per second with burst, per minute (sliding window), per day and per month, counted in the region's Redis. Over a limit the caller gets 429 with
RateLimit-*andRetry-Afterheaders. - Fail closed: while gateway-auth is not installed or not ready, key and JWT routes answer 503
auth_unavailableand never reach the function unchecked. The gateway showspendingorerrorwith the reason, and a job opens the routes once gateway-auth is ready. - Usage per hour, consumer, key and route in
GET /api/api-usage: the calls let through and refused, and what the function answered to the calls let through — status classes, errors (5xx and calls without an answer), latency p50 / p95 / p99 and average, request and answer bytes (a WebSocket's traffic included, counted when it closes). gateway-auth's metering proxy (runr-gateway-auth-proxy) measures them after the check, which stays in front so a route still fails closed; the check hands it a signed ticket, without which it serves nothing. Percentiles come from a histogram per row, so any grouping adds up. With the Billing module the usage is also sent as signedapi.usage.hourly,api.usage.monthlyandapi.usage.quota_exceededevents with the same fields. The console's Usage tab,runr gateway usageand the MCP toolapi_usageshow them. RunR measures; billing stays with an external system. - Only through the gateway: a route can make its project's Knative Service cluster-local, so its own address stops serving.
- A route is created with its authentication:
POST /api/gateways/{gateway}/routestakesauth_mode,jwtandgateway_only, validated as on a change, and the route reaches the region with its forwardAuth Middleware applied before the IngressRoute that references it — a key or JWT route is never public, not even between two requests. The console's route form,runr gateway routes add --auth …, the MCP toolgateway_route_add, Terraform'srunr_gateway_routeand the organization import create it in one call (the CLI and the provider set it right after the creation against an earlier API). The Kind suite checks it (api-key-on-create). - Route answers are the route as the gateway shows it:
POSTandPATCHof a route andGET /api/gateways/{gateway}/routesanswer itsurl,auth_mode,jwt,gateway_only, project and environment likeGET /api/gateways/{gateway}, sorunr gateway routes addprints the URL. A path another route of the gateway has is a 422 on creation and on a change of the path — also when two requests race for it (the unique index's refusal, on SQLite and PostgreSQL) — not a 500; so is a second route to the project's own workload or to one environment (a partial unique index for the project's own route: central migration2026_10_23_100001, tenant migration0109). - The gateway's custom domain is set through the API and served only after its TXT record and CNAME / address are verified.
- Developer page: an optional public page per gateway with its routes and how to call them, and, when the owner switches it on (
developer_page_metrics,runr gateway page --metrics), each key / JWT route's success rate and latency of the last 7 days — never counts or consumers. region:reconcilerebuilds the routes of every gateway an earlier release built (stepgateway-routes, by the routing revision on its IngressRoute) and applies gateway-auth again when its Deployment changed, not only its image.- Console (Settings → API gateways),
runr gateway …, eleven MCP tools, the permissionapi_products.manage, audit events, Helm valuesapiProduct.*, and the Kind suiteruntime/ci.sh test-api-product.
- Every gateway route is
- Organization export and import (docs/org-export.md). An owner packs the organization into one versioned file and imports it into another organization, another RunR installation (self-hosted to hosted, one site to another) or the same one after a disaster.
- The package is a tar compressed with zstd. Its first entry,
manifest.json, records the schema version, the RunR version and a SHA-256 for every entry. The importer reads its own schema version and older ones, and refuses a newer one with a clear message. - Always in the package: projects with every version and its code, service configs (every column), env vars, custom domains, gateways and routes (with their authentication: a key or JWT route is imported as one, never public), workflow definitions, binding attachments, observability settings, uptime checks, project access, runtime pins and a deployment history summary; custom roles, groups, the default role, alert channels and rules, the status page.
- Optional: members (e-mails and roles for re-invitation), the prebuilt images of the deployed and latest versions, and binding data: Postgres
pg_dump, KV keys, object storage objects. The export shows a size estimate first. - Run history (optional,
include_history): every deployment, the workflow runs RunR keeps with their steps, the stored trigger events, the usage records and — with the Billing module — the organization's bills. The import lands them as history that never acts: deployments in a final state marked imported and never metered, workflow runs in a final state the engine leaves alone, trigger events that are only delivered when replayed, usage and bills read-only (GET /api/org-transfers/{id}/history), never charged or billed again. What the target's retention would prune is left out and shown in the plan (Run history). - Secrets are excluded by default. When included they are sealed with a passphrase (Argon2id + XChaCha20-Poly1305) and never written in clear.
- Destinations: a streamed download (resumable with
Range), an object binding of the organization, or an external S3 bucket whose credentials are used for that one export only. - Import:
- A resumable chunked upload — or RunR reads the package straight from an object binding of the organization (a transfer pod in its region) or an S3 bucket (ranged reads with
If-Match), resumable after the bytes it has and checked by SHA-256; S3 credentials stay in the encrypted job only. Then a dry-run plan: region mapping, slug and binding-name collisions with suggested renames, capacity / quota / residency checks, missing runtimes, and the manual steps (DNS of custom domains, secrets, alert channels, git links, members). - KV data needs a Redis of the same or a newer version: each KV dump records its Redis version, and the plan refuses a target region whose KV Redis is older (or does not tell its version), naming both; one binding's data can be left out (
skip_binding_data). - Apply creates everything once per entry, so applying again after a failure resumes it. It can also restore binding data, push images into the target registry, invite members and deploy.
- A resumable chunked upload — or RunR reads the package straight from an object binding of the organization (a transfer pod in its region) or an S3 bucket (ranged reads with
- Permissions and limits: owners only (new
org.transferpermission for tokens and custom roles). Both sides are audited. One export and one import run per organization at a time. - Clients and tools:
/api/org-transfers, the MCP toolsorg_export_start,org_export_status,org_import_start(from an object binding),org_import_planandorg_import_apply,runr org export|import|transfers|transfer(import --from binding:NAME|s3://bucket/key,--resume,--skip-data,--no-history,export --history,transfer history), and Settings → Export / Import in the console (upload or a bucket as the source, live progress, plan and report). - Migrations: central
2026_10_22_520001(org_transfers.import_source) and2026_10_22_520002/ tenant0107(deployments.imported_at,imported_history). - Region side: a new
transfermode ofruntime/image-upload(image 1.1.0) serves dumps and images through the Kubernetes API server and receives restores, so the region needs no route to RunR. Itspush-when-readycommand pushes an imported image. - API image: now includes
zstd. - Kind e2e:
ci.sh test-org-transfer. The tenant-to-tenant case runs intest:e2e-tenancy.
- The package is a tar compressed with zstd. Its first entry,
- Queue consumers can be charged, per region and organization (Queue consumers). A queue trigger's consumer (50m CPU, 64 MiB, running as long as the project is deployed) is now metered as a usage record of its own. It is charged only where the region charges queue consumers (
charge_queue_consumers, set by the region's owner), the platform has not switched it off for the tenant on a platform region (PUT /api/billing/regions/{region}/organizations/{tenant}, which the platform's bill to the tenant follows too), and the organization has not switched it off for itself in that region (PUT /api/billing/regions/{region}/queue-consumers). Then it is a line of its own on a pay-as-you-go bill, and in a fixed region it counts against the project's allocation (until now the allocation always held it). Off by default: the upgrade raises no bill. The console sets both in Region settings → Billing and Settings → Billing → Regions & prices. Tenant migration0074.- Queue consumer choices go with their tenant and region (Billing module, Queue consumers). A deleted tenant left the platform's choices for it in
billing_tenant_region_overrides; they are now deleted with the tenant's other central rows. The end of a region's share (detach) removes the platform's choice for that tenant and the tenant's own choice for the region (billing_region_overrides), and a deleted region takes every choice made for it along, centrally and in the tenants' copies.
- Queue consumer choices go with their tenant and region (Billing module, Queue consumers). A deleted tenant left the platform's choices for it in
- Terraform provider: every setting a project or an organization declares (docs/terraform.md). Each new attribute is optional and keeps the server's value when it is left out, so a configuration and state of provider 0.5.10 plan no change (a test applies one, strips its state to the 0.5.10 schema and plans; another runs the 0.5.10 binary itself). What is left out of Terraform are actions and reports, listed with the CLI or API way in What the provider does not manage.
runr_project:triggerssets the whole trigger list — HTTP, schedules, the queue, workflows, webhook inboxes, topics, platform events and the event sources kafka, nats, rabbitmq, mqtt, cdc, email and s3 — with each trigger's JSON config and write-only secrets (urlof a webhook's inbox computed);network(access mode, IP allow list, internal callers by internal name),deploy_strategy(blue-green and canary with their options) andglobal(regions, mode, failover; a regional project that gets it is replaced).trigger.typetakesworkflow.- Environments:
runr_environment(branch, approval, region, overrides, timed warmer, deploy strategy;name = "production"adopts the project's own approval),runr_environment_secret, andenvironmentonrunr_deployment,runr_binding_attachment,runr_domain,runr_gateway_routeand the newrunr_private_connection.runr_binding_attachment.credentialsswitches a Postgres attachment to dynamic credentials. - API product:
runr_gateway(rate limit, developer page, custom domain),runr_gateway_route(public, API key or JWT authentication, gateway-only),runr_api_plan,runr_api_consumer,runr_api_key(the key in the create answer; destroy revokes it). - Secrets, billing, workflows:
runr_secret_storeand therunr_secret_store_trustdata source,runr_env_var.reference(a variable that refers to a value in a secret store),runr_cost_limit(resource and data source),runr_workflow(a workflow declared through the API, paused or active) andrunr_workflow_app(the engine's endpoint override),runr_topic. - Workflow definitions keep their source: one definition is declared or archived on its own with the new
PUT/DELETE /api/projects/{project}/workflows/{name}(workflows.manage, auditworkflow.declare/workflow.archive);runr_workflowandrunr_workflow_triggeruse them, so the project's other definitions are never sent and what the function's SDK reported stayssdk(against an earlier API they declare the set as before). Declaring the set (PUT …/workflows,runr workflows declare) keeps the source of a definition whose content does not change; only new and changed definitions becomeapiuntil the next sync. - Reports as data sources:
runr_runtime_patchesandrunr_rightsizing;data.runr_projectreads the trigger list, the network, the deploy strategy and the placement. - Client and fake API: the CLI client creates, changes and deletes secret stores, reads and sets cost limits, changes plans, keys, gateway rate limits and routes, reads and changes a topic and sends trigger secrets with a whole list; the fake reproduces the API for each of them (trigger configs normalized per type, the event sources' required settings and secrets, the stored list kept in step with
config.trigger_type, a region's default TLS mode for new domains).
Changed#
- Terraform:
runr_domain.tls_modeleft out is RunR's choice. A new domain withouttls_modestarts in the mode the API gives it —cert-managerwhere the region issues certificates — and keeps the mode it has; the provider no longer forcesnone. A domain that provider 0.5 created withnonekeeps it, and no change is planned. A configured mode other than the region's default is now set at create (provider 0.5 compared the modes after reading the API's answer and failed with "inconsistent result after apply"); provider 0.5 against this API therefore cannot create a domain in a region that issues certificates unlesstls_mode = "cert-manager": upgrade the provider first. - The API document types a region.
KnativeRegionResource(GET /api/knative-regions, the region in the answers of the region endpoints) is an object with its fields and their types instead ofstring: the fields every region has, and — for a region of the request's own context, not a tenant's synced copy of a platform region — the rest. The answers carry the same fields as before;knative_settingsis{}instead of[]when a region has no settings. runr workflows app signing-key --envis now--dotenv(prints the key as an env file line):--envnames the environment in everyrunr workflowscommand. Scripts that wrote--env >> .envwrite--dotenv >> .env.- Deployments, versions and projects record the runtime image digest they run (
runtime_image,runtime_digest,runtime_patch_id; migrations of the central and every tenant schema). Without a known patch — the feed off, a runtime version it does not list, region builds, uploaded images, custom runtimes, CaaS — workloads run the version image by its tag as before.runr projects getshows the runtime patch state of a project (Runtime patch), andruntimes_listthe newest patch of each runtime version.
- Sign in with the user name or the e-mail address. The console's sign-in field is E-mail or user name (E-posta veya kullanıcı adı) and takes either, so the administrator an install created signs in as
admin(admin.username,ADMIN_USERNAME) without knowing the e-mail address typed at install time.POST /api/auth/login(login) andrunr login --with-password --usernametake either as well, in any letter case (security.md).- An identifier with
@is an e-mail address first, then a user name; without@it is a user name. One identifier names one account at most: of accounts that differ in letter case only, the exactly spelled one signs in. - The sign-in rate limit counts per client address and account, so switching between the user name and the e-mail address no longer starts a new budget.
- Registration, the user management and the profile refuse a user name or an e-mail address that differs from another account's in letter case only. User names keep their characters (letters, digits,
-,_; never@), and the first-administrator bootstrap creates no account whoseADMIN_USERNAMEcontains@. Existing accounts sign in as before. php artisan runr:horizon-link --userandtenancy:setup --adminfind the account a user name or e-mail address names the same way: one account, in any letter case.- A migration adds indexes on
lower(username)andlower(email)of the users table.
- An identifier with
- The console's navigation is grouped into a main menu with mega menus. The header no longer hides the administration pages in the avatar menu and a settings page with ~25 tabs.
- Projects is the first main-menu item, on its own. Build & Run (Sandboxes, Sandbox settings, Models, Runtimes, Images, Events), Data & Integrations (Data, Git providers, AI agents (MCP)), Observability (Alerts, Tracing, Status page) and Administration (Members: Users, Roles, Groups · Access & security: API Tokens, Auth Providers, Audit log · Organization: Regions & domain, Organization Branding, Billing · Platform: Regions & settings, Tenants, Resource Usage, Branding, Signup) open mega menus of labelled sections, each link with an icon and a one-line description.
- The header keeps the tenant switcher, the notifications and the user menu: Account settings (formerly User Settings), My bills, Support Tickets, the theme, the language, the tutorial and sign-out. The Models and Sandboxes header links and the theme, language and tutorial buttons moved into the menus.
- The menus follow the user's permissions: a link the user may not open is left out, and a section or group left empty disappears.
- Keyboard: ←/→ move along the main menu, ↓/↑ open a mega menu and move in it, Esc closes it; the menus carry
aria-expanded/aria-controlsand mark the current page. - Below 1024 px wide the menus collapse into a drawer opened with the menu button, the groups as an accordion.
- The settings page shows its menu group as the title and the pages of the same section as tabs instead of every tab at once. Its URLs did not change (
/settings?tab=<key>). - Console modules place their pages in the menus with
group,sectionanddescKeyonsettingsTabsor the newnavLinksregistry list (Architecture); the docs name the new locations. - The main menu fits a 1024 px window in every language. In Turkish, with the tenant switcher in the header, the group labels of the new main menu ran into the header's right side. When the labels do not fit the free width, the groups now show their icons (the label stays the button's name for screen readers and its tooltip); with room, the labels come back.
- Console hints name the new menu locations. Hints that still sent users to "Settings → …" (adding a Git provider, a secret store or a topic, uploading an image, installing tracing, enabling a runtime, revoking an MCP token, the queue consumer setting of a shared region, a region without sandbox runtimes) now name the main menu's path, in English and Turkish.
- The queue consumer is the project's event gateway (
runtime/queue-consumer, same imageghcr.io/brf-tech/runr/queue-consumer, same Deployment<slug>-qc). One delivery core — CloudEvents over HTTP, retries with backoff, dead letters, per-trigger concurrency, Prometheus metrics on/metrics— with pluggable sources: Redis Streams (queues, topics), the API's inbox (webhooks, platform events) and schedules. A project whose only gateway trigger is one queue keeps exactly the manifests of earlier versions; other gateways readgateway.jsonfrom the Secret<slug>-qc-config.- A schedule next to other triggers is delivered by the event gateway as a CloudEvent (
dev.runr.schedule.tick) to the project's service instead of running a CronJob pod. A project whose only trigger is a schedule keeps its CronJob. - The Knative Service of a project without an HTTP (or workflow) trigger is cluster-local, as for queue projects before. Custom domains, previews and request replay need an enabled HTTP trigger.
- The event gateway's legacy env mode keeps the queue consumer's guarantees (triggers.md). The dead-letter write and the acknowledgement of a queue entry are one
MULTI/EXECagain (also for queue and topic triggers ingateway.json), so a gateway that stops in between no longer copies an entry to the dead letter twice; withoutSINK_URLit delivers tohttp://localhost:8080as the queue consumer did instead of not starting.CLAIM_MIN_IDLE(30s) sets when another consumer's pending entry is claimed.
- A schedule next to other triggers is delivered by the event gateway as a CloudEvent (
- Preview environments are billed. The region agent now tracks the pods of preview environments (
runr/preview-of), and metering counts them like the project's own scale-to-zero sessions, at the project's size. They appear on the project's bill as a line of their own (component: preview); in a fixed region the allocation covers them, as it already held them. Until now previews were neither metered nor billed. Tenant migration0073lets a session belong to a preview instead of a deployment. An agent takes this up when its pod restarts with the new image.- Previews of projects with long slugs are metered. A preview's Knative Service
<slug>-pr-<number>keeps to 63 characters, so the slug of a project from an earlier release (longer than today's 44) lost its end — in a tenant, the tenant's slug with it — and RunR, which matched the region agent's session events by slug, found no project for them: the preview was not metered or billed. The agent now sends the pod's project id and organization (labelsrunr/preview-of,runr/project-idandrunr/tenant, which preview pods now carry) with every session event, and RunR finds the project by its id; previews deployed before carry norunr/tenantand are found by their project id among the tenants (remembered for an hour).region:reconcile's internal-network step gives such a preview its project's internal name instead of one parsed from the shortened Service name. A new preview of a tenant's long slug keeps the slug's-<organization>end in its name (the tenant's own domain alias answers only names ending in it), and a preview keeps the name it was created with.
- Previews of projects with long slugs are metered. A preview's Knative Service
- Kata hypervisors the region's nodes cannot run are not offered. RunR's node check (the
kata-kvm-probeDaemonSet) now also records each node's/dev/shmsize and whether its systemd has the D-Bus system bus (shm_mb=anddbus=in therunr.dev/kvm-probeannotation). QEMU backs its guest's memory with a file in/dev/shmand needs at least 1 GiB there: a node that runs in a container (Kind, k3d) has Docker's 64 MiB, where QEMU dies of SIGBUS while the guest boots. Dragonball (Kata's Rust runtime) needs the D-Bus system bus on nodes that run systemd; Kind's node image has none. Where every Kata node with KVM lacks it, the hypervisor issmall_shmorno_dbus: the console's hypervisor selector says why, and the install API refuses it and lists it underunavailablein its 422, so the other hypervisors can still be installed. Nodes checked by an earlier release, without the size, keep QEMU offered (/dev/shm (QEMU), D-Bus (Dragonball)).
Security#
- The API image is built on PHP 8.3.36, which fixes two remote code execution flaws in PHP (CVE-2026-109618, CVE-2026-109619), and its build fails on an older PHP. The API also turns off PHP's session upload progress tracking, turns on
session.use_strict_modeand no longer sendsX-Powered-By. The opcache JIT is off (opcache stays on): under the image's thread-safe PHP 8.3.36 the tracing JIT crashed API workers (upgrade.md). - The PHP runtime images run PHP 8.2.35, 8.3.36, 8.4.27 and 8.5.12 under their tags (
exec-php8.2 to 8.5). A PHP function gets the fixed release with its next deploy. - Cluster-local services were reachable from outside the cluster. The region Traefik served the cluster-local hosts of every Knative Service (
<service>.<namespace>,.svc,.svc.cluster.local) on the entry point its LoadBalancer publishes, so a request to the region's external address with such aHostheader reached queue-trigger sinks and model services. They are now served only on a private entry point (internal, published by the ClusterIP Servicetraefik-internal); through the external address they answer404, and Knative's placeholder Services point in-cluster callers attraefik-internal.- Custom domains, gateway routes and the second address of a base-domain migration hand requests on to the project's public host instead of its cluster-local one, so they reach public HTTP projects only. The status check keeps their host on the service's URL when its domain changes. Gateway routes are served again: they pointed at the Knative Service's
ExternalNameplaceholder, which Traefik does not route to. - NetworkPolicies in each region admit traffic to revision pods only from Knative Serving's Traefik, activator and autoscaler, and to the activator only from Traefik and the autoscaler. Before, any pod of the cluster could call another project's pods directly (
<revision>-private, the pod address) or through the activator, past the route and its visibility. They need a network plugin that enforces NetworkPolicies (regions.md). - The web shell's wake-up and keep-alive and the model playground call the service's cluster-local host through
traefik-internalinstead of the activator or the revision's pods. region:reconcile(after every upgrade) moves existing regions: stepsprivate-ingressandnetwork-policy. Traefik is upgraded twice — cluster-local hosts on both entry points while the KIngresses are made to nametraefik-internal, then on the private one only — and its pods roll twice. On Kind the move took 154 s without a failed request or queue delivery (runtime/ci.sh test-network).
- Custom domains, gateway routes and the second address of a base-domain migration hand requests on to the project's public host instead of its cluster-local one, so they reach public HTTP projects only. The status check keeps their host on the service's URL when its domain changes. Gateway routes are served again: they pointed at the Knative Service's
- An unknown account answers a password sign-in after the same work as a wrong password, so the answer time does not tell which user names and e-mail addresses exist. Both get the same 401 message.
- Pass-through: RunR's database no longer stores the values of project secret variables. A value goes straight into the Kubernetes Secret
<slug>-secret-envof the project's region, written key by key, and lives there only; RunR keeps the name, a version, who changed it when and a keyed fingerprint (HMAC-SHA256 withRUNR_SECRET_FINGERPRINT_KEYor a key derived fromAPP_KEY, never a plain hash). Reading a value back reads the region and says whether it is still the value RunR wrote (matches_fingerprint); a region that does not answer fails the request (503) and changes nothing. A deploy fails with the names of values that are not in the region; stopping a project keeps them, deleting it removes them; a region migration copies them to the new cluster and a rollback back, in memory; moving a project to another region copies them; previews read the project's Secret instead of a copy of it. RunR's own keys (RUNR_WORKFLOW_SIGNING_KEY,RUNR_MCP_GATEWAY_KEY) stay encrypted in its database — RunR signs with them — and deploys write them next to the values (security.md). The Kind suiteruntime/ci.sh test-region-movechecks the copies both ways: values in the region's Secret and in its OpenBao, values set while the region ran on the new cluster, references, and what the rollback leaves on the new cluster.php artisan secrets:move-to-regionmoves the values earlier releases stored into their regions: written, read back and compared by fingerprint, then cleared from the database; per tenant schema, idempotent and resumable, audited assecrets.moved_to_region. The Helm chart's bootstrap Job runs it after every upgrade.- Downgrade guard for pass-through (Rolling back to 0.5.10). An installation upgraded from 0.5.10 keeps an encrypted copy of each value in
project_secrets.value(rollback_copy, shown by the API) until the upgrade is finalized, so a rollback to 0.5.10 deploys the values instead of empty strings; RunR itself keeps reading the region. A copy that 0.5.10 changed after a rollback is recognised by its fingerprint and moved again as a new version (also into OpenBao).secrets:move-to-region --finalize(Helm valueprojectSecrets.finalize) checks every value in its region, restores one the region lost from its copy, drops the copies and recordssecrets.pass_through_finalized; until then references to secret stores are refused, and an organization import lists them as a manual step. The Helm notes of an upgrade say how to finalize. Migration2026_10_22_500001(tenant schemas:0105).
Fixed#
- Long jobs run on their own queue. An organization export or import (4 h by default) and the copy of an uploaded image into another region of a global project (1 h) ran on the default queue, whose
retry_after(1900 s) handed a job still running to a second worker that failed it. They now run on theredis-longconnection (queuelong, Horizon'ssupervisor-long), whose limits followRUNR_ORG_TRANSFER_JOB_TIMEOUTandRUNR_GLOBAL_IMAGE_COPY_TIMEOUT; the copy's timeout isRUNR_GLOBAL_IMAGE_COPY_TIMEOUTitself. - A region move routes the custom domains on the new cluster. The move deployed the projects there but applied only the domains that were waiting to be routed: an active custom domain had no route on the new cluster and stopped answering once DNS pointed there. Every active custom domain of the region's projects is now applied on the new cluster (an uploaded certificate is copied from the old one), and on the restored cluster after a rollback (Regions).
- Gateways of upgraded regions get the current routes without a deploy. Their IngressRoutes were rebuilt only on the gateway's next sync (a deploy or a route change);
region:reconcilenow rebuilds every gateway whose routes an earlier release built — the Host rewrite per route, the forwardAuth Middlewares and the direct routes of allowlist and gateway-only projects. - Custom domains, domain aliases and gateways follow a domain migration at once. When a base-domain or organization-domain migration completed or rolled back, custom domains and gateway routes followed the new hosts only on the next status check (up to a minute of 404). They now point at the new hosts before Knative switches over (the migration's alias answers there until then), an organization's alias that sent to the old domain sends to the new one, the region's gateways are rebuilt on the new domain, and the status check no longer moves a rewrite back to a domain the region stopped serving while Knative catches up.
- A gateway route with several methods is served. Its rule named them in one
Method()matcher, which the region Traefik's v3 rule syntax refuses; each method has its own matcher now. - A scale-to-zero project keeps its maximum scale (#627). The console's Zero Scale mode always sent
max_scale: 1, so such a project could never run more than one container under load. It now has its own maximum (1-1000). The API checks thatmin_scaleis not abovemax_scaleon create, update and deploy, instead of Knative refusing the revision. - Gateways use their custom domain and the region's function domain (#628). The gateway's IngressRoute was built from the base domain, while the project URL came from the region's function domain: on a region with its own function domain the two disagreed. Both now use the function domain. A gateway's
custom_domainwas stored but never routed. It now gets its own IngressRoute (gw-<id>-domain) with the gateway's routes and TLS as a project custom domain: a cert-manager certificate from the region's ClusterIssuer, else the region's default certificate. - A rollback restores the configuration with the code (#626). Deployments record the effective configuration they were applied with (
config_snapshot); a rollback (POST /projects/{project}/deployments/{deployment}/rollback, the console's Roll back,runr rollback, MCProllback) deploys the older version with the scale, env vars, resources, trigger and timeout it ran with, unless keep the current configuration is chosen. A CaaS project's image is its version's code: a deploy of an older version now runs that version's image instead of the one configured now. Secret variables and bindings stay the current ones (Rollback). - Scale-to-zero projects were metered with no CPU or memory. Metering counted a session's CPU and memory from figures the region agent never sends, so pay-as-you-go bills of scale-to-zero projects showed 0 vCPU-hours and 0 GB-hours for them (GPU time was counted). A session now counts the size its deployment was deployed with for the time its pod ran — CPU, memory and GPUs alike, the way always-on projects count their reserved size (Metering and the fixed bill). This needs nothing from the agent, so the sessions of agents that are not updated yet are metered the same way.
- Runs of a schedule trigger are metered. The agent did not report the pods of a project's CronJob, so scheduled runs were not metered at all. It now reports each run as a session, from its pod's start until the run finishes (not when the finished pod is removed later). An agent takes this up when its pod restarts and pulls the new image (
kubectl -n observability rollout restart deployment/runr-agenton the region's cluster). - Sessions of a queue trigger are metered under
queue, with the queue multiplier. The agent reports every Knative pod as HTTP; RunR now files a session under the trigger its deployment was deployed with. - Long-running pods are metered for the whole time they run. A session counted on the day it started, and every session open for 30 minutes was closed as stale. Its real end corrected it later, but the metering job only recalculates today and yesterday, so a busy service's pod that ran past the next day lost the rest of its time, and time past a month's end could belong to a bill that was already closed. Each day now counts the part of a session that ran on it, and a session still running counts until now.
- Sessions are closed by the agent's heartbeat. The agent reports the sessions it still sees running every minute. A session is closed as stale only when these reports stop for 10 minutes (the agent or the region is gone), as of when its pod was last seen. A heartbeat reopens a session closed too early and starts the session of a pod whose start was lost while the agent was disconnected. Sessions of agents that send no heartbeat keep the 30-minute close. Tenant schemas get
deployment_sessions.last_seen_atwith tenant migration0072. - The metrics summary reports what the sessions used.
GET /api/projects/{id}/metrics/summaryansweredsessions.total_cpu_msandtotal_memory_mb_msfrom figures nothing records, always 0; they are now the deployment's size × the sessions' pod time, as metering counts them.runr metricsshows the CPU as vCPU time. - Days metered before the upgrade keep their values; today's usage (and yesterday's, in the first hour of a day) is recalculated by the metering job as usual.
php artisan usage:recalculate FROM [TO]recalculates earlier days of existing deployments: open bills read the new values, closed bills do not change. - Metrics name their units, and the sessions' size × time says what it is. Since the summary reports the sessions' CPU and memory as the deployment's size × their pod time, the console's metrics history shows the averages in cores, MiB and requests per second and the sessions' vCPU-hours and GiB-hours at the deployment's size; memory reads MiB / GiB (it said MB / GB for the same numbers), and the live bar's last-hour averages are cores and MiB, no longer "per hour".
runr metricsprints the sessions' size × time on a line of its own (vCPU time and MiB- or GiB-hours of memory) andrunr metrics historyheads its CPU column "CPU CORES". The MCP toolmetrics_summaryaddsunitsand a summary line in those units.
- Runs of a schedule trigger are metered. The agent did not report the pods of a project's CronJob, so scheduled runs were not metered at all. It now reports each run as a session, from its pod's start until the run finishes (not when the finished pod is removed later). An agent takes this up when its pod restarts and pulls the new image (
- A change to a project that is deleted meanwhile answers 404, not 500. A config change whose project another request deleted while it ran (an e2e cleanup did, while the console's save was on its way) failed with a server error, "Call to a member function update() on null": the config row had gone with the project. A project's config change (
PUT /projects/{project}, also a rightsizing apply), deploy strategy and network change now answer 404 as for an unknown project, and so does any other request whose project is gone by the time it fails — such a request is not reported as an error. - A refused preview says why in the commit status. A preview that was not allowed to start (the fixed allocation or the organization quota leaves no room, the preview limit, forks off, …) reported "Replaced by a newer deploy"; it now reports "Preview #12 not deployed:
". - New MCP server: the console showed "A project with this name already exists" before it opened the new server. The API broadcasts
project.createdas soon as the project exists, while the modal still saves the starter files and the gateway settings. The projects list behind the modal got the project, and the modal's name check counted it as taken. The check now leaves out the project the modal created. A setup step that fails now stays in the modal: the project is listed, and Continue the setup finishes that project instead of creating a second one. The modal cannot be closed while the setup runs, and it no longer navigates when the page was left meanwhile. - Late and out-of-order answers in the console. Pages that load again when a filter, a page, a tab or a route parameter changes now show the answer of the newest request. Polls stop when the page is left. An action's answer is not undone by a poll that was sent before it. Specifically:
- Project pages:
- Opening another project from a project page (a clone, a notification, a link) now loads that project. Before, the page kept showing the old project. Leaving with unsaved code asks first here too, and the clone's permissions are loaded.
- The editor shows the files of the version clicked last. Before, a slower answer for an older version could fill the editor of the latest one, and saving it reverted the code.
- Code typed while a save runs stays as unsaved changes.
- A deploy whose follow-up reload fails closes the deploy modal; a deployments list read before the deploy no longer hides the new deployment.
- The MCP tab and its call log, the history chart range and the starter preview show what was picked last.
- Polls that kept running after their page was left:
- the deployment log panel, the shell terminal (socket and sandbox keepalive);
- the workflow runs and run detail;
- the project security tab and the region supply-chain tab;
- the bindings and data services pages, the runtime builds and the tenant deletion list.
- Forms the background poll overwrote while being edited: git deploy settings, the GPU settings of a region, the scaling of a model service, the sandbox runtime mapping, other rows of the sandbox quotas and pools, branding texts after a logo upload, and the billing webhook URL.
- Double clicks that sent an action twice: SSO provider toggle, platform admin toggle, tracing install, supply-chain key generation and admission install, binding restore, demo data, billing redelivery and markup removal, 2FA confirm (Enter) and recovery code regeneration, the runtime enable toggle, notification mark-read, and the organization switcher.
- An error shown after the action succeeded, when only the follow-up reload failed: tenant create/delete, region quota assignment (a retry now edits the assignment), SSO group mappings, project access, tracing removal, git disconnect.
- Broadcasts during a reload:
- A deployment status, a new deployment or a finished build that is broadcast while the project page reloads its lists (for example right after a deploy) is no longer lost.
- The projects list keeps a project created, changed or deleted while it loads, also from another tab or by someone else.
- The models page's poll no longer undoes a deploy, stop or delete for a few seconds.
- Mirroring or deleting an offline copy no longer writes back the model as it was at the click.
- Deleting a project from its page keeps the page until the projects list opens (the page went blank in between). A project deleted elsewhere opens the projects list with a note.
- Organization switch: the onboarding wizard loads the new organization's onboarding once; it loaded twice and could show the first answer.
- Install status: a job that finished before its install request answered (sandbox runtimes, Wasm, Knative components, region demo data) no longer goes back to "installing". The region setup wizard no longer misses a fast failure.
- Shared state after an organization switch or sign-in:
- the effective permissions, the settings and the runtime catalog drop answers read before the change;
- the data residency of region pickers is read again for the new organization;
- notifications are no longer received twice after signing in.
- Search, filter and detail views (sandboxes, files, metrics, bills, events, audit log, users, tickets, alert deliveries, uploaded images, Hub models, groups, SSO buttons on the sign-in page, model upload abort) show the newest answer. Ticket replies no longer vanish or show twice.
- Project pages:
- Region setup wizard: a failed storage or registry test left the wizard waiting. The setup job recorded the failure but broadcast nothing, so the wizard (which follows the setup by its events) never showed the error. It now broadcasts
region.create_failedwith the setup's id, like every other failure. - The registry setup gives the StorageClass time to bind its test volume. The PVC test waited a fixed 60 s, and a
WaitForFirstConsumerclass binds only once the consumer pod is scheduled, its image pulled and the volume provisioned: on a busy cluster or slow storage the setup failed with "did not provision a volume within 60s". It now waitsRUNR_REGISTRY_PVC_TEST_SECONDS(180 s by default, 30 to 900), and a volume that does not bind is reported with the real wait and what the test saw: the claim's phase, the consumer pod's phase and why it waits, and the claim's last event (e.g.ProvisioningFailedwith its message). The registry setup jobs' timeout grows with the wait;POST /registry-setup/test-pvcandrunr regions registry test pvcwait as long (Regions). - Invitations and SSO name accounts by their e-mail address in any letter case (like the password sign-in). Tenant invitations, their acceptance, the member search, SSO sign-ins (OAuth, OIDC, SAML) with auto-registration, the organization import's member grants and
tenancy:create --owner-emailmatched addresses exactly, soAda@Example.comcould get a second account next toada@example.com. The address now names the existing account; an invitation is stored, and a new account created, with the address trimmed and in lower case. An address that several accounts of an earlier release share in different letter case names none of them: the invitation or SSO sign-in is refused (email_ambiguous) instead of creating a third. - A new account from an invitation gets a free user name. Accepting an invitation set the user name to the part of the address before
@without checking it, so a name another account had failed with a database error, and characters such as.or+made a user name the registration would refuse. It is now built from letters, digits,-and_of the local part, free in any letter case and of waiting signups, with-2,-3, … when it is taken; SSO auto-registration uses the same rule. - A sandbox the container runtime cannot start fails with the runtime's error. A pod whose sandbox the runtime cannot create (a Kata VM that does not boot, a runtime handler the node lacks) stays pending while the kubelet retries, and its status never says why: the create answered 202
pending, the sandbox failed as a plain start timeout up to 6 minutes later and counted towards the organization's quota until then. The start now reads the pod's events every 5 s (sandbox.start_events_seconds); after threeFailedCreatePodSandBoxevents, in about 30 s, the sandbox fails with the runtime's latest error — the create answers 502start_failed— and no longer counts towards the quota. A warm pool pod in that state is replaced with the same reason, and a background start that times out names the pod's last warning event (Sandbox API). - A Kata install on a region without gVisor no longer fails with 422. The
/dev/kvmcheck runs before anything else of RunR's sandbox runtimes is on the cluster, and its DaemonSet could not be applied while therunr-sandboxnamespace did not exist ("Cannot check the cluster for Kata"); the console's check failed the same way. The check creates the namespace now. A namespace RunR created was also remembered by name for the whole process, so a queue worker that had createdrunr-sandboxon one region's cluster skipped it on the next; it is remembered per cluster now.
0.5.10 - 2026-10-21#
Changed#
- The runtime images have their own repository, BRF-Tech/runr-runtimes. It holds the language and ML runtimes (
exec-python,-nodejs,-go,-php,-pytorch,-tensorflow), the WebAssembly build images (exec-wasm-*), the universal wrapper, hf-serve, the manifests of the built-in catalog and the reqs mirror (ghcr.io/brf-tech/runr/reqs/). Its GitHub Actions build, smoke-test and publish them under the same names and tags, so installs and the API's runtime discovery see no change.runtime/keeps RunR's own images (agent, queue consumer, image-upload, supply-chain tools, the Spin shim, nginx-test) and the end-to-end harness.- The API ships the catalog as before (
api/resources/runtimes/builtin.json). After a manifest changes in runr-runtimes,RUNR_RUNTIME_CATALOG=<checkout>/runtimes php artisan runtimes:bundleregenerates it. runtime/ci.sh build,wrapper,wasm,ml-runtimes,mirror-reqs,mirror-basesandeolmoved there, and the GitLab jobsbuild:runtimeandbuild:ml-runtimesare gone.
- The Kind e2e suites pull the published runtime images instead of building them.
test-runtimespulls the wrapper,test-wasmthe Rust and JavaScript build images,test-imageswasm-exec andtest-aihf-serve, at the tags pinned inruntime/lib/runtime-images.sh.runtime/test/runtime_images_test.shchecks that these pins are the product's defaults.RUNTIME_IMAGES_LOCAL=1uses a local build from runr-runtimes instead.
ci/check-reqs-mirror.shresolves every referenced reqs image anonymously on ghcr.io instead of looking it up in a mirror list. Resolved references are cached for a day. A missing image says where to add it in runr-runtimes, and a network error is reported apart from missing images.
Fixed#
- A region migration rollback left the projects running on the new cluster. The rollback deployed the projects on the old cluster again, but what the move had deployed for them on the new cluster kept running there: their functions answered on that cluster's own address, and nothing metered them. After a rollback RunR now removes them from the new cluster in the background — the Knative Services of the projects and of their previews, their CronJobs and queue consumers, and the ConfigMaps, Secrets and build Jobs these read or built — found by the labels RunR sets, for the projects of every organization on a platform region and for a project created while the region ran there. The cluster's installs and the data services with their volumes, credentials and data stay: they may hold what was written during the move. Nothing is removed when the new cluster is the restored one under another address (the same
kube-systemnamespace). The removal does not hold the rollback up and is tried three times; the operator who rolled back gets a notification with what was removed, or with the reason and what to remove by hand (What the rollback removes from the new cluster).
0.5.9 - 2026-10-20#
Upgrade notes#
- The chart's API and UI images follow the chart version. An upgrade from 0.5.8 with
--reuse-valueskeeps the image tags stored in the release —latest, the default of the charts up to 0.5.8, or the version an install set with--set api.image.tag=…— and with them the old images. Add--set api.image.tag= --set ui.image.tag=to that upgrade once, so both tags follow the chart from then on, or set them explicitly on every upgrade (Procedure).
Added#
- e2e: uptime checks from a region (
runtime/ci.sh test-observability,obs-uptime-region). An uptime check with location "regions" is fetched and run by the region's agent, which calls RunR at the API container's address on the Kind network; its results must arrive with the region's name as location and the agent's log must have noRunR answered 400— the 0.5.7 refusal of the agent's host that 0.5.8 fixed. - Queued deploys in the API. A deploy of a version that is built again for its region's registry answers
202withmessageandqueued_deployinstead of a deployment;GET /api/projects/{id}/queued-deploys/{queued_deploy}(projects.view) follows it todeployedwith itsdeployment_id, or tofailed/cancelledwith the reason. The MCPdeployandrollbacktools follow it until the deployment starts (Older versions). runr deployandrunr rollbackfollow a queued deploy — the build, then the deployment it starts — within--timeout(--no-wait:queued_deployin-o json,queued_deploy_id=andstatus=buildingin--output-env); the Go client'sDeploydoes so too, andStartDeployreturns either (cli.md).runr regions migration statuslists what the move did with each project — project, organization, outcome, version, deployment status and message — while the move can be rolled back (projectsin-o json; cli.md).
Changed#
- Helm:
api.image.tagandui.image.tagdefault to the chart'sappVersioninstead oflatest.helm install --version Xruns the images of RunR X andhelm rollbackrolls them back with the chart; a tag that is set,latestincluded, is used as it is. The multi-tenant edition runs<appVersion>-mtby default, and for the taglatest, which has no multi-tenant variant, as before. The install and upgrade instructions no longer pin the image tags next to--version.runtime/ci.sh deploysets the taglatestof the images it builds explicitly.
Fixed#
- A refused deploy of an older version during a region move left the project running on the old cluster. While the move's deploy of a project waits for its build on the new cluster, a deploy of an older version replaces it. When that deploy's build failed or a deploy check refused it, nothing shut down the project's deployment on the cluster the region left: it stayed live and metered, and
migration-statusshowed the project ascancelled. Such a deploy now takes the move's place (queued_deploys.replaces_reason, in the tenant schemas too): when it fails, that deployment is shut down with "Not moved to the new cluster of the region: …" and the reason, as for a project the move refused, and the report showsnot_movedwith the reason; when it starts, the report showsdeployedwith its version (Older versions). - Helm: the agent of RunR's own cluster got 400 after each token rotation. About every 48 minutes Kubernetes renews the API pod's service account volume with a new token: it points the volume at a new directory and deletes the old one. PHP's realpath cache kept resolving the volume's files into the deleted directory for up to two minutes, so the API lost its namespace, the in-cluster gateway (
runr-gateway.<namespace>.svc.cluster.local) dropped out of the hosts the region callbacks accept, and the agent's fetch of its uptime checks got 400 once or twice (uptime: fetching checks: RunR answered 400). RunR's own token failed to read in that window too, and a check whether a region is RunR's own cluster made then answered "no" for five minutes. A failed read of the service account volume is now retried with that cache cleared. The in-cluster gateway and the configured addresses never depend on the database or the cache, and the lookup of the regions'agent_ws_urlandapp_urlkeeps no empty or failed result. Upgrading the API is enough (regions). - A git deploy replaced before it went live stayed "pending" on the provider. Such a deploy ends as skipped, but no commit status followed. It now ends the commit status "runr" as "Replaced by a newer deploy" ("Stopped before it went live" when the project was stopped):
canceledon GitLab,warningon Forgejo and Gitea,successon GitHub, whose commit statuses have no canceled state (errorwould mark a commit as broken that nothing failed in, and a late report could paint a live commit of the same SHA red); the GitHub deployment it started becomesinactive(Status on the provider). - A region migration rollback left projects serving without metering. A project the move did not move (its redeploy refused, its deployments shut down) still had its Knative Service on the old cluster, which the rollback made the region's cluster again: it served with no live deployment. The rollback now deploys such a project again with the version and overrides it ran with, metered from when it is ready; when the deploy checks still refuse it, or the project was stopped while the region was on the new cluster, its workloads are removed from the restored cluster. A project whose redeploy the rollback itself refuses is removed from there too, its deployments shut down with "Not deployed again on the cluster the migration rollback restored" (Moving a region to another cluster).
- A region migration rollback kept the new cluster's registry address. The move points the region at the Zot it installs on the new cluster; the rollback gave back only the kubeconfig and the external address, so every redeploy and build of the region's projects on the restored cluster used the new cluster's registry. A move now keeps what the region held for its old cluster (
knative_regions.migration_old_state, in the tenant schemas too) and the rollback restores it before the projects are deployed again: the registry address, the dedicated workers and capacity, the runr-agent's token (the old cluster's agent was refused after a rollback), and the install state of Knative, Redis, the sandbox runtimes, WebAssembly, GPU support and function TLS. The copies of a platform region in its organizations get the restored values. A move started before the upgrade has no kept state: its rollback restores the kubeconfig and the external address only (Moving a region to another cluster). - Moving a platform region left the organizations' projects on it behind. In a multi-tenant install the move and its rollback deployed only the projects of the platform's own schema; the projects the organizations run on the region, in their own schemas, kept their old deployments and were deployed on neither cluster. Both now deploy them too, each organization's in its own context through its copy of the region, with the same stop and admission checks; the rollback also deploys again or removes an organization's project the move did not move. An organization's own region still moves inside that organization (Moving a region to another cluster).
- A region migration installed Kata Containers with Cloud Hypervisor only. A region on QEMU, Dragonball or Firecracker got
clhon the new cluster. The move now installs the region's hypervisors. A Kata install that fails on the new cluster leaves Kata inerrorthere instead ofreadywith the old cluster's hypervisors, and no longer keeps gVisor from being installed. - A region migration redeployed built versions the new cluster cannot pull. A version built for the region (dependencies, a compile step, WebAssembly) has its image in the old cluster's registry, which the new cluster's nodes are not set up for, and the move deployed it there all the same. When the move gives the region another registry, such a version is now built again on the new cluster and deployed there once the build is done, through the deploy checks of every deploy; until then the project keeps running on the old cluster, and a build or deploy that fails leaves it not moved, with the reason as its deployment's message. A version without a build of its own on a runtime image RunR built in the region waits for that image to be built again there; a build running when the region switches is submitted again on the new cluster. The new deploy-after-build step (
queued_deploys, in the tenant schemas too) deploys a version once its build, its runtime image and its supply-chain checks are ready, and is replaced by any newer deploy of the project.GET /api/knative-regions/{id}/migration-statusnow lists what the move did with each project (projects: moved, building, deployed, not moved — with the reason —, or cancelled), and the console's Migration tab shows it (Images the new cluster cannot pull). - Uploaded images of a moved region were offered and deployed as if they were there. An upload is in the registry of the cluster it was pushed to only, and RunR keeps no copy of it, so a move cannot bring it along. Each uploaded image now records that registry (
registry_address; the upgrade fills it in): one the region left showsin_registry: false, a CaaS project on it is not moved (and, after a rollback, one switched to an image uploaded on the new cluster is not deployed again on the old one), with a message that says to upload the image to the region again; choosing such an image for a CaaS project, a runtime version or a wasm-exec function finds it not uploaded in the region. Uploading the same name again puts it into the region's registry (Moving a region to another cluster). - Builds running on the new cluster at a migration rollback ended as failed builds. Their check looked for the Kaniko Job on the restored cluster and failed them with "Build Job not found", announced as a build failure. They now end with the new build status
cancelledand "Cancelled by the migration rollback: …", without a failure notification; a git push waiting for one ends as skipped, and the Job is deleted from the cluster the region left. A version the move built again gets the image back that the old cluster's registry has, and the runtime image builds come back as they were (Completing and rolling back the move). - A region migration set Firecracker's thin-pool up with the default 20 GiB. The size chosen at the Kata install was not stored, so the move reinstalled
fcon the new cluster with the default, and adding hypervisors to a working install sent the default along too. The region now records the size its nodes were set up with (sandbox_kata_devmapper_pool_gb, in the tenant schemas too) and every later install uses it: adding hypervisors,region:reconcile, the move (for a pool set up before this release, the size in the old cluster'skata-devmapperDaemonSet), and the rollback restores it with the region's other state. Uninstalling Kata clears it (Firecracker and devmapper). - A migration rollback deployed versions the old cluster cannot pull. A version built while the region was on the new cluster has its image in the new cluster's registry only; the rollback deployed it on the old cluster, where its pods could not start. The rollback now deploys the project's last deployment whose version is in the restored registry again (usually what ran before the move), with its overrides, and shuts the deployments of the other version down with "… was built into the registry of the cluster the rollback left; vN runs instead". A project with no such deployment is removed from the restored cluster.
- A failed region migration started again lost the old cluster. A move in
failedcould be started again without a rollback, and the new attempt saved the half-moved region — the new cluster's kubeconfig included — as the state to roll back to. A move started again now keeps what its first attempt saved. - A migration rollback kept the new cluster's status for data services, the admission policy and tracing. The move now also keeps the status of the region's data services, admission policy and tracing backend, and the rollback restores them; a data service or tracing backend installed on the new cluster only is not installed after the rollback.
- Two deploys created at the same instant could both stay active. When the older one went live before the newer one was created, nothing replaced it (metering already counted it only up to the newer one's start). A deployment now becomes active under its project's row lock, only while no newer deployment of the project is live, and its activation replaces every older live deployment of the project in the same transaction: the readiness check, a CronJob project and a demo region alike. At most one deployment of a project is active.
- Older versions of a moved region failed at the image pull. A region migration builds again only the version each project runs; a version built before the move still names the old cluster's registry, so deploying it later — a deploy or rollback in the console, the CLI or the MCP tools,
POST /api/projects/{id}/deploy, a git push or Deploy again whose tree equals it, the resume of a project stopped before the move — started pods that could not pull their image. Such a version is now built again into the region's registry first and deployed when the build is ready, through the deploy checks of every deploy, with "vN is built again for this region's registry: … It deploys when the build is ready." (the deploy answers with a queued deploy, a git deploy waits inbuilding, a resume answersredeploy_code: version_rebuilding). A deploy the checks refuse once the version is built notifies the user who asked for it; a migration rollback gives a version built again this way its old image back (Older versions). - A region migration left the preview environments on the old cluster. The previews of pull/merge requests kept running on the cluster the region left, and their next deploy went to the new cluster with images it cannot pull. The move now deploys every live preview again on the new cluster through the preview path (as Redeploy does), building a preview version again first when its image is in the old cluster's registry or its build ran there, and marks a preview whose build had failed as failed with the reason; a rollback does the same on the old cluster and deletes the preview builds still running on the new one. A preview on a runtime image RunR builds in the region now waits for that image instead of failing (Preview environments).
- A migration rollback did not cancel a queued deploy that was starting. A deploy waiting for its build that a worker had just claimed (status
starting) was left alone, and could create a deployment on the cluster the rollback left. The worker now claims the deployment it creates first: the rollback cancels every deploy queued since the move that has not created its deployment yet — the worker then starts nothing — and one that has is replaced by the rollback's redeploy like any other deployment (Completing and rolling back the move). - A build submitted again by a region migration could be failed by the old build's checks. The checks of the build that ran on the old cluster kept polling the version and failed it with "Build timeout" at the old build's deadline, while the build submitted on the new cluster still ran. The checks of a build (a version's and a preview's) now follow their own submission (
versions.build_token) and stop when the version is submitted again.
0.5.8 - 2026-10-19#
Upgrade notes#
- Read 0.5.7 → 0.5.8 before the upgrade. In short: a multi-tenant install upgraded with
--reuse-valuesadds--set ui.image.multiTenantSuffix=-mt; inactive and maintenance regions stop serving onceregion:reconcileapplies their status block; SSO providers need thehttps://callback URL, and a Google provider with a Hosted Domain admits that domain only; switching to the multi-tenant edition converts the installation into its first tenant (--timeout 20m); a Compose install takes the newCaddyfile. - Billing is rebuilt as the optional Billing module (off by default). Plans, subscriptions, credits, purchases, add-ons, capacity deals and charges of 0.5.7 are archived in
billing_legacy_archiveand their tables dropped; the cost configuration becomes the platform's prices and closed charges become closed tenant bills. SetBILLING_CURRENCYto the currency of your 0.5.7 cost configuration in the upgrade: the import runs also with billing off and warns when they differ (Billing). - Back up the database before the upgrade. 0.5.8 cannot be rolled back with migrations (they stop with a message instead of half-undoing the billing changes); going back to 0.5.7 is restoring that backup (Rolling back).
Added#
Billing module (
api/Modules/Billing,vue/src/modules/billing; off by default, Helmbilling.enabled,BILLING_ENABLED). RunR calculates one bill per project and calendar month, paid by the project's owner, and an account bill per user for bindings, sandboxes and model services; bills close daily at 02:30 after their month and are never recalculated. Every region has one billing type: pay as you go (metered vCPU-hours, GB-hours, GPU-hours, binding storage, sandboxes) or fixed (a project requests vCPU, memory and GPUs, sees the monthly price before submitting, the region's seller approves, the price is prorated by day from the approval and the project cannot exceed its allocation — its worst case, every pod up to a requiredmax_scalewith a deploy's overrides, its previews and GPUs, is checked at every deploy, resume, preview, region migration and size change, and a request or approval smaller than what runs is refused until the project scales down, one while it runs withoutmax_scaleuntil it is set and redeployed; administrators see the metered cost next to the fixed price). Prices per region and default prices are versions with an effective date and never missing: region → tenant default → platform default → built-in defaults seeded on the first start. With tenancy the platform bills each tenant for the shared platform regions; a shared region is an offer the tenant's administrators accept with a markup (default markup prefilled, changes with an effective date), and only then can its users use it. RunR only calculates: an external invoicing system reads bills, usage, the rollup per owner and fixed requests through/api/billing/…, setspayment_urland the paid status of each bill and receives signed events (bill.closed,bill.paid,bill.payment_url_set,fixed_request.approved/.rejected,region_share.accepted,project.stopped, …). New permissionsbilling.approve,projects.billing;billing.view/billing.manageand the Billing role now exist in both editions (docs/billing.md).How the Billing module counts. Amounts are exact — decimal arithmetic, each line rounded once, half up, to the currency's minor unit, the bill's total the sum of its lines — and the quote of fixed resources is the bill's own calculation. Days billing was off are not billed: the bills open before end with the last active day, a month inside the off period gets none, billing starts again on the day it comes back. A single-organization installation converted into its first tenant bills every project once, in the tenant, from the conversion day. Fixed allocations end with their project (deleted or moved) also while billing is off, and with their tenant on the day it is deleted. Billing webhooks never reach private or cluster-internal addresses — a tenant's never, the platform's only with
observability.alerts.allowPrivateTargets— checked on save and before every delivery. Accepting a region offer writes the markup and activates the share in one step, announced after it succeeded (docs/billing.md).Project owner. Every project has an owner (
owner_id, the creator by default), the payer of its bills — also of the running month's bill, which belongs to whoever owns the project when the month closes; the owner or a holder ofprojects.manage_accesstransfers it to a member of the organization who sees the project (a platform administrator's access to every organization does not count), who is notified (PUT /api/projects/{project}/owner, audited asproject.owner.transfer) (docs/billing.md).Project stop and resume. Administrators (inside a tenant, its administrators) and platform administrators stop a project with a required reason — its workloads and previews are shut down, deploys queued before the stop do nothing and every deploy path (console, git pushes, previews, region migrations) is refused until it is resumed, which redeploys the running version with its overrides through every deploy check — with the new permission
projects.stop(POST /api/projects/{project}/stop/resume, audited; the reason is shown to the project's members). A stop by a platform administrator is the platform's: the console says so, and only a platform administrator resumes it; a platform administrator takes an organization's stop over with Stop for the platform on the project page (docs/billing.md).The console hides the Ticket and Billing pages at run time when the API reports the module off (
features.tickets/features.billingofGET /api/branding), so the published UI images are built with both screens.Bills of deleted tenants are kept. A tenant's project and account bills live in its schema, which tenant deletion drops; the deletion job now copies them first into the central
billing_archived_bills— line items, amounts, payer, paid status, payment link, closing time and the tenant's id, slug and name (open bills with the estimate of that moment). When the copy fails the tenant staysdelete_failed, also with Force delete. Platform administrators read them withGET /api/billing/archived-bills(?tenant=<id or slug>) andGET /api/billing/archived-bills/{id}(docs/billing.md).Helm: the API volume can be ReadWriteMany.
api.storage.accessModes(default[ReadWriteOnce], the mode of every earlier install) sets the access modes ofrunr-api-storage. The API, Horizon, scheduler and Reverb pods are colocated on the volume's node only with ReadWriteOnce; with[ReadWriteMany](and a ReadWriteMany StorageClass) they run on any node. An upgrade that asks an existing claim for other access modes or another StorageClass stops with a message instead of the API server's "spec is immutable"; docs/install.md moves an existing volume.database.postgres.storageClassNamesets the StorageClass of the bundled PostgreSQL's volume.Helm: passwords of an external database, Redis and SMTP server from Secrets.
database.password,redis.passwordandmail.passwordare written to the chart's Secret;*.existingSecret/*.existingSecretPasswordKeyreference a Secret of yours. The pods read them as Secret env vars (DB_PASSWORD,REDIS_PASSWORD,MAIL_PASSWORD), never from the ConfigMap;mail.username,mail.encryption(starttls/tls) andredis.usernamecomplete the settings, and the bundled Redis requires a configured password. The generatedAPP_KEYandREVERB_APP_SECRETstay with the chart and are kept on every upgrade: an external database no longer needssecret.create=false.extraEnventries keep working (upgrade note).Helm: NetworkPolicies.
networkPolicy.enabled(default off) adds policies that allow exactly the connections of docs/install.md — ingress controller → gateway (networkPolicy.gatewayFrom), gateway → UI / API / Reverb, the API, Horizon, scheduler, Reverb and hook pods → database, Redis, SMTP, Reverb, DNS and outbound to the regions (networkPolicy.egress,networkPolicy.extraEgress) — and deny the rest.Horizon dashboard for administrators.
/horizonadmits platform administrators and, in the single-tenant edition, Owners, signed in with a single-use link (valid 5 minutes, stored as a hash) fromphp artisan runr:horizon-linkorPOST /api/horizon/link(audited ashorizon.link.create). The session it starts is checked against the account on every request (docs/install.md). Before, the dashboard answered 403 to everyone in production, and its gate would have let any signed-in user in.Helm:
ingress.tls.mode: secretreferences akubernetes.io/tlsSecret you manage (ingress.tls.secretName); the legacymode: ""keeps working.gateway.affinity,api.affinityandui.affinityreplaceplacement.affinityfor one component.Helm: more than one Reverb replica. With
reverb.replicasabove 1 the replicas share broadcasts over Redis (REVERB_SCALING_ENABLED); shell, sandbox exec and agent sessions stay on the replica their WebSocket reached.Helm:
cors.allowedOriginsfor browser pages on other origins that call the API.Helm:
kubeVersion: ">=1.23.0-0"inChart.yaml, the minimum of the APIs the templates use (autoscaling/v2).Linked accounts in Account settings. With the AuthProvider module, Account settings list the external accounts a user signs in with: Link
connects another OAuth or OpenID Connect account — also one with another e-mail address — and Unlink removes one. The provider's callback hands the console a one-time code that only the user who started the link can confirm ( POST /api/oauth/link/complete); an account linked to another user is refused.GET /api/oauth/{driver}/link, which answered a plain sign-in URL, now links. Audited associal_account.link(docs/modules.md).Platform administrators open an organization as its administrator. The tenant switcher or Settings → Tenants → Open as administrator opens any organization; a bar names it until Stop. Opening and leaving are audit-logged (
impersonation.start,impersonation.stop).POST /api/impersonate/{tenant}andPOST /api/impersonate/stopwork again, andGET /api/tenant/currentreportsimpersonating, andmemberfor each organization (docs/tenancy.md).Organization domain in the console. Members with
settings.manageset and clear their organization's own domain in Settings → Organization domain;GET /api/settingsreturnstenant_base_domainandplatform_base_domaininside an organization (docs/tenancy.md).Support ticket assignment and tags in the console. The staff view of a ticket assigns it to a staff member and edits its tags;
GET /api/admin/support-tickets/stafflists the assignee choices (docs/modules.md).runr whoaminames the organization the session works in (organizationin its JSON output); a platform administrator outside any organization seesplatform (no organization).The CLI shows stopped projects and refusal codes.
runr statusandrunr projects getsay who stopped a project (the platform or the organization), when and why (stoppedin their JSON output),runr projects lslists it asstopped, and an error the API answers with a code prints the code after the message, e.g.(project_stopped)or(fixed_allocation_exceeded)(docs/cli.md).Helm: the bootstrap Job converts an installation that ran single-tenant. In the multi-tenant edition it runs
php artisan tenancy:setup --forcebefore the administrator check, so switchingeditiontomulti-tenantmakes the installation its first tenant (Default, the slug of the git remote owner); fresh, converted and multi-tenant installations are left alone, and a conversion that stopped continues on the next run.bootstrap.tenancySetup=falseleaves the conversion to the operator (docs/tenancy.md).
Changed#
Usage metering is part of the core and runs on every installation, single-tenant included (
usage-aggregationevery 5 minutes, central and every tenant schema). It records quantities only; the cost columns of the usage records are no longer filled. Usage of tenant projects in platform regions is copied to the central database for the platform's bills to tenants. An always-on deployment counts its reservation for the time it ran that day with the size it was deployed with (deployments.sizing,deployments.ended_at), no longer today's configuration for 24 hours;min_scale≥ 1 counts as always-on, as documented.php artisan usage:recalculatecalculates earlier days again; a deployment applied before the upgrade keeps the scale mode and size its 0.5.7 record of the day shows, never today's configuration (docs/monitoring.md).Sharing a platform region with a tenant is an offer while billing is on: the region reaches the tenant's users once a tenant administrator accepted it. Existing assignments stay active, and with billing off offers are active at once (pending ones within five minutes) and stay active when billing returns (docs/tenancy.md).
White-label keeps branding, self-service signup and the onboarding wizard, without plans: the wizard assigns the first region with the default signup quota.
The source is public at https://github.com/BRF-Tech/runr. The single-host install starts from a checkout of the release tag (compose/README.md).
CORS follows the console's domain. The API allowed a fixed list of development origins in every environment; it now allows the origin of
APP_URL/FRONTEND_URL(Helm:https://<domain>) plusCORS_ALLOWED_ORIGINS(Helm:cors.allowedOrigins), and the development servers only withAPP_ENV=local. The console is same-origin and unaffected.A changed organization domain moves like the platform base domain. The functions keep their URLs and answer on the new domain too — through an alias of the organization's own on platform regions, through the base-domain migration on its own regions — until DNS of the new
<region>.<domain>answers, or until Complete now in the Organization domain card (PATCH /api/settings{"complete_domain_migration": true}). Setting the previous domain back rolls the move back;GET /api/settingsshows it astenant_domain_migration(docs/tenancy.md).
Fixed#
- Helm: uptime checks run in the region that is RunR's own cluster. Its agent calls the API through the in-cluster gateway (
runr-gateway.<namespace>.svc.cluster.local), a host the API answered with 400 (uptime: fetching checks: RunR answered 400), so checks with location "regions" never ran there; the replay captures of that cluster's function pods, and the callbacks of any region whose agent address is not the console's host (agent_ws_url,RUNR_AGENT_WS_URL), failed the same way. The region callbacks under/api/internal/now also accept the hosts of the addresses RunR hands out: the in-cluster gateway, every region'sagent_ws_urlandapp_url,RUNR_AGENT_WS_URL,observability.replay.ingestUrlandWORKFLOWS_PUBLIC_API_URL; every other route still answers only for the console's host. Upgrading the API is enough, the agents keep their address (regions). - Helm: switching a module on in an upgrade creates its tables in every tenant schema. The pre-upgrade migration Job ran before Helm applied the new ConfigMap and so migrated with the previous release's module switches: an upgrade that turned billing (or tickets) on left the tenant schemas without the module's tables. The migration and bootstrap Jobs now take
TENANCY_ENABLED,AUTH_PROVIDER_ENABLED,TICKET_ENABLED,BILLING_ENABLEDandBILLING_CURRENCYfrom the values, the Billing and Ticket modules register their tenant migrations whether switched on or not, and the bootstrap Job runstenancy:migrateafter its module switches as well (upgrade). - A redeploy no longer counts the day's usage again. Metering kept a usage record per deployment but filled each with all of the project's sessions of the day, so a project redeployed (or moved, or switched between scale modes or triggers) N times had its morning counted N times, and an always-on project got another full day of reservation per deploy. Every session and every second of a reservation now counts once, and a recalculated day gives the same totals. The run for yesterday during the first hour of a day no longer applies today's state: a project first deployed after midnight has no usage the day before, and a resize after midnight does not change it (upgrade note).
- A redeploy while the previous deploy is still starting leaves one live deployment. A deploy replaced only the active deployment: one still starting went live next to the new one once its readiness check found the project's Knative Service ready, and both were metered. Every live deployment of the project is now replaced (console, CLI, API, git pushes, resumes, runtime test deploys), the readiness check of a replaced one ends it without activating it, and a git deploy replaced before it went live ends as
skippedinstead ofdeploying. - Helm: SSO callback and MCP URLs are
https. The gateway overwrote the ingress controller'sX-Forwarded-Protowithhttp, so the API senthttp://SSO redirect URIs (which Google rejects) and publishedhttp://MCP metadata. It now keeps the header of a trusted ingress controller. Providers registered with thehttp://callback need thehttps://one; SAML URLs came fromAPP_URLand do not change (upgrade note). - Helm:
api.pdbno longer blocks node drains. With the ReadWriteOnce API volume the API pods cannot leave the volume's node, so the budget held a drain forever; it is created only with a ReadWriteMany volume, andhelm statussays when it is skipped. - Helm: upgrades keep the replica count of an autoscaler. The Deployments of the gateway, API and UI set
replicason every upgrade, resetting what their HorizontalPodAutoscaler had scaled to; with the HPA on, the chart leaves the count to it. - Helm:
api.port,ui.portandreverb.porttake effect. Octane, the UI's Caddy and Reverb listened on 80, 80 and 8080 whatever the values said; they listen on the configured ports (RUNR_API_PORT,RUNR_UI_PORT,REVERB_SERVER_PORT), the defaults unchanged. - Reverb logs every message only while debugging. It ran with
--debugin production; now only withAPP_DEBUG=trueorAPP_ENV=local. - A generated administrator password is never logged. With
secret.create=falseand noADMIN_PASSWORD, the seeder printed the generated password into the bootstrap Job's log; the Job now stores it in the Secretrunr-admin(it may read and patch that one Secret) andhelm statusprints the command that reads it. The seeder shows a password it generates only on an interactive terminal; the Compose migrator still prints it once on the console that runs it. - Helm:
values.schema.jsonvalidatesimages.*andsupplyChain.*, the keys the templates read; it checkedapi.imagesandapi.supplyChain, which nothing reads (still accepted). - A Compose install has a first administrator.
docker compose run --rm migratorran the seeders, which create accounts only in a local development environment: a production install ended without any account. The migrator now creates the first administrator while the installation has none, with the rules of the Helm chart's bootstrap job:ADMIN_USERNAME,ADMIN_EMAILandADMIN_PASSWORDfromapi.env(new inapi.env.example; defaultsadminandadmin@example.com); with an emptyADMIN_PASSWORDit generates a password and prints it once at the end of its output, never to the log. Nothing happens when that account exists, another administrator exists (without the Tenancy module's administrator flag: any user) or the e-mail address belongs to another account, so the migrator stays safe to run after every update and an existing install with users gets no new account.php artisan runr:bootstrap-admindoes the same by hand (compose/README.md). - HTTPS for the Compose install. The guide served plain HTTP only; the example host proxy configurations were not part of it and disagreed with the default port. Section 7 of compose/README.md sets up a Caddy on the host for the console domain (Let's Encrypt or an own certificate) and, for a region whose ingress only that host reaches (a kind cluster), the wildcard function domain:
Caddyfile-hostforwards to the gateway on127.0.0.1:8080(PORTin.env),Caddyfile-wildcardis appended to it. The gateway (compose/Caddyfile) now keeps theX-Forwarded-ProtoandX-Forwarded-Forof a proxy on a private address instead of overwriting them, so behind the HTTPS proxy the API buildshttpsURLs (the SSO callback URL among them) and, withTRUSTED_PROXY_HOPS=2(new inapi.env.example, default1), records the client's address; clients that connect to the gateway directly are handled as before. The examples ofAPP_URL,FRONTEND_URLandSANCTUM_STATEFUL_DOMAINSmatch the default port. - Compose module toggles reach the console.
compose/build.shbuilt the UI without the module flags, soTENANCY_ENABLED,AUTH_PROVIDER_ENABLEDandTICKET_ENABLEDswitched the API only. The build now reads them fromcompose/.env(an environment variable of the same name takes precedence), with the defaults ofdocker-compose.yml, and builds the UI with the modules the API enables;compose/deploy.shreads them from the server's.env. The guide sets the toggles before the build (compose/README.md). An existing install keeps its.env; its next build shows the modules its API already runs. - The Compose gateway serves the Horizon dashboard.
/horizonwent to the console; it reaches the API as on the Helm gateway, signed in with the single-use link ofphp artisan runr:horizon-linkorPOST /api/horizon/link(compose/README.md). - SSO callbacks reach the API. The sign-in callback of an OAuth / OpenID Connect provider is the API's
/oauth/<provider>/callback. The Helm chart's gateway sent it to the UI, so a provider sign-in ended on the console's start page; the Compose gateway sent every/oauth/*path to the API, the console's own/oauth/callbackpage that completes the sign-in included, which then answered 404. Both gateways now route/oauth/<provider>/callbackto the API and leave/oauth/callbackto the console;ci/check-helm-chart.shchecks the chart's route. edition: multi-tenantruns the multi-tenant console. The modules of the console are built into its image, and the chart appendedui.image.multiTenantSuffix(default empty) to the UI tag of the multi-tenant edition, so--set edition=multi-tenantalone ran the single-tenant console without the tenant and SSO pages. The default suffix is now-mt, the published multi-tenant UI images (ghcr.io/brf-tech/runr/ui:<version>-mt); with the default taglatest, which has no multi-tenant variant, the multi-tenant edition runs the chart's release (<appVersion>-mt). Single-tenant installs and installs that set the suffix keep their image.helm upgrade --reuse-valueskeeps the empty suffix of an earlier release: add--set ui.image.multiTenantSuffix=-mtto that upgrade. A UI image of your own that carries the modules under its plain tag needsui.image.multiTenantSuffix="".- Helm value
ticket.enabled. The Ticket module (support tickets) had no chart value: the API always enabled it, andruntime/ci.sh deploy --no-ticket(its default) did not reach the API.ticket.enabledsetsTICKET_ENABLEDfor every API container and hook job; its defaulttrueis the module state every earlier install had, also for reused values without the key. Installs deployed withci.sh deploywithout--ticketcarryticket.enabled=false: their API now disables the module, as their console already did. The console shows the ticket pages when its UI image is built withVITE_TICKET_ENABLED=true. - OpenID Connect sign-in. The Generic OIDC provider failed at every sign-in: no
oidcdriver existed. RunR now signs in against the identity provider's discovery document with the authorization code flow, PKCE,stateandnonce, verifies the ID token (signature against the published keys, issuer, audience, expiry, nonce) and reads e-mail, name and the groups claim of SSO group mapping from it and the userinfo response (docs/modules.md). - Google hosted domain. The provider's Hosted Domain was not sent to Google, so any Google account could sign in. The sign-in now asks for the domain (
hd) and refuses an account whosehdclaim is another domain or missing. - SSO sign-in on a tenant's sign-in page. The redirect used the platform's provider settings and the callback the tenant's: a provider that exists only in the tenant failed. Redirect and callback now use the same provider of the same organization.
- SAML button on the sign-in page. It called an OAuth route that excludes SAML and failed; it now starts at the provider's SAML login.
- Several providers of one driver. A sign-in always took one, arbitrary enabled provider of its driver. The sign-in page's button now names its provider (
?provider=<id>on/api/oauth/{driver}/redirect; without it the first by order, then name), so two OpenID Connect identity providers, or a gitlab.com and a self-managed GitLab, can be enabled side by side. OpenID Connect users are identified per provider and GitLab users per self-managed instance; accounts linked earlier keep working. A self-managed GitLab's Instance URL without a trailing slash works. - Sign-in errors at the provider. A cancelled sign-in or a code the provider refuses returned an error page; the sign-in page now shows the reason. A provider address marked unverified (
email_verified: false) is no longer matched to an existing account. - An SSO sign-in from a tenant's sign-in page starts the console in that tenant. The one-time code of the OAuth, OpenID Connect and SAML handoff names the tenant, and in a multi-tenant install
POST /api/oauth/exchangeanswers like the password sign-in, withtenantsandactive_tenantnext to the token. A tenant's page admits its members and platform administrators only; anyone else gets 403 and the session is revoked. The session is not pinned to the tenant, so the console still switches organizations (docs/security.md). - An SSO sign-in after another session replaces that session's organization and brand in the console with the ones of the new sign-in.
- Converting a single-organization installation into a multi-tenant one.
php artisan tenancy:setupmigrated the new tenant schema with the central migrations (the nexttenancy:migrateof every upgrade failed, and a second run created every central table in every tenant schema) and copied nine tables. It now creates the schema with the tenant migrations and copies every table the tenant schema has, in foreign-key order with the ids they had, gives every account its organization role as membership, assigns and syncs the regions, binds the organization's central rows (sandboxes, git installations and repository links, MCP, status pages) and API tokens to it and movesstorage/gitand the git remotes into the organization's directories. The central schema keeps its rows.--dry-runshows the plan and changes nothing; a conversion that stopped continues where it stopped; a finished conversion, a fresh installation and an installation with organizations are left alone. The slug defaults to the git remote owner, so existing git remotes keep their URLs (docs/tenancy.md). - An installation switched to the multi-tenant edition has platform administrators. The Tenancy migrations added
users.is_adminas false for every account and the chart's bootstrap creates an administrator only on installs without accounts. A new Tenancy migration makes the owners and admins of the single organization platform administrators when the installation has none (nothing changes where one exists),tenancy:setuppromotes them too, andtenancy:setup --admin=<account>makes any account one. tenancy:migrate --freshempties the tenant's schema only. It ranmigrate:freshin tenant context, whose search path includespublic: the central tables were dropped too. It now drops and recreates the tenant's schema, and asks for confirmation in production unless--force.- New organizations get every module's tenant migrations. The console, self-service signups and
tenancy:create-tenantran only Tenancy's tenant migrations; the Ticket module's waited for the next upgrade. They now run the same migrations astenancy:migrate. - Single-tenant installs have a support staff view. Members whose role holds
tickets.manage(Owner, Admin, custom roles) list, answer, assign and tag tickets in Settings → Support Tickets; it was shown to platform administrators only, which single-tenant installs do not have. - Ticket attachments are kept per organization. Ticket ids are per organization, so their files shared
support-attachments/<ticket id>/. Files now go tosupport-attachments/<tenant id>/<ticket id>/, a download serves only files recorded on that ticket of the requesting organization (internal-note files to support staff only), and the upgrade moves each organization's existing files into its directory. - Changing an organization's domain keeps the platform's domain entry. The previous domain was read with the platform's as fallback, so an organization's first domain removed the platform's
<region>.<platform domain>entry from the regions; the platform's own domain and malformed host names are refused. GET /api/settingsshows the capacity and allocations of every platform region (assignable_regions) to platform administrators only.- Deleting an organization removes what it kept outside its schema: its sandboxes (pods deleted in the region), warm pools and sandbox quota, uploads in progress and uploaded images (registry manifests included), custom runtimes with their registry images, hosted MCP servers with their calls and sessions, git installations and repository links, status page lookups, its branding override and assets, its support ticket attachments, its notifications and the settings of its own regions. The audit log and billing records are kept; a sandbox pod or image that cannot be removed keeps the organization as
delete_failedfor a retry (docs/tenancy.md). - Organizations create projects, previews and sandboxes on their own regions. The organization quota check refused a region the organization added itself, because it has no quota assignment; own regions are limited by their capacity only.
- Regions an organization adds are recorded as its own (
owner_tenant_id): the installation's organization CPU/RAM ceiling is no longer applied to an organization's own cluster. The upgrade marks existing ones. - A platform region's status reaches the organizations. Its status (active, inactive, maintenance) and migration state are synced to the organizations' copies, so their deploys respect a region in maintenance.
- The status checks of a platform region reach the organizations' copies. The installation, cluster and DNS status the status jobs write stayed in the platform's row; only changes made through the region API were synced.
- Ending a region assignment or deleting a platform region removes the organization's copy. The organization's projects there are torn down and stay without a region, its API gateways are removed from the region, and the region leaves its list.
- Detaching a platform region from an organization, or deleting an organization without a domain of its own, removed the platform's
<region>.<base domain>entry from the region's Knative domain configuration; only the organization's entries are removed now. - Reading an unreachable cluster removed its kubeconfig.
GET /api/knative-regions/{id}/knative-status(runr regions status) andGET …/nodes(the Workers tab) set the region's kubeconfig to null on any error, so a network hiccup left the region without access to its cluster. They now answer the error and record the cluster as unreachable; the kubeconfig stays. - The full Knative install deleted the Knative Services of a cluster that already ran Knative.
POST …/install-knative(runr regions knative install REGION) installed the operator without an adoption check, and the operator install starts by removing the previous operator and the Knative Serving CRDs. It now adopts a running operator and a Ready Knative Serving like the component installs; the operator install itself adopts a running operator, and its cleanup of a broken install keeps the CRDs while Knative Services exist (docs/regions.md). - A changed base domain did not move the regions. The new
<region>.<base domain>was added next to the old one (Knative serves only one), the regions' function domain — project URLs, function TLS, the status block, request replay — kept the old one, the DNS check never ran, the old domain was removed after 48 hours whatever DNS said, and the first base domain replaced a region's own domain. A changed base domain now migrates the regions that follow it: both domains are served (the new one byrunr-domain-aliasin the project namespace) until DNS of the new one answers the region's external address, or the migration is completed by hand (Migration tab,runr regions migration complete REGION --domain,POST …/complete-domain-migration); then the new domain becomes the function domain. Setting the old base domain back rolls the migration back; a region with a domain of its own is never touched (docs/regions.md). - Dedicated workers could not be cleared.
POST …/workersrefused an empty list, sorunr regions nodes set REGION --alland deselecting every node in the Workers tab failed with 422. An empty list releases every node. - The console's "In-cluster" Redis was refused. The Redis tab sent the mode
in-cluster; the API takesin_cluster. - Upgrade Serving did nothing. The Serving tab's Upgrade answered "already installed". It upgrades Knative Serving to the next minor version the cluster supports (
POST …/upgrade-serving,runr regions knative upgrade REGION --version): the operator follows when it is older, the version's CRDs are applied, then the KnativeServing resource (docs/regions.md). - The region mode chosen in the wizard was dropped.
POST /api/registry-setup/start(the console wizard) ignoredregion_mode; with the Tenancy module it is stored like onPOST /api/knative-regions. region_modeis a platform administrator's choice. In a multi-tenant install anyone else'sregion_mode(managed or infinite organization quotas) is ignored on region create and update and onPOST /api/registry-setup/start, so an organization cannot lift the quotas of a region.- Region settings had no console form, and
agent_ws_urlneeded a manual redeploy. The General tab has the settings form (request timeout, agent address, cert-manager ClusterIssuer, trusted proxies); a changedagent_ws_urldeploys the region's agent again.PUT …/settingsno longer takesnamespace, which was never read (docs/regions.md). request_timeoutdid not raise the function timeout. The region Traefik cut every response after 120 s and Knative refused a revision timeout above 600 s, whatever the setting (default 300, up to 3600). Traefik's read and write timeouts follow the longest setting of the regions on the cluster plus 10 s, and Knative's maximum revision timeout is raised to it — at install, on a settings change and withregion:reconcile(Traefik's pods roll once) (docs/regions.md).- Registry, monitoring and function TLS could not change after a region was created.
POST …/registry(the Registry tab,runr regions registry set) sets up the registry of an existing region, andPOST /api/knative-regionswith registry fields sets it up instead of only storing them;PUT …/monitoring(General tab,runr regions set --set prometheus_url=…) switches between an existing Prometheus / Loki and RunR's own;DELETE …/function-tls(runr regions function-tls rm) switches function TLS off; the Isolation tab adds Kata hypervisors to a working install (docs/regions.md). - A region created inactive was refused deploys while it showed active.
is_active: false(runr regions create --inactive) leftregion_statusactive; it isinactivenow, and the upgrade fixes existing regions. - Inactive and maintenance regions kept serving. The status block's route used Traefik v2 rule syntax, which never matches on the region Traefik, covered only
*.<function domain>, was shared by the regions of a cluster and was applied only on a status change. It now matches with Traefik v3 syntax every host of the region — function domain, project custom domains, the domains of its Knative Services — one route per region, and it is applied when the region's Serving becomes ready and byregion:reconcile, which also removes a leftover from an active region (docs/regions.md). region:reconcilekeeps the status block of the organizations' own regions. It applied and removed the block of platform regions only; it now does both for every organization's own regions too, in the organization's context. The tenant schemas get theregion_statusfix of regions created withis_active: false.- A region moved to another cluster lost part of its setup. A migration now also brings the dedicated workers that exist on the new cluster, WebAssembly and GPU support, the data services RunR runs, the admission policy, function TLS, the Skupper site and the status block; the Migration tab sends the new cluster's data location, and a region whose cluster does not answer can be opened in the console to move it (docs/regions.md).
- The DNS check never passed behind a load balancer with a host name. It compared
gethostbyname()with the external address, sodns_resolvedstayed false and a migration to such a cluster waited the full 48 hours. A CNAME to the load balancer, or one of its addresses, now counts. - A removed tracing backend came back.
DELETE /api/observability/regions/{id}/tracingkept the region's tracing mode andregion:reconcileinstalled the backend again; the removal sets the mode tooff. uninstall-knativeleft the region showing Knative installed. The operator and Serving statuses stayed ready; they are reset tonot_installed.- A replaced kubeconfig was not checked.
PUT /api/knative-regions/{id}withkubeconfig(runr regions set --kubeconfig) took any value; it is checked like a new region's and has to reach the same cluster (docs/regions.md). - The fallback Knative version needed Kubernetes 1.32. Without the chart repository RunR installed Knative 1.20 on any cluster; it installs the newest pinned Knative version the cluster supports. RunR's Prometheus (kube-prometheus-stack 92.10.1) and Loki (loki-stack 2.10.3) charts are pinned instead of the newest the repository has.
- The region-mode hint of the wizard said every tenant gets unlimited access to an
infiniteregion; only the tenants it is assigned to use it.
Removed#
Plans and the 0.5.7 billing: plans and plan limits, plan renewal, subscriptions, trials, credits, purchases, add-ons, capacity deals, cost configurations, charges, usage thresholds and the store link, with Settings → Plans & Billing, Settings → Subscription, Settings → Charges and the old cost overview; the endpoints
/api/admin/plans,/api/admin/billing-settings,/api/plans/public,/api/tenant/subscription,/api/tenant/billing,/api/cost-configs,/api/tenants/{tenant}/billing/*,/api/tenants/{tenant}/capacity-deals,/api/regions/{region}/profit-lossand the 0.5.7/api/billing/tenants,/chargesand/purchasesroutes; the Helm valuesplans.enabled,billing.usageThresholds,billing.storeUrland the variablesPLANS_ENABLED,BILLING_USAGE_THRESHOLDS,BILLING_STORE_URL,BILLING_DEFAULT_CURRENCY(nowBILLING_CURRENCY). The data is kept inbilling_legacy_archive(docs/upgrade.md).Cost fields of the core usage endpoints:
final_cost,raw_costandcurrencyofGET /api/bindings/{id}/usageandGET /api/sandbox-usage, which were 0 for every record since metering records quantities only, and the COST column ofrunr bindings get --usageandrunr sandbox usage(which now totals time, CPU and memory hours). What usage costs comes from the Billing module:GET /api/billing/usageand the bills (upgrade note).Helm: SQLite is not a chart option.
database.driver: sqlitenever worked with the chart (no database file on the API volume, one database per pod); the API, Horizon, scheduler and Reverb pods share one PostgreSQL server, and any other driver stops the install with a message.
0.5.7 - 2026-10-17#
Changed#
- The CLI downloads from runr.sh.
curl -fsSL https://runr.sh/install.sh | shinstalls therunrCLI on Linux and macOS (amd64, arm64): it reads the latest release fromhttps://runr.sh/dl/cli/latest.txt(RUNR_VERSIONpins one), checks the archive's SHA-256 against the release'schecksums.txtand installs into/usr/local/binwhen that is writable, else into~/.local/bin(RUNR_INSTALL_DIRsets another directory). The archives of every platform, Windows included, are underhttps://runr.sh/dl/cli/<tag>/.runr version --check-update(andupdate_check: true) reads the latest release from thatlatest.txt(docs/cli.md). - Terraform provider address
runr.sh/brf-tech/runr. runr.sh serves Terraform's provider registry protocol: withsource = "runr.sh/brf-tech/runr",terraform initdownloads the archive for the platform and checks it against the release'sSHA256SUMS, signed with the RunR release key (key ID2588846B241BC095). A configuration that usedbrf-tech/runrchangessourceand moves its state once withterraform state replace-provider registry.terraform.io/brf-tech/runr runr.sh/brf-tech/runr(docs/terraform.md). - The GitHub Action downloads the CLI from runr.sh.
cli-base-urldefaults tohttps://runr.sh/dl/cliandversion: latestis the tag in<cli-base-url>/latest.txt; thegithub-tokeninput is gone (a workflow that still sets it gets an "unexpected input" warning) (docs/github-action.md). - Documentation on runr.sh. The documentation is published at https://runr.sh/docs/ and the security policy at https://runr.sh/security/.
- The Helm chart comes from BRF's public Helm repository. The chart
runris installed without credentials:helm repo add brf-tech https://helm.brf.sh, thenhelm upgrade --install runr brf-tech/runr --set domain=…; the install, upgrade, key rotation and white-label examples use the chartbrf-tech/runrinstead of./helmof a repository checkout (README, docs/upgrade.md).
0.5.6 - 2026-10-17#
Changed#
- The CLI's own API token expires.
runr login --with-passwordcreated itsrunr-cli@<host>API token without an expiry. The token now expires 90 days after the login;--token-ttlsets another lifetime (30d,720h;0orneverkeeps a token without expiry).runr loginandrunr whoamishow when it expires (token_expires_atin their JSON output, and in the context of the config file); once it has expired, a command the API refuses says that the token of the context expired and to log in again, andrunr logoutno longer tries to revoke it. A newrunr logininto a context revokes the token an earlierrunr login --with-passwordcreated for it, asrunr logoutdoes: logging in again replaces a token from an earlier release with an expiring one.--expiresofrunr tokens createand--token-ttlrefuse a fractional day count such as1.5dinstead of reading it as1d(docs/cli.md).
Fixed#
ci.sh ml-runtimes --pushpushed untested images. The push built the PyTorch and TensorFlow images in a separate builder and pushed them right away; the smoke test, and on a GPU host its GPU check, ran for local builds only.--pushnow builds every image into the local Docker, smoke-tests it like a local build (with the GPU check wheredocker run --gpus allgives a GPU, required withML_SMOKE_GPU=1) and pushes only once every image passed:docker pushof the image the smoke test ran, its image ID checked again before the push.--push --no-smokeis refused (docs/runtimes.md).- A deleted user's tokens stayed in the database. An administrator deleting a user (
DELETE /api/users/{id}) left the user's sign-in sessions and API tokens behind; no request could use them without the user, but the rows stayed. Deleting a user — by an administrator or, as before, by the user (DELETE /api/account) — now deletes every sign-in session and API token of the user in the same transaction as the account, then the user's MCP server sessions. In the multi-tenant edition, a deleted account's copies in the organizations it belonged to go as well, with its groups and role assignments there (only the memberships went before); a member removed from an organization loses the API tokens they created in it (these work in that organization only; sign-in sessions and other organizations' tokens stay), and deleting an organization deletes the tokens created in it. The dailyphp artisan tokens:prunealso removes the tokens of users that no longer exist and, in the multi-tenant edition, the tokens created in an organization that no longer exists or whose user is no longer a member (a platform administrator's token stays), and the organizations' copies and role rows of users that no longer exist; an organization whose schema is missing is skipped. The rows earlier deletions left go after the upgrade without any action (docs/security.md).
0.5.5 - 2026-10-17#
Security#
- Signing out did not end the session. From 0.3.0 to 0.5.4,
POST /api/auth/logoutanswered "Logged out." and deleted the session's token, but the API's session tracking (client address, user agent and device of a token), which runs after every request, saved the deleted token again: the token stayed valid. The same kept the sign-in session alive thatrunr login --with-passwordends once it has created its API token. The session tracking now updates the token's row and never re-creates it, and writes only when the address, user agent or device changed. Revoking sessions, deleting API tokens and disabling or deleting the account deleted their tokens as intended; tests now cover each of these paths. - Sign-in sessions expire. A sign-in session — a password sign-in of the console or the CLI, an SSO sign-in, a self-service signup, an accepted invitation — ends 30 days after the sign-in (
RUNR_LOGIN_TOKEN_TTL_DAYS, Helm valueauth.loginTokenTtlDays;0keeps sessions until they are signed out). They had no expiry before. The lifetime counts from the sign-in for the sessions that exist at the upgrade as well: those older than 30 days end with the upgrade, among them the sessions that were signed out but stayed valid, and their users sign in again. Operators need to do nothing. A session signed out during the 30 days before the upgrade stays valid until it is 30 days old; its user can end it at once under Account → Active Sessions (Revoke). API tokens keep the expiry they were created with. - Changing the password ends the user's other sign-in sessions (their browsers are signed out); the session that changed it and the API tokens stay.
php artisan tokens:prune, scheduled daily, removes the sign-in sessions that ended and API tokens 30 days after they expired. API tokens cannot be namedinvitation-accept(norapi, as before): these names mark sign-in sessions, and the API token list no longer shows the session of an accepted invitation, which Account → Active Sessions lists (docs/security.md).
0.5.4 - 2026-10-17#
Upgrade notes#
- The last region workloads on RunR's earlier image names move too.
region:reconcile(stepimage-registry) left the node installers of Kata (with Firecracker's devmapper setup) and gVisor and the region's queue Redis onghcr.io/runrsh/ghcr.io/brf-techuntil their next install. Their pod templates now move toghcr.io/brf-tech/runras well, and their running pods keep running: the installer DaemonSets get the new images together with the update strategyOnDelete(a new pod would re-run the install on its node; kata-deploy's cleanup and reinstall restart the container runtime); the queue Redis, whose streams, consumer groups and KV bindings' data live in memory only, keeps its pod — the Deployment is paused while its ReplicaSet takes the new image, then takes it as well and is resumed without starting a pod. A pod created later (a new node, a deleted or evicted pod) starts fromghcr.io/brf-tech/runr. A gVisor install, a Kata install that re-runs kata-deploy and Kata's Firecracker setup applyRollingUpdateagain; Kata's rollout status counts the pinned installer pods by their config hash, and its in-place drop-in update keeps them pinned (docs/upgrade.md). - hf-serve 1.1.1. The defaults move to
ghcr.io/brf-tech/runr/hf-serve:1.1.1and…:1.1.1-cuda(ai.images.hfServe,ai.images.hfServeCuda). Installations that set these values, for example to a registry mirror of an air-gapped region, update them and mirror the new images. - Components on dedicated workers restart once.
region:reconcile(stepdedicated-worker-toleration) gives the registry, Prometheus, Loki, Tempo and the data services the toleration of the dedicated-worker taint where a pod of them runs on a dedicated worker; that pod restarts once, on the same node (below, Fixed). Components on the other nodes keep running as they are.php artisan region:reconcile --dry-runlists them. exec-tensorflow:2.21-gpuis published again under the same tag (below, Fixed). A function ontensorflow2.21-gpuruns on the new image from its next deploy: Knative resolves the tag to the new digest for the new revision. Installations that mirror RunR's images copy the tag again.
Changed#
hf-serve starts a cached model without the Hub. In online mode hf-serve asked the Hub at every start, also when the pinned commit was already on the model volume, so a start from zero depended on the Hub's reachability and latency. After a load of a pinned commit it now records the load next to the cached repository (per engine, task and pipeline arguments), and a later start with the same settings loads the cached snapshot directly; when that does not load, it loads from the Hub as before. A revision that is a branch or tag is still resolved by the Hub at each start (docs/huggingface.md).
The GPU case of
ci.sh test-aileaves Docker's default runtime alone. The Kind worker with the GPU gets the driver's files thatnvidia-container-cli listnames, mounted read-only (Kind's privileged nodes see the host's/dev/nvidia*already), so a host with the NVIDIA driver and Container Toolkit runs the case as it is configured. Where the NVIDIA runtime is Docker's default runtime and accepts device requests as volume mounts, the worker gets the GPU that way, as before (docs/ai.md).The GPU case of
ci.sh test-aineeds far less disk. Its check image is the new GPU CaaS example (below) instead of CUDA PyTorch (3.3 GB compressed), it goes into the Kind GPU worker only instead of every node, and the host's copy is removed once the worker has it.kind loadwrites its temporary archives toRUNR_E2E_TMPDIR(default/var/tmp) instead of/tmp, which is a RAM-backed tmpfs on some hosts. Before the cluster is created the suite adds up what the run writes, from the built images' sizes and per Kind node, plusRUNR_E2E_GPU_RESERVE_GB(default 12) for the rest; it does not start when Docker's root has less free and prints both figures. This replaces the fixedRUNR_E2E_GPU_MIN_FREE_GB. The case now also asserts that the container ran the kernel on the GPU and returned the CPU's result (docs/ai.md).The GPU CaaS example runs a CUDA kernel in a 43 MB image.
examples/ai/gpu-caaswas CUDA PyTorch with a small inference: 3.3 GB compressed, about 9 GB unpacked. It is now Python's slim image with a standard-library server that runs a SAXPY kernel through the CUDA driver API (the driver compiles the kernel's PTX for its GPU, so no CUDA toolkit is in the image) and compares the result with the CPU's: 43 MB compressed, 134 MB unpacked, for NVIDIA GPUs from compute capability 5.0 on. The answer keepscuda,deviceanddevice_count; the PyTorch fields give way tocompute_capability,memory_mb,driver_cuda_version,computed_on,checksum,expected_checksumandresult_ok(examples/ai/gpu-caas).ci.sh ml-runtimessmoke-tests the-gpuimages on the GPU. On a host wheredocker run --gpus allgives containers a GPU, the handler of a-gpuimage's smoke test runs with it, over HTTP and as a scheduled run: in both, the framework has to see the GPU and run a mean and a matrix product on it. Without a GPU the CUDA build is checked as before and the output says the GPU check was skipped;ML_SMOKE_GPU=1requires it,ML_SMOKE_GPU=0leaves it out (docs/runtimes.md).Pinned helm and kubectl in the API image. The image took kubectl from the latest stable Kubernetes release at build time, so two builds of one tag could differ. helm v3.16.0 and kubectl v1.35.9 are now pinned and checked against their published SHA-256 for amd64 and arm64, and the downloads retry on network errors.
ci.sh test-aikeeps etcd in memory. The suite setsKIND_ETCD_IN_MEMORY=1unless it is set: etcd keeps its data on the Kind control-plane's tmpfs. On a GPU host whose disk other jobs kept busy, the region setup's Knative Operator install failed withetcdserver: request timed out; with etcd in memory the same run got past it.KIND_ETCD_IN_MEMORY=0puts etcd on the disk again (docs/ai.md).Organization quotas overcommit scale-to-zero workloads. In the multi-tenant edition an organization's CPU and memory quota on a region splits into the same two budgets as the region capacity, with the region's overcommit factors. Always-on projects and model services, the Kata pod overhead, queue consumers, sandboxes and warm pool pods count in full against the quota; one pod of every scale-to-zero project, model service and git deploy preview counts against quota × the factor (CPU 3, memory 1.5 by default) minus what the always-on part holds. A factor of 1 on a region keeps the quotas there hard. The quotas on a region still add up to at most its capacity, stored quotas stay as they are, and billing is unchanged. A refusal (422) names the budget that is used up (
budget:always_onorfaas) and what it has free (docs/architecture.md).
Fixed#
- The region registry stopped after a worker selection. Zot installed before a region's workers were selected had no toleration of the dedicated-worker taint. Its node-bound volume (local-path, k3s's default class) keeps it on its node, and when that node was selected as a dedicated worker (
runr.dev/dedicated=projects:NoSchedule), the next Zot pod — after the restart ofregion:reconcile'sregistry-sync-config, the roll ofimage-registry,registry:rolling-restartor a Helm upgrade — stayed Pending ("untolerated taint" and "volume node affinity conflict"): builds failed and the registry pre-warm Job gave up after 600 s. Prometheus, Loki, Tempo and the data services (KV Redis, Garage, the CloudNativePG clusters) had the same exposure. All of them now tolerate the taint and prefer the nodes that are not dedicated workers. The worker selection adds the toleration to such a component whose pod runs on a selected node, andregion:reconcile(new first stepdedicated-worker-toleration) to those on any dedicated worker; a pod Pending on the taint is replaced, as a StatefulSet does not replace a pod that never became ready. The registry restart ofregistry-sync-configcarries the toleration as well (docs/architecture.md). - The registry and Knative after deselecting workers. A registry installed while dedicated workers were selected, its node mirror and Knative's system deployments select the workers by the label
runr.dev/dedicated=projects. Once the last worker was deselected their next pods found no node, and a registry whose volume sat on a deselected node could not start there again. The worker selection andregion:reconcile(stepdedicated-worker-toleration) now move such a registry to any node (the toleration kept, the other nodes preferred, a Pending pod replaced), apply the node mirror for every node once no worker is left — projects then run on any node and pull from the registry there — and drop Knative's nodeSelector (docs/architecture.md). - RunR's own pods on a node selected as a dedicated worker. When RunR runs in a cluster it also uses as a region, the node holding PostgreSQL's volume or the API storage could be selected as a dedicated worker; its taint then kept the next pods (a restart, an upgrade) from starting there. Every pod of the chart — the Deployments and the migration and bootstrap hook Jobs — now tolerates
runr.dev/dedicated=projects:NoScheduleand prefers the nodes that are not dedicated workers (placement.tolerateDedicatedWorkers, on by default). New valuesplacement.tolerations,placement.affinityandplacement.nodeSelectorapply to every pod; the chart adds its toleration, preference and the API storage colocation to them (docs/architecture.md). - The registry pre-warm Job's log. While the registry did not answer, every try printed wget's "can't connect to remote host: Connection refused". The Job now prints one line per wait instead:
waiting for registry http://… to answer: next try in 4 s (2 s waited). - Executor images in the Kind cluster of
ci.sh test. The preload of the default executor images matchedbrf-tech/…and loaded none since RunR's images moved toghcr.io/brf-tech/runr/. It loads the default general runtimes from there again; an ML runtime (several GB) is pulled by the node that runs it. - The TensorFlow
-gpuruntime ran on the CPU. Inexec-tensorflow:2.21-gpuTensorFlow registered no GPU and ran every function on the CPU, without an error: it opens the CUDA libraries of NVIDIA's pip packages by soname through its RUNPATH, which lacks the cuSOLVER directory, and one library it cannot open makes it skip all GPUs. The image now puts everysite-packages/nvidia/*/libdirectory into the dynamic loader's cache, whatever the entrypoint and a function's ownLD_LIBRARY_PATH, and its build fails unless every library there is in the cache. - diffusers model services and
dtype. hf-serve passed the dtype to diffusers astorch_dtype, which diffusers 0.41 deprecates (removed in 1.0); with adtypeinRUNR_PIPELINE_KWARGSthe load failed, because diffusers refuses both arguments together, and a dtype name such as"float16"fell back to float32. hf-serve 1.1.1 passesdtypeto both engines, takestorch_dtypeof the pipeline arguments asdtype, and gives diffusers the torch dtype a name stands for. - pip's dependency conflict in hf-serve's GPU image. The PyTorch base image ships
spin0.18, a developer CLI that requires click below 8.4, and hf-serve's server brings click 8.5: the build printed pip's dependency-conflict error andpip checkfailed in the image. hf-serve 1.1.1-cuda leavesspinout, which nothing in the image uses, and its build fails unlesspip checkpasses. - GPU nodes of
ci.sh test-ai. Every Kind node counted as a GPU node, because all of them are privileged and see/dev/nvidia0: on a GPU host each would have got the NVIDIA Container Toolkit, the GPU label and a--gpu-nodeargument. Only Kind nodes wherenvidia-smi -Llists a GPU count now — the worker with the driver. - Resource bars on overcommitted regions. The projects page added up the projects' reservations in the browser against the region capacity and capped the bar at 100 %, so a region holding more scale-to-zero reservations than its capacity looked full while it still admitted projects. "Total Resources" now shows the always-on and the FaaS overcommit budget per resource as the API admits against them, summed over the regions and per region, with the bars of Settings → Regions → (region) → Workers. In an organization a platform region shows the organization's quota there, split the same way (
capacitywithcapacity_scope: "quota"inGET /api/settings), and no longer shows as unreachable. - An organization's quota usage seen by platform administrators. Tenants → (tenant) → Quotas (
GET /api/tenants/{id}/regions) counted the projects of the organization the administrator was working in, not those of the organization opened. It now counts in that organization's own schema and shows its quota's always-on and FaaS overcommit budgets (capacityper region) with the bars of the projects page. - Quota refusals for a resource without a quota. A 422 of the organization quota (and a sandbox's
quota_region) reported 9223372036854775807 as what was left of a resource with quota 0. It now givesremaining_cpu_m/remaining_memory_mbonly for the resources with a quota and names only the resource that is short. - The disk check of
ci.sh test-aicounts the whole run. It counted hf-serve and the GPU image per Kind node plus a 12 GiB reserve, and let a GPU run start with 23 GiB free under Docker's root; that run was down to 5 GiB free by its first AI case, with the other AI cases, GPU support and the GPU case still ahead. The check now also counts each Kind node's own data (1 GiB: Kubernetes' images, copied into every node), the Compose stack (3 GiB) and, unlessKIND_PRELOAD=0, the preloaded images on every node and the host (1.4 GiB each): about 35 GiB with the default Kind cluster, 25 GiB withKIND_WORKERS=1 KIND_PRELOAD=0. Its refusal names both options (docs/ai.md). - A Kind e2e run stopped from outside cleans up.
ci.sh test, its suites (test-ai,test-runtimes, …) andtest-helm-upgraderemove their Kind cluster, Compose project, Kind lock and Cloud Provider KIND also when they are stopped: on TERM or HUP at once, also in the middle of a step, and on INT (Ctrl-C) once the running command has stopped. A run whose output pipe was stopped together with it ended at the cleanup's first line and left all of them behind; INT to the run while a command ran could let it go on with its next step; a stop duringdocker compose upleft the Compose stack; and a run stopped while it waited for the Kind lock deleted the cluster of the same name that the lock holder used. The cleanup touches only what the run created, and when the run's output is gone it writes to${TMPDIR:-/tmp}/<cluster>-cleanup.log(docs/development.md). - A short hiccup of the Kubernetes API server no longer fails the Knative Operator install. The operator's Helm install failed on its first error, and with it the region setup: on a cluster whose etcd could not keep up with a busy disk, Helm got
etcdserver: request timed outonce and the region had no Knative. The install is now tried again on transient errors of the API server — etcd timeouts and leader changes, 5xx and 429 answers, timeouts and dropped connections on the way to it — five tries in all, 2, 4, 8 and 16 s apart, each retry in the operator's install log. Any other failure, Helm's own--waitrunning out included, fails the install at once, as before.
0.5.3 - 2026-10-16#
Fixed#
- Tempo crash loop on storage that ignores fsGroup. With a storage driver that does not apply the pod's
fsGroup, Tempo's volume stayed owned by root and Tempo restarted with "mkdir /var/tempo/traces: permission denied". Tempo gets an init containerfix-permissionsthat hands/var/tempoto Tempo's user, as Loki's does;region:reconcile(steptracing) adds it to installed backends. A Tempo pod that crash-loops with the earlier template kept its StatefulSet from rolling: the step replaces it, and the new pod starts with the init container. - Tracing requests on small regions. On a single-node region, Knative's own pods stayed Pending on "Insufficient cpu" once the tracing backend was installed. The collector and Tempo request 50m CPU each (100m before), in line with the other region add-ons; the limits are unchanged.
region:reconcile(steptracing) moves installed backends to these requests, and the region capacity subtracts them like every other platform pod. - Registry pre-warm while the registry restarts. The
registry-prewarmJob started beforeimage-registryrolled the registry, so every image failed with "Connection refused", and the failed Job kept later reconciles from starting another one for 24 hours. The step now runs last inregion:reconcile; the Job waits for the registry to answer before it starts and before each further try of an image (three tries, backing off), a failed pod is retried twice, and a Job that failed is deleted and started again by the next reconcile. runr regionsshowed 0 projects to a tenant. A platform region assigned to a tenant answered withoutproject_countin the tenant's view (GET /api/knative-regions). It now counts the tenant's own projects there.
0.5.2 - 2026-10-16#
Upgrade notes#
- Image namespace ghcr.io/brf-tech/runr. Stored image references move automatically, from 0.5.1 (
ghcr.io/brf-tech/<name>) and from earlier releases (ghcr.io/runrsh/<name>): Helm values (runr.image), the database (migrationmove_images_to_runr_namespace, tenant migration0055,images:move-registryin the bootstrap job) and the region workloads (region:reconcile). A registry of your own that holds copies of RunR's images under their paths needs them under the new paths: copy them fromghcr.io/brf-tech/runr(docs/upgrade.md). - ML runtime versions. PyTorch 2.2 / 2.3 and TensorFlow 2.15 / 2.16 and their
-gpuversions, listed by the local and demo seeders only and never published as images, leave the catalog (migrationretire_unpublished_ml_runtime_versions). Projects that name one keep running as deployed and take the runtime's default version at their next deploy.executor:synclists the published versions: PyTorch2.14/2.14-gpu, TensorFlow2.21/2.21-gpu(docs/runtimes.md). - Tracing backend on existing regions.
region:reconcile(steptracing) installs the OpenTelemetry Collector and Grafana Tempo where tracing is enabled and the backend is missing — the region's own tracing mode, or for a region without one (Knative Serving installed by 0.4.x) the platform'sobservability.tracing.defaultModewhileinstallWithServingis on — and moves installed backends to collector chart 0.176.0 (collector 0.162.0) and Tempo chart 3.2.0 (Tempo 3.1.0); the collector and Tempo pods roll once. Tempo chart 3.2.0 appliestempoRetentionHours(default 72 hours) to the deletion of old traces, which chart 3.1.0 left at Tempo's 14 days. To keep tracing off on regions without their own setting, setobservability.tracing.installWithServing=falsein the upgrade (docs/upgrade.md).
Added#
- PyTorch and TensorFlow runtime images.
ghcr.io/brf-tech/runr/exec-pytorch:2.14(torch 2.14.1, torchvision 0.29.1) and…:2.14-gpu(the same torch for CUDA 12.6),ghcr.io/brf-tech/runr/exec-tensorflow:2.21(TensorFlow 2.21.0, CPU build) and…:2.21-gpu(CUDA 12): Python 3.12 with Keras (TensorFlow), NumPy, SciPy, pandas, Matplotlib, scikit-learn and Pillow, on the Python runtime's handler server, for linux/amd64.runtime/ci.sh ml-runtimes [--push] [--list]builds them; a build fails unlesspip checkpasses and the framework and its CUDA build match the tag, and a local build answers a tensor-op handler over HTTP and as a scheduled run.executor:syncreads aX.Y-gputag as the GPU version ofX.Y. The node image pre-pull and the region registry pre-warm leave the ML runtime images out: the nodes their functions run on pull them (docs/runtimes.md). - Billing events, newest first.
GET /api/billing/events?order=desclists the newest events first;before=<event id>bounds the list asafter=does, andmeta.next_beforecontinues a newest-first list (meta.next_afteran oldest-first one) (docs/billing.md). The console's billing settings read their 20 newest events with one request. - Registry pre-warm.
region:reconcilehas the internal registry of each region fetch the base images of the release's function builds before a first build needs them (stepregistry-prewarm, a Job in the registry namespace) (docs/runtimes.md). - Any sandbox command on a terminal.
POST /api/sandboxes/{id}/exec/wstakestty,colsandrows: the command runs on a terminal of that size, so shells, REPLs, editors and pagers work as in a terminal window.{"type":"resize","cols","rows"}frames change the size,stdin_closesends Ctrl+D,TERMdefaults toxterm-256color, and the start frame and the answer say whether the command has a terminal. Withoutttythe exec WebSocket works as before (docs/sandbox-api.md). runr sandbox exec -it.-truns the command on a terminal over the exec WebSocket; with-iton a terminal runr's terminal is raw while the command runs (every key, Ctrl+C included, goes to the command), the size follows the window (SIGWINCHon Linux and macOS, the console size on Windows), the terminal is restored on every way out and runr exits with the command's exit code (docs/cli.md).- Exec WebSocket in the sandbox SDKs. Python
Sandbox.exec_ws()/AsyncSandbox.exec_ws()and Nodesandbox.execWs()/runCodeWs()run a command over the exec WebSocket: input goes to the command while it runs (send_stdin/sendStdin,close_stdin/closeStdin), the output arrives as events or throughwait(),kill()stops the command, and with a terminal (tty,cols,rows)resize()changes its size. The Python SDK depends onwebsockets13 or newer; the Node SDK uses the runtime'sWebSocket(Node.js 22+, Bun, Deno) or its own client on Node.js 18 and 20 and keeps no dependencies (sdk/sandbox/python, sdk/sandbox/node). runr metrics livereads what a deployment uses right now — CPU, memory and requests per second from the region's Prometheus, with the averages of the last hour (--quickwithout them) — for the live deployment or--deployment, once or every--watchinterval,-o json(docs/cli.md).- Region lifecycle in the CLI.
runr regionscreates, checks, changes and removes regions (create,check,get,set,rm,status), saves a region's kubeconfig to a file only its owner can read (kubeconfig;--stdoutprints it) and chooses the worker nodes projects run on (nodes set). It installs, adopts and removes Knative (knative install|uninstall|detect) and manages the queue Redis, the image registry (also regions created with one, and checks of their storage), the move to another cluster (migration), function-domain TLS, Skupper sites, the FaaS overcommit factors (capacity), demo projects and WebAssembly support (wasm [status]|install|uninstall). Kubeconfigs, passwords and keys come from files or stdin and never reach the trace; removals ask for confirmation unless--yes, and--waitfollows background installs (docs/cli.md). - Mirror fallback for Docker Hub's pull limit. When Docker Hub refuses a copy,
ci.sh mirror-reqsandmirror-basescopy the image frommirror.gcr.ioorpublic.ecr.aws/docker, by the digest Docker Hub reports for it, so the copy is the same image (REQS_UPSTREAMSsets the sources and their order).mirror-reqstakes image names to copy only those and exits 1 when a copy fails.
Changed#
- Build deadline. Function build Jobs (Kaniko and WebAssembly toolchain builds) run up to
BUILD_DEADLINE_SECONDS, 900 s by default; the build checks and the stale-build check follow it. A build that runs out of time while the region registry still fetches its base image is built once more, and itsbuild_errorsays so meanwhile. - Sandbox start. A starting sandbox pod is read every 150 ms (
SANDBOX_START_POLL_MS, 100–200) for its first 30 s and once a second after that, so a sandbox is reported running at most one interval after its pod runs (docs/sandbox-api.md). - Warm pools. A pod a pool refill starts can be claimed as soon as it runs; the next per-minute refill no longer has to mark it warm first (docs/sandbox-api.md).
- Model docblocks. A
migraterunside-helper:models --write --resetonly withAPP_ENV=localandIDE_HELPER_POST_MIGRATE=true, so migrations in test containers leave the models alone (docs/development.md). - Tracing images from the reqs mirror. The region collector and Tempo run
ghcr.io/brf-tech/runr/reqs/opentelemetry-collector-contrib:0.162.0andghcr.io/brf-tech/runr/reqs/tempo:3.1.0, RunR's copies of the Docker Hub images; the Helm valuesobservability.tracing.collectorImage/tempoImagepoint them at a registry of your own.collectorChartVersion/tempoChartVersionare empty by default: the versions RunR pins. - Build base images from the reqs mirror. The WebAssembly build images, the Spin shim build, hf-serve's GPU image and the GPU images of the pytorch and tensorflow runtimes build on
ghcr.io/brf-tech/runr/reqscopies of their Docker Hub bases (TinyGo, Node.js, Python, Rust, PyTorch, CUDA).ci/check-dockerfiles.sh(CI jobcheck:build-scripts) fails on a Docker Hub base image. - Image builds pull their base images. Every image build of
runtime/ci.sh(local and--push, the release tars ofbuild-deploy) and ofcompose/build.sh/compose/deploy.shruns with--pull, so an older copy of a base image in the build host's Docker is never built on. - Runtime versions from the image registry.
executor:synclists the versions of the built-in runtimes from the tags of their images underghcr.io/brf-tech/runr(the registry's OCI distribution API, read anonymously) instead of Docker Hub'srunrsh/exec-*repositories, so every version published in RunR's image namespace reaches the catalog, PyTorch and TensorFlow included.versionDiscovery.repositorynames the whole image repository (ghcr.io/brf-tech/runr/exec-python); one without a registry host is on Docker Hub (docs/runtimes.md). - Image namespace ghcr.io/brf-tech/runr. RunR publishes every image under
ghcr.io/brf-tech/runr, so the organization's other packages never share a name with RunR's:reqs/<image>,exec-<runtime>,hf-serve,image-upload,containerd-shim-spinkeep their names there,runr-wrapper,runr-agent,runr-queue-consumerandrunr-supply-chainbecomewrapper,agent,queue-consumerandsupply-chain, andrunr/apiandrunr/uistay where they are.runtime/ci.shpushes toREGISTRY=ghcr.io/brf-tech/runrby default,runr devpulls from it, and the system images policy passesghcr.io/brf-tech/runr/**.
Security#
- ML runtimes.
pip-auditfinds no known vulnerability in the packages of the four ML runtime images. The PyTorch 2.3 package set they replace had 20 advisories in torch (among them CVE-2025-32434) and the TensorFlow 2.16 set one in protobuf that TensorFlow 2.16'sprotobuf < 5pin cannot take the fix of; pip and setuptools are upgraded to 26.2.1 and 84.0.0. The PyTorch weights example (examples/ai/hf-python/weights) takes transformers 5.20.
Fixed#
- Registry leftovers. Deleting a runtime deletes its images (
runr-runtimes/<id>) from the registries of the regions it was built in. Builds no longer open an upload before they are built, so a failed build leaves no empty repository, and a daily job hands build repositories without a tag to Zot's garbage collection, which removes them. - Sandbox API e2e. The Kata guest-size check creates its sandboxes with
cpuandmemory, so they have the size whose vCPUs it checks. - Upgrade hook while PostgreSQL restarts. An upgrade that restarts the control plane's PostgreSQL (a new image, or its volume moving to another node) failed in the post-upgrade
runr-bootstraphook, which started before the database accepted connections. The hook now waits up to 4 minutes for the database, and its deadline is 10 minutes. - WebAssembly build images on a docker-container builder. A buildx builder with the docker-container driver (
ci.sh wasm --push, CI) gives every build stepOTEL_EXPORTER_OTLP_TRACES_ENDPOINT=unix:///dev/otel-grpc.sock, and Spin 4.1 exited with "Failed to initialize telemetry: invalid URI". The build steps ofruntime/wasm-*run Spin withOTEL_SDK_DISABLED=true(a build argument: the images do not carry it). --pushexited 1 after a successful push.ci.sh build,queue-consumer,wrapper,build-agent,test-imageandsupply-chainended with a test that failed when pushing; they exit 0 once the push is done.
0.5.1 - 2026-10-16#
Upgrade notes#
- Region capacity admission. Scale-to-zero projects and model services no longer hold the region capacity: each counts one pod against a FaaS overcommit budget of capacity × 3 (CPU) and × 1.5 (memory), set per region. Every existing region gets these defaults; a factor of 1 keeps every reservation against the capacity. Sandbox and model service pods are no longer subtracted as platform pods, so a region's capacity rises by what they request at its next status check (docs/architecture.md).
- Image namespace
ghcr.io/brf-tech. RunR's images are published underghcr.io/brf-tech— the same names, tags and digests as underghcr.io/runrshup to 0.5.0 — and the upgrade moves every reference an existing installation holds (exact prefix only; other registries stay): image values of the Helm release, also the ones--reuse-valueskeeps (chart helperrunr.image); stored references in the central and every tenant schema (migrationmove_images_to_brf_tech_registry,php artisan images:move-registryin the bootstrap job); and the workloads in the regions (region:reconcile). Knative Services get a new revision that takes the traffic once it is ready; the other workloads roll once. The node installers of Kata and gVisor and the region's in-memory queue Redis keep running as they are, a restart would cost more than the move (docs/upgrade.md). - Terraform. The provider's address is
brf-tech/runr: changesourceand runterraform state replace-provider registry.terraform.io/runrsh/runr registry.terraform.io/brf-tech/runronce. - Model images. The defaults move to TEI 1.9.4 (
ai.images.teiCpu…:cpu-1.9.4,ai.images.teiGpu…:{arch}1.9.4) and hf-serve 1.1.0 (ghcr.io/brf-tech/hf-serve:1.1.0,…:1.1.0-cuda). Installations that set these values, for example to a registry mirror of an air-gapped region, update them and mirror the new images: TEI 1.5 can no longer download models from the Hugging Face Hub.
Added#
- Why a model service does not start. While a model service deploys, RunR reads its pods and shows Kubernetes' reason next to the status —
CrashLoopBackOffwith the engine's last output,OOMKilledwith the memory limit,ImagePullBackOff/ErrImagePullwith the image and the registry's answer,Unschedulablewith the scheduler's message — in the console, the API (status_reason, details instatus_message) and the MCP tools (docs/huggingface.md). - Model image pre-pull. Setting up a region's model storage pulls the hf-serve image onto the nodes projects run on, and creating or deploying a model service pulls its engine image onto the nodes it can run on (GPU nodes for GPU services), so a first start no longer waits for a half-gigabyte image (docs/huggingface.md).
- TEI on Blackwell GPUs. Compute capability 10.0, 12.0 and 12.1 get TEI's
100-,120-and121-images; any other GPU getscuda-, which picks its kernels when it starts. TEI services answer with their served name in/v1/embeddings.
Changed#
- Region capacity: always-on and FaaS overcommit budgets. Always-on projects (
min_scale≥ 1 orscale_to_zero: false) reserve (size + queue-proxy + Kata overhead) ×max(1, min_scale)against the region capacity without overcommit, together with the Kata pod overhead of scale-to-zero projects, queue consumers, always-on model services, sandboxes and warm pool pods. Scale-to-zero projects and model services count one pod (size + queue-proxy) against capacity × the region's overcommit factor, minus the always-on reservations. Refusals name the budget and what is still free. - Overcommit settings.
GET/PUT /api/knative-regions/{id}/capacity(regions.view/regions.manage) shows both budgets and setsfaas_cpu_overcommitandfaas_memory_overcommit(1–10); Settings → Regions → (region) → Workers shows and sets them. The region resource carries the factors and acapacityobject;used_cpu_m/used_memory_mbnow include the sandboxes and may exceed the total on an overcommitted region. Capacity alerts measure the fuller budget. Audit eventregion.capacity.update. - Sandbox API. A sandbox that needs a new pod, and every warm pool pod, needs room on the region's always-on budget; otherwise the create is refused with
quota_capacity. - Repository. The repository is github.com/brf-tech/runr: Go module paths
github.com/brf-tech/runr/…, the GitHub Actionbrf-tech/runr/actions/deploy, CLI release downloads and update checks, the Terraform providerbrf-tech/runr(mirror repositorybrf-tech/terraform-provider-runr), chart and SDK metadata. - Images. Images under
ghcr.io/brf-tech: the API and UI, the runtime and WebAssembly build images, the wrapper, region agent, queue consumer, image-upload, supply-chain, hf-serve and Spin shim images and thereqsmirror; the CLI'srunr devand every region component use them. - Region reconcile.
region:reconcilehas two more steps:admission-policybrings the supply-chain admission policy of a region to this release's image globs,image-registrymoves region workloads to RunR's image namespace;traefik-imagealso moves a Traefik of another version fromghcr.io/runrsh, andwasm-sandboxre-runs the WebAssembly installer for its shim image. - Model sizes count the weights an engine loads. A Hub repository that holds one model in several formats (safetensors, PyTorch, TensorFlow, Flax, ONNX, OpenVINO) counts one of them: a 268 MB model gets 1536 MiB on CPU instead of 3328 MiB. Hub models already in the library are sized again the next time a one-click deploy reuses them (docs/huggingface.md).
- Model service threads follow the CPU limit. Model service containers get
OMP_NUM_THREADS/MKL_NUM_THREADS= the CPU limit rounded up, and hf-serve reads its cgroup quota when they are missing: a 1-CPU hf-serve service no longer runs one PyTorch thread per node CPU. - hf-serve 1.1.0 restarts on a failed model load. It writes the reason to the termination log and exits, so the pod retries with back-off and the service status shows why, instead of staying unready.
Security#
- Dependency updates for published advisories: the console's code editor takes monaco-editor 0.57 (its bundled DOMPurify 3.4.15) and the console builds with Tailwind CSS 4 (it looks as before; braces, micromatch, chokidar, fast-glob and postcss-selector-parser leave the build tooling), the workflow SDK tests with Vitest 4 (development needs npm 11), every Go module requires go1.26.9 (GO-2026-6603 … 6617 in net/http, HTTP/2, net/textproto and crypto/tls) and the CLI takes
golang.org/x/sys0.48.npm auditreports one low advisory: DOMPurify'sIN_PLACEmode in the copy bundled with Monaco, which does not use that mode. With Tailwind CSS 4 the console needs Safari 16.4, Chrome 111, Firefox 128 or newer. - hf-serve 1.1.0 (the
transformers/diffusersengine and the mirror and upload helpers): PyTorch 2.5.1 → 2.14.1 (19 advisories, among them CVE-2025-32434), transformers 4.49 → 5.20 (24), Starlette 0.52 → 1.8 with FastAPI 0.143 (5), huggingface_hub 1.33, pip 26.2 and setuptools 84 in the image (8); the GPU image is built on PyTorch 2.14.1 with CUDA 12.6 and diffusers 0.41.pip-auditfinds no known vulnerability in the new image; hf-serve's tests run on pytest 9.1.
Fixed#
- Kata overhead of the deployed RuntimeClass. Capacity counted a project's Kata pod overhead from the region's current sandbox mapping; it now follows the RuntimeClass the project (or model service) was last deployed with, so a mapping change counts from each workload's next deploy.
- Model services and sandboxes counted twice. The capacity refresh subtracted model service and sandbox pods as platform pods while admission counted them as well (a region showed 3800m used of 3405m). They are counted once now, from their records.
- TEI embedding services crash-looped with "relative URL without a base": the Hugging Face Hub answers downloads with a relative redirect that TEI 1.5 cannot follow. RunR runs TEI 1.9.4.
0.5.0 - 2026-10-15#
Upgrade notes#
- Upgrading. Upgrade with
helm upgradeof the running release (--reuse-valuesworks), never uninstall and install again; back up the database and do not edit projects or deploy while the upgrade rolls out (docs/upgrade.md).ci.sh test-helm-upgraderepeats the upgrade frommainon a Kind cluster. - Roles and permissions. In the single-tenant edition the administrator account becomes Owner and every other user Admin (0.4.x had no permission checks); in the multi-tenant edition tenant
owner,adminandviewermap to the built-in roles of the same name, and tenantmembermaps to a custom role Member, created in every tenant that has members: what members could do before (the Developer permissions withoutmodels.deployand the hosted MCP server permissions, plussettings.manageandsso.manage). Tenant viewers, who could change and deploy projects in 0.4.x, are read-only now. API tokens with*are unchanged; tokens with 0.4.x scopes (projects:read,deployments:write, …) are translated to permissions, also when an older client still sends them — and are enforced now, so aprojects:readtoken no longer deploys. Users the previous version creates while the upgrade rolls out get the default role from the bootstrap job (php artisan rbac:assign-missing). See docs/permissions.md. - Secrets at rest. The upgrade migrations encrypt every stored secret in the central and every tenant schema with
APP_KEY. Keep a copy of the key — without it the encrypted columns cannot be read. Rows written in plaintext by an older pod during the rollout are encrypted on their next save, or runphp artisan secrets:encryptonce the rollout is done; after a key change,php artisan secrets:encrypt --rotatere-encrypts with the new key (docs/security.md). - Supply chain. Regions that exist at the upgrade keep SBOM generation and scanning off, so an upgrade never starts Jobs whose tools image a region cannot pull; enable them per region in the region's Supply chain tab. New regions start with both on (docs/supply-chain.md).
- Private broadcast channels. Every WebSocket channel is private. A custom client that subscribes to
platform,notifications,deployment.{projectId}or the ticket channels must authorize the subscription withPOST /api/broadcasting/authand the user's API token; a user's own notifications arrive onnotifications.{userId}(docs/security.md). - SSO code exchange. OAuth, OIDC and SAML sign-ins redirect to
/oauth/callback?code=<one-time code>instead of carrying the API token in the URL. A custom front end exchanges the code (valid for 60 seconds, one use) withPOST /api/oauth/exchange(docs/security.md). - New endpoints on the console domain. The Helm gateway and the Compose Caddyfile route
/git/*,/mcp,/mcp/*,/oauth2/*and/.well-known/oauth-*to the API, streamed. A proxy of your own in front of the console must do the same, without buffering or compressing them (with ingress-nginx:proxy-buffering: "off"and aproxy-read-timeoutof at least 900 throughingress.annotations). The MCP server and its OAuth server, including dynamic client registration, are on by default (MCP_ENABLED,MCP_OAUTH_ENABLED,MCP_OAUTH_OPEN_REGISTRATION);gitDeploy.enabled: falseturns the git remotes and provider webhooks off. - Per-region features. GPU support, WebAssembly support and the data services of bindings are installed per region when you need them; KV, object and Postgres bindings are refused in a region until its data service is ready. Custom runtimes stay off until a holder of
runtimes.manageswitches Custom Runtimes on (Settings → Platform). Existing tenants get no GPU limit (gpu_quotaempty). - Triggers. The
workflowtrigger type runs durable workflows; theeventtrigger type is rejected by validation (422). Existing rows of either type stay readable and deletable. - Billing (multi-tenant). Payment records become charges (
turn_payment_records_into_charges): the billing service's invoice id is kept asexternal_ref, cancelled records are void, records of an ended period are closed with their invoiced (else recorded) amount. RunR no longer calls a billing service to create invoices, poll payment status or refund: your billing system reads closed charges through the billing API or thebilling.period.closedevent and invoices them.billing.gatewayUrlandbilling.gatewayTokenare not used any more;billing.webhookSecretnow signs the billing events sent to the newbilling.webhookUrl./api/tenants/{tenant}/paymentsand/api/tenant/paymentsgive way to/api/billing/chargesand/api/tenant/billing(docs/upgrade.md). - Defaults. The first administrator's default e-mail is
admin@example.com(ADMIN_EMAIL, Helmadmin.email); existing installs keep theirs.api/.env.dockerships without anAPP_KEY(scripts/devready.shgenerates one) andcompose/deploy.shtakes its target fromDEPLOY_SERVER,DEPLOY_USERandDEPLOY_DIR.
Added#
Functions and runtimes#
- Pluggable runtimes (docs/runtimes.md): every language is a plugin — a
runtime.yamlmanifest plus an image — that runs inhttp,execorpersistentmode; inexecandpersistentmode the universal wrapperrunr-wrapperturns HTTP requests, queue CloudEvents and schedule ticks into invocations over stdin / stdout. The built-in runtimes are plugins of the same kind; platform administrators register platform runtimes and organizations custom ones (Settings → Runtimes,/api/runtimes, permissionsruntimes.manage,runtimes.use_custom), with manifest validation, runtime image builds and test deploys. Example runtimes for C, Rust, Bash and the toy language Q inexamples/runtimes;ci.sh test-runtimes. - WebAssembly functions (docs/wasm.md): Rust, Go (TinyGo), JavaScript / TypeScript and Python as Spin components on the runwasi Spin shim. Install in the region's Isolation tab puts the shim on the selected nodes, creates the
wasmtime-spin-v2RuntimeClass and turns on the Knative features Wasm functions need; the runtimeswasm-rust,wasm-go,wasm-jsandwasm-pythonthen appear in that region's runtime picker.ci.sh test-wasm. - RunR's build of the Spin shim, containerd-shim-spin v0.26.0-runr.1 (
ghcr.io/runrsh/containerd-shim-spin,runtime/wasm-shim-spin,ci.sh wasm-shim): upstream v0.26.0 with its runwasi library giving the Linux containers of a Wasm pod — Knative's queue-proxy — the environment of their pod spec, which the upstream releases do not; built from pinned sources and toolchains, Apache-2.0 with LICENSE and NOTICE in the image. Thewasm-installertakes it from the image (init container) and checks it against the image's SHA256SUMS (docs/wasm.md). region:reconcilebrings the shims on the nodes of a region with WebAssembly support up to the release: when the region'swasm-installerinstalls other shim binaries or is missing, it queues the region's WebAssembly install with the same shims and nodes, which replaces the binaries in place (the container runtime restarts only where RunR's configuration block changes).wasm-execruntime (docs/wasm.md): any WASI command — a preview 1 module from C / C++ (wasi-sdk) or Zig, a WASI 0.2 component from Go (TinyGo with wasm-tools), a projectbuild.shor an uploaded.wasm— on Spin's WAGI executor (the request on stdin and in CGI variables, the response on stdout); runtime manifests on the Spin shim may declarewagi. Examplesexamples/wasm/c-execandexamples/wasm/q-exec.- Uploaded images (docs/images.md):
docker savearchives, OCI image-layout tars and.wasmmodules uploaded through Settings → Images,POST /api/images/uploadsorrunr images pushinto a region's registry in resumable chunks — also for regions without internet access. Uploaded images run as CaaS images, as runtime versions (upload:source) and aswasm-execmodules, get the supply-chain pipeline of a build and count against a per-organization storage quota. Permissionsimages.view,images.upload,images.delete; Helm valuesimages.*.
Deploy#
- Git deploy (docs/git-deploy.md): every project has a git remote
https://<console>/git/<organization>/<project>.git(smart HTTP, an API token as password); a push to the deploy branch becomes a version with its commit, appliesrunr.yaml, builds and deploys. Projects connect to repositories on GitHub (GitHub App), GitLab or Forgejo / Gitea with push-to-deploy, commit statuses and a deploy history; monorepos use a root directory. Permissionsgit.pull,git.push,git.manage,git_providers.manage; Helm valuesgitDeploy.*. - Preview environments per pull / merge request:
<project>-pr-<number>on the function domain with environment overrides and scale-to-zero, a PR comment kept up to date, removal on close, merge or after a TTL, a per-project limit and the organization quota; secrets reach forks only when allowed. Permissionpreviews.manage. runrCLI (docs/cli.md): log in, scaffold (runr init), run locally on the runtime images (runr dev, also for runtime plugins) and deploy (runr deploywithrunr.yaml), plus commands for every feature of this release — secret variables, git deploy and previews, runtimes, workflows, bindings, models, sandboxes, uploaded images, traces, uptime checks and alerts, MCP clients and hosted MCP servers, supply-chain reports, data residency, members, roles, groups and the audit log. Releases for Linux, macOS and Windows.- Terraform provider
terraform-provider-runr(docs/terraform.md) for projects, environment variables (write-only secrets withvalue_wo), domains, deployments, bindings, runtimes, workflow triggers, model services, hosted MCP servers, roles, groups, project access and branding. - GitHub Action
actions/deploy(docs/github-action.md): deploys a project with the CLI on push and runs pull request preview environments.
Data#
- Bindings (docs/bindings.md): managed KV (a Redis ACL user and key prefix), object storage (a Garage bucket or an external S3 endpoint), Postgres (CloudNativePG, shared or dedicated cluster) and external credentials, attached to projects with an env prefix and injected through the Secret
<project>-bindings; credential rotation with a rolling restart, deletion with retention and restore, health, usage metering and per-organization quotas. Console: project Bindings tab, Settings → Data, region Data services. Permissionsbindings.view,bindings.manage,bindings.view_credentials,regions.data_services.manage; Helm valuesbindings.*. - Durable workflows (docs/workflows.md): code-first, step-based workflows whose progress survives crashes, redeploys and long sleeps — memoized steps with retries and backoff, sleeps, waits for events, child workflows and events — started by the API, events, cron, queue messages or other workflows. Python (
runr-workflows) and TypeScript (@runr/workflows) SDKs on a language-neutral protocol; a project Workflows tab with run timelines and an Events view. Permissionsworkflows.view,workflows.run,workflows.manage,events.send.
AI#
- GPU workers (docs/ai.md): GPU nodes, products, GPU memory and allocation per region; Install GPU support puts the NVIDIA device plugin (with GPU Feature Discovery and optional time-slicing) or the NVIDIA GPU Operator on a region. Permission
regions.gpu.manage. - GPU workloads:
config.gpu(count,type,memory_min) on CaaS projects and on functions of GPU runtime versions — GPU limits, node selection by product and memory, runc or gVisor with nvproxy — admitted against the region's GPU nodes and the tenant GPU quota, and metered as GPU time in the usage aggregation. - Hugging Face models (docs/huggingface.md): a Hub browser, one-click model services (vLLM, TGI, TEI, RunR's
hf-servefor transformers / diffusers, or a custom image) online or offline, offline copies in the region's model volume (mirror Jobs, chunked uploads and imports for air-gapped regions), a playground, and model bindings that give functions in any languageRUNR_MODEL_<ALIAS>_*environment variables. Permissionsmodels.view,models.deploy,models.manage; examples inexamples/ai. - RunR MCP server (docs/mcp.md): a built-in Model Context Protocol server at
/mcp(Streamable HTTP, sessions, progress for long-running tools) with tools, resources and prompts over the RunR API; the tools a client sees follow its token's permissions and every call is audited. MCP clients connect with an API token or through OAuth 2.1 (dynamic client registration, PKCE, a consent page in the console). Console: Settings → AI agents (MCP). - Hosted MCP servers: any HTTP project served at
/mcp/servers/<slug>behind a gateway with public, RunR-token or OAuth access, a signed caller identity, a call log and an inspector in the project's MCP tab; Python and TypeScript starters as the runtimesmcp-pythonandmcp-node(examples/mcp). Permissionsmcp.servers.manage,mcp.servers.call. - Sandbox API (docs/sandbox-api.md): isolated sandboxes for untrusted code on your own cluster — pods under the region's strongest sandbox runtime (Kata, else gVisor) with Pod Security
restricted, default-deny networking with an optional egress allowlist and resource limits. Exec as JSON, Server-Sent Events or WebSocket, background processes, files, a code interpreter, a console terminal, pause / resume, TTL and idle expiry, warm pools, organization quotas and usage metering. Python (runr-sandbox) and TypeScript (@runr/sandbox) SDKs; permissionssandboxes.view,sandboxes.create,sandboxes.exec,sandboxes.manage; Helm valuessandboxApi.*.
Security and enterprise#
- Roles and permissions (docs/permissions.md): one permission catalog for every API route, built-in roles (Owner, Admin, Developer, Viewer, Billing), custom roles, groups, per-project access and API tokens limited to a selection of their owner's permissions. SSO providers (OIDC, SAML) map identity-provider groups to RunR groups and roles and provision members just in time;
RBAC_DEFAULT_ROLEsets the role of new members. - Encryption at rest for every stored secret (docs/security.md): environment variable values, version snapshots, region kubeconfigs, Redis / registry passwords and S3 credentials, cold-storage credentials, provider credentials and white-label keys, with key rotation through
APP_PREVIOUS_KEYSandphp artisan secrets:encrypt --rotate. Helm valuessecret.appKey,secret.previousAppKeys. - Secret environment variables: write-only values per project, delivered through the Kubernetes Secret
<project>-secret-envandsecretKeyRefentries; reading a value back needssecrets.view_values(runr env set --secret,secret = truein Terraform). - Audit log: an append-only record of sign-ins, account and token changes, project, deploy and rollback, shell, domain, gateway, region, runtime, member, impersonation, settings, billing and SSO actions, branding, signup, plan and billing-integration changes (field names only, never secret values) and authorization denials, with filters, CSV / JSON export, a daily retention prune (Helm
audit.retentionDays, default 365 days) and optional JSON-line forwarding to a SIEM (Helmaudit.siemForwarding). Console: Settings → Audit log; permissionsaudit.view,audit.export. - Software supply chain (docs/supply-chain.md): an SBOM (SPDX and CycloneDX), a vulnerability scan (also with an offline database) and a cosign signature with the region's key for every image RunR builds or receives; a policy gate before deploy, re-scans of deployed images and signed-image admission in the region. WebAssembly functions (Spin apps) are signed and verified like images; they get no SBOM or scan, and the gate does not require one of them. Permissions
security.view,security.override_policy,regions.supply_chain.manage; Helm valuessupplyChain.*. - Data residency: organizations limited to allowed regions and data locations, enforced on projects, deploys, bindings, model services, uploads and pushes (permission
tenants.residency.manage, multi-tenant edition).
Operations#
- Observability (docs/observability.md): an OpenTelemetry Collector and Grafana Tempo per region (or an existing backend), auto-instrumentation of functions, a Traces tab with search and waterfall, request replay of captured failed requests with a response diff, uptime checks run by the region agent, a public status page per organization, and alert channels and rules. Permissions
traces.view,replay.manage,uptime.manage,alerts.manage,status_page.manage,regions.observability.manage; Helm valuesobservability.*. - White-label (docs/white-label.md): the installation's brand (name, logos, colours, login texts, e-mails) without a rebuild and, in the multi-tenant edition, per tenant; self-service signup with e-mail verification, a domain allowlist, invite-only mode, CAPTCHA and terms, a tenant per signup with an onboarding wizard; optional plans (off by default) with monthly / yearly prices, trial days, limits (CPU, memory, projects, GPUs, invocations per month, code storage) and feature flags. Helm values
branding.*,signup.*,plans.*; permissionsbranding.manage,signup.manage. - Billing (docs/billing.md, multi-tenant): RunR calculates what each tenant owes and leaves invoicing and payment to the operator's billing system. Charges per tenant and calendar month — metered usage priced with the cost configuration (functions, model services, bindings, sandboxes), plan fees, capacity deals and one-off charges — close after the month with final line items, the credit they draw and the amount due. The billing API (
/api/billing/…) reads usage of any period, charges, plans and tenants, changes a tenant's plan, records purchases made elsewhere with their reference (credit, prepaid plan terms, add-ons that raise plan limits) and stores the billing system's reference on charges; signed billing events (billing.period.closed,billing.plan.changed,billing.usage.threshold,billing.charge.voided) go to a webhook with retries and stay readable in an event log. Platform administrators connect a system with an API token limited tobilling.vieworbilling.manage; tenants see their plan, limits, charges, credit and purchases in the console and a link to the operator's store. Helm valuesbilling.webhookUrl,billing.webhookSecret,billing.usageThresholds,billing.storeUrl. - Scheduled lifecycle tasks (docs/monitoring.md):
close-stale-sessions(every ten minutes) closes deployment sessions open for more than 30 minutes without a reported end, in the central schema and every tenant schema, and the pod's real end later corrects a session closed too early;capacity-deal-expiry(hourly, multi-tenant) sets active capacity deals past their end date toexpired. - Tests and CI: Vitest and Playwright suites for the console, ESLint, Pint and Larastan for the API, Go tests for the agent, the queue consumer, the wrapper, the CLI, the Terraform provider and the Sentry tunnel, and end-to-end suites on kind (
runtime/ci.sh test-*), run as quality, security and e2e gates in GitLab CI and GitHub Actions; ESLint fails on warnings, and the API and console tests on coverage below their floor (docs/development.md).
Changed#
- Documentation lives in
docs/in English, with an index (docs/README.md) and an architecture overview; MITLICENSE,CONTRIBUTING.md,SECURITY.md,CODE_OF_CONDUCT.md, issue and merge request templates; English comments in the Helmvalues.yaml. - The console, page title, favicon, API docs and image labels carry the RunR name and wordmark.
- API tokens carry permissions of the catalog instead of the 0.4.x scopes (see Upgrade notes).
- The development Compose network is
runr_default. - API caching (docs/architecture.md): the trace endpoints answer
Cache-Control: no-storewithout anETag, so neither the response cache nor a proxy keeps a trace that is still receiving spans; the public status page (public, max-age=30) andGET /api/branding(public, max-age=60) keep their directive next to theETag, so a proxy or CDN may cache them. - Code and file contents (editor saves, runtime files, sandbox file writes, MCP tool arguments) are no longer trimmed by the API: they are stored byte for byte.
- Audit events list the added, removed and changed environment variable names in
metadata.changes[].env_vars, never values (docs/security.md). PUT /api/tenants/{tenant}/security-policydrops repeated region ids and locations instead of refusing the request.- The API test suite sets
memory_limit=1GandAPP_URL=http://localhostitself (api/phpunit.xml), whatever the.envsays. - Text files are stored and checked out with LF on every platform (root
.gitattributes). - The region Traefik runs its image from RunR's mirror (
ghcr.io/runrsh/reqs/traefik:v3.7.4) instead of Docker Hub, like RunR's other third-party images;region:reconcilemoves existing regions running the same Traefik version, andknative.traefikImage(KNATIVE_TRAEFIK_IMAGE) names another registry (docs/upgrade.md). The Bash and Rust example runtimes use the mirroredbash:5.2andrust:1.83-alpine.
Security#
- Cached API responses are served only to the user they were cached for; an unauthenticated request no longer receives a cached project list.
- Every WebSocket channel is private and authorized with the user's permissions; a user's notifications go to their own channel and internal ticket notes to a ticket-manager channel.
- SSO callbacks (OAuth, OIDC, SAML) redirect with a one-time code instead of the API token in the URL.
- Broadcasts mask environment variable values and secret fields; version and deployment broadcasts carry no environment variables; the git snapshot of a version lists environment variable names only; queue payloads with kubeconfigs or credentials and the shell session handoff are encrypted; the SSO client secret and linked-account OAuth tokens are never returned by the API.
- The queue consumer reads the Redis address and password from the Secret
<slug>-qc-redis, and the signing key of a workflows project and the queue bridge's sink URL reach the workload from Kubernetes Secrets, instead of plain values in the spec. - Tenant invitation tokens are stored as SHA-256 hashes; links already sent keep working.
- A tenant's copy of a platform region no longer double-encrypts the Redis and registry passwords.
- The development Sentry tunnel (
proxy/) forwards envelopes only to the host of a configured DSN; without one it refuses them instead of relaying them to whatever host they name. - Dependency updates for published advisories: the API's Composer packages (Laravel 12.70 with Horizon 5.50, whose workers declare the
queue:workoptions Laravel 12.70 added, Guzzle, Symfony 7.4, league/commonmark, phpseclib and others:composer auditreports none),golang.org/x/netand gRPC in the region agent and the Terraform provider, Go 1.26 for the agent, queue consumer, wrapper and test images, and Pillow, FastAPI, Starlette, transformers, huggingface_hub, safetensors and diffusers inhf-serve.TRUSTED_PROXIES=private_rangeskeeps meaning the private, loopback and link-local networks: newer Symfony releases add publicly routable prefixes to their private-subnet list, which RunR no longer follows. - An IPv4-mapped peer address (
::ffff:a.b.c.d, how dual-stack listeners such as Octane / FrankenPHP report IPv4 clients) is matched againstTRUSTED_PROXIESas its IPv4 address: a mapped private peer is a trusted proxy, a mapped public peer is the client, and itsX-Forwarded-For,-Protoand-Hostare not used. - The notification list, its unread count, marking read and deleting cover the signed-in user's own notifications and the shared ones only; a member no longer sees or clears another user's (the result of a job someone else started, each admin's copy of a ticket notification).
- Revoking a session that is not the user's answers 404 and no longer signs out the console that holds that token.
Fixed#
An offline model copy whose Job waits for a model volume that cannot be bound (for example a
ReadWriteManyclaim the local-path provisioner cannot provision) shows Pending with the reason from the claim's events instead of Copying with no message, and so do the offline services waiting for it; the region's model volume status gives the same reason.Dragonball (Kata runtime-rs) guests get 1 vCPU + the pod's CPU limits like Cloud Hypervisor and QEMU guests; with kata-deploy 3.28 they had 1 vCPU whatever the limit. Every workload RunR writes on Dragonball carries the vCPU count as Kata pod annotations, which Dragonball's drop-in enables on containerd nodes (docs/kata-hypervisors.md); existing Dragonball functions get it on their next deploy.
The login page no longer reloads itself endlessly: the console subscribed to the (now private) broadcast channels while signed out, the 401 of the channel authorization sent the browser to the login page again, and the API's rate limit started answering 429. The console subscribes once a session exists.
Installing Knative Serving on a region whose Traefik trusts no proxy (no
ingress_trusted_ips, the default) no longer fails at the Traefik release withports.web.proxyProtocol.trustedIPs: Invalid type. Expected: array, given: object.A build or readiness check that outruns its own 30 s limit (a slow region API server) checks again instead of marking the build or the deployment failed; the overall timeouts are unchanged.
A change the console hears about live (saved code, settings, deployment status, notifications, support tickets) no longer answers 500 while the WebSocket server (Reverb) is unreachable: the change was saved all the same, and a client that retried saved it twice. The failed broadcast is reported in the log.
Migrating a region to a new cluster reinstalls the region's managed Redis there.
A plain HTTP request on the shell or agent WebSocket route is answered with 400 instead of failing.
runr:patch-cluster-observabilitywaits for the Cilium compatibility DaemonSet instead of logging a failure.Log search highlights queries that contain
<,>or&; a container command with repeated or surrounding spaces no longer sends empty arguments.On SQLite (the Valet development setup) the usage aggregation added another usage record of the day on every run instead of updating it, and billing summed them.
Under the Turkish locale, upper-cased labels turned the i of English terms into İ (KNATİVE BÖLGELERİ, BASE DOMAİN, BELLEK (MİB)); the terms keep English capitals now.
The development Sentry tunnel no longer crashes, at every start, on a queued envelope whose DSN has no http(s) scheme; it pauses between retries while Sentry answers 5xx, refuses envelopes over 10 MiB (413) instead of forwarding them cut off, and its health check fails while Redis is down.
Removed#
- Leftover AI screens (DAG visualizer, GPU metrics, agent observability, blueprint editor) and dead code paths of the AI project types retired in 0.4.0; their rows keep a read-only detail page with delete. A deployment of a retired type fails instead of being marked active, and region migrations skip them.
- The duplicate Node.js Sentry tunnel (the Go implementation in
proxy/ships). - The unused setting
KNATIVE_MIN_K8S_MAJOR(Helmknative.minK8sMajor).
[0.4.4] - 2026-10-01#
Added#
- Hardware virtualization check for Kata: a short-lived, unprivileged DaemonSet (non-root, no capabilities, the node's
/devread-only) checks every Kata target node for the CPU'svmx/svmflag and/dev/kvmor the KVM module, and records the result as the node labelrunr.dev/kvm=true|falseand the annotationrunr.dev/kvm-probe. Node Feature Discoverycpu-cpuidlabels are used for nodes RunR has not checked; new nodes are checked in the background. - The console's Kata card shows hardware virtualization on the region's nodes: without a KVM-capable node Kata is shown as unavailable (no Install button, no hypervisor list, not offered in the project-type mapping) and an existing Kata install gets a warning to move its mappings to gVisor or runc; on mixed clusters the card names the nodes Kata pods run on.
Changed#
- Every Kata RuntimeClass also selects
runr.dev/kvm=true, so on mixed clusters Kata pods are only scheduled on nodes that can start them.region:reconcileupdates the nodeSelector of existing RuntimeClasses in place (no restart) and prints a warning for Kata installs on regions without KVM; they are never uninstalled automatically.
Fixed#
- Kata no longer installs "successfully" on nodes that cannot run virtual machines (for example cloud VMs without nested virtualization), where its pods never started: the install API returns 422 for Kata when no target node has KVM, gVisor and runc are unaffected, and a working Kata install stays ready.
[0.4.3] - 2026-10-01#
Fixed#
- Adding a hypervisor (for example Firecracker) to a ready Kata install no longer breaks it when the addition fails before the nodes' Kata configuration or the RuntimeClasses change: the region stays ready with its previous hypervisors and the
kata-*mappings keep working; the install log ends with the error, the job completes with a warning and the console shows it on the ready card. - Firecracker's devmapper setup no longer takes a stale pod's result from an earlier install for its own: the previous
kata-devmapperDaemonSet is deleted with its pods first, and RunR only waits on pods of the current DaemonSet and pod template. - Installing Firecracker again after a rollback or cleanup sets up the devmapper thin pool again instead of failing with "thin-pool runr-thinpool is not active":
runr-devmapper.serviceis stopped and reset before it is removed, and the install restarts it.
[0.4.2] - 2026-10-01#
Changed#
- queue-proxy gets a 1-CPU limit over its 100m request on every Knative Service RunR renders (all project types and triggers, runc, gVisor and Kata), so it can use idle node CPU while a pod starts and under load. Project pods become Burstable instead of Guaranteed; admission, tenant quotas, capacity checks and the organization ceiling count requests only. Revisions from before the upgrade get the limit on their next deploy.
- Kata guests on every hypervisor boot the Kata agent as init instead of systemd; the image's
sysctl.dsettings andRLIMIT_NOFILE1048576 are still applied, and Dragonball gets the cgroup v2 kernel parameters its guest needs. - A change to RunR's Kata configuration drop-ins is written on every node in place by a short-lived DaemonSet instead of running kata-deploy again, so it no longer restarts the container runtime; 0.4.1 installs are recognised and updated without a restart.
- The organization ceiling's ResourceQuota limits
requests.cpuonly (thelimits.cpuentry is dropped);region:reconcileupdates existing regions.
Fixed#
- Region capacity admission checks the effective size (the value sent, else the stored one, else the service type's default) on create, update, deploy, scaling and trigger changes and region moves; projects created or deployed without an explicit size skipped the check before. The region demo checks that all four demo projects fit before creating any of them.
- The Kata and gVisor install log is kept after the install, with the result appended as its last line; the console shows the log of a ready runtime in a collapsed "Install log" section and the whole log on an error.
- Firecracker's devmapper check accepts containerd 2.x config dumps. A failed setup no longer crash-loops its DaemonSet: the error is shown in the region's Kata status, and the thin pool, unit and containerd config of a first setup are removed again without a restart.
- A failed live-progress broadcast (for example while the WebSocket server restarts) no longer fails sandbox installs and other jobs; the install log and the job result are stored either way.
region:reconcileHelm upgrades pin the deployed chart version, read fromhelm list, and leave a release alone when that version cannot be determined, instead of upgrading to the newest chart in the repository.- Single-tenant installs no longer run the Tenancy module's migrations, which logged
Database connection [central] not configuredon every migrate.
[0.4.1] - 2026-09-30#
Added#
- Firecracker (
fc) as a Kata hypervisor: RunR sets up containerd's devmapper snapshotter on the nodes (sparse thin pool under/var/lib/runr/devmapper, boot unit, containerd config) before kata-deploy runs. The Kata card has a thin-pool size field (devmapper_pool_gb), and all four hypervisors (clh,qemu,dragonball,fc) can be installed together. The card states that installing or uninstalling Kata restarts the container runtime on every node and asks for confirmation. region:reconcilebrings existing Kata installs to the current configuration: RuntimeClass-only changes are applied in place, an installer change queues the region's Kata install once and reports the container runtime restarts it causes.
Changed#
- Kata installs: a region is ready only after kata-deploy has finished on every target node (20-minute timeout naming the pending nodes); RunR's runtimes keep the shim, VMM and virtiofsd inside the pod's cgroup with fixed VM sizing; the
kata-<hypervisor>RuntimeClasses declare their pod overhead; unknown hypervisors are rejected (422); uninstall is refused while pods run with a Kata RuntimeClass; node detection covers every node, including dedicated workers. - Projects mapped to a Kata runtime reserve that RuntimeClass's pod overhead per pod in tenant quota admission, region capacity validation and usage reporting.
- Region capacity is node allocatable minus the requests of platform pods, so admission and tenant quotas no longer count CPU and memory that system pods already request; build Job pods count with their project.
- On regions with the internal registry, builds pull their base images through it: the registry also caches
ghcr.io(region:reconcileupdates existing registries). Kaniko is pinned tov1.28.5. - Python runtime images ship precompiled bytecode for the standard library, so a cold start does not compile the modules it imports; the PyTorch and TensorFlow image builds are fixed.
- Queue workers block on Redis instead of sleeping between polls, and build and deployment readiness checks start 1 s after submit and repeat every 2 s (previously 5 s).
Fixed#
- Function dependencies are installed in the executor image the function runs on, so native npm modules, compiled wheels and bytecode match its interpreter; Composer resolves PHP packages for the runtime's PHP version.
- Log retention deletes from the region's own Loki (RunR-managed or reused, also for remote regions) through the Kubernetes API proxy and logs refused deletes; a RunR-managed Loki is configured to accept deletes (
region:reconcileapplies it, restarting Loki once), andkeep_last_n_deploymentsprotects the logs of the newest deployments.
Security#
- The region Traefik trusts PROXY protocol and
X-Forwarded-*headers only from the new region settingingress_trusted_ips(default: nobody); before, any client could set the client IP seen by functions, the gateway rate limit and the API.region:reconcileupdates existing installs. - The API takes the client IP from known proxy hops (
TRUSTED_PROXIES,TRUSTED_PROXY_HOPS; chart valuesclientIp.trustedProxiesandclientIp.proxyHops) instead of trusting every proxy, so a spoofedX-Forwarded-Forchanges neither the client IP nor its rate-limit bucket.
[0.4.0] - 2026-09-30#
Added#
- SAML 2.0 single sign-on: SP-initiated login with SP metadata, login and ACS endpoints (also per tenant), optionally signed AuthnRequests and strict response checks (InResponseTo, single use, destination, audience, issuer, validity window, signed assertion). Configured under Auth Providers; the login page shows a SAML button.
- Custom domains for HTTP-triggered projects: ownership is verified with a TXT record and the routing record is checked; TLS per domain is the region's default certificate, an uploaded PEM pair or a cert-manager certificate from the region's ClusterIssuer. A Domains tab on the project page lists the records to create.
- Private connections from functions to Managed PostgreSQL through Skupper v2 (detected on the region, never installed by RunR): credentials are kept only in a Secret and exposed as
DATABASE_URLandPG*environment variables; new "Private connections" project tab. - API rate limiting per tenant, per user and per IP for unauthenticated calls, plus a login limiter, with JSON 429 responses and
Retry-After/X-RateLimit-*headers (API_RATE_LIMIT_*,AUTH_RATE_LIMIT_PER_MINUTE); shared gateways can set an optional Traefik rate limit. - External billing integration (chart values
billing.gatewayUrl,billing.gatewayToken,billing.webhookSecret; off when empty): closed periods are invoiced through the billing gateway, payment status arrives through a signed webhook, and admins can refund; tenants see their payments and invoices in the console. - Organization resource ceiling: chart values
quota.cpuandquota.memoryGiBbecome a ResourceQuota and LimitRange in each region's project namespace; tenant quotas on a region cannot add up to more than the region capacity or the ceiling. region:reconcile [--region=] [--dry-run], run by the Helm bootstrap job on every upgrade (time-limited, never blocks the upgrade), brings existing regions to the release's region configuration: Knative revision GC and deployment settings, removal of the old Prometheus NodePort, organization ceiling.
Changed#
- Instance sizes: six reference sizes (nano to xlarge) are offered as presets when creating, editing and cloning projects; new XaaS and FaaS projects default to micro (250m / 256 MiB), CaaS to small (500m / 512 MiB). A project reserves its size × max(1,
min_scale) plus queue-proxy (now 100m), its queue consumer or schedule pod, with one formula for quota admission, region capacity and usage; existing projects are grandfathered. - Tenant deletion runs as a queued job (202): it tears down the tenant's projects, domains, labelled workloads and the agent of tenant-owned regions, and drops the database only when every teardown succeeded (or with
force). The tenant list shows "Deleting…" and "Deletion failed" with Retry and Force delete actions; writes to a tenant being deleted return 409.
Removed#
- The AI project types (AaaS, AgentFlow, MaaS) can no longer be created, changed or deployed; existing rows still list and can be deleted. Tenant plan labels are removed from the API, the CLI and the console.
Fixed#
- Registry retention keeps the newest ready builds plus every image still in use (active deployments, live Knative revisions, CronJobs, queue consumers); pruned versions are marked "pruned" and deploying one returns 422.
- Tenant-owned regions store the monitoring and function-TLS settings introduced in 0.3.9, so Prometheus, Loki and cAdvisor are installed on them like on platform regions; optional Prometheus scrape resources no longer fail a Serving install.
Security#
- Platform regions are read-only for tenant users: region-changing endpoints (including Skupper site setup) and the kubeconfig download require a platform admin, or the owner or admin role on a tenant-owned region. Reused Prometheus and Loki URLs must be in-cluster Services of the region's cluster, are read through the region's Kubernetes API proxy, and only platform admins or the tenant owner may set them; the region setup wizard accepts them together with a Loki tenant ID (
X-Scope-OrgID). - Function-domain TLS checks that the certificate matches the key, is unexpired and covers
*.<function domain>; it refuses a Traefik TLSStoredefaultin another namespace and replaces one RunR did not create only on explicit opt-in (a checkbox in the console). Console mode reads the console TLS Secret of the active ingress TLS mode. - The
runr-apiServiceAccount keeps only the two reads it performs (thekube-systemnamespace and the console TLS Secret), and pods that need no token no longer mount one. The generated admin password is no longer written to logs, anddeploy.shinstalls get a generated one.
[0.3.9] - 2026-09-30#
Added#
- Regions can reuse an existing Prometheus and/or Loki instead of installing RunR's monitoring stack: a
prometheus_urlorloki_urlgiven at region creation marks that component as unmanaged. Existing regions keep their RunR-managed stack. - Function-domain TLS per region (
PUT /api/knative-regions/{id}/function-tls): upload a certificate and key for the region's function hosts, served through the region Traefik's default TLSStore with the key kept only in a Kubernetes Secret, or reuse the console certificate when the region runs on the console's cluster. Regions that never use it are unchanged. - Build image retention for the internal registry: after each successful build the newest
registry.retain_versions(default 10) build tags are kept, and a deleted project's images are removed. External registries are never touched.
Fixed#
- Deleting a tenant tears down every project's workloads (Knative Services, CronJobs, queue consumers) and storage before the tenant database is dropped.
Security#
- The control-plane ClusterRole is narrowed to the Knative resource kinds RunR uses (
servicesinserving.knative.devandknativeservings), without wildcards or grants on Knative's internal API groups.
[0.3.8] - 2026-09-30#
Changed#
- Knative Serving keeps a bounded number of non-active revisions (time-based retention with a hard ceiling) instead of every revision.
- queue-proxy runs with CPU and memory request equal to limit, so project pods get Guaranteed QoS; quota and capacity accounting include this per-pod overhead.
Fixed#
- The max scale field in the console accepts values up to 1000, as the API does.
- The region demo's contact form shows the backend's message instead of a generic "Sent!".
Security#
- Prometheus is no longer exposed through an unauthenticated NodePort (30090); RunR reads it in-cluster or through the Kubernetes API proxy and removes the old NodePort Service.
- No default
admin123administrator password: whenADMIN_PASSWORDis unset, a random password is generated and printed once.
[0.3.7] - 2026-09-29#
Fixed#
- Removing the region demo inside a tenant deletes the demo projects instead of failing with an SQL error.
- Seeding the region demo is refused while the region's demo projects still exist, so the demo is never created twice.
[0.3.6] - 2026-09-29#
Added#
- Console TLS modes in the chart (
ingress.tls.mode):letsencrypt(cert-manager ClusterIssuer, defaultletsencrypt-prod),custom(uploaded certificate and key, validated at install) andnone(the ingress controller's default certificate). An empty mode keeps the previous behaviour, also withhelm upgrade --reuse-values. - Chart values
storage.applicationandstorage.databasefor the API and PostgreSQL volume sizes, used by the Advanced settings of marketplace install forms.
Changed#
- New regions get a managed Redis by default, so queue triggers and the region demo work right away;
redis_mode(managed,in_cluster,externalornone) withredis_addressandredis_passwordselects another setup.
Fixed#
- Saving the files of a FaaS project whose handler is not
main.*no longer fails with "Main file (main.php) cannot be deleted". - The region demo's frontend calls its backend through the in-cluster service address (no public DNS or publicly trusted certificate needed), and all demo components use the tenant-aware queue stream name.
[0.3.5] - 2026-09-29#
Added#
- The queue consumer retries failed deliveries with exponential backoff (
RETRY_BACKOFF,RETRY_BACKOFF_MAX), moves a message to a dead-letter stream (DLQ_STREAM, default<stream>-dlq) afterMAX_RETRIES(default 5) attempts, claims the pending messages of restarted consumers and reports its counters onGET /stats.
Changed#
- Project, code and deploy input is validated against Kubernetes limits with clear messages: project name (DNS label of at most 44 characters, unique), cron syntax, environment variable names and size, timeout up to the region's request timeout, CPU quantity and code size (below 1 MB); the console shows the same hints. Deploy-time
cpu,memory_mbandtimeout_secondsoverrides are applied to the workload.
Fixed#
- gVisor installs the sidecar binaries (
gvisor-bin/) that current runsc releases require; gVisor pods failed with "sidecar gvisor_sentry not usable" before. Earlier installs are completed on the next install run. - Uninstalling gVisor, or a failed first install, cleans the nodes: RunR's runsc configuration is removed from containerd (checked with
containerd config dumpbefore the file is replaced), the runtime is restarted only where needed and the runsc binaries are removed; uninstall is refused while gVisor pods run. - Tenant projects on a platform region run with the sandbox runtime the region maps to their project type.
- Deleting a project removes everything it runs (Knative Service, CronJob, queue consumer, code ConfigMaps) whatever its deployment status; a redeploy only removes what the new deployment does not reuse.
- Queued jobs no longer end in unresolved states: jobs run once unless they are safe to repeat, long jobs are not handed to a second worker, and failed deploys, builds and readiness checks mark the deployment or build as failed with the reason.
- Usage metering works without a cost config (zero cost in
BILLING_DEFAULT_CURRENCY, default USD), and one tenant's failure no longer stops the others. - A scheduled task interrupted by a pod restart runs again once its overlap lock expires, after 10 to 120 minutes depending on its frequency, instead of 24 hours.
[0.3.4] - 2026-09-29#
Changed#
registry:rolling-restartrestarts containerd in place (no eviction) when no target node can be drained, for example when RunR runs on every node or the cluster has a single schedulable node; when only some nodes were skipped it prints the--in-placecommand for them.
Fixed#
- The in-cluster agent URL uses the gateway port again; the Kubernetes service-link variable
RUNR_GATEWAY_PORThad turned it into port 0. The settings are nowRUNR_AGENT_GATEWAY_SERVICEandRUNR_AGENT_GATEWAY_PORT. - gVisor installs download the sha512-verified release tarball, as upstream no longer publishes the individual binaries; the installer is ready only after runsc is loaded, and a failed install removes the
gvisorRuntimeClass again. - Each deploy is applied once: the deployment listeners were registered twice, which created an extra Knative revision per deploy and ran teardowns twice.
- Region capacity comes from the schedulable nodes when no dedicated workers are selected, so tenant quotas can be set on such regions; the Serving version is read from the KnativeServing CR for manual and adopted installs.
- Helm install and upgrade run pending tenant migrations in the multi-tenant edition; a failure stops the upgrade before new code runs on an old tenant schema.
[0.3.3] - 2026-09-29#
Added#
imagePullSecretsFromDefaultServiceAccount(on by default): every pod inherits the pull secrets attached to the install namespace'sdefaultServiceAccount, as marketplace installers set them up, also on--reuse-valuesupgrades.imagePullSecretsaccepts names or{name: …}entries and also applies to PostgreSQL and Redis.- Agent connections without public DNS: the agent uses the region setting
agent_ws_url, else the in-cluster gateway when the region is RunR's own cluster, elseRUNR_AGENT_WS_URL, elseAPP_URL. - gVisor and Kata installers support containerd 2.x and k3s / RKE2 (configuration drop-ins or templates, restarting the k3s / RKE2 unit); CRI-O and mixed runtimes are refused up front, and a node's error becomes the region's sandbox status message.
Changed#
registry:rolling-restartnever drains RunR's own node or the only node of a single-node cluster (--in-placerestarts containerd there without cordon or drain); k3s and RKE2 need no restart.- The region Traefik chart is pinned to 40.3.0. When another controller owns the
traefikIngressClass, the region Traefik installs without one, and uninstalling Serving keeps a Traefik that still serves other Ingresses (remove_traefik: trueremoves it anyway).
Fixed#
- The registry mirror works on containerd 2.x and k3s / RKE2 without producing an invalid containerd config (existing duplicate keys are repaired); the Zot chart is pinned and its PVC size and storage class are honoured; cAdvisor uses the distribution's containerd socket.
- Serving counts as ready only with a ready KnativeServing CR;
GET /api/knative-regions/{id}no longer returns 500; failed Knative installs record their error status and log.
Security#
- Each region's agent authenticates with its own random token, of which only the SHA-256 hash is stored; requests without a token or with an unknown or wrong one are rejected. Agents deployed before 0.3.3 are redeployed with
php artisan agent:redeploy, which the chart's bootstrap hook runs on upgrade. - Production installs of the multi-tenant edition no longer seed a test login and demo tenants; the bootstrap hook creates the configured admin unless an administrator already exists.
[0.3.2] - 2026-09-28#
Chart-only release; the application images stay at 0.3.0.
Fixed#
- Marketplace installs create the administrator from the install form: a new
runr-bootstrappost-install / post-upgrade hook seeds it while the users table is empty and changes nothing once any user exists (logins failed with 401 before).bootstrap.enabled: falseturns it off.
[0.3.1] - 2026-09-28#
Chart-only release; the application images stay at 0.3.0.
Changed#
- The
fix-permissionsandfix-pgdatainit containers useghcr.io/runrsh/reqs/busyboxinstead of pullingbusyboxfrom Docker Hub; configurable withinitImage.repositoryandinitImage.tag.
[0.3.0] - 2026-09-28#
Initial tagged release.
Added#
- Projects on Knative Serving in three types: Code Snippet (XaaS, a single file), Code Project (FaaS, multiple files with a web IDE and platform-internal Git versioning) and Container (CaaS, any container image).
- HTTP, schedule (cron) and Redis-stream queue triggers, with scale to zero and configurable minimum and maximum scale.
- Runtimes for Python, Node.js, Go and PHP, plus PyTorch and TensorFlow images; builds run as Kaniko Jobs, and interpreted code without dependencies deploys straight from a ConfigMap.
- Every saved change creates a Version, and any Version can be deployed; deployments have logs, live metrics and a browser shell.
- Regions: connect a Kubernetes cluster by kubeconfig; RunR installs the Knative Operator and Serving, Traefik, Prometheus, Loki and Redis, can place projects on dedicated workers, and moves a region to a new cluster (cluster migration with rollback).
- An internal Zot registry per region (PVC or S3 storage) with a node registry mirror, or an external registry.
- Sandbox runtimes mapped per project type and region: runc by default, gVisor or Kata Containers.
- A runr-agent per region reports metrics, sessions and events; the console updates live over WebSockets.
- Multi-tenant edition: tenants with their own database schema, invitations, per-region quotas, usage metering and cost configuration.
- Accounts with two-factor authentication, session management and API tokens with permissions; sign-in through Google, GitHub, Microsoft, GitLab and OIDC providers.
- API gateways with routes, in-app notifications and support tickets.
- Helm chart with a single-tenant / multi-tenant edition, an install form (
values.schema.json), generated secrets and a Traefik ingress; a Docker Compose setup for development.