16 Commits
Author SHA1 Message Date
mram 482731ed9e Pin compose example to v0.1.8
ci / test (push) Successful in 14s
ci / docker (push) Successful in 1m11s
ci / release (push) Successful in 15s
2026-09-21 17:19:16 +02:00
mram e4bdc92ece Game detection: foreign GPU holders take an external lock hold (GAME_PROCS, GPU_FOREIGN_VRAM_MB) 2026-09-21 17:16:56 +02:00
mram 14120bf4a4 Supervisor probes before spawning: an external server on the port is used, never fought or killed 2026-09-21 16:06:28 +02:00
mram d7566329ae Managed ComfyUI: COMFY_CMD starts it on demand, idle stop frees VRAM (internal/supervise) 2026-09-21 13:48:07 +02:00
mram e43ad02fc4 Ignore staged-update leftovers next to the dev exe 2026-09-21 13:14:33 +02:00
mram 232f5b61f2 Periodic upstream health checks (HEALTH_INTERVAL, default 30s); log down/recovered transitions 2026-09-21 13:14:18 +02:00
mram 5e7a042cad CFG_VER is always a concrete version: dev builds stamp v0.0.0, never "dev" 2026-09-21 12:42:32 +02:00
mram 14c2e30478 Clarify listener wording: Ollama-compatible clients, not Ollama-facing 2026-09-21 12:38:03 +02:00
mram dcdb2cc72e Pin compose example to v0.1.7
ci / test (push) Successful in 13s
ci / docker (push) Successful in 1m7s
ci / release (push) Successful in 16s
2026-09-21 10:04:29 +02:00
mram 44a8e9fbdc CFG_VER/APP_VER in the env file: invalid configs replaced, updates follow APP_VER (dev/stable/pin) 2026-09-21 09:28:29 +02:00
mram 272a3a467d force-update: single-shot with a 30s fail-fast timeout 2026-09-21 09:09:27 +02:00
mram cedf28a809 force-update does not require consumer URLs (ErrNoConsumer sentinel) 2026-09-21 09:05:46 +02:00
mram e46e92bee8 Sample env carries a version marker; install appends settings missing since the writing version 2026-09-21 08:54:47 +02:00
mram cf48796183 Install writes a fully commented sample env file when no config exists 2026-09-21 08:48:51 +02:00
mram dd3187f951 Install sets a default LOG_FILE in the env file on Windows (no console as a service) 2026-09-21 08:37:25 +02:00
mram 1394a76eea Make service install converge: stop running service, refresh binary/config only on change, restart only if it was running 2026-09-21 08:30:55 +02:00
27 changed files with 2322 additions and 151 deletions
+3
View File
@@ -4,3 +4,6 @@
/compose.yml
/signing/
/gpu-turnstile.exe.old
/gpu-turnstile.exe.new
/gpu-turnstile.exe~
+80 -12
View File
@@ -2,9 +2,10 @@
GPU arbitration proxy for Ollama + ComfyUI. One consumer GPU is shared by an
LLM server (Ollama) and an image generator (ComfyUI); gpu-turnstile sits in
front of both and guarantees the GPU is always in exactly one of three states:
`idle`, `llm` (N ≥ 1 Ollama requests in flight), or `image` (exactly one
ComfyUI job, Ollama models unloaded). See [SPEC.md](SPEC.md) for the full
front of both and guarantees the GPU is always in exactly one of four states:
`idle`, `llm` (N ≥ 1 Ollama requests in flight), `image` (exactly one
ComfyUI job, Ollama models unloaded), or `external` (a foreign process such
as a game holds the GPU). See [SPEC.md](SPEC.md) for the full
design.
gpu-turnstile listens on the ports the services normally use; the actual
@@ -32,8 +33,9 @@ Open WebUI / n8n ────► :8188 ───┘
Each consumer is enabled by setting its URL (`OLLAMA_URL`, `COMFY_URL`) and
disabled by leaving it empty — at least one is required. With only Ollama
the proxy is a pass-through (no image jobs can arrive); with only ComfyUI
the Ollama unload/warm steps are skipped. Future consumers (e.g. local game
detection) plug into the same lock the same way.
the Ollama unload/warm steps are skipped. A third, URL-less consumer —
detection of foreign GPU holders such as games — is enabled by `GAME_PROCS`
and/or `GPU_FOREIGN_VRAM_MB` (see below).
## Configuration
@@ -45,8 +47,8 @@ override file values. Invalid values fail at startup.
| Var | Default | Meaning |
|---|---|---|
| `LISTEN_OLLAMA` | `:11434` | Ollama-facing listener |
| `LISTEN_COMFY` | `:8188` | ComfyUI-facing listener |
| `LISTEN_OLLAMA` | `:11434` | Listener for Ollama-compatible clients |
| `LISTEN_COMFY` | `:8188` | Listener for ComfyUI clients |
| `OLLAMA_URL` | _(empty = disabled)_ | Ollama upstream; set to enable the Ollama consumer |
| `COMFY_URL` | _(empty = disabled)_ | ComfyUI upstream; set to enable the ComfyUI consumer |
| `UNLOAD_TIMEOUT` | `60s` | Wait for Ollama to unload before an image job |
@@ -56,12 +58,21 @@ override file values. Invalid values fail at startup.
| `LLM_BUSY_STATUS` | `503` | HTTP status for rejected LLM requests in reject mode (400599, e.g. 429) |
| `BUSY_RETRY_AFTER` | `30` | Seconds sent as `Retry-After` on busy responses (both modes) |
| `WARM_MODEL` | _(empty)_ | Model to reload after an image job (off by default) |
| `COMFY_CMD` | _(empty = unmanaged)_ | Supervise ComfyUI: start on demand, stop when idle to free VRAM. Requires `COMFY_URL` |
| `COMFY_DIR` | _(empty)_ | Working directory for `COMFY_CMD` |
| `COMFY_IDLE_TIMEOUT` | `5m` | Stop the managed ComfyUI after this long idle |
| `COMFY_START_TIMEOUT` | `2m` | Max wait for the managed ComfyUI to come up |
| `GAME_PROCS` | _(empty = disabled)_ | Process names (comma-separated); while any runs, the GPU counts as held: requests wait, Ollama unloads, managed ComfyUI stops |
| `GPU_FOREIGN_VRAM_MB` | `0` (disabled) | Also treat the GPU as held when a non-ignored process uses more VRAM than this (needs nvidia-smi) |
| `GPU_IGNORE_PROCS` | `ollama,ollama app,ollama_llama_server,python,pythonw` | Process names never counted as foreign GPU users |
| `GAME_POLL_INTERVAL` | `5s` | How often game/VRAM detection runs |
| `LOGLEVEL` | `warn` | `info` logs every request (colored arrows in text mode), `debug` adds lock transitions. `LOG_LEVEL` works as an alias |
| `LOG_FORMAT` | `text` | `json` for structured JSON logs |
| `LOG_FILE` | _(empty)_ | Append logs to this file instead of stderr |
| `UNLOAD_POLL_INTERVAL` | `500ms` | `/api/ps` poll interval while unloading |
| `HISTORY_POLL_INTERVAL` | `1s` | `/history/<id>` poll interval while a job runs |
| `PROBE_TIMEOUT` | `5s` | Startup probe of both upstreams |
| `PROBE_TIMEOUT` | `5s` | Startup probe of both upstreams (also per-probe health check timeout) |
| `HEALTH_INTERVAL` | `30s` | Periodic upstream probe; down/recovered changes are logged |
| `FREE_TIMEOUT` | `30s` | `POST /free` call after an image job |
| `WARM_TIMEOUT` | `2m` | Warm-model reload after an image job |
| `SHUTDOWN_TIMEOUT` | `10s` | Graceful shutdown on SIGINT/SIGTERM |
@@ -72,10 +83,11 @@ override file values. Invalid values fail at startup.
| `UPDATE_INTERVAL` | `6h` | Auto-update check interval |
| `UPDATE_REPO` | `https://git.rambossek.at/PUBLIC/gpu-turnstile` | Repository checked for releases |
| `UPDATE_ASSET` | `gpu-turnstile.exe` | Release asset to download |
| `APP_VER` | `stable` | `dev` disables updates, `stable` tracks latest, or pin an exact `vX.Y.Z` |
## Observability
- `GET /healthz` (both listeners): `{"state":"idle|llm|image","llm_inflight":N,"image_pending":B}`
- `GET /healthz` (both listeners): `{"state":"idle|llm|image|external","llm_inflight":N,"image_pending":B}`
- `GET /metrics` (both listeners): Prometheus text format — `gpu_turnstile_state`,
`gpu_turnstile_llm_inflight`, `gpu_turnstile_image_pending`,
`gpu_turnstile_image_jobs_total`, `gpu_turnstile_lock_wait_seconds`
@@ -87,6 +99,52 @@ override file values. Invalid values fail at startup.
class) in text mode, which renders in `docker compose logs` on Windows
Terminal. Set `NO_COLOR` to disable colors.
## Managed ComfyUI (`COMFY_CMD`)
Don't want ComfyUI running 24/7 (it holds VRAM even when idle — and the
Desktop app kills its server when you close it)? Point `COMFY_CMD` at a
standalone launch command and gpu-turnstile supervises it: the first
request starts it, it stops again after `COMFY_IDLE_TIMEOUT` (default 5m)
without work, freeing the GPU for games or the LLM. Example for a Desktop
install (run it once manually to confirm it works):
```
COMFY_URL=http://127.0.0.1:8188
COMFY_CMD="C:\ComfyUI\.venv\Scripts\python.exe ComfyUI\main.py --port 8188"
COMFY_DIR=C:\ComfyUI
```
`POST /prompt` waits for the server to answer before taking the GPU lock
(LLM traffic keeps flowing while torch loads); crashes are logged and the
next request respawns. Shutting gpu-turnstile down stops the child too.
Running the ComfyUI Desktop app alongside is safe: if something already
answers on the port, gpu-turnstile just uses it instead of spawning
(and never kills it — it only ever stops its own child). If the managed
instance already holds the port when you open the desktop app, the
desktop's server is the one that fails to bind.
## Game detection
Want to game on the same GPU without Ollama/ComfyUI squatting on the VRAM?
gpu-turnstile can watch for foreign GPU holders and, while one is active,
make LLM/image requests wait (or 503, per `LLM_BUSY_MODE`), unload Ollama's
models and stop the managed ComfyUI so the game gets the memory. Two
detection paths, each optional, polled every `GAME_POLL_INTERVAL` (5s):
```
GAME_PROCS=cyberpunk2077.exe,bg3.exe # the reliable way on Windows
GPU_FOREIGN_VRAM_MB=1024 # catch-all via nvidia-smi
```
`GAME_PROCS` matches running process names (case-insensitive, `.exe`
optional). `GPU_FOREIGN_VRAM_MB` asks nvidia-smi which processes hold GPU
memory and treats anything not in `GPU_IGNORE_PROCS` above the threshold as
foreign — handy as a catch-all, but note that under Windows' WDDM driver
graphics-only games may not show up in nvidia-smi's per-process list, so
name your games in `GAME_PROCS` there; on Linux both paths work. When the
game exits, requests resume automatically.
## Build and run
```sh
@@ -112,12 +170,19 @@ gpu-turnstile.exe --remove-service
```
Layout: `C:\Program Files\gpu-turnstile\` holds the exe and
`gpu-turnstile.env`, logs go to `C:\ProgramData\gpu-turnstile\` (set
`LOG_FILE` in the env file — there is no console). The service always runs
`gpu-turnstile.env`, logs go to `C:\ProgramData\gpu-turnstile\`. If you
install without a config, the installer writes a sample env file with every
setting commented and explained — only `LOG_FILE` is active (a service has
no console). Your own `LOG_FILE` setting is always kept. The service always
runs
as the virtual account `NT SERVICE\gpu-turnstile` (low-privilege,
per-service, no password); the installer automatically grants it write
access to the install and data directories — nothing else to do.
Re-running `--install-service` is safe: it stops a running service,
replaces the installed binary only if it changed, fixes the registration
only where it drifted, and restarts the service only if it was running.
### Run natively on Linux (systemd)
The same binary works on Linux. Install it as a systemd service as root:
@@ -144,7 +209,10 @@ containers and interactive shells.
release on startup and every `UPDATE_INTERVAL`, verifies the Ed25519
signature of the download against the public key embedded at build time,
and — once the GPU lock is idle — restarts the service onto the new
version. Disable with `AUTO_UPDATE=false`. Releases are signed by CI with
version. `APP_VER` controls the target: `dev` disables updates, `stable`
(the default) tracks the latest release, and an exact `vX.Y.Z` pins that
release (even as a downgrade or to replace a dev build). Disable entirely
with `AUTO_UPDATE=false`. Releases are signed by CI with
OpenSSL; the matching public key lives in `internal/update/pubkey.go`
(one-time setup: `openssl genpkey -algorithm ed25519 -out private.pem`,
`openssl pkey -in private.pem -pubout -out public.pem`; private key goes
+125 -22
View File
@@ -10,11 +10,12 @@ becomes very slow; on Linux it would OOM instead.
## Goal
A single Go binary that sits in front of **both** services and guarantees that
at any moment the GPU is in exactly one of three states:
at any moment the GPU is in exactly one of four states:
- `idle` — nothing in flight
- `llm` — N ≥ 1 Ollama inference requests in flight (concurrency allowed)
- `image` — exactly one ComfyUI job in flight, Ollama models unloaded
- `external` — a foreign process (e.g. a game) holds the GPU; new work waits
Clients (LiteLLM, Open WebUI, n8n) point at gpu-turnstile instead of at the
services. gpu-turnstile is transparent for everything that does not touch the
@@ -53,9 +54,10 @@ consumer gets no listener, no startup probe, and no lock participation:
blocks since no image jobs can arrive.
- **Only `COMFY_URL`**: image jobs are tracked and ComfyUI's VRAM is freed
afterwards, but the Ollama unload and warm-reload steps are skipped.
- Future consumers (e.g. detecting a local game holding VRAM) plug into the
same lock the same way: enabled by their config knob, excluded when
absent.
- **Game detection** is a third, optional consumer without a URL: enabled by
`GAME_PROCS` and/or `GPU_FOREIGN_VRAM_MB` it watches for foreign processes
holding the GPU (see below) and plugs into the same lock the same way —
excluded when both knobs are unset.
### Lock semantics
@@ -75,6 +77,11 @@ are LLM requests and the single "writer" is an image job):
requests start), waits until n == 0, sets state := `image`. Released after
the ComfyUI job finished and models were freed.
- Concurrent image jobs queue FIFO behind each other.
- **External hold**: game detection calls `SetExternal(holder)` while a
foreign process holds the GPU. New LLM and image grants block (same busy
handling as above) until `ClearExternal()`; in-flight work is not
preempted, it drains. Snapshot reports state `external` once nothing else
is in flight.
- All waits are context-aware: a client that disconnects while waiting is
removed from the queue.
@@ -121,6 +128,68 @@ state is `idle`, send `POST /api/generate {"model":WARM_MODEL,"keep_alive":-1}`
with empty prompt to reload the chat model so the next chat doesn't pay the
load time. Off by default.
### Managed ComfyUI (`COMFY_CMD`)
When `COMFY_CMD` is set, gpu-turnstile runs ComfyUI as a supervised child
process instead of expecting an always-on server:
- **Start on demand**: any ComfyUI request spawns it (double quotes in the
command line group arguments with spaces; `COMFY_DIR` sets the working
directory). `POST /prompt` additionally waits for readiness
(`/system_stats`) for up to `COMFY_START_TIMEOUT` *before* taking the GPU
lock, so LLM traffic flows while torch loads. Other requests are bridged
by the normal retry backoff. Spawn failure or a readiness timeout
answers 502.
- **Idle stop**: after `COMFY_IDLE_TIMEOUT` without requests or finished
jobs — and only while the GPU lock is idle — the process tree is killed,
freeing the VRAM ComfyUI holds. The next request restarts it.
- **Crash**: an unexpected exit is logged; the next request respawns.
gpu-turnstile's own shutdown stops the child too.
- **Coexistence**: before spawning, the URL is probed — if another server
already answers (e.g. the ComfyUI desktop app), it is used as-is and no
child is spawned; the idle watcher and shutdown only ever stop the
supervisor's own process, never the external one. The other direction —
starting the desktop app while the managed instance holds the port —
makes the *desktop* server fail to bind; gpu-turnstile is unaffected.
- Its stdout/stderr is forwarded to the log at INFO. The health check
skips the intentionally-stopped/starting states; a failed probe while
the process is alive and was previously ready is logged as DOWN.
## Game detection (foreign GPU holders)
Games and other foreign GPU users sit outside the URL-based consumer model —
nothing proxies through gpu-turnstile for them. Two independent detection
paths, polled every `GAME_POLL_INTERVAL` (default 5 s); either one being
configured enables the feature:
- **Process watch list** (`GAME_PROCS`, comma-separated, case-insensitive,
`.exe` optional): while any listed process runs, the GPU counts as held.
This is the reliable path on Windows.
- **Foreign VRAM threshold** (`GPU_FOREIGN_VRAM_MB`): `nvidia-smi
--query-compute-apps` lists per-process GPU memory; any process not in
`GPU_IGNORE_PROCS` (default: Ollama and python — ComfyUI runs under python)
holding more than the threshold counts as a foreign holder. Needs
nvidia-smi on the PATH (absent: logged once, path disabled) and works best
on Linux — under Windows' WDDM driver, graphics-only games may not appear
in the per-process list.
While a holder is detected, gpu-turnstile:
1. takes the external hold on the lock (`SetExternal`), so new LLM/image
requests wait or are rejected per `LLM_BUSY_MODE`; busy responses name the
holder,
2. once in-flight work has drained, **frees VRAM for the foreign process**:
the managed ComfyUI is stopped (an external server on its port is never
touched) and Ollama's resident models are unloaded,
3. refuses to spawn the managed ComfyUI; ComfyUI requests that would need a
spawn are answered 503 + `Retry-After` (a running desktop instance keeps
being proxied),
4. logs the transitions at WARN ("GPU held by an external process …" /
"released the GPU; resuming") and reports state `external` in `/healthz`
and `/metrics`.
When the holder disappears, the hold is lifted and queued requests proceed.
## Configuration (env)
Configuration comes from environment variables and/or an `.env`-style
@@ -131,8 +200,8 @@ override file values. A missing file is fine; a malformed one is fatal.
| Var | Default | Meaning |
|---|---|---|
| `LISTEN_OLLAMA` | `:11434` | Ollama-facing listener |
| `LISTEN_COMFY` | `:8188` | ComfyUI-facing listener |
| `LISTEN_OLLAMA` | `:11434` | listener for Ollama-compatible clients |
| `LISTEN_COMFY` | `:8188` | listener for ComfyUI clients |
| `OLLAMA_URL` | _(empty = disabled)_ | Ollama upstream; set to enable the Ollama consumer |
| `COMFY_URL` | _(empty = disabled)_ | ComfyUI upstream; set to enable the ComfyUI consumer |
| `UNLOAD_TIMEOUT` | `60s` | wait for Ollama to unload |
@@ -142,12 +211,21 @@ override file values. A missing file is fine; a malformed one is fatal.
| `LLM_BUSY_STATUS` | `503` | HTTP status for rejected LLM requests in reject mode (400599, e.g. 429) |
| `BUSY_RETRY_AFTER` | `30` | seconds sent as `Retry-After` on busy responses (both modes) |
| `WARM_MODEL` | `` | optional model to reload after an image job |
| `COMFY_CMD` | _(empty = unmanaged)_ | spawn and supervise ComfyUI on demand: first request starts it, idle stop after `COMFY_IDLE_TIMEOUT` frees its VRAM. Requires `COMFY_URL` |
| `COMFY_DIR` | `` | working directory for `COMFY_CMD` |
| `COMFY_IDLE_TIMEOUT` | `5m` | stop the managed ComfyUI after this long without requests or jobs |
| `COMFY_START_TIMEOUT` | `2m` | how long a request waits for the managed ComfyUI to come up |
| `GAME_PROCS` | _(empty = disabled)_ | comma-separated process names (case-insensitive, `.exe` optional); while any runs, the GPU counts as held by it: requests wait, Ollama unloads, the managed ComfyUI stops |
| `GPU_FOREIGN_VRAM_MB` | `0` (disabled) | also treat the GPU as held when a process not in `GPU_IGNORE_PROCS` uses more VRAM than this; needs nvidia-smi |
| `GPU_IGNORE_PROCS` | `ollama,ollama app,ollama_llama_server,python,pythonw` | process names never counted as foreign GPU users |
| `GAME_POLL_INTERVAL` | `5s` | how often game/VRAM detection runs |
| `LOGLEVEL` | `warn` | `info` logs every request (colored arrows in text mode), `debug` adds lock transitions. `LOG_LEVEL` is accepted as an alias |
| `LOG_FORMAT` | `text` | `json` for structured JSON logs |
| `LOG_FILE` | `` | append logs to this file instead of stderr (useful as a service) |
| `UNLOAD_POLL_INTERVAL` | `500ms` | `/api/ps` poll interval while unloading |
| `HISTORY_POLL_INTERVAL` | `1s` | `/history/<id>` poll interval while a job runs |
| `PROBE_TIMEOUT` | `5s` | startup probe of both upstreams |
| `PROBE_TIMEOUT` | `5s` | startup probe of both upstreams (also the per-probe health check timeout) |
| `HEALTH_INTERVAL` | `30s` | periodic probe of enabled upstreams; status changes (down/recovered) are logged |
| `FREE_TIMEOUT` | `30s` | `POST /free` call after an image job |
| `WARM_TIMEOUT` | `2m` | warm-model reload after an image job |
| `SHUTDOWN_TIMEOUT` | `10s` | graceful shutdown on SIGINT/SIGTERM |
@@ -158,6 +236,8 @@ override file values. A missing file is fine; a malformed one is fatal.
| `UPDATE_INTERVAL` | `6h` | auto-update check interval |
| `UPDATE_REPO` | `https://git.rambossek.at/PUBLIC/gpu-turnstile` | repository to check for releases |
| `UPDATE_ASSET` | `gpu-turnstile.exe` | release asset to download |
| `APP_VER` | `stable` | version to run: `dev` disables updates, `stable` tracks the latest release, or an exact `vX.Y.Z` pin (up- or downgraded to) |
| `CFG_VER` | _(installer-managed)_ | config format reference written by `--install-service` (always a concrete `vX.Y.Z`; a dev build stamps `v0.0.0`); missing = the file is replaced with a fresh sample (backup `.bak`) |
Startup fails fast on unparsable values and when neither consumer URL is
set. Enabled upstreams are probed once at start (`/api/version`,
@@ -176,12 +256,27 @@ prompt instead of failing — the command relaunches itself elevated, waits
for the child, and mirrors its exit code. The legacy form
`gpu-turnstile service install|remove` does the same thing.
Re-running install on an already-registered service converges instead of
failing: a running service is stopped first, the installed binary copy is
refreshed only when the content differs, the registration (Windows service
config / systemd unit) is updated only where it drifted, and the service is
started again only if it was running before.
By default install creates the canonical layout and copies the binary into
it (Windows: `%ProgramFiles%\gpu-turnstile\`, plus
`%ProgramData%\gpu-turnstile\` for logs; Linux: `/var/lib/gpu-turnstile/`
with the config at `/etc/gpu-turnstile.env`). An existing config in the
target location is never overwritten. `--no-copy` registers the current
executable location as-is instead.
with the config at `/etc/gpu-turnstile.env`). If there is no config at all,
install writes a sample env file covering every setting — each with a
comment line, everything commented out — except the `CFG_VER`/`APP_VER`
header and `LOG_FILE`, which is active on Windows
(`%ProgramData%\gpu-turnstile\gpu-turnstile.log`) since a service has no
console; on Linux it stays commented because stderr goes to the journal.
The first line is `CFG_VER=vX.Y.Z`, recording the installer version. When a
later version's install finds an older `CFG_VER`, it appends every setting
the file does not mention (commented or not) at the end and updates
`CFG_VER`; a file without `CFG_VER` is invalid and gets replaced by a fresh
sample, with the old content kept as `<file>.bak`. `--no-copy` registers
the current executable location as-is and leaves the config untouched.
### Windows
@@ -189,8 +284,9 @@ executable location as-is instead.
`%ProgramData%\gpu-turnstile\`, copies the exe and (if none exists there
yet) the `gpu-turnstile.env` into the Program Files directory, and
registers that copy as a Windows service; recovery actions restart it
5 s after any failure. Logs go to the ProgramData directory via
`LOG_FILE` since there is no console.
5 s after any failure. The install ensures the env file sets `LOG_FILE`
to `%ProgramData%\gpu-turnstile\gpu-turnstile.log` since there is no
console — an existing `LOG_FILE` setting is kept.
- **Account**: the service always runs as the virtual account
`NT SERVICE\gpu-turnstile` — a per-service low-privilege identity the
SCM manages (no password, automatic logon-as-a-service right, no admin
@@ -238,19 +334,25 @@ executable location as-is instead.
`RestartSec=5s` brings up the staged binary after the updater exits with
code 3.
- **Auto-update**: on startup and every `UPDATE_INTERVAL`, the binary
checks `UPDATE_REPO`'s latest release; if its tag is a newer `vX.Y.Z`,
it downloads `UPDATE_ASSET` plus its `.sig` (and `.sha256` when present)
and verifies an Ed25519 signature against the public key embedded in
consults `APP_VER`: `dev` disables updates; `stable` (the default)
fetches `UPDATE_REPO`'s latest release and applies it when its tag is a
newer `vX.Y.Z` (a `dev` binary cannot be compared and is replaced by the
latest release); a `vX.Y.Z` pin fetches that exact tag and stages it on
any difference, including downgrades. Applying means downloading
`UPDATE_ASSET` plus its `.sig` (and `.sha256` when present) and verifying
an Ed25519 signature against the public key embedded in
`internal/update/pubkey.go`. A verified binary is swapped in next to the
running exe (rename-aside, allowed on Windows), and once the GPU lock is
idle the process exits with code 3 so the service recovery restarts it
on the new version. Interactive runs only log "restart to apply".
`dev` builds and builds without an embedded public key never update.
- **`--force-update`** runs the same check immediately: it downloads,
verifies and stages a newer release, and if the service is running it
restarts it right away (otherwise the new version applies on next
start). On Windows it elevates via UAC only when the stage or restart
needs permissions the caller does not have.
Builds without an embedded public key never update.
- **`--force-update`** runs the same check immediately, single-shot: one
attempt with a 30 s timeout, then exit — "up to date" (exit 0) or the
error (exit 1), no retries. When a newer release is found it downloads,
verifies and stages it, and if the service is running it restarts it
right away (otherwise the new version applies on next start). On Windows
it elevates via UAC only when the stage or restart needs permissions the
caller does not have.
- **Signing setup (one time)**: `openssl genpkey -algorithm ed25519 -out
private.pem`; `openssl pkey -in private.pem -pubout -out public.pem`.
Private key → repo secret `RELEASE_SIGNING_KEY`; public key → committed into
@@ -334,7 +436,8 @@ are new.
receives the first chunk before the last is sent (no buffering).
- `internal/config`: env-file parsing, precedence, fail-fast values.
- `internal/update`: fake Gitea releases API; staged update happy path,
tampered signature rejected, older versions and dev builds skipped.
tampered signature rejected, older versions skipped, APP_VER=dev and
pinned releases honored.
## Build and CI
+205 -11
View File
@@ -21,11 +21,13 @@ import (
"gpu-turnstile/internal/comfy"
"gpu-turnstile/internal/config"
"gpu-turnstile/internal/game"
"gpu-turnstile/internal/lock"
"gpu-turnstile/internal/metrics"
"gpu-turnstile/internal/ollama"
"gpu-turnstile/internal/proxy"
"gpu-turnstile/internal/service"
"gpu-turnstile/internal/supervise"
"gpu-turnstile/internal/update"
)
@@ -299,6 +301,9 @@ func forceUpdateCommand(configPath string, elevatedChild bool) int {
defer waitForEnter()
}
cfg, err := loadMergedConfig(configPath)
if errors.Is(err, config.ErrNoConsumer) {
err = nil // update settings do not depend on a consumer URL
}
if err != nil {
fmt.Fprintf(os.Stderr, "%s\n\ngpu-turnstile: %v\n", versionLine(), err)
return 1
@@ -310,9 +315,15 @@ func forceUpdateCommand(configPath string, elevatedChild bool) int {
}
log, _, logCloser := newLogger(cfg)
defer logCloser.Close()
u := &update.Updater{Repo: cfg.UpdateRepo, Asset: cfg.UpdateAsset, Version: version, Log: log}
if cfg.AppVersion == "dev" {
fmt.Printf("%s: APP_VER=dev, updates disabled\n", versionLine())
return 0
}
u := &update.Updater{Repo: cfg.UpdateRepo, Asset: cfg.UpdateAsset, Version: version, Desired: cfg.AppVersion, Log: log}
ctx, cancel := context.WithTimeout(context.Background(), 2*time.Minute)
// Single-shot: one attempt, fail fast when the server is unreachable
// instead of hanging in a TCP connect for minutes.
ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
defer cancel()
staged, err := u.Check(ctx, exePath)
if err != nil && isPermission(err) && !service.Elevated() {
@@ -381,7 +392,7 @@ func serviceCommand(configPath string, install, noCopy, elevatedChild bool) int
if abs, absErr := filepath.Abs(path); absErr == nil {
path = abs
}
err = service.Install(path, !noCopy)
err = service.Install(path, !noCopy, version)
} else {
err = service.Remove()
}
@@ -419,6 +430,7 @@ func run(ctx context.Context, cfg config.Config, log *slog.Logger, logOut io.Wri
"unload_poll_interval", cfg.UnloadPollInterval,
"history_poll_interval", cfg.HistoryPollInterval,
"probe_timeout", cfg.ProbeTimeout,
"health_interval", cfg.HealthInterval,
"free_timeout", cfg.FreeTimeout,
"warm_timeout", cfg.WarmTimeout,
"shutdown_timeout", cfg.ShutdownTimeout,
@@ -426,6 +438,14 @@ func run(ctx context.Context, cfg config.Config, log *slog.Logger, logOut io.Wri
"backoff_max", cfg.BackoffMax,
"prompt_capture_limit", cfg.PromptCaptureLimit,
"warm_model", cfg.WarmModel,
"comfy_cmd", cfg.ComfyCmd,
"comfy_dir", cfg.ComfyDir,
"comfy_idle_timeout", cfg.ComfyIdleTimeout,
"comfy_start_timeout", cfg.ComfyStartTimeout,
"game_procs", cfg.GameProcs,
"gpu_foreign_vram_mb", cfg.GPUForeignVRAMMB,
"gpu_ignore_procs", cfg.GPUIgnoreProcs,
"game_poll_interval", cfg.GamePollInterval,
"auto_update", cfg.AutoUpdate,
"update_interval", cfg.UpdateInterval,
"update_repo", cfg.UpdateRepo,
@@ -452,12 +472,31 @@ func run(ctx context.Context, cfg config.Config, log *slog.Logger, logOut io.Wri
}
}
// With COMFY_CMD set, ComfyUI runs as a managed child: started on
// demand by the proxy, stopped after COMFY_IDLE_TIMEOUT idle (and on
// shutdown) so its VRAM is freed.
var comfySup *supervise.Process
if cfg.ComfyCmd != "" {
var err error
comfySup, err = supervise.New("comfy", cfg.ComfyCmd, cfg.ComfyDir, comfyClient.Probe, cfg.ComfyStartTimeout, log)
if err != nil {
return err
}
defer comfySup.Stop()
gpuIdle := func() bool {
state, _, pending := lk.Snapshot()
return state == lock.StateIdle && !pending
}
go comfySup.WatchIdle(ctx, cfg.ComfyIdleTimeout, gpuIdle)
}
srv, err := proxy.New(proxy.Config{
OllamaURL: cfg.OllamaURL,
ComfyURL: cfg.ComfyURL,
Lock: lk,
Ollama: ollamaClient,
Comfy: comfyClient,
ComfySup: comfySup,
Metrics: metrics.New(),
Log: log,
LogColor: !cfg.LogJSON && cfg.LogFile == "" && os.Getenv("NO_COLOR") == "",
@@ -481,19 +520,52 @@ func run(ctx context.Context, cfg config.Config, log *slog.Logger, logOut io.Wri
return err
}
// Probe the enabled upstreams once; failure is logged, not fatal.
probeCtx, probeCancel := context.WithTimeout(ctx, cfg.ProbeTimeout)
// Probe the enabled upstreams once; failure is logged, not fatal. A
// managed ComfyUI is intentionally down at startup — the first request
// starts it — so neither the startup probe nor the health check treats
// that as an outage.
probes := map[string]func(context.Context) error{}
if ollamaClient != nil {
if err := ollamaClient.Probe(probeCtx); err != nil {
log.Warn("ollama probe failed", "url", cfg.OllamaURL, "err", err)
}
probes["ollama"] = ollamaClient.Probe
}
if comfyClient != nil {
if err := comfyClient.Probe(probeCtx); err != nil {
log.Warn("comfy probe failed", "url", cfg.ComfyURL, "err", err)
if comfySup == nil {
probes["comfy"] = comfyClient.Probe
} else {
// Managed upstream: an idle-stopped or still-starting server is
// not an outage, so it is skipped until it has answered once
// (Ready resets on every spawn/stop). After that, a failed
// probe while the process lives is a real "DOWN".
probes["comfy"] = func(ctx context.Context) error {
if !comfySup.Ready() {
if comfySup.Running() {
if err := comfyClient.Probe(ctx); err == nil {
comfySup.MarkReady()
}
}
return errManagedDown
}
return comfyClient.Probe(ctx)
}
}
}
probeCtx, probeCancel := context.WithTimeout(ctx, cfg.ProbeTimeout)
for name, probe := range probes {
if err := probe(probeCtx); err != nil && !errors.Is(err, errManagedDown) {
log.Warn(name+" probe failed", "err", err)
}
}
probeCancel()
if cfg.HealthInterval > 0 {
go healthLoop(ctx, cfg.HealthInterval, cfg.ProbeTimeout, log, probes)
}
// Foreign GPU holders (games, other ML jobs) — enabled by GAME_PROCS
// and/or GPU_FOREIGN_VRAM_MB — hold the lock externally while they run.
if len(cfg.GameProcs) > 0 || cfg.GPUForeignVRAMMB > 0 {
det := game.New(cfg.GameProcs, cfg.GPUForeignVRAMMB, cfg.GPUIgnoreProcs, log)
go gameLoop(ctx, cfg, log, det, lk, ollamaClient, comfySup)
}
// Bind the listeners up front so a port conflict fails fast and the
// readiness notification below really means "accepting connections".
@@ -551,6 +623,128 @@ func run(ctx context.Context, cfg config.Config, log *slog.Logger, logOut io.Wri
return nil
}
// errManagedDown marks a managed upstream that is intentionally stopped
// (idle); health checks skip it instead of logging an outage.
var errManagedDown = errors.New("managed upstream intentionally stopped")
// gameLoop polls for foreign GPU holders (a game, another ML job). While one
// is detected it holds the lock externally so new LLM and image requests
// wait (or are rejected per LLM_BUSY_MODE), and — once in-flight work has
// drained — frees VRAM for it: the managed ComfyUI is stopped and Ollama's
// resident models are unloaded.
func gameLoop(ctx context.Context, cfg config.Config, log *slog.Logger, det *game.Detector, lk *lock.Lock, ollamaClient *ollama.Client, comfySup *supervise.Process) {
ticker := time.NewTicker(cfg.GamePollInterval)
defer ticker.Stop()
held, freed := false, false
for {
select {
case <-ctx.Done():
return
case <-ticker.C:
}
holders, err := det.Check(ctx)
if err != nil && ctx.Err() == nil {
log.Warn("game detection failed", "err", err)
}
switch {
case len(holders) > 0 && !held:
held = true
lk.SetExternal(summarizeHolders(holders))
log.Warn("GPU held by an external process; new LLM/image requests wait",
"holders", summarizeHolders(holders))
case len(holders) == 0 && held:
held, freed = false, false
lk.ClearExternal()
log.Warn("external process released the GPU; resuming")
}
if held && !freed {
if state, _, _ := lk.Snapshot(); state != lock.StateLLM && state != lock.StateImage {
freed = true
freeVRAM(ctx, cfg.UnloadTimeout, log, ollamaClient, comfySup)
}
}
}
}
// summarizeHolders joins holder descriptions for logs and busy responses,
// capping the list so a process name matching dozens of PIDs (system
// services) does not flood the log.
func summarizeHolders(holders []string) string {
const max = 5
if len(holders) > max {
return strings.Join(holders[:max], "; ") + fmt.Sprintf("; +%d more", len(holders)-max)
}
return strings.Join(holders, "; ")
}
// freeVRAM stops the managed ComfyUI (never an external server on its port)
// and unloads Ollama's resident models so the foreign process gets the GPU
// memory.
func freeVRAM(ctx context.Context, unloadTimeout time.Duration, log *slog.Logger, ollamaClient *ollama.Client, comfySup *supervise.Process) {
if comfySup != nil && comfySup.Running() {
log.Warn("stopping the managed ComfyUI to free VRAM")
comfySup.Stop()
}
if ollamaClient == nil {
return
}
uctx, cancel := context.WithTimeout(ctx, unloadTimeout)
defer cancel()
if models, err := ollamaClient.LoadedModels(uctx); err != nil || len(models) == 0 {
return // nothing resident (or ollama unreachable); nothing to free
}
elapsed, err := ollamaClient.UnloadAll(uctx)
if err != nil {
log.Warn("ollama unload incomplete; continuing", "err", err)
return
}
log.Warn("ollama models unloaded to free VRAM", "seconds", elapsed.Seconds())
}
// healthLoop probes the enabled upstreams every interval and logs status
// transitions — "is DOWN" when a previously healthy upstream stops
// answering, "recovered" when it comes back. The first round only
// establishes the baseline; the startup probe already reported that state.
func healthLoop(ctx context.Context, interval, probeTimeout time.Duration, log *slog.Logger, probes map[string]func(context.Context) error) {
ticker := time.NewTicker(interval)
defer ticker.Stop()
up := map[string]bool{}
managed := map[string]bool{}
for {
select {
case <-ctx.Done():
return
case <-ticker.C:
}
for name, probe := range probes {
pctx, cancel := context.WithTimeout(ctx, probeTimeout)
err := probe(pctx)
cancel()
if errors.Is(err, errManagedDown) {
managed[name] = true // intentionally stopped; not an outage
continue
}
now := err == nil
if managed[name] {
// First real probe after an idle stop only re-baselines —
// an on-demand start is not a "recovery".
managed[name] = false
up[name] = now
continue
}
was, seen := up[name]
if seen && now != was {
if now {
log.Warn(name + " upstream recovered")
} else {
log.Warn(name+" upstream is DOWN", "err", err)
}
}
up[name] = now
}
}
}
// updateLoop checks for signed updates on startup and every UPDATE_INTERVAL.
// In service mode a staged update is applied by exiting with exitCodeUpdate
// once the GPU lock is idle; the service recovery configuration restarts the
@@ -561,7 +755,7 @@ func updateLoop(ctx context.Context, cfg config.Config, log *slog.Logger, lk *lo
log.Warn("auto-update disabled: cannot locate executable", "err", err)
return
}
u := &update.Updater{Repo: cfg.UpdateRepo, Asset: cfg.UpdateAsset, Version: version, Log: log}
u := &update.Updater{Repo: cfg.UpdateRepo, Asset: cfg.UpdateAsset, Version: version, Desired: cfg.AppVersion, Log: log}
for {
staged, err := u.Check(ctx, exePath)
if err != nil && ctx.Err() == nil {
+1 -1
View File
@@ -6,7 +6,7 @@
# ComfyUI --listen 0.0.0.0 --port 8189).
services:
gpu-turnstile:
image: git.rambossek.at/public/gpu-turnstile:v0.1.6
image: git.rambossek.at/public/gpu-turnstile:v0.1.8
restart: unless-stopped
environment:
# Each consumer is enabled by setting its URL; leave one unset to
+88 -1
View File
@@ -4,6 +4,7 @@ package config
import (
"bufio"
"errors"
"fmt"
"io"
"log/slog"
@@ -25,6 +26,7 @@ type Config struct {
UnloadPollInterval time.Duration
HistoryPollInterval time.Duration
ProbeTimeout time.Duration
HealthInterval time.Duration
FreeTimeout time.Duration
WarmTimeout time.Duration
ShutdownTimeout time.Duration
@@ -37,6 +39,11 @@ type Config struct {
UpdateRepo string
UpdateAsset string
// AppVersion is the version the user wants to run: "dev" disables
// updates, "stable" tracks the latest release, anything else is an
// exact vX.Y.Z release to pin. From APP_VER; defaults to "stable".
AppVersion string
// LLMBusyMode is "wait" (hold requests until the lock is free or
// LLMWaitTimeout expires) or "reject" (immediately answer with
// LLMBusyStatus + Retry-After when an image job is active or pending).
@@ -44,6 +51,27 @@ type Config struct {
LLMBusyStatus int
BusyRetryAfter int
// ComfyCmd spawns and supervises a ComfyUI server on demand (empty =
// unmanaged, the current behavior). ComfyDir is its working directory.
// The managed server is stopped after ComfyIdleTimeout without
// requests, freeing its VRAM; ComfyStartTimeout bounds how long a
// request waits for it to come up.
ComfyCmd string
ComfyDir string
ComfyIdleTimeout time.Duration
ComfyStartTimeout time.Duration
// GameProcs (GAME_PROCS) is a watch list of process names; while any of
// them runs, the GPU is treated as held by a foreign process. The
// nvidia-smi path (GPUForeignVRAMMB, GPU_FOREIGN_VRAM_MB) does the same
// when a process not in GPUIgnoreProcs (GPU_IGNORE_PROCS) holds more than
// that many MiB of VRAM. GamePollInterval (GAME_POLL_INTERVAL) is how
// often both checks run.
GameProcs []string
GPUForeignVRAMMB int
GPUIgnoreProcs []string
GamePollInterval time.Duration
WarmModel string
LogLevel slog.Level
LogJSON bool
@@ -64,6 +92,7 @@ func Defaults() Config {
UnloadPollInterval: 500 * time.Millisecond,
HistoryPollInterval: time.Second,
ProbeTimeout: 5 * time.Second,
HealthInterval: 30 * time.Second,
FreeTimeout: 30 * time.Second,
WarmTimeout: 2 * time.Minute,
ShutdownTimeout: 10 * time.Second,
@@ -75,15 +104,29 @@ func Defaults() Config {
UpdateInterval: 6 * time.Hour,
UpdateRepo: "https://git.rambossek.at/PUBLIC/gpu-turnstile",
UpdateAsset: "gpu-turnstile.exe",
AppVersion: "stable",
LLMBusyMode: "wait",
LLMBusyStatus: 503,
BusyRetryAfter: 30,
ComfyIdleTimeout: 5 * time.Minute,
ComfyStartTimeout: 2 * time.Minute,
// ComfyUI runs under python; excluding it (and Ollama) by name keeps
// our own consumers from tripping the foreign-VRAM check.
GPUIgnoreProcs: []string{"ollama", "ollama app", "ollama_llama_server", "python", "pythonw"},
GamePollInterval: 5 * time.Second,
LogLevel: slog.LevelWarn,
}
}
// ErrNoConsumer is returned by Load when neither OLLAMA_URL nor COMFY_URL
// is set. Commands that never talk to an upstream (--force-update) may
// ignore it and proceed with the rest of the configuration.
var ErrNoConsumer = errors.New("at least one of OLLAMA_URL or COMFY_URL must be set (each URL enables its consumer)")
// ParseEnvFile parses a .env-style file: KEY=VALUE lines, blank lines and
// #-comments are ignored, no quoting. A line without '=' is an error.
func ParseEnvFile(r io.Reader) (map[string]string, error) {
@@ -110,6 +153,17 @@ func ParseEnvFile(r io.Reader) (map[string]string, error) {
return values, scanner.Err()
}
// splitList parses a comma-separated setting into trimmed, non-empty items.
func splitList(v string) []string {
var out []string
for _, item := range strings.Split(v, ",") {
if item = strings.TrimSpace(item); item != "" {
out = append(out, item)
}
}
return out
}
func envDuration(getenv func(string) string, name string, dst *time.Duration) error {
v := getenv(name)
if v == "" {
@@ -136,6 +190,8 @@ func Load(getenv func(string) string) (Config, error) {
{"OLLAMA_URL", &cfg.OllamaURL},
{"COMFY_URL", &cfg.ComfyURL},
{"WARM_MODEL", &cfg.WarmModel},
{"COMFY_CMD", &cfg.ComfyCmd},
{"COMFY_DIR", &cfg.ComfyDir},
{"UPDATE_REPO", &cfg.UpdateRepo},
{"UPDATE_ASSET", &cfg.UpdateAsset},
{"LOG_FILE", &cfg.LogFile},
@@ -154,17 +210,34 @@ func Load(getenv func(string) string) (Config, error) {
{"UNLOAD_POLL_INTERVAL", &cfg.UnloadPollInterval},
{"HISTORY_POLL_INTERVAL", &cfg.HistoryPollInterval},
{"PROBE_TIMEOUT", &cfg.ProbeTimeout},
{"HEALTH_INTERVAL", &cfg.HealthInterval},
{"FREE_TIMEOUT", &cfg.FreeTimeout},
{"WARM_TIMEOUT", &cfg.WarmTimeout},
{"SHUTDOWN_TIMEOUT", &cfg.ShutdownTimeout},
{"BACKOFF_INITIAL", &cfg.BackoffInitial},
{"BACKOFF_MAX", &cfg.BackoffMax},
{"COMFY_IDLE_TIMEOUT", &cfg.ComfyIdleTimeout},
{"COMFY_START_TIMEOUT", &cfg.ComfyStartTimeout},
{"UPDATE_INTERVAL", &cfg.UpdateInterval},
{"GAME_POLL_INTERVAL", &cfg.GamePollInterval},
} {
if err := envDuration(getenv, e.name, e.dst); err != nil {
return cfg, err
}
}
if v := getenv("GAME_PROCS"); v != "" {
cfg.GameProcs = splitList(v)
}
if v := getenv("GPU_IGNORE_PROCS"); v != "" {
cfg.GPUIgnoreProcs = splitList(v)
}
if v := getenv("GPU_FOREIGN_VRAM_MB"); v != "" {
n, err := strconv.Atoi(v)
if err != nil || n < 0 {
return cfg, fmt.Errorf("GPU_FOREIGN_VRAM_MB: must be a non-negative integer (MiB, 0 = disabled)")
}
cfg.GPUForeignVRAMMB = n
}
if v := getenv("PROMPT_CAPTURE_LIMIT"); v != "" {
n, err := strconv.ParseInt(v, 10, 64)
if err != nil || n < 0 {
@@ -199,6 +272,17 @@ func Load(getenv func(string) string) (Config, error) {
}
cfg.BusyRetryAfter = n
}
if v := getenv("APP_VER"); v != "" {
switch {
case v == "dev" || v == "stable":
cfg.AppVersion = v
default:
if _, ok := parseVersion(v); !ok {
return cfg, fmt.Errorf("APP_VER: must be \"dev\", \"stable\" or a vX.Y.Z version")
}
cfg.AppVersion = "v" + strings.TrimPrefix(v, "v")
}
}
// LOGLEVEL is the canonical spelling; LOG_LEVEL is kept as an alias.
logLevelValue := getenv("LOGLEVEL")
if logLevelValue == "" {
@@ -218,8 +302,11 @@ func Load(getenv func(string) string) (Config, error) {
default:
return cfg, fmt.Errorf("LOG_FORMAT: must be \"text\" or \"json\"")
}
if cfg.ComfyCmd != "" && cfg.ComfyURL == "" {
return cfg, fmt.Errorf("COMFY_CMD requires COMFY_URL to be set (the proxy needs somewhere to forward)")
}
if cfg.OllamaURL == "" && cfg.ComfyURL == "" {
return cfg, fmt.Errorf("at least one of OLLAMA_URL or COMFY_URL must be set (each URL enables its consumer)")
return cfg, ErrNoConsumer
}
return cfg, nil
}
+106
View File
@@ -1,6 +1,7 @@
package config
import (
"errors"
"log/slog"
"strings"
"testing"
@@ -39,6 +40,61 @@ func TestLoadRequiresConsumer(t *testing.T) {
if err == nil || !strings.Contains(err.Error(), "OLLAMA_URL") {
t.Fatalf("err = %v, want missing-consumer error", err)
}
if !errors.Is(err, ErrNoConsumer) {
t.Fatalf("err = %v, want errors.Is(err, ErrNoConsumer)", err)
}
}
func TestAppVersion(t *testing.T) {
load := func(appVer string) (Config, error) {
return Load(func(k string) string {
switch k {
case "OLLAMA_URL":
return "http://127.0.0.1:11435"
case "APP_VER":
return appVer
}
return ""
})
}
cfg, err := load("")
if err != nil || cfg.AppVersion != "stable" {
t.Fatalf("default AppVersion = %q, err %v; want stable", cfg.AppVersion, err)
}
for _, v := range []string{"dev", "stable"} {
if cfg, err := load(v); err != nil || cfg.AppVersion != v {
t.Fatalf("APP_VER=%s: got %q, err %v", v, cfg.AppVersion, err)
}
}
if cfg, err := load("1.2.3"); err != nil || cfg.AppVersion != "v1.2.3" {
t.Fatalf("APP_VER=1.2.3: got %q, err %v; want normalized v1.2.3", cfg.AppVersion, err)
}
if _, err := load("nightly"); err == nil {
t.Fatal("APP_VER=nightly: want validation error")
}
}
func TestComfyCmdRequiresURL(t *testing.T) {
_, err := Load(func(k string) string {
if k == "COMFY_CMD" {
return "python main.py"
}
return ""
})
if err == nil || !strings.Contains(err.Error(), "COMFY_CMD requires COMFY_URL") {
t.Fatalf("err = %v, want COMFY_CMD/COMFY_URL validation error", err)
}
if _, err := Load(func(k string) string {
switch k {
case "COMFY_CMD":
return "python main.py"
case "COMFY_URL":
return "http://127.0.0.1:8188"
}
return ""
}); err != nil {
t.Fatalf("COMFY_CMD with COMFY_URL must load: %v", err)
}
}
func TestParseEnvFile(t *testing.T) {
@@ -101,6 +157,8 @@ func TestLoadErrors(t *testing.T) {
{"LLM_BUSY_MODE", "bogus"},
{"LLM_BUSY_STATUS", "200"},
{"BUSY_RETRY_AFTER", "0"},
{"GPU_FOREIGN_VRAM_MB", "-1"},
{"GAME_POLL_INTERVAL", "bogus"},
} {
_, err := Load(func(k string) string {
if k == tc.key {
@@ -116,3 +174,51 @@ func TestLoadErrors(t *testing.T) {
}
}
}
func TestGameDetectionSettings(t *testing.T) {
cfg, err := Load(func(k string) string {
switch k {
case "OLLAMA_URL":
return "http://127.0.0.1:11435"
case "GAME_PROCS":
return " cyberpunk2077.exe, hl2.exe ,, "
case "GPU_FOREIGN_VRAM_MB":
return "1024"
case "GPU_IGNORE_PROCS":
return "ollama, my-trainer"
}
return ""
})
if err != nil {
t.Fatal(err)
}
if len(cfg.GameProcs) != 2 || cfg.GameProcs[0] != "cyberpunk2077.exe" || cfg.GameProcs[1] != "hl2.exe" {
t.Fatalf("GameProcs = %v", cfg.GameProcs)
}
if cfg.GPUForeignVRAMMB != 1024 {
t.Fatalf("GPUForeignVRAMMB = %d", cfg.GPUForeignVRAMMB)
}
if len(cfg.GPUIgnoreProcs) != 2 || cfg.GPUIgnoreProcs[1] != "my-trainer" {
t.Fatalf("GPUIgnoreProcs = %v", cfg.GPUIgnoreProcs)
}
if cfg.GamePollInterval != 5*time.Second {
t.Fatalf("GamePollInterval = %v, want 5s default", cfg.GamePollInterval)
}
// Defaults: both detection paths off, ignore list covers our consumers.
cfg, err = Load(func(k string) string {
if k == "OLLAMA_URL" {
return "http://127.0.0.1:11435"
}
return ""
})
if err != nil {
t.Fatal(err)
}
if len(cfg.GameProcs) != 0 || cfg.GPUForeignVRAMMB != 0 {
t.Fatalf("detection must be off by default: %v %d", cfg.GameProcs, cfg.GPUForeignVRAMMB)
}
if len(cfg.GPUIgnoreProcs) == 0 {
t.Fatal("GPUIgnoreProcs default must not be empty")
}
}
+192
View File
@@ -0,0 +1,192 @@
package config
import (
"fmt"
"strconv"
"strings"
)
// sampleEntry is one line pair in the sample env file: a comment and the
// (usually commented-out) KEY=VALUE line.
type sampleEntry struct {
name string
value string
comment string
active bool // rendered uncommented
}
// appVerComment documents APP_VER wherever it is rendered.
const appVerComment = `Version to run: "dev" disables updates, "stable" tracks the latest release, or pin an exact release like v0.1.7`
// sampleEntries lists every setting in sample order. logFile activates the
// LOG_FILE line (Windows install); empty keeps it commented like the rest.
// CFG_VER and APP_VER are not entries — they head the file, always active.
func sampleEntries(logFile string) []sampleEntry {
return []sampleEntry{
{"LISTEN_OLLAMA", ":11434", "Listen address for Ollama-compatible clients (gpu-turnstile poses as Ollama here)", false},
{"LISTEN_COMFY", ":8188", "Listen address for ComfyUI clients (gpu-turnstile poses as ComfyUI here)", false},
{"OLLAMA_URL", "http://127.0.0.1:11434", "Ollama upstream URL; setting it enables the Ollama consumer (default: empty = disabled)", false},
{"COMFY_URL", "http://127.0.0.1:8188", "ComfyUI upstream URL; setting it enables the ComfyUI consumer (default: empty = disabled)", false},
{"WARM_MODEL", "", "Optional model to reload after an image job (default: empty = none)", false},
{"COMFY_CMD", `"C:\ComfyUI\.venv\Scripts\python.exe" ComfyUI\main.py --port 8188`, "Spawn and supervise ComfyUI on demand: the first request starts it, it stops after COMFY_IDLE_TIMEOUT to free VRAM (default: empty = unmanaged)", false},
{"COMFY_DIR", `C:\ComfyUI`, "Working directory for COMFY_CMD (default: empty = inherit)", false},
{"COMFY_IDLE_TIMEOUT", "5m", "Stop the managed ComfyUI after this long without requests or jobs (frees VRAM)", false},
{"COMFY_START_TIMEOUT", "2m", "How long a request waits for the managed ComfyUI to come up", false},
{"GAME_PROCS", "cyberpunk2077.exe,hl2.exe", "While a listed process runs, the GPU counts as held by it: requests wait, Ollama unloads, managed ComfyUI stops (default: empty = disabled)", false},
{"GPU_FOREIGN_VRAM_MB", "1024", "Also treat the GPU as held when a process not in GPU_IGNORE_PROCS uses more VRAM than this (needs nvidia-smi; 0/empty = disabled)", false},
{"GPU_IGNORE_PROCS", "ollama,ollama app,ollama_llama_server,python,pythonw", "Process names never counted as foreign GPU users (ComfyUI runs under python)", false},
{"GAME_POLL_INTERVAL", "5s", "How often game/VRAM detection runs", false},
{"UNLOAD_TIMEOUT", "60s", "How long to wait for Ollama to unload a model", false},
{"JOB_TIMEOUT", "15m", "Maximum time to wait for a ComfyUI job", false},
{"LLM_WAIT_TIMEOUT", "10m", "Max time an LLM request waits for the GPU before being answered 503 (wait mode)", false},
{"LLM_BUSY_MODE", "wait", `How blocked LLM requests are handled: "wait" holds them, "reject" fails them immediately`, false},
{"LLM_BUSY_STATUS", "503", "HTTP status for rejected LLM requests in reject mode (400-599, e.g. 429)", false},
{"BUSY_RETRY_AFTER", "30", "Seconds sent as Retry-After on busy responses (both modes)", false},
{"LOGLEVEL", "warn", `Log verbosity: debug, info, warn, error; "info" logs every request (LOG_LEVEL works too)`, false},
{"LOG_FORMAT", "text", `Log format: "text" or "json"`, false},
{"LOG_FILE", logFile, "Append logs to this file instead of stderr (a Windows service has no console)", logFile != ""},
{"UNLOAD_POLL_INTERVAL", "500ms", "/api/ps poll interval while unloading", false},
{"HISTORY_POLL_INTERVAL", "1s", "/history/<id> poll interval while a job runs", false},
{"PROBE_TIMEOUT", "5s", "Probe timeout for the startup probe and the periodic health check", false},
{"HEALTH_INTERVAL", "30s", "How often enabled upstreams are probed; status changes are logged", false},
{"FREE_TIMEOUT", "30s", "Timeout for the POST /free call after an image job", false},
{"WARM_TIMEOUT", "2m", "Timeout for the warm-model reload after an image job", false},
{"SHUTDOWN_TIMEOUT", "10s", "Graceful shutdown timeout on SIGINT/SIGTERM", false},
{"BACKOFF_INITIAL", "1s", "First retry wait when an upstream connection fails", false},
{"BACKOFF_MAX", "60s", "Cap for the exponential retry backoff", false},
{"PROMPT_CAPTURE_LIMIT", "65536", "Bytes of the /prompt response buffered to find prompt_id (pass-through unaffected)", false},
{"AUTO_UPDATE", "true", "Poll the releases API for signed updates", false},
{"UPDATE_INTERVAL", "6h", "Auto-update check interval", false},
{"UPDATE_REPO", "https://git.rambossek.at/PUBLIC/gpu-turnstile", "Repository to check for releases", false},
{"UPDATE_ASSET", "gpu-turnstile.exe", "Release asset to download", false},
}
}
// SampleEnv renders a sample .env file covering every setting, each with a
// comment line. Everything is commented out — so all defaults apply —
// except the CFG_VER/APP_VER header and LOG_FILE when logFile is non-empty
// (a Windows service has no console). CFG_VER records the version that
// wrote the file so later installs can upgrade it.
func SampleEnv(version, logFile string) string {
// CFG_VER is always a concrete vX.Y.Z — never "dev". A dev build
// stamps v0.0.0, which sorts older than any release, so the next
// release install upgrades the file and stamps a proper version.
cfgVer := version
if cfgVer == "" || cfgVer == "dev" {
cfgVer = "v0.0.0"
}
var b strings.Builder
fmt.Fprintf(&b, "CFG_VER=%s\n", cfgVer)
b.WriteString("# Config format reference, written by the installer — do not edit.\n")
b.WriteString("# The installer uses it to append newly added settings on updates.\n\n")
b.WriteString("# gpu-turnstile configuration\n")
b.WriteString("# KEY=VALUE lines; \"#\" starts a comment. Every setting below is at its\n")
b.WriteString("# default and commented out — remove the \"#\" to change it.\n")
b.WriteString("# At least one of OLLAMA_URL / COMFY_URL must be set for the proxy to start.\n\n")
writeEntry(&b, sampleEntry{"APP_VER", "stable", appVerComment, true})
for _, e := range sampleEntries(logFile) {
writeEntry(&b, e)
}
return b.String()
}
func writeEntry(b *strings.Builder, e sampleEntry) {
fmt.Fprintf(b, "# %s\n", e.comment)
if e.active {
fmt.Fprintf(b, "%s=%s\n\n", e.name, e.value)
} else {
fmt.Fprintf(b, "#%s=%s\n\n", e.name, e.value)
}
}
// SyncSample upgrades an installer-written env file: when its CFG_VER says
// it was written by an older gpu-turnstile, every setting the file does not
// mention — commented or not — is appended at the end, and CFG_VER is
// updated to version. Files without CFG_VER (hand-written or foreign),
// up-to-date files and "dev" builds are returned unchanged; changed reports
// whether the returned content differs.
func SyncSample(data, version, logFile string) (string, bool) {
if version == "" || version == "dev" {
return data, false
}
values, err := ParseEnvFile(strings.NewReader(data))
if err != nil || values["CFG_VER"] == "" {
return data, false // not installer-written; the caller decides
}
if compareVersions(values["CFG_VER"], version) >= 0 {
return data, false // same or newer
}
present := make(map[string]bool)
for _, line := range strings.Split(data, "\n") {
line = strings.TrimSpace(strings.TrimPrefix(strings.TrimSpace(line), "#"))
if name, _, ok := strings.Cut(line, "="); ok {
present[strings.TrimSpace(name)] = true
}
}
lines := strings.Split(data, "\n")
for i, l := range lines {
if strings.HasPrefix(strings.TrimSpace(l), "CFG_VER=") {
lines[i] = "CFG_VER=" + version
break
}
}
out := strings.Join(lines, "\n")
if !strings.HasSuffix(out, "\n") {
out += "\n"
}
var b strings.Builder
if !present["APP_VER"] {
fmt.Fprintf(&b, "\n# Added by gpu-turnstile %s:\n", version)
writeEntry(&b, sampleEntry{"APP_VER", "stable", appVerComment, true})
}
for _, e := range sampleEntries(logFile) {
if !present[e.name] {
fmt.Fprintf(&b, "\n# Added by gpu-turnstile %s:\n", version)
writeEntry(&b, e)
}
}
out += b.String()
return out, out != data
}
// compareVersions orders two vX.Y.Z version strings. Unparseable versions
// (including "dev") sort before any release.
func compareVersions(a, b string) int {
pa, oka := parseVersion(a)
pb, okb := parseVersion(b)
if oka != okb {
if oka {
return 1
}
return -1
}
for i := range pa {
if pa[i] != pb[i] {
if pa[i] > pb[i] {
return 1
}
return -1
}
}
return 0
}
func parseVersion(v string) ([3]int, bool) {
var out [3]int
v = strings.TrimPrefix(strings.TrimSpace(v), "v")
parts := strings.Split(v, ".")
if len(parts) != 3 {
return out, false
}
for i, p := range parts {
n, err := strconv.Atoi(p)
if err != nil || n < 0 {
return out, false
}
out[i] = n
}
return out, true
}
+156
View File
@@ -0,0 +1,156 @@
package config
import (
"strings"
"testing"
)
const sampleLogPath = `C:\ProgramData\gpu-turnstile\gpu-turnstile.log`
var allSettingNames = []string{
"LISTEN_OLLAMA", "LISTEN_COMFY", "OLLAMA_URL", "COMFY_URL",
"WARM_MODEL", "UNLOAD_TIMEOUT", "JOB_TIMEOUT", "LLM_WAIT_TIMEOUT",
"LLM_BUSY_MODE", "LLM_BUSY_STATUS", "BUSY_RETRY_AFTER",
"LOGLEVEL", "LOG_FORMAT", "LOG_FILE",
"UNLOAD_POLL_INTERVAL", "HISTORY_POLL_INTERVAL", "PROBE_TIMEOUT",
"FREE_TIMEOUT", "WARM_TIMEOUT", "SHUTDOWN_TIMEOUT",
"BACKOFF_INITIAL", "BACKOFF_MAX", "PROMPT_CAPTURE_LIMIT",
"AUTO_UPDATE", "UPDATE_INTERVAL", "UPDATE_REPO", "UPDATE_ASSET",
}
func TestSampleEnv(t *testing.T) {
sample := SampleEnv("v0.1.7", sampleLogPath)
if !strings.HasPrefix(sample, "CFG_VER=v0.1.7\n") {
t.Errorf("first line does not carry CFG_VER: %q", strings.SplitN(sample, "\n", 2)[0])
}
// Every setting known to Load must appear.
for _, name := range allSettingNames {
if !strings.Contains(sample, name+"=") {
t.Errorf("sample is missing %s", name)
}
}
// The sample must parse cleanly; active values are CFG_VER, APP_VER
// and LOG_FILE.
values, err := ParseEnvFile(strings.NewReader(sample))
if err != nil {
t.Fatalf("sample does not parse: %v", err)
}
want := map[string]string{"CFG_VER": "v0.1.7", "APP_VER": "stable", "LOG_FILE": sampleLogPath}
if len(values) != len(want) {
t.Fatalf("active values = %v, want %v", values, want)
}
for k, v := range want {
if values[k] != v {
t.Errorf("%s = %q, want %q", k, values[k], v)
}
}
// Without a log path LOG_FILE stays commented out.
values, err = ParseEnvFile(strings.NewReader(SampleEnv("v0.1.7", "")))
if err != nil {
t.Fatalf("sample without log path does not parse: %v", err)
}
if len(values) != 2 || values["LOG_FILE"] != "" {
t.Fatalf("active values = %v, want only CFG_VER and APP_VER", values)
}
}
func TestSyncSample(t *testing.T) {
old := SampleEnv("v0.1.6", sampleLogPath)
// Simulate a setting that did not exist yet when the file was written.
old = strings.Replace(old, "#UPDATE_ASSET=gpu-turnstile.exe\n", "", 1)
out, changed := SyncSample(old, "v0.1.7", sampleLogPath)
if !changed {
t.Fatal("older installer file was not upgraded")
}
if !strings.Contains(out, "\nCFG_VER=v0.1.7\n") && !strings.HasPrefix(out, "CFG_VER=v0.1.7\n") {
t.Error("CFG_VER was not updated to the new version")
}
if !strings.Contains(out, "#UPDATE_ASSET=gpu-turnstile.exe") {
t.Error("missing setting was not appended")
}
if !strings.Contains(out, "LOG_FILE="+sampleLogPath) {
t.Error("existing active LOG_FILE was lost")
}
if !strings.Contains(out, "Added by gpu-turnstile v0.1.7") {
t.Error("appended section is not attributed")
}
if _, err := ParseEnvFile(strings.NewReader(out)); err != nil {
t.Fatalf("upgraded file does not parse: %v", err)
}
// A file written by the same or a newer version is left alone.
if _, changed := SyncSample(SampleEnv("v0.1.7", sampleLogPath), "v0.1.7", sampleLogPath); changed {
t.Error("same-version file was modified")
}
if _, changed := SyncSample(SampleEnv("v0.2.0", sampleLogPath), "v0.1.7", sampleLogPath); changed {
t.Error("newer-version file was modified")
}
// Files without CFG_VER are not installer-written; the installer
// replaces them, SyncSample leaves them alone.
user := "OLLAMA_URL=http://host:11434\n"
if out, changed := SyncSample(user, "v0.1.7", sampleLogPath); changed || out != user {
t.Error("file without CFG_VER was modified")
}
// A dev build never upgrades.
if _, changed := SyncSample(old, "dev", sampleLogPath); changed {
t.Error("dev build modified the file")
}
}
func TestSyncSampleAppendsMissingAppVer(t *testing.T) {
old := SampleEnv("v0.1.6", sampleLogPath)
old = strings.Replace(old, "# "+appVerComment+"\n", "", 1)
old = strings.Replace(old, "APP_VER=stable\n", "", 1)
out, changed := SyncSample(old, "v0.1.7", sampleLogPath)
if !changed {
t.Fatal("file without APP_VER was not upgraded")
}
values, err := ParseEnvFile(strings.NewReader(out))
if err != nil {
t.Fatalf("upgraded file does not parse: %v", err)
}
if values["APP_VER"] != "stable" {
t.Fatalf("APP_VER = %q, want appended default \"stable\"", values["APP_VER"])
}
}
func TestSampleEnvDevStampsConcreteVersion(t *testing.T) {
// A dev build must never write CFG_VER=dev: it stamps v0.0.0, and the
// next release install upgrades the file to a proper version.
dev := SampleEnv("dev", sampleLogPath)
if !strings.HasPrefix(dev, "CFG_VER=v0.0.0\n") {
t.Errorf("dev sample first line: %q", strings.SplitN(dev, "\n", 2)[0])
}
out, changed := SyncSample(dev, "v0.1.7", sampleLogPath)
if !changed || !strings.HasPrefix(out, "CFG_VER=v0.1.7\n") {
t.Errorf("dev-stamped file was not upgraded to v0.1.7 (changed=%v)", changed)
}
}
func TestCompareVersions(t *testing.T) {
cases := []struct {
a, b string
want int
}{
{"v0.1.6", "v0.1.7", -1},
{"v0.1.7", "v0.1.7", 0},
{"v1.0.0", "v0.9.9", 1},
{"0.1.7", "v0.1.6", 1},
{"dev", "v0.1.7", -1},
{"v0.1.7", "dev", 1},
{"dev", "dev", 0},
}
for _, c := range cases {
if got := compareVersions(c.a, c.b); got != c.want {
t.Errorf("compareVersions(%q, %q) = %d, want %d", c.a, c.b, got, c.want)
}
}
}
+159
View File
@@ -0,0 +1,159 @@
// Package game detects processes outside gpu-turnstile's control that hold
// the GPU — typically a game — so the proxy can block new GPU work and free
// VRAM while they run. Two detection paths: an explicit process watch list
// (GAME_PROCS) and a foreign-VRAM threshold via nvidia-smi
// (GPU_FOREIGN_VRAM_MB) that catches anything not on the ignore list.
package game
import (
"context"
"errors"
"fmt"
"log/slog"
"os/exec"
"strconv"
"strings"
)
// Process is one running OS process.
type Process struct {
PID int
Name string
}
// computeApp is one process holding GPU memory, as reported by nvidia-smi.
type computeApp struct {
PID int
UsedMB int
}
// Detector checks whether a foreign process holds the GPU. The zero value
// (no watch list, no threshold) never detects anything; main only starts the
// poll loop when at least one path is configured.
type Detector struct {
procs map[string]bool // normalized names from GAME_PROCS
vramMB int // foreign VRAM threshold; 0 = disabled
ignore map[string]bool // normalized names never counted as foreign
log *slog.Logger
noNvidia bool // nvidia-smi was not found; VRAM path disabled for good
}
// New builds a Detector from the configured watch list, VRAM threshold in
// MiB (0 disables the nvidia-smi path) and ignore list. Names are matched
// case-insensitively, with or without a trailing ".exe".
func New(procs []string, vramMB int, ignore []string, log *slog.Logger) *Detector {
if log == nil {
log = slog.Default()
}
return &Detector{
procs: nameSet(procs),
vramMB: vramMB,
ignore: nameSet(ignore),
log: log,
}
}
// normName lowercases a process name and strips a trailing ".exe" so the
// watch and ignore lists match on Windows and Linux spellings alike.
func normName(s string) string {
return strings.TrimSuffix(strings.ToLower(strings.TrimSpace(s)), ".exe")
}
func nameSet(names []string) map[string]bool {
set := make(map[string]bool, len(names))
for _, n := range names {
if n = normName(n); n != "" {
set[n] = true
}
}
return set
}
// Check looks once for foreign GPU holders and returns a human-readable
// description of each (empty when the GPU is free for gpu-turnstile's
// consumers). A failing nvidia-smi call is returned as an error only when
// the process list found nothing; a missing nvidia-smi binary disables the
// VRAM path permanently (logged once).
func (d *Detector) Check(ctx context.Context) ([]string, error) {
ps, psErr := processes()
if d.vramMB <= 0 || d.noNvidia {
return d.detect(ps, nil), psErr
}
apps, err := queryComputeApps(ctx)
if errors.Is(err, exec.ErrNotFound) {
d.noNvidia = true
d.log.Warn("GPU_FOREIGN_VRAM_MB is set but nvidia-smi was not found; VRAM detection disabled")
return d.detect(ps, nil), nil
}
if err != nil {
return d.detect(ps, nil), err
}
return d.detect(ps, apps), nil
}
// detect is the pure core of Check: given the process table and (optionally)
// the nvidia-smi compute-apps list, it returns the foreign holders.
func (d *Detector) detect(ps []Process, apps []computeApp) []string {
var holders []string
for _, p := range ps {
if d.procs[normName(p.Name)] {
holders = append(holders, fmt.Sprintf("%s (pid %d)", p.Name, p.PID))
}
}
if d.vramMB > 0 && apps != nil {
names := make(map[int]string, len(ps))
for _, p := range ps {
names[p.PID] = p.Name
}
for _, a := range apps {
name := names[a.PID]
if d.ignore[normName(name)] || a.UsedMB < d.vramMB {
continue
}
if name == "" {
name = "unknown process"
}
holders = append(holders, fmt.Sprintf("%s (pid %d) using %d MiB VRAM", name, a.PID, a.UsedMB))
}
}
return holders
}
// queryComputeApps runs nvidia-smi and parses the per-process VRAM list.
// Note: under Windows' WDDM driver, nvidia-smi only sees compute
// allocations, so graphics-only games may not appear there — GAME_PROCS is
// the reliable path on Windows; on Linux both work.
func queryComputeApps(ctx context.Context) ([]computeApp, error) {
out, err := exec.CommandContext(ctx, "nvidia-smi",
"--query-compute-apps=pid,used_memory", "--format=csv,noheader,nounits").Output()
if err != nil {
return nil, err
}
return parseComputeApps(string(out))
}
// parseComputeApps parses "pid, used_memory" CSV lines (no header, MiB
// units). Unsupported rows ("N/A" on WDDM) are skipped.
func parseComputeApps(out string) ([]computeApp, error) {
var apps []computeApp
for _, line := range strings.Split(out, "\n") {
line = strings.TrimSpace(line)
if line == "" {
continue
}
pidStr, memStr, ok := strings.Cut(line, ",")
if !ok {
return nil, fmt.Errorf("nvidia-smi: unexpected line %q", line)
}
pid, err := strconv.Atoi(strings.TrimSpace(pidStr))
if err != nil {
return nil, fmt.Errorf("nvidia-smi: unexpected pid in %q", line)
}
mem, err := strconv.Atoi(strings.TrimSpace(memStr))
if err != nil {
continue // "N/A" and friends: unsupported under WDDM
}
apps = append(apps, computeApp{PID: pid, UsedMB: mem})
}
return apps, nil
}
+96
View File
@@ -0,0 +1,96 @@
package game
import (
"runtime"
"slices"
"testing"
)
func TestNormName(t *testing.T) {
for in, want := range map[string]string{
"Cyberpunk2077.exe": "cyberpunk2077",
"ollama": "ollama",
"OLLAMA APP.EXE": "ollama app",
" python.exe ": "python",
"hl2": "hl2",
} {
if got := normName(in); got != want {
t.Errorf("normName(%q) = %q, want %q", in, got, want)
}
}
}
func TestParseComputeApps(t *testing.T) {
apps, err := parseComputeApps("1234, 512\n 42 , 8192 \n\n")
if err != nil {
t.Fatal(err)
}
want := []computeApp{{PID: 1234, UsedMB: 512}, {PID: 42, UsedMB: 8192}}
if !slices.Equal(apps, want) {
t.Errorf("got %+v, want %+v", apps, want)
}
// WDDM reports "N/A" for memory; those rows are skipped, not fatal.
apps, err = parseComputeApps("1234, N/A\n42, 1024\n")
if err != nil {
t.Fatal(err)
}
if !slices.Equal(apps, []computeApp{{PID: 42, UsedMB: 1024}}) {
t.Errorf("got %+v", apps)
}
if _, err := parseComputeApps("garbage\n"); err == nil {
t.Error("expected an error for a malformed line")
}
}
func TestDetect(t *testing.T) {
d := New([]string{"Cyberpunk2077.exe", "hl2"}, 1024,
[]string{"ollama", "python", "pythonw"}, nil)
ps := []Process{
{PID: 10, Name: "ollama.exe"},
{PID: 20, Name: "python.exe"},
{PID: 30, Name: "cyberpunk2077.exe"},
}
apps := []computeApp{
{PID: 10, UsedMB: 8192}, // ignored: ollama
{PID: 20, UsedMB: 4096}, // ignored: python (ComfyUI)
{PID: 40, UsedMB: 2048}, // foreign, above threshold
{PID: 50, UsedMB: 100}, // foreign but below threshold
}
holders := d.detect(ps, apps)
if len(holders) != 2 {
t.Fatalf("got %v, want 2 holders", holders)
}
if holders[0] != "cyberpunk2077.exe (pid 30)" {
t.Errorf("holders[0] = %q", holders[0])
}
if holders[1] != "unknown process (pid 40) using 2048 MiB VRAM" {
t.Errorf("holders[1] = %q", holders[1])
}
}
func TestDetectNothingConfigured(t *testing.T) {
d := New(nil, 0, nil, nil)
if got := d.detect([]Process{{PID: 1, Name: "game.exe"}}, nil); len(got) != 0 {
t.Errorf("got %v, want none", got)
}
}
func TestProcessesLive(t *testing.T) {
if runtime.GOOS != "windows" && runtime.GOOS != "linux" {
t.Skip("no process listing on this platform")
}
ps, err := processes()
if err != nil {
t.Fatal(err)
}
if len(ps) == 0 {
t.Fatal("no processes listed")
}
for _, p := range ps {
if p.Name == "" {
t.Errorf("pid %d has an empty name", p.PID)
}
}
}
+30
View File
@@ -0,0 +1,30 @@
//go:build linux
package game
import (
"os"
"strconv"
"strings"
)
// processes lists the running processes from /proc/<pid>/comm.
func processes() ([]Process, error) {
entries, err := os.ReadDir("/proc")
if err != nil {
return nil, err
}
var ps []Process
for _, e := range entries {
pid, err := strconv.Atoi(e.Name())
if err != nil {
continue
}
comm, err := os.ReadFile("/proc/" + e.Name() + "/comm")
if err != nil {
continue // process vanished mid-walk
}
ps = append(ps, Process{PID: pid, Name: strings.TrimSpace(string(comm))})
}
return ps, nil
}
+7
View File
@@ -0,0 +1,7 @@
//go:build !windows && !linux
package game
// processes is unsupported on this platform; the process watch list never
// matches and the nvidia-smi path reports PIDs without names.
func processes() ([]Process, error) { return nil, nil }
+35
View File
@@ -0,0 +1,35 @@
//go:build windows
package game
import (
"unsafe"
"golang.org/x/sys/windows"
)
// processes lists the running processes via the Toolhelp32 snapshot API.
func processes() ([]Process, error) {
h, err := windows.CreateToolhelp32Snapshot(windows.TH32CS_SNAPPROCESS, 0)
if err != nil {
return nil, err
}
defer windows.CloseHandle(h) //nolint:errcheck // best effort
var entry windows.ProcessEntry32
entry.Size = uint32(unsafe.Sizeof(entry))
if err := windows.Process32First(h, &entry); err != nil {
return nil, err
}
var ps []Process
for {
ps = append(ps, Process{
PID: int(entry.ProcessID),
Name: windows.UTF16ToString(entry.ExeFile[:]),
})
if err := windows.Process32Next(h, &entry); err != nil {
break // ERROR_NO_MORE_FILES ends the walk
}
}
return ps, nil
}
+43 -8
View File
@@ -1,6 +1,8 @@
// Package lock implements the two-mode GPU arbitration lock: any number of
// concurrent LLM requests ("readers") or exactly one image job ("writer"),
// with image jobs taking priority over newly arriving LLM requests.
// with image jobs taking priority over newly arriving LLM requests. An
// external hold (SetExternal) blocks new grants of both kinds while a
// foreign process — e.g. a game — holds the GPU; in-flight work drains.
package lock
import (
@@ -13,9 +15,10 @@ import (
type State string
const (
StateIdle State = "idle"
StateLLM State = "llm"
StateImage State = "image"
StateIdle State = "idle"
StateLLM State = "llm"
StateImage State = "image"
StateExternal State = "external"
)
type imageWaiter struct{ id uint64 }
@@ -30,6 +33,7 @@ type Lock struct {
imageActive bool // an image job holds the GPU
imageQ []imageWaiter
nextID uint64
external string // non-empty: a foreign process (e.g. a game) holds the GPU
log *slog.Logger
}
@@ -52,12 +56,41 @@ func (l *Lock) logTransition(msg string, args ...any) {
}
}
// SetExternal records that a process outside gpu-turnstile's control (a
// game, another ML job) holds the GPU: new LLM and image grants block until
// ClearExternal. In-flight work is not preempted. holder describes the
// process for logs and busy responses.
func (l *Lock) SetExternal(holder string) {
l.mu.Lock()
l.external = holder
l.broadcast()
l.mu.Unlock()
l.logTransition("lock transition", "state", StateExternal, "holder", holder)
}
// ClearExternal lifts the external hold; waiting LLM and image requests
// proceed.
func (l *Lock) ClearExternal() {
l.mu.Lock()
l.external = ""
l.broadcast()
l.mu.Unlock()
l.logTransition("lock transition", "state", StateIdle)
}
// External returns the current external holder, or "" when none.
func (l *Lock) External() string {
l.mu.Lock()
defer l.mu.Unlock()
return l.external
}
// AcquireLLM blocks until no image job is active or pending, then registers
// one in-flight LLM request. Returns ctx.Err() if the context is cancelled
// while waiting; no state is changed in that case.
func (l *Lock) AcquireLLM(ctx context.Context) error {
l.mu.Lock()
for l.imageActive || len(l.imageQ) > 0 {
for l.imageActive || len(l.imageQ) > 0 || l.external != "" {
ch := l.change
l.mu.Unlock()
select {
@@ -76,10 +109,10 @@ func (l *Lock) AcquireLLM(ctx context.Context) error {
// TryAcquireLLM acquires one in-flight LLM slot without waiting and
// reports whether it succeeded. It fails when an image job is active or
// pending.
// pending or an external hold is set.
func (l *Lock) TryAcquireLLM() bool {
l.mu.Lock()
if l.imageActive || len(l.imageQ) > 0 {
if l.imageActive || len(l.imageQ) > 0 || l.external != "" {
l.mu.Unlock()
return false
}
@@ -120,7 +153,7 @@ func (l *Lock) AcquireImage(ctx context.Context) error {
for {
l.mu.Lock()
if l.imageQ[0].id == w.id && l.n == 0 && !l.imageActive {
if l.imageQ[0].id == w.id && l.n == 0 && !l.imageActive && l.external == "" {
l.imageQ = l.imageQ[1:]
l.imageActive = true
l.mu.Unlock()
@@ -166,6 +199,8 @@ func (l *Lock) Snapshot() (state State, llmInflight int, imagePending bool) {
state = StateImage
case l.n > 0:
state = StateLLM
case l.external != "":
state = StateExternal
default:
state = StateIdle
}
+67
View File
@@ -226,3 +226,70 @@ func TestRace(t *testing.T) {
t.Fatalf("leaked lock state: state=%s n=%d pending=%v", state, n, pending)
}
}
func TestExternalHoldBlocksBoth(t *testing.T) {
lk := New(nil)
ctx := context.Background()
lk.SetExternal("game.exe (pid 42)")
if lk.TryAcquireLLM() {
t.Fatal("TryAcquireLLM succeeded during external hold")
}
if got := lk.External(); got != "game.exe (pid 42)" {
t.Fatalf("External() = %q", got)
}
if state, _, _ := lk.Snapshot(); state != StateExternal {
t.Fatalf("state=%s, want external", state)
}
llmAcquired := make(chan struct{})
go func() {
if err := lk.AcquireLLM(ctx); err != nil {
t.Error(err)
}
close(llmAcquired)
}()
assertBlocked(t, llmAcquired, "LLM acquire during external hold")
imageAcquired := make(chan struct{})
go func() {
if err := lk.AcquireImage(ctx); err != nil {
t.Error(err)
}
close(imageAcquired)
}()
assertBlocked(t, imageAcquired, "image acquire during external hold")
lk.ClearExternal()
// The queued image job wins over the LLM waiter (image priority).
waitFor(t, imageAcquired, "image acquire after ClearExternal")
assertBlocked(t, llmAcquired, "LLM acquire while image active")
lk.ReleaseImage()
waitFor(t, llmAcquired, "LLM acquire after image release")
lk.ReleaseLLM()
}
func TestExternalHoldDoesNotPreempt(t *testing.T) {
lk := New(nil)
ctx := context.Background()
if err := lk.AcquireLLM(ctx); err != nil {
t.Fatal(err)
}
lk.SetExternal("game.exe")
// In-flight LLM work keeps the llm state; the hold blocks new grants.
if state, _, _ := lk.Snapshot(); state != StateLLM {
t.Fatalf("state=%s, want llm while work in flight", state)
}
if lk.TryAcquireLLM() {
t.Fatal("TryAcquireLLM succeeded during external hold")
}
lk.ReleaseLLM()
if state, _, _ := lk.Snapshot(); state != StateExternal {
t.Fatalf("state=%s, want external after drain", state)
}
lk.ClearExternal()
if state, _, _ := lk.Snapshot(); state != StateIdle {
t.Fatalf("state=%s, want idle", state)
}
}
+1 -1
View File
@@ -89,7 +89,7 @@ func (m *Metrics) Render(w io.Writer, state string, llmInflight int, imagePendin
fmt.Fprint(w, `# HELP gpu_turnstile_state Current GPU state (1 for the active state).
# TYPE gpu_turnstile_state gauge
`)
for _, s := range []string{"idle", "llm", "image"} {
for _, s := range []string{"idle", "llm", "image", "external"} {
v := 0
if s == state {
v = 1
+62 -3
View File
@@ -26,6 +26,7 @@ import (
"gpu-turnstile/internal/lock"
"gpu-turnstile/internal/metrics"
"gpu-turnstile/internal/ollama"
"gpu-turnstile/internal/supervise"
)
// defaultCaptureLimit bounds how much of a /prompt response body is
@@ -44,6 +45,11 @@ type Config struct {
Metrics *metrics.Metrics
Log *slog.Logger
// ComfySup, when non-nil, is the managed ComfyUI process: any ComfyUI
// request starts it on demand, /prompt additionally waits for
// readiness before taking the GPU lock. nil = unmanaged upstream.
ComfySup *supervise.Process
// LogColor enables ANSI colors in per-request log lines. Ignored when
// the log level is above INFO (request lines are not emitted at all).
LogColor bool
@@ -438,7 +444,7 @@ func isLLMRequest(r *http.Request) bool {
return r.Method == http.MethodPost && llmPaths[r.URL.Path]
}
// OllamaHandler serves the Ollama-facing listener.
// OllamaHandler serves the listener for Ollama-compatible clients.
func (s *Server) OllamaHandler() http.Handler {
return s.logRequests("ollama", http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
switch r.URL.Path {
@@ -458,10 +464,14 @@ func (s *Server) OllamaHandler() http.Handler {
if s.busyMode == "reject" {
if !s.cfg.Lock.TryAcquireLLM() {
s.cfg.Metrics.ObserveLockWait("llm", time.Since(start).Seconds())
msg := "GPU busy: image job active or queued"
if holder := s.cfg.Lock.External(); holder != "" {
msg = "GPU busy: " + holder
}
s.log.Info("llm request rejected; GPU busy",
"path", r.URL.Path, "status", s.busyStatus)
w.Header().Set("Retry-After", strconv.Itoa(s.busyRetryAfter))
http.Error(w, "GPU busy: image job active or queued", s.busyStatus)
http.Error(w, msg, s.busyStatus)
return
}
s.cfg.Metrics.ObserveLockWait("llm", time.Since(start).Seconds())
@@ -485,7 +495,7 @@ func (s *Server) OllamaHandler() http.Handler {
}))
}
// ComfyHandler serves the ComfyUI-facing listener.
// ComfyHandler serves the listener for ComfyUI clients.
func (s *Server) ComfyHandler() http.Handler {
return s.logRequests("comfy", http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
switch r.URL.Path {
@@ -500,6 +510,21 @@ func (s *Server) ComfyHandler() http.Handler {
s.handlePrompt(w, r)
return
}
if s.cfg.ComfySup != nil {
// Any other ComfyUI request also wakes the managed server; the
// retry backoff bridges the time it needs to come up. While a
// foreign process holds the GPU we refuse to spawn it — the
// request gets the busy answer instead of fighting for VRAM.
if holder := s.cfg.Lock.External(); holder != "" && !s.cfg.ComfySup.Running() {
w.Header().Set("Retry-After", strconv.Itoa(s.busyRetryAfter))
http.Error(w, "GPU busy: "+holder, http.StatusServiceUnavailable)
return
}
if err := s.cfg.ComfySup.EnsureRunning(); err != nil {
http.Error(w, fmt.Sprintf("cannot start ComfyUI: %v", err), http.StatusBadGateway)
return
}
}
s.comfyProxy.ServeHTTP(w, r)
}))
}
@@ -539,6 +564,23 @@ func (w *captureWriter) Unwrap() http.ResponseWriter { return w.ResponseWriter }
func (s *Server) handlePrompt(w http.ResponseWriter, r *http.Request) {
log := s.log.With("op", "image")
// Normally the managed server is brought up *before* taking the GPU
// lock: torch can take a minute to load, and LLM traffic should keep
// flowing in the meantime. While a foreign process holds the GPU, LLM
// traffic is blocked anyway and a fresh ComfyUI would fight it for
// VRAM — so the lock comes first in that case.
comfyFirst := s.cfg.ComfySup != nil && s.cfg.Lock.External() == ""
if comfyFirst {
if err := s.cfg.ComfySup.EnsureRunning(); err != nil {
http.Error(w, fmt.Sprintf("cannot start ComfyUI: %v", err), http.StatusBadGateway)
return
}
if err := s.cfg.ComfySup.WaitReady(r.Context()); err != nil {
http.Error(w, fmt.Sprintf("ComfyUI did not become ready: %v", err), http.StatusBadGateway)
return
}
}
start := time.Now()
if err := s.cfg.Lock.AcquireImage(r.Context()); err != nil {
if errors.Is(err, context.DeadlineExceeded) {
@@ -549,6 +591,19 @@ func (s *Server) handlePrompt(w http.ResponseWriter, r *http.Request) {
s.cfg.Metrics.ObserveLockWait("image", time.Since(start).Seconds())
log.Info("image lock acquired")
if s.cfg.ComfySup != nil && !comfyFirst {
if err := s.cfg.ComfySup.EnsureRunning(); err != nil {
s.cfg.Lock.ReleaseImage()
http.Error(w, fmt.Sprintf("cannot start ComfyUI: %v", err), http.StatusBadGateway)
return
}
if err := s.cfg.ComfySup.WaitReady(r.Context()); err != nil {
s.cfg.Lock.ReleaseImage()
http.Error(w, fmt.Sprintf("ComfyUI did not become ready: %v", err), http.StatusBadGateway)
return
}
}
if s.cfg.Ollama != nil {
uctx, ucancel := context.WithTimeout(r.Context(), s.cfg.UnloadTimeout)
elapsed, uerr := s.cfg.Ollama.UnloadAll(uctx)
@@ -608,6 +663,10 @@ func (s *Server) finishImageJob(promptID string) {
s.cfg.Lock.ReleaseImage()
log.Info("image lock released")
if s.cfg.ComfySup != nil {
// The idle clock starts when the job ends, not when it began.
s.cfg.ComfySup.NoteActivity()
}
if s.cfg.Ollama != nil && s.cfg.WarmModel != "" {
if state, _, _ := s.cfg.Lock.Snapshot(); state == lock.StateIdle {
+2 -2
View File
@@ -46,8 +46,8 @@ type fakes struct {
rec *recorder
ollama *httptest.Server
comfy *httptest.Server
server *httptest.Server // Ollama-facing gpu-turnstile listener
comfySrv *httptest.Server // ComfyUI-facing gpu-turnstile listener
server *httptest.Server // gpu-turnstile listener for Ollama-compatible clients
comfySrv *httptest.Server // gpu-turnstile listener for ComfyUI clients
freeCh chan struct{}
chatCh chan struct{}
historyMu sync.Mutex
+47
View File
@@ -0,0 +1,47 @@
// Shared env-file handling for the installers (Windows and Linux).
package service
import (
"fmt"
"os"
"strings"
"gpu-turnstile/internal/config"
)
// syncEnvFile ensures the installed env file at path is a current
// installer-written sample. A missing file is created; a file without a
// CFG_VER line (hand-written or from before versioning) is invalid and
// replaced by a fresh sample after a .bak backup; a file written by an
// older version gets newly added settings appended via config.SyncSample.
// logPath activates the LOG_FILE line (Windows); empty leaves it commented.
func syncEnvFile(path, version, logPath string) error {
data, err := os.ReadFile(path)
switch {
case os.IsNotExist(err):
return writeEnvFile(path, config.SampleEnv(version, logPath))
case err != nil:
return fmt.Errorf("read %s: %w", path, err)
}
values, perr := config.ParseEnvFile(strings.NewReader(string(data)))
if perr == nil && values["CFG_VER"] != "" {
synced, changed := config.SyncSample(string(data), version, logPath)
if !changed {
return nil
}
return writeEnvFile(path, synced)
}
// No readable CFG_VER: the file is invalid — keep a backup and start
// from a fresh sample.
if err := os.WriteFile(path+".bak", data, 0o644); err != nil {
return fmt.Errorf("back up %s: %w", path, err)
}
return writeEnvFile(path, config.SampleEnv(version, logPath))
}
func writeEnvFile(path, content string) error {
if err := os.WriteFile(path, []byte(content), 0o644); err != nil {
return fmt.Errorf("write %s: %w", path, err)
}
return nil
}
+69 -11
View File
@@ -6,6 +6,7 @@
package service
import (
"bytes"
"context"
"fmt"
"io"
@@ -126,12 +127,18 @@ func copyFile(src, dst string, mode os.FileMode) error {
// copyBin=false the current executable location and config path are
// registered as-is instead. Needs root.
//
// Re-running install converges an existing unit instead of failing: it is
// stopped first if active, the installed binary is replaced only when the
// content differs, the unit file is rewritten (followed by daemon-reload)
// only when it changed, and the service is started again only if it was
// active before.
//
// The binary lives in the StateDirectory rather than /usr/local/sbin on
// purpose: replacing a running binary needs write access to its
// *directory*, and granting the sandboxed service user write access to a
// shared system directory would let a compromised service overwrite other
// binaries. /var/lib/gpu-turnstile is exclusively ours.
func Install(configPath string, copyBin bool) error {
func Install(configPath string, copyBin bool, version string) error {
exe, err := os.Executable()
if err != nil {
return err
@@ -139,6 +146,16 @@ func Install(configPath string, copyBin bool) error {
if abs, absErr := filepath.Abs(exe); absErr == nil {
exe = abs
}
unit := Name + ".service"
_, statErr := os.Stat(unitPath)
fresh := os.IsNotExist(statErr)
wasRunning := exec.Command("systemctl", "is-active", "--quiet", unit).Run() == nil
if wasRunning {
if out, err := exec.Command("systemctl", "stop", unit).CombinedOutput(); err != nil {
return fmt.Errorf("systemctl stop (run as root): %w (%s)", err, out)
}
}
cfg := etcConfig
if copyBin {
if err := os.MkdirAll(stateDir, 0o755); err != nil {
@@ -146,33 +163,74 @@ func Install(configPath string, copyBin bool) error {
}
installedExe := filepath.Join(stateDir, Name)
if exe != installedExe {
if err := copyFile(exe, installedExe, 0o755); err != nil {
return fmt.Errorf("install binary to %s: %w", installedExe, err)
if same, _ := sameFileContent(exe, installedExe); !same {
if err := copyFile(exe, installedExe, 0o755); err != nil {
return fmt.Errorf("install binary to %s: %w", installedExe, err)
}
}
}
exe = installedExe
if _, err := os.Stat(etcConfig); os.IsNotExist(err) && configPath != "" {
// Missing config is not fatal: the service fails fast with a
// clear "no consumer URL" error until the user writes one.
copyFile(configPath, etcConfig, 0o644) //nolint:errcheck // best effort
}
// Missing configs get a fully commented sample (LOG_FILE stays
// commented — stderr goes to the journal on Linux);
// installer-written ones from older versions get new settings
// appended; anything without CFG_VER is invalid and gets replaced
// (backup kept as .bak).
if err := syncEnvFile(etcConfig, version, ""); err != nil {
return err
}
} else if configPath != "" {
if abs, absErr := filepath.Abs(configPath); absErr == nil {
cfg = abs
}
}
if err := os.WriteFile(unitPath, []byte(renderUnit(exe, cfg)), 0o644); err != nil {
return fmt.Errorf("write %s (run as root): %w", unitPath, err)
rendered := renderUnit(exe, cfg)
if old, _ := os.ReadFile(unitPath); string(old) != rendered {
if err := os.WriteFile(unitPath, []byte(rendered), 0o644); err != nil {
return fmt.Errorf("write %s (run as root): %w", unitPath, err)
}
if out, err := exec.Command("systemctl", "daemon-reload").CombinedOutput(); err != nil {
return fmt.Errorf("systemctl daemon-reload: %w (%s)", err, out)
}
}
if out, err := exec.Command("systemctl", "daemon-reload").CombinedOutput(); err != nil {
return fmt.Errorf("systemctl daemon-reload: %w (%s)", err, out)
if fresh {
if out, err := exec.Command("systemctl", "enable", "--now", unit).CombinedOutput(); err != nil {
return fmt.Errorf("systemctl enable --now: %w (%s)", err, out)
}
return nil
}
if out, err := exec.Command("systemctl", "enable", "--now", Name+".service").CombinedOutput(); err != nil {
return fmt.Errorf("systemctl enable --now: %w (%s)", err, out)
if exec.Command("systemctl", "is-enabled", "--quiet", unit).Run() != nil {
if out, err := exec.Command("systemctl", "enable", unit).CombinedOutput(); err != nil {
return fmt.Errorf("systemctl enable: %w (%s)", err, out)
}
}
if wasRunning {
if out, err := exec.Command("systemctl", "start", unit).CombinedOutput(); err != nil {
return fmt.Errorf("systemctl start: %w (%s)", err, out)
}
}
return nil
}
// sameFileContent reports whether two files hold identical bytes. A missing
// destination is simply "different".
func sameFileContent(a, b string) (bool, error) {
ba, err := os.ReadFile(a)
if err != nil {
return false, err
}
bb, err := os.ReadFile(b)
if os.IsNotExist(err) {
return false, nil
}
if err != nil {
return false, err
}
return bytes.Equal(ba, bb), nil
}
// Remove stops and disables the service and deletes the unit file and the
// installed binary. The config file in /etc is left in place (user data).
func Remove() error {
+1 -1
View File
@@ -29,7 +29,7 @@ func Run(run func(ctx context.Context) error) error {
}
// Install is unsupported on non-Windows, non-Linux platforms.
func Install(string, bool) error { return errUnsupported }
func Install(string, bool, string) error { return errUnsupported }
// Remove is unsupported on non-Windows, non-Linux platforms.
func Remove() error { return errUnsupported }
+179 -57
View File
@@ -8,6 +8,7 @@
package service
import (
"bytes"
"context"
"errors"
"fmt"
@@ -109,7 +110,13 @@ func (h *handler) Execute(_ []string, requests <-chan svc.ChangeRequest, status
// config file if it lives elsewhere. The grants must come after
// CreateService: the virtual account's SID only exists once the service is
// registered.
func Install(configPath string, copyBin bool) error {
//
// Re-running install on an existing service converges instead of failing:
// the service is stopped first if running (so the binary can be replaced),
// the installed binary is refreshed only when the content differs, the
// registration is updated only where it drifted, and the service is
// started again only if it was running before.
func Install(configPath string, copyBin bool, version string) error {
exe, err := os.Executable()
if err != nil {
return err
@@ -123,62 +130,191 @@ func Install(configPath string, copyBin bool) error {
}
}
installDir, _ := installDirs()
if copyBin && !strings.EqualFold(filepath.Dir(exe), installDir) {
if err := os.MkdirAll(installDir, 0o755); err != nil {
return fmt.Errorf("create %s: %w", installDir, err)
}
installedExe := filepath.Join(installDir, "gpu-turnstile.exe")
if err := copyFile(exe, installedExe); err != nil {
return fmt.Errorf("copy binary to %s: %w", installedExe, err)
}
exe = installedExe
targetCfg := filepath.Join(installDir, "gpu-turnstile.env")
if configPath != "" && !strings.EqualFold(configPath, targetCfg) {
if _, statErr := os.Stat(targetCfg); os.IsNotExist(statErr) {
copyFile(configPath, targetCfg) //nolint:errcheck // best effort
}
configPath = targetCfg
}
}
m, err := mgr.Connect()
if err != nil {
return fmt.Errorf("connect to service manager (run as administrator): %w", err)
}
defer m.Disconnect()
binPath := fmt.Sprintf(`"%s" -config "%s"`, exe, configPath)
s, err := m.CreateService(Name, binPath, mgr.Config{
StartType: mgr.StartAutomatic,
DisplayName: "gpu-turnstile",
Description: "GPU arbitration proxy for Ollama and ComfyUI",
ServiceStartName: virtualAccount,
})
if err != nil {
return fmt.Errorf("create service: %w", err)
// An existing service is converged, not an error. Stop it first so the
// binary copy can be replaced, and remember whether to start it again.
var s *mgr.Service
wasRunning := false
if existing, openErr := m.OpenService(Name); openErr == nil {
s = existing
defer s.Close()
if st, qErr := s.Query(); qErr == nil &&
(st.State == svc.Running || st.State == svc.StartPending) {
wasRunning = true
if err := stopAndWait(s); err != nil {
return err
}
}
}
defer s.Close()
restart := mgr.RecoveryAction{Type: mgr.ServiceRestart, Delay: 5 * time.Second}
if err := s.SetRecoveryActions([]mgr.RecoveryAction{restart, restart, restart}, 24*60*60); err != nil {
installDir, dataDir := installDirs()
if copyBin {
targetCfg := filepath.Join(installDir, "gpu-turnstile.env")
if !strings.EqualFold(filepath.Dir(exe), installDir) {
if err := os.MkdirAll(installDir, 0o755); err != nil {
return fmt.Errorf("create %s: %w", installDir, err)
}
installedExe := filepath.Join(installDir, "gpu-turnstile.exe")
if same, _ := sameFileContent(exe, installedExe); !same {
if err := copyFile(exe, installedExe); err != nil {
return fmt.Errorf("copy binary to %s: %w", installedExe, err)
}
}
exe = installedExe
if configPath != "" && !strings.EqualFold(configPath, targetCfg) {
if _, statErr := os.Stat(targetCfg); os.IsNotExist(statErr) {
copyFile(configPath, targetCfg) //nolint:errcheck // best effort
}
}
}
// A service has no console: without LOG_FILE the output vanishes, so
// the sample pre-wires it to ProgramData. Missing configs get a
// fresh sample; installer-written ones from older versions get new
// settings appended; anything without CFG_VER is invalid and gets
// replaced (backup kept as .bak).
logPath := filepath.Join(dataDir, "gpu-turnstile.log")
if err := syncEnvFile(targetCfg, version, logPath); err != nil {
return err
}
configPath = targetCfg
}
binPath := fmt.Sprintf(`"%s" -config "%s"`, exe, configPath)
if s == nil {
s, err = m.CreateService(Name, binPath, mgr.Config{
StartType: mgr.StartAutomatic,
DisplayName: "gpu-turnstile",
Description: "GPU arbitration proxy for Ollama and ComfyUI",
ServiceStartName: virtualAccount,
})
if err != nil {
return fmt.Errorf("create service: %w", err)
}
defer s.Close()
if err := ensureRecovery(s); err != nil {
s.Delete() // roll back so a retry starts clean
return err
}
if err := grantAll(exe, configPath); err != nil {
s.Delete() // roll back so a retry starts clean
return err
}
// Best effort: start now instead of waiting for the next boot. A
// missing config (no consumer URLs) fails the start; the service stays
// registered and can be started once the config exists.
s.Start()
return nil
}
// Existing service: update the registration only where it drifted.
cur, err := s.Config()
if err != nil {
return fmt.Errorf("query service config: %w", err)
}
const displayName = "gpu-turnstile"
const description = "GPU arbitration proxy for Ollama and ComfyUI"
if cur.BinaryPathName != binPath ||
cur.StartType != mgr.StartAutomatic ||
cur.ServiceStartName != virtualAccount ||
cur.DisplayName != displayName ||
cur.Description != description {
upd := cur
upd.BinaryPathName = binPath
upd.StartType = mgr.StartAutomatic
upd.ServiceStartName = virtualAccount
upd.DisplayName = displayName
upd.Description = description
if err := s.UpdateConfig(upd); err != nil {
return fmt.Errorf("update service config: %w", err)
}
}
if err := ensureRecovery(s); err != nil {
return err
}
// Idempotent: re-assert the virtual account's ACLs (granting an
// existing ACE is a no-op).
if err := grantAll(exe, configPath); err != nil {
return err
}
if wasRunning {
if err := s.Start(); err != nil {
return fmt.Errorf("start service: %w", err)
}
}
return nil
}
// ensureRecovery makes sure the service restarts 5s after a failure — the
// mechanism that also brings up a staged update. The current actions are
// queried first so an up-to-date service is left untouched.
func ensureRecovery(s *mgr.Service) error {
want := mgr.RecoveryAction{Type: mgr.ServiceRestart, Delay: 5 * time.Second}
actions, err := s.RecoveryActions()
onFailure, flagErr := s.RecoveryActionsOnNonCrashFailures()
if err == nil && flagErr == nil && onFailure && len(actions) == 3 {
ok := true
for _, a := range actions {
if a.Type != want.Type || a.Delay != want.Delay {
ok = false
}
}
if ok {
return nil
}
}
if err := s.SetRecoveryActions([]mgr.RecoveryAction{want, want, want}, 24*60*60); err != nil {
return fmt.Errorf("set recovery actions: %w", err)
}
if err := s.SetRecoveryActionsOnNonCrashFailures(true); err != nil {
return fmt.Errorf("set failure actions flag: %w", err)
}
if err := grantAll(exe, configPath); err != nil {
s.Delete() // roll back so a retry starts clean
return err
}
// Best effort: start now instead of waiting for the next boot. A
// missing config (no consumer URLs) fails the start; the service stays
// registered and can be started once the config exists.
s.Start()
return nil
}
// stopAndWait stops the service and waits up to 30s for the stopped state.
func stopAndWait(s *mgr.Service) error {
if _, err := s.Control(svc.Stop); err != nil {
return fmt.Errorf("stop service: %w", err)
}
deadline := time.Now().Add(30 * time.Second)
for {
st, err := s.Query()
if err != nil {
return fmt.Errorf("query service: %w", err)
}
if st.State == svc.Stopped {
return nil
}
if time.Now().After(deadline) {
return fmt.Errorf("service did not stop within 30s")
}
time.Sleep(300 * time.Millisecond)
}
}
// sameFileContent reports whether two files hold identical bytes. A missing
// destination is simply "different".
func sameFileContent(a, b string) (bool, error) {
ba, err := os.ReadFile(a)
if err != nil {
return false, err
}
bb, err := os.ReadFile(b)
if os.IsNotExist(err) {
return false, nil
}
if err != nil {
return false, err
}
return bytes.Equal(ba, bb), nil
}
// copyFile copies src to dst (0755 on the new file).
func copyFile(src, dst string) error {
in, err := os.Open(src)
@@ -374,22 +510,8 @@ func RestartIfRunning() (bool, error) {
if st.State != svc.Running && st.State != svc.StartPending {
return false, nil
}
if _, err := s.Control(svc.Stop); err != nil {
return false, fmt.Errorf("stop service: %w", err)
}
deadline := time.Now().Add(30 * time.Second)
for {
st, err = s.Query()
if err != nil {
return false, fmt.Errorf("query service: %w", err)
}
if st.State == svc.Stopped {
break
}
if time.Now().After(deadline) {
return false, fmt.Errorf("service did not stop within 30s")
}
time.Sleep(300 * time.Millisecond)
if err := stopAndWait(s); err != nil {
return false, err
}
if err := s.Start(); err != nil {
return false, fmt.Errorf("start service: %w", err)
+273
View File
@@ -0,0 +1,273 @@
// Package supervise runs an upstream server (ComfyUI) as a managed child
// process: started on demand when a request needs it, stopped after an
// idle timeout so the GPU memory it holds is freed, and stopped with the
// parent. Crashes are logged; the next request respawns it.
package supervise
import (
"context"
"fmt"
"io"
"log/slog"
"os/exec"
"runtime"
"strings"
"sync"
"time"
)
// Process is one managed child process.
type Process struct {
name string
argv []string
dir string
probe func(context.Context) error
startTimeout time.Duration
log *slog.Logger
mu sync.Mutex
cmd *exec.Cmd
stopping bool
ready bool
external bool // someone else serves the port; not our process
lastActivity time.Time
}
// New parses cmdLine (double quotes group arguments containing spaces) and
// prepares a managed process. probe reports whether the server answers
// (e.g. the comfy client's Probe); startTimeout bounds WaitReady. dir is
// the child's working directory; empty inherits ours.
func New(name, cmdLine, dir string, probe func(context.Context) error, startTimeout time.Duration, log *slog.Logger) (*Process, error) {
argv, err := splitCommandLine(cmdLine)
if err != nil {
return nil, fmt.Errorf("%s command: %w", name, err)
}
if len(argv) == 0 {
return nil, fmt.Errorf("%s command is empty", name)
}
if log == nil {
log = slog.Default()
}
if startTimeout <= 0 {
startTimeout = 2 * time.Minute
}
return &Process{name: name, argv: argv, dir: dir, probe: probe, startTimeout: startTimeout, log: log}, nil
}
// splitCommandLine splits a command line on whitespace, treating
// double-quoted sections as one argument (quotes removed). Backslashes are
// literal — this matches Windows paths.
func splitCommandLine(s string) ([]string, error) {
var argv []string
var cur strings.Builder
inQuote := false
have := false
flush := func() {
if have {
argv = append(argv, cur.String())
cur.Reset()
have = false
}
}
for _, r := range s {
switch {
case r == '"':
inQuote = !inQuote
have = true
case (r == ' ' || r == '\t') && !inQuote:
flush()
default:
cur.WriteRune(r)
have = true
}
}
if inQuote {
return nil, fmt.Errorf("unterminated quote in %q", s)
}
flush()
return argv, nil
}
// Running reports whether the child process is currently alive.
func (p *Process) Running() bool {
p.mu.Lock()
defer p.mu.Unlock()
return p.cmd != nil
}
// Ready reports whether the server has answered a probe since its last
// (re)start. Health checks use it to tell "starting up" from "outage".
func (p *Process) Ready() bool {
p.mu.Lock()
defer p.mu.Unlock()
return p.ready
}
// MarkReady records that the server answered.
func (p *Process) MarkReady() {
p.mu.Lock()
p.ready = true
p.mu.Unlock()
}
// NoteActivity resets the idle clock; called for every request served.
func (p *Process) NoteActivity() {
p.mu.Lock()
p.lastActivity = time.Now()
p.mu.Unlock()
}
// EnsureRunning starts the child if it is not running. It returns as soon
// as the process is spawned; readiness is WaitReady's job (and the proxy's
// retry backoff bridges the gap for plain proxied requests). When the URL
// already answers — e.g. the ComfyUI desktop app grabbed the port — no
// child is spawned: the external server is used as-is, and the idle
// watcher never touches it (it only kills its own child).
func (p *Process) EnsureRunning() error {
p.mu.Lock()
defer p.mu.Unlock()
p.lastActivity = time.Now()
if p.cmd != nil {
return nil
}
pctx, cancel := context.WithTimeout(context.Background(), 2*time.Second)
err := p.probe(pctx)
cancel()
if err == nil {
p.ready = true
if !p.external {
p.external = true
p.log.Info(p.name + " is already served externally; not spawning a managed instance")
}
return nil
}
p.external = false
cmd := exec.Command(p.argv[0], p.argv[1:]...)
cmd.Dir = p.dir
stdout, err := cmd.StdoutPipe()
if err != nil {
return err
}
stderr, err := cmd.StderrPipe()
if err != nil {
return err
}
if err := cmd.Start(); err != nil {
return fmt.Errorf("start %s: %w", p.name, err)
}
p.cmd = cmd
p.stopping = false
p.ready = false
go p.pipeLog(stdout)
go p.pipeLog(stderr)
go func() {
err := cmd.Wait()
p.mu.Lock()
p.cmd = nil
p.ready = false
intentional := p.stopping
p.mu.Unlock()
if intentional {
p.log.Info(p.name + " stopped")
} else {
p.log.Warn(p.name+" exited unexpectedly; the next request restarts it", "err", err)
}
}()
p.log.Info(p.name+" starting", "pid", cmd.Process.Pid, "cmd", strings.Join(p.argv, " "))
return nil
}
// WaitReady blocks until the probe succeeds, ctx ends, or the start
// timeout passes.
func (p *Process) WaitReady(ctx context.Context) error {
ctx, cancel := context.WithTimeout(ctx, p.startTimeout)
defer cancel()
for {
pctx, pcancel := context.WithTimeout(ctx, 5*time.Second)
err := p.probe(pctx)
pcancel()
if err == nil {
p.NoteActivity()
p.MarkReady()
return nil
}
select {
case <-ctx.Done():
return fmt.Errorf("%s did not become ready: %w", p.name, ctx.Err())
case <-time.After(500 * time.Millisecond):
}
}
}
// Stop kills the child process (the whole tree on Windows) if running.
func (p *Process) Stop() {
p.mu.Lock()
cmd := p.cmd
if cmd == nil {
p.mu.Unlock()
return
}
p.stopping = true
p.ready = false
p.mu.Unlock()
stopTree(cmd)
}
// WatchIdle stops the child after idleTimeout without activity, but only
// when gpuIdle reports the GPU lock is free (no active or pending work).
// Returns when ctx ends.
func (p *Process) WatchIdle(ctx context.Context, idleTimeout time.Duration, gpuIdle func() bool) {
ticker := time.NewTicker(5 * time.Second)
defer ticker.Stop()
for {
select {
case <-ctx.Done():
return
case <-ticker.C:
}
p.mu.Lock()
idleFor := time.Since(p.lastActivity)
running := p.cmd != nil
p.mu.Unlock()
if running && idleFor > idleTimeout && gpuIdle() {
p.log.Info(p.name+" idle; stopping to free the GPU", "idle_for", idleFor.Round(time.Second))
p.Stop()
}
}
}
// pipeLog forwards one child output stream to the log at INFO, line by
// line, prefixed with the process name.
func (p *Process) pipeLog(r io.Reader) {
buf := make([]byte, 4096)
var line string
for {
n, err := r.Read(buf)
line += string(buf[:n])
for {
i := strings.IndexByte(line, '\n')
if i < 0 {
break
}
p.log.Info(p.name + ": " + strings.TrimRight(line[:i], "\r"))
line = line[i+1:]
}
if err != nil {
if strings.TrimSpace(line) != "" {
p.log.Info(p.name + ": " + line)
}
return
}
}
}
// stopTree kills cmd's process, including its children on Windows (python
// launchers tend to spawn some). The Wait goroutine reaps it.
func stopTree(cmd *exec.Cmd) {
if runtime.GOOS == "windows" {
exec.Command("taskkill", "/T", "/F", "/PID",
fmt.Sprint(cmd.Process.Pid)).Run() //nolint:errcheck // best effort
return
}
cmd.Process.Kill() //nolint:errcheck // best effort
}
+192
View File
@@ -0,0 +1,192 @@
package supervise
import (
"context"
"errors"
"log/slog"
"net/http"
"net/http/httptest"
"os"
"testing"
"time"
)
func TestSplitCommandLine(t *testing.T) {
cases := []struct {
in string
want []string
}{
{`python main.py --port 8188`, []string{"python", "main.py", "--port", "8188"}},
{`"C:\Program Files\py\python.exe" main.py`, []string{`C:\Program Files\py\python.exe`, "main.py"}},
{` spaced out `, []string{"spaced", "out"}},
{`a "b c" d`, []string{"a", "b c", "d"}},
{"", nil},
}
for _, c := range cases {
got, err := splitCommandLine(c.in)
if err != nil {
t.Errorf("splitCommandLine(%q): %v", c.in, err)
continue
}
if len(got) != len(c.want) {
t.Errorf("splitCommandLine(%q) = %v, want %v", c.in, got, c.want)
continue
}
for i := range got {
if got[i] != c.want[i] {
t.Errorf("splitCommandLine(%q) = %v, want %v", c.in, got, c.want)
break
}
}
}
if _, err := splitCommandLine(`"unterminated`); err == nil {
t.Error("expected error for unterminated quote")
}
}
// TestHelperProcess is the child executed by the lifecycle tests: it just
// sleeps. The env marker is set only around the spawn, so in the normal
// test run this returns immediately.
func TestHelperProcess(t *testing.T) {
if os.Getenv("GO_HELPER_PROCESS") != "1" {
return
}
time.Sleep(30 * time.Second)
os.Exit(0)
}
func newHelper(t *testing.T, name string) *Process {
t.Helper()
// The probe always fails: nothing external serves the port, so
// EnsureRunning spawns the helper child.
p, err := New(name, `"`+os.Args[0]+`" -test.run=TestHelperProcess`, "",
func(context.Context) error { return errors.New("nothing there") }, 5*time.Second, slog.Default())
if err != nil {
t.Fatal(err)
}
return p
}
// startHelper spawns the child with the marker set; exec.Command inherits
// the environment at spawn time, so it can be unset right after.
func startHelper(t *testing.T, p *Process) {
t.Helper()
os.Setenv("GO_HELPER_PROCESS", "1")
defer os.Unsetenv("GO_HELPER_PROCESS")
if err := p.EnsureRunning(); err != nil {
t.Fatal(err)
}
}
func waitStopped(t *testing.T, p *Process, timeout time.Duration) {
t.Helper()
deadline := time.Now().Add(timeout)
for p.Running() && time.Now().Before(deadline) {
time.Sleep(50 * time.Millisecond)
}
if p.Running() {
t.Fatal("process still running")
}
}
func TestEnsureRunningAndStop(t *testing.T) {
p := newHelper(t, "helper")
if p.Running() {
t.Fatal("Running before start")
}
startHelper(t, p)
if !p.Running() {
t.Fatal("not Running after EnsureRunning")
}
if err := p.EnsureRunning(); err != nil {
t.Fatal("second EnsureRunning must be a no-op")
}
p.Stop()
waitStopped(t, p, 5*time.Second)
}
func TestEnsureRunningPrefersExternalServer(t *testing.T) {
// The port is already served (e.g. the ComfyUI desktop app): no child
// is spawned, the supervisor reports ready, and Stop is a no-op.
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
w.WriteHeader(http.StatusOK)
}))
defer srv.Close()
probe := func(ctx context.Context) error {
req, _ := http.NewRequestWithContext(ctx, http.MethodGet, srv.URL, nil)
resp, err := http.DefaultClient.Do(req)
if err != nil {
return err
}
resp.Body.Close()
return nil
}
p, err := New("external", `"`+os.Args[0]+`"`, "", probe, 5*time.Second, slog.Default())
if err != nil {
t.Fatal(err)
}
if err := p.EnsureRunning(); err != nil {
t.Fatal(err)
}
if p.Running() {
t.Fatal("spawned a child even though the port is already served")
}
if !p.Ready() {
t.Fatal("external server should count as ready")
}
p.Stop() // must not touch the external server
}
func TestWaitReady(t *testing.T) {
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
w.WriteHeader(http.StatusOK)
}))
defer srv.Close()
probe := func(ctx context.Context) error {
req, _ := http.NewRequestWithContext(ctx, http.MethodGet, srv.URL, nil)
resp, err := http.DefaultClient.Do(req)
if err != nil {
return err
}
resp.Body.Close()
return nil
}
p, err := New("ready", `"`+os.Args[0]+`"`, "", probe, 5*time.Second, slog.Default())
if err != nil {
t.Fatal(err)
}
if err := p.WaitReady(context.Background()); err != nil {
t.Fatal(err)
}
failing, err := New("failing", `"`+os.Args[0]+`"`, "",
func(context.Context) error { return errors.New("no") }, 500*time.Millisecond, slog.Default())
if err != nil {
t.Fatal(err)
}
if err := failing.WaitReady(context.Background()); err == nil {
t.Fatal("expected timeout error from WaitReady")
}
}
func TestWatchIdleStops(t *testing.T) {
p := newHelper(t, "idle")
startHelper(t, p)
ctx, cancel := context.WithCancel(context.Background())
defer cancel()
go p.WatchIdle(ctx, 200*time.Millisecond, func() bool { return true })
waitStopped(t, p, 10*time.Second)
}
func TestWatchIdleRespectsBusyGPU(t *testing.T) {
p := newHelper(t, "busy")
startHelper(t, p)
defer p.Stop()
ctx, cancel := context.WithCancel(context.Background())
go p.WatchIdle(ctx, 100*time.Millisecond, func() bool { return false })
time.Sleep(600 * time.Millisecond)
cancel()
if !p.Running() {
t.Fatal("process was stopped while the GPU was busy")
}
}
+41 -16
View File
@@ -26,11 +26,15 @@ import (
// maxAssetSize bounds release asset downloads.
const maxAssetSize = 512 << 20
// Updater checks one Gitea repository for newer releases.
// Updater checks one Gitea repository for releases.
type Updater struct {
Repo string // e.g. https://git.rambossek.at/PUBLIC/gpu-turnstile
Asset string // e.g. gpu-turnstile.exe
Version string // current version, e.g. v0.1.2 ("dev" disables updates)
Version string // current binary version, e.g. v0.1.2 ("dev" cannot be compared)
// Desired is the APP_VER policy: "dev" disables updates, "stable" (or
// empty) tracks the latest release, anything else is an exact vX.Y.Z
// release tag to pin — staging it even when that means a downgrade.
Desired string
Log *slog.Logger
Client *http.Client
}
@@ -167,15 +171,20 @@ func CleanupOld(exePath string) {
os.Remove(exePath + ".new")
}
// Check performs a single update check. staged is true when a newer,
// Check performs a single update check. staged is true when a
// signature-verified binary has been swapped into place at exePath; the
// caller should then restart the process. A nil error with staged=false
// means "no action" (up to date, disabled, or dev build); a non-nil error
// means the check failed and the running binary is untouched.
// means "no action" (up to date, APP_VER=dev, or no embedded public key);
// a non-nil error means the check failed and the running binary is
// untouched.
func (u *Updater) Check(ctx context.Context, exePath string) (staged bool, err error) {
log := u.logger()
if u.Version == "" || u.Version == "dev" {
log.Debug("auto-update: dev build, skipping")
desired := u.Desired
if desired == "" {
desired = "stable"
}
if desired == "dev" {
log.Debug("auto-update: APP_VER=dev, skipping")
return false, nil
}
if publicKeyPEM == "" {
@@ -187,21 +196,37 @@ func (u *Updater) Check(ctx context.Context, exePath string) (staged bool, err e
return false, err
}
body, err := u.get(ctx, api+"/releases/latest")
pinned := desired != "stable"
endpoint := api + "/releases/latest"
if pinned {
endpoint = api + "/releases/tags/" + desired
}
body, err := u.get(ctx, endpoint)
if err != nil {
return false, fmt.Errorf("fetch latest release: %w", err)
return false, fmt.Errorf("fetch release: %w", err)
}
var rel release
if err := json.Unmarshal(body, &rel); err != nil {
return false, fmt.Errorf("parse release: %w", err)
}
newer, err := newerVersion(u.Version, rel.TagName)
if err != nil {
return false, err
}
if !newer {
log.Debug("auto-update: up to date", "version", u.Version, "latest", rel.TagName)
return false, nil
if pinned {
// Pin mode: any difference from the target tag means stage it —
// including downgrades and replacing a dev binary.
if u.Version == rel.TagName {
log.Debug("auto-update: already on pinned version", "version", u.Version)
return false, nil
}
} else if u.Version != "" && u.Version != "dev" {
// Stable mode: only strictly newer releases count; a dev binary
// cannot be compared and is always replaced by the latest release.
newer, err := newerVersion(u.Version, rel.TagName)
if err != nil {
return false, err
}
if !newer {
log.Debug("auto-update: up to date", "version", u.Version, "latest", rel.TagName)
return false, nil
}
}
urls := make(map[string]string, len(rel.Assets))
+62 -5
View File
@@ -43,7 +43,7 @@ func newFakeGitea(t *testing.T, tag string, assetContent []byte) *fakeGitea {
sign := func() []byte { return ed25519.Sign(priv, f.asset) }
mux := http.NewServeMux()
mux.HandleFunc("/api/v1/repos/o/r/releases/latest", func(w http.ResponseWriter, r *http.Request) {
serveRelease := func(w http.ResponseWriter, r *http.Request) {
assets := []map[string]string{
{"name": "gpu-turnstile.exe", "browser_download_url": f.srv.URL + "/dl/exe"},
{"name": "gpu-turnstile.exe.sha256", "browser_download_url": f.srv.URL + "/dl/sha"},
@@ -52,7 +52,9 @@ func newFakeGitea(t *testing.T, tag string, assetContent []byte) *fakeGitea {
assets = append(assets, map[string]string{"name": "gpu-turnstile.exe.sig", "browser_download_url": f.srv.URL + "/dl/sig"})
}
json.NewEncoder(w).Encode(map[string]any{"tag_name": f.tag, "assets": assets})
})
}
mux.HandleFunc("/api/v1/repos/o/r/releases/latest", serveRelease)
mux.HandleFunc("/api/v1/repos/o/r/releases/tags/"+f.tag, serveRelease)
mux.HandleFunc("/dl/exe", func(w http.ResponseWriter, r *http.Request) { w.Write(f.asset) })
mux.HandleFunc("/dl/sig", func(w http.ResponseWriter, r *http.Request) {
sig := sign()
@@ -73,6 +75,12 @@ func (f *fakeGitea) updater(version string) *Updater {
return &Updater{Repo: f.srv.URL + "/o/r", Asset: "gpu-turnstile.exe", Version: version}
}
func (f *fakeGitea) updaterDesired(version, desired string) *Updater {
u := f.updater(version)
u.Desired = desired
return u
}
func fakeExe(t *testing.T) string {
t.Helper()
exe := filepath.Join(t.TempDir(), "gpu-turnstile.exe")
@@ -153,12 +161,61 @@ func TestCheckSkipsWithoutPublicKey(t *testing.T) {
}
}
func TestCheckSkipsDevBuild(t *testing.T) {
func TestCheckDevBuildGetsStable(t *testing.T) {
// A dev binary cannot be compared; APP_VER=stable replaces it with the
// latest release.
f := newFakeGitea(t, "v9.9.9", []byte("new-binary"))
withPublicKey(t, f.pubPEM)
staged, err := f.updater("dev").Check(context.Background(), fakeExe(t))
exe := fakeExe(t)
staged, err := f.updater("dev").Check(context.Background(), exe)
if err != nil {
t.Fatal(err)
}
if !staged {
t.Fatal("expected dev binary to be replaced by the latest release")
}
content, _ := os.ReadFile(exe)
if string(content) != "new-binary" {
t.Fatalf("exe content = %q", content)
}
}
func TestCheckDesiredDevDisables(t *testing.T) {
f := newFakeGitea(t, "v9.9.9", []byte("new-binary"))
withPublicKey(t, f.pubPEM)
for _, version := range []string{"dev", "v0.1.2"} {
staged, err := f.updaterDesired(version, "dev").Check(context.Background(), fakeExe(t))
if err != nil || staged {
t.Fatalf("version %s: staged=%v err=%v, want no action with APP_VER=dev", version, staged, err)
}
}
}
func TestCheckPinned(t *testing.T) {
// Pin mode stages the exact tag — up or down — and skips when the
// binary already matches.
for _, version := range []string{"v0.1.2", "v9.9.9", "dev"} {
f := newFakeGitea(t, "v0.5.0", []byte("pinned-binary"))
withPublicKey(t, f.pubPEM)
exe := fakeExe(t)
staged, err := f.updaterDesired(version, "v0.5.0").Check(context.Background(), exe)
if err != nil {
t.Fatalf("version %s: %v", version, err)
}
if !staged {
t.Fatalf("version %s: expected pinned v0.5.0 to be staged", version)
}
content, _ := os.ReadFile(exe)
if string(content) != "pinned-binary" {
t.Fatalf("version %s: exe content = %q", version, content)
}
}
f := newFakeGitea(t, "v0.5.0", []byte("pinned-binary"))
withPublicKey(t, f.pubPEM)
staged, err := f.updaterDesired("v0.5.0", "v0.5.0").Check(context.Background(), fakeExe(t))
if err != nil || staged {
t.Fatalf("staged=%v err=%v, want no action for dev build", staged, err)
t.Fatalf("staged=%v err=%v, want no action when already on the pinned version", staged, err)
}
}