Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
21e40a1774 | ||
|
|
bdc844872d | ||
|
|
299b6dc0bb | ||
|
|
9589e58ce6 | ||
|
|
be0bb36317 | ||
|
|
9b267533c9 | ||
|
|
6f092ddc12 | ||
|
|
0228ccc296 | ||
|
|
9997913929 | ||
|
|
482731ed9e | ||
|
|
e4bdc92ece | ||
|
|
14120bf4a4 | ||
|
|
d7566329ae | ||
|
|
e43ad02fc4 | ||
|
|
232f5b61f2 | ||
|
|
5e7a042cad | ||
|
|
14c2e30478 | ||
|
|
dcdb2cc72e | ||
|
|
44a8e9fbdc | ||
|
|
272a3a467d | ||
|
|
cedf28a809 | ||
|
|
e46e92bee8 | ||
|
|
cf48796183 | ||
|
|
dd3187f951 | ||
|
|
1394a76eea | ||
|
|
da02457fd5 | ||
|
|
6a6359a630 | ||
|
|
e4e4348e8d | ||
|
|
b421bb7bfb | ||
|
|
909918657f | ||
|
|
a0435c858e | ||
|
|
fbab0bba33 | ||
|
|
802a64280f | ||
|
|
a88955e35c | ||
|
|
97624470eb |
@@ -4,3 +4,6 @@
|
|||||||
/compose.yml
|
/compose.yml
|
||||||
/signing/
|
/signing/
|
||||||
|
|
||||||
|
/gpu-turnstile.exe.old
|
||||||
|
/gpu-turnstile.exe.new
|
||||||
|
/gpu-turnstile.exe~
|
||||||
|
|||||||
@@ -2,9 +2,10 @@
|
|||||||
|
|
||||||
GPU arbitration proxy for Ollama + ComfyUI. One consumer GPU is shared by an
|
GPU arbitration proxy for Ollama + ComfyUI. One consumer GPU is shared by an
|
||||||
LLM server (Ollama) and an image generator (ComfyUI); gpu-turnstile sits in
|
LLM server (Ollama) and an image generator (ComfyUI); gpu-turnstile sits in
|
||||||
front of both and guarantees the GPU is always in exactly one of three states:
|
front of both and guarantees the GPU is always in exactly one of four states:
|
||||||
`idle`, `llm` (N ≥ 1 Ollama requests in flight), or `image` (exactly one
|
`idle`, `llm` (N ≥ 1 Ollama requests in flight), `image` (exactly one
|
||||||
ComfyUI job, Ollama models unloaded). See [SPEC.md](SPEC.md) for the full
|
ComfyUI job, Ollama models unloaded), or `external` (a foreign process such
|
||||||
|
as a game holds the GPU). See [SPEC.md](SPEC.md) for the full
|
||||||
design.
|
design.
|
||||||
|
|
||||||
gpu-turnstile listens on the ports the services normally use; the actual
|
gpu-turnstile listens on the ports the services normally use; the actual
|
||||||
@@ -32,8 +33,9 @@ Open WebUI / n8n ────► :8188 ───┘
|
|||||||
Each consumer is enabled by setting its URL (`OLLAMA_URL`, `COMFY_URL`) and
|
Each consumer is enabled by setting its URL (`OLLAMA_URL`, `COMFY_URL`) and
|
||||||
disabled by leaving it empty — at least one is required. With only Ollama
|
disabled by leaving it empty — at least one is required. With only Ollama
|
||||||
the proxy is a pass-through (no image jobs can arrive); with only ComfyUI
|
the proxy is a pass-through (no image jobs can arrive); with only ComfyUI
|
||||||
the Ollama unload/warm steps are skipped. Future consumers (e.g. local game
|
the Ollama unload/warm steps are skipped. A third, URL-less consumer —
|
||||||
detection) plug into the same lock the same way.
|
detection of foreign GPU holders such as games — is enabled by `GAME_PROCS`
|
||||||
|
and/or `GPU_FOREIGN_VRAM_MB` (see below).
|
||||||
|
|
||||||
## Configuration
|
## Configuration
|
||||||
|
|
||||||
@@ -45,8 +47,8 @@ override file values. Invalid values fail at startup.
|
|||||||
|
|
||||||
| Var | Default | Meaning |
|
| Var | Default | Meaning |
|
||||||
|---|---|---|
|
|---|---|---|
|
||||||
| `LISTEN_OLLAMA` | `:11434` | Ollama-facing listener |
|
| `LISTEN_OLLAMA` | `:11434` | Listener for Ollama-compatible clients |
|
||||||
| `LISTEN_COMFY` | `:8188` | ComfyUI-facing listener |
|
| `LISTEN_COMFY` | `:8188` | Listener for ComfyUI clients |
|
||||||
| `OLLAMA_URL` | _(empty = disabled)_ | Ollama upstream; set to enable the Ollama consumer |
|
| `OLLAMA_URL` | _(empty = disabled)_ | Ollama upstream; set to enable the Ollama consumer |
|
||||||
| `COMFY_URL` | _(empty = disabled)_ | ComfyUI upstream; set to enable the ComfyUI consumer |
|
| `COMFY_URL` | _(empty = disabled)_ | ComfyUI upstream; set to enable the ComfyUI consumer |
|
||||||
| `UNLOAD_TIMEOUT` | `60s` | Wait for Ollama to unload before an image job |
|
| `UNLOAD_TIMEOUT` | `60s` | Wait for Ollama to unload before an image job |
|
||||||
@@ -56,12 +58,21 @@ override file values. Invalid values fail at startup.
|
|||||||
| `LLM_BUSY_STATUS` | `503` | HTTP status for rejected LLM requests in reject mode (400–599, e.g. 429) |
|
| `LLM_BUSY_STATUS` | `503` | HTTP status for rejected LLM requests in reject mode (400–599, e.g. 429) |
|
||||||
| `BUSY_RETRY_AFTER` | `30` | Seconds sent as `Retry-After` on busy responses (both modes) |
|
| `BUSY_RETRY_AFTER` | `30` | Seconds sent as `Retry-After` on busy responses (both modes) |
|
||||||
| `WARM_MODEL` | _(empty)_ | Model to reload after an image job (off by default) |
|
| `WARM_MODEL` | _(empty)_ | Model to reload after an image job (off by default) |
|
||||||
|
| `COMFY_CMD` | _(derived from `COMFY_DIR`; both empty = unmanaged)_ | Supervise ComfyUI: start on demand, stop when idle to free VRAM. Requires `COMFY_URL` |
|
||||||
|
| `COMFY_DIR` | _(empty)_ | Standard venv install root: set alone to supervise ComfyUI with the derived command (`.venv` + `main.py`); also the working directory for `COMFY_CMD` |
|
||||||
|
| `COMFY_IDLE_TIMEOUT` | `5m` | Stop the managed ComfyUI after this long idle |
|
||||||
|
| `COMFY_START_TIMEOUT` | `2m` | Max wait for the managed ComfyUI to come up |
|
||||||
|
| `GAME_PROCS` | _(empty = disabled)_ | Process names (comma-separated); while any runs, the GPU counts as held: requests wait, Ollama unloads, managed ComfyUI stops |
|
||||||
|
| `GPU_FOREIGN_VRAM_MB` | `0` (disabled) | Also treat the GPU as held when a non-ignored process uses more VRAM than this (needs nvidia-smi) |
|
||||||
|
| `GPU_IGNORE_PROCS` | `ollama,ollama app,ollama_llama_server,python,pythonw` | Process names never counted as foreign GPU users |
|
||||||
|
| `GAME_POLL_INTERVAL` | `15s` | How often game/VRAM detection runs (don't go below ~10s — nvidia-smi polls keep the GPU awake) |
|
||||||
| `LOGLEVEL` | `warn` | `info` logs every request (colored arrows in text mode), `debug` adds lock transitions. `LOG_LEVEL` works as an alias |
|
| `LOGLEVEL` | `warn` | `info` logs every request (colored arrows in text mode), `debug` adds lock transitions. `LOG_LEVEL` works as an alias |
|
||||||
| `LOG_FORMAT` | `text` | `json` for structured JSON logs |
|
| `LOG_FORMAT` | `text` | `json` for structured JSON logs |
|
||||||
| `LOG_FILE` | _(empty)_ | Append logs to this file instead of stderr |
|
| `LOG_FILE` | _(empty)_ | Append logs to this file instead of stderr |
|
||||||
| `UNLOAD_POLL_INTERVAL` | `500ms` | `/api/ps` poll interval while unloading |
|
| `UNLOAD_POLL_INTERVAL` | `500ms` | `/api/ps` poll interval while unloading |
|
||||||
| `HISTORY_POLL_INTERVAL` | `1s` | `/history/<id>` poll interval while a job runs |
|
| `HISTORY_POLL_INTERVAL` | `1s` | `/history/<id>` poll interval while a job runs |
|
||||||
| `PROBE_TIMEOUT` | `5s` | Startup probe of both upstreams |
|
| `PROBE_TIMEOUT` | `5s` | Startup probe of both upstreams (also per-probe health check timeout) |
|
||||||
|
| `HEALTH_INTERVAL` | `30s` | Periodic upstream probe; down/recovered changes are logged |
|
||||||
| `FREE_TIMEOUT` | `30s` | `POST /free` call after an image job |
|
| `FREE_TIMEOUT` | `30s` | `POST /free` call after an image job |
|
||||||
| `WARM_TIMEOUT` | `2m` | Warm-model reload after an image job |
|
| `WARM_TIMEOUT` | `2m` | Warm-model reload after an image job |
|
||||||
| `SHUTDOWN_TIMEOUT` | `10s` | Graceful shutdown on SIGINT/SIGTERM |
|
| `SHUTDOWN_TIMEOUT` | `10s` | Graceful shutdown on SIGINT/SIGTERM |
|
||||||
@@ -72,10 +83,11 @@ override file values. Invalid values fail at startup.
|
|||||||
| `UPDATE_INTERVAL` | `6h` | Auto-update check interval |
|
| `UPDATE_INTERVAL` | `6h` | Auto-update check interval |
|
||||||
| `UPDATE_REPO` | `https://git.rambossek.at/PUBLIC/gpu-turnstile` | Repository checked for releases |
|
| `UPDATE_REPO` | `https://git.rambossek.at/PUBLIC/gpu-turnstile` | Repository checked for releases |
|
||||||
| `UPDATE_ASSET` | `gpu-turnstile.exe` | Release asset to download |
|
| `UPDATE_ASSET` | `gpu-turnstile.exe` | Release asset to download |
|
||||||
|
| `APP_VER` | `stable` | `dev` disables updates, `stable` tracks latest, or pin an exact `vX.Y.Z` |
|
||||||
|
|
||||||
## Observability
|
## Observability
|
||||||
|
|
||||||
- `GET /healthz` (both listeners): `{"state":"idle|llm|image","llm_inflight":N,"image_pending":B}`
|
- `GET /healthz` (both listeners): `{"state":"idle|llm|image|external","llm_inflight":N,"image_pending":B}`
|
||||||
- `GET /metrics` (both listeners): Prometheus text format — `gpu_turnstile_state`,
|
- `GET /metrics` (both listeners): Prometheus text format — `gpu_turnstile_state`,
|
||||||
`gpu_turnstile_llm_inflight`, `gpu_turnstile_image_pending`,
|
`gpu_turnstile_llm_inflight`, `gpu_turnstile_image_pending`,
|
||||||
`gpu_turnstile_image_jobs_total`, `gpu_turnstile_lock_wait_seconds`
|
`gpu_turnstile_image_jobs_total`, `gpu_turnstile_lock_wait_seconds`
|
||||||
@@ -87,60 +99,145 @@ override file values. Invalid values fail at startup.
|
|||||||
class) in text mode, which renders in `docker compose logs` on Windows
|
class) in text mode, which renders in `docker compose logs` on Windows
|
||||||
Terminal. Set `NO_COLOR` to disable colors.
|
Terminal. Set `NO_COLOR` to disable colors.
|
||||||
|
|
||||||
|
## Managed ComfyUI (`COMFY_CMD` / `COMFY_DIR`)
|
||||||
|
|
||||||
|
Don't want ComfyUI running 24/7 (it holds VRAM even when idle — and the
|
||||||
|
Desktop app kills its server when you close it)? gpu-turnstile can supervise
|
||||||
|
it: the first request starts it, it stops again after `COMFY_IDLE_TIMEOUT`
|
||||||
|
(default 5m) without work, freeing the GPU for games or the LLM.
|
||||||
|
|
||||||
|
The easy way — point `COMFY_DIR` at a standard venv install (a folder with
|
||||||
|
`.venv` and `main.py`, or `.venv` and `ComfyUI\main.py`) and the launch
|
||||||
|
command is derived from it, including `--port` from `COMFY_URL`:
|
||||||
|
|
||||||
|
```
|
||||||
|
COMFY_URL=http://127.0.0.1:8189
|
||||||
|
COMFY_DIR=C:\ComfyUI
|
||||||
|
```
|
||||||
|
|
||||||
|
For other layouts, spell the command out yourself (run it once manually to
|
||||||
|
confirm it works):
|
||||||
|
|
||||||
|
```
|
||||||
|
COMFY_URL=http://127.0.0.1:8189
|
||||||
|
COMFY_CMD="C:\ComfyUI\.venv\Scripts\python.exe" main.py --port 8189
|
||||||
|
COMFY_DIR=C:\ComfyUI
|
||||||
|
```
|
||||||
|
|
||||||
|
`POST /prompt` waits for the server to answer before taking the GPU lock
|
||||||
|
(LLM traffic keeps flowing while torch loads); crashes are logged and the
|
||||||
|
next request respawns. Shutting gpu-turnstile down stops the child too.
|
||||||
|
|
||||||
|
Running the ComfyUI Desktop app alongside is safe: if something already
|
||||||
|
answers on the port, gpu-turnstile just uses it instead of spawning
|
||||||
|
(and never kills it — it only ever stops its own child). If the managed
|
||||||
|
instance already holds the port when you open the desktop app, the
|
||||||
|
desktop's server is the one that fails to bind.
|
||||||
|
|
||||||
|
One catch when gpu-turnstile runs as a service: the sandboxed service
|
||||||
|
account may not enter your user profile, so a ComfyUI install under
|
||||||
|
`C:\Users\...` (or `/home/...`) fails with "Access is denied".
|
||||||
|
`--install-service` fixes that automatically — it grants
|
||||||
|
`NT SERVICE\gpu-turnstile` recursive access to `COMFY_DIR` on Windows and
|
||||||
|
adds a `BindPaths=` to the systemd unit on Linux. Re-run it after changing
|
||||||
|
`COMFY_DIR`; or grant by hand from an admin shell:
|
||||||
|
`icacls "<COMFY_DIR>" /grant "NT SERVICE\gpu-turnstile:(OI)(CI)M" /T`.
|
||||||
|
|
||||||
|
## Game detection
|
||||||
|
|
||||||
|
Want to game on the same GPU without Ollama/ComfyUI squatting on the VRAM?
|
||||||
|
gpu-turnstile can watch for foreign GPU holders and, while one is active,
|
||||||
|
make LLM/image requests wait (or 503, per `LLM_BUSY_MODE`), unload Ollama's
|
||||||
|
models and stop the managed ComfyUI so the game gets the memory. Two
|
||||||
|
detection paths, each optional, polled every `GAME_POLL_INTERVAL` (15s):
|
||||||
|
|
||||||
|
```
|
||||||
|
GAME_PROCS=cyberpunk2077.exe,bg3.exe # the reliable way on Windows
|
||||||
|
GPU_FOREIGN_VRAM_MB=1024 # catch-all via nvidia-smi
|
||||||
|
```
|
||||||
|
|
||||||
|
`GAME_PROCS` matches running process names (case-insensitive, `.exe`
|
||||||
|
optional). `GPU_FOREIGN_VRAM_MB` asks nvidia-smi which processes hold GPU
|
||||||
|
memory and treats anything not in `GPU_IGNORE_PROCS` above the threshold as
|
||||||
|
foreign — handy as a catch-all, but note that under Windows' WDDM driver
|
||||||
|
graphics-only games may not show up in nvidia-smi's per-process list, so
|
||||||
|
name your games in `GAME_PROCS` there; on Linux both paths work. When the
|
||||||
|
game exits, requests resume automatically.
|
||||||
|
|
||||||
## Build and run
|
## Build and run
|
||||||
|
|
||||||
```sh
|
```sh
|
||||||
go build ./cmd/gpu-turnstile
|
go build ./cmd/gpu-turnstile
|
||||||
./gpu-turnstile
|
OLLAMA_URL=http://127.0.0.1:11435 COMFY_URL=http://127.0.0.1:8189 ./gpu-turnstile
|
||||||
```
|
```
|
||||||
|
|
||||||
|
Running the binary with no arguments in a terminal prints the help screen
|
||||||
|
(same as `-h`/`--help`); without a terminal (services, containers) a bare
|
||||||
|
invocation starts the proxy.
|
||||||
|
|
||||||
### Run natively on Windows (current primary deployment)
|
### Run natively on Windows (current primary deployment)
|
||||||
|
|
||||||
Download `gpu-turnstile.exe` from a release, put a `gpu-turnstile.env`
|
Download `gpu-turnstile.exe` from a release and install it as a Windows
|
||||||
next to it, and run it — or install it as a Windows service from an
|
service — no admin shell needed, a UAC prompt appears automatically and
|
||||||
elevated shell:
|
the elevated child does the work (its window waits for Enter so you can
|
||||||
|
read the result):
|
||||||
|
|
||||||
```sh
|
```sh
|
||||||
gpu-turnstile.exe --install-service # auto-start service, recovery = restart
|
gpu-turnstile.exe --install-service # installs into Program Files, auto-start
|
||||||
|
gpu-turnstile.exe --install-service --no-copy # register in place instead
|
||||||
gpu-turnstile.exe --remove-service
|
gpu-turnstile.exe --remove-service
|
||||||
```
|
```
|
||||||
|
|
||||||
The service uses the config file (services have no convenient
|
Layout: `C:\Program Files\gpu-turnstile\` holds the exe and
|
||||||
environment); set `LOG_FILE` in it since there is no console.
|
`gpu-turnstile.env`, logs go to `C:\ProgramData\gpu-turnstile\`. If you
|
||||||
|
install without a config, the installer writes a sample env file with every
|
||||||
|
setting commented and explained — only `LOG_FILE` is active (a service has
|
||||||
|
no console). Your own `LOG_FILE` setting is always kept. The service always
|
||||||
|
runs
|
||||||
|
as the virtual account `NT SERVICE\gpu-turnstile` (low-privilege,
|
||||||
|
per-service, no password); the installer automatically grants it write
|
||||||
|
access to the install and data directories — nothing else to do.
|
||||||
|
|
||||||
Suggested layout: `C:\Program Files\gpu-turnstile\` for the exe and
|
Re-running `--install-service` is safe: it stops a running service,
|
||||||
`gpu-turnstile.env`, logs under `C:\ProgramData\gpu-turnstile\` via
|
replaces the installed binary only if it changed, fixes the registration
|
||||||
`LOG_FILE`. The service runs as `LocalSystem` by default, which can write
|
only where it drifted, and restarts the service only if it was running.
|
||||||
the install directory for self-updates. For least privilege, run it as the
|
|
||||||
virtual account `NT SERVICE\gpu-turnstile` and grant write access to just
|
|
||||||
those two directories.
|
|
||||||
|
|
||||||
### Run natively on Linux (systemd)
|
### Run natively on Linux (systemd)
|
||||||
|
|
||||||
The same binary works on Linux. Install it as a systemd service as root:
|
The same binary works on Linux. Install it as a systemd service as root:
|
||||||
|
|
||||||
```sh
|
```sh
|
||||||
gpu-turnstile --install-service # writes + enables + starts the unit
|
gpu-turnstile --install-service # installs into /var/lib/gpu-turnstile, enables + starts
|
||||||
|
gpu-turnstile --install-service --no-copy # register in place instead
|
||||||
gpu-turnstile --remove-service
|
gpu-turnstile --remove-service
|
||||||
```
|
```
|
||||||
|
|
||||||
The unit (`/etc/systemd/system/gpu-turnstile.service`) is `Type=notify`:
|
The unit (`/etc/systemd/system/gpu-turnstile.service`) is `Type=notify`:
|
||||||
`systemctl start` blocks until the listeners are actually bound, a 30 s
|
`systemctl start` blocks until the listeners are actually bound, a 30 s
|
||||||
watchdog restarts the process if it wedges, and logs land in the journal
|
watchdog restarts the process if it wedges, and logs land in the journal
|
||||||
(`journalctl -u gpu-turnstile -f`) unless `LOG_FILE` is set. Put the
|
(`journalctl -u gpu-turnstile -f`) unless `LOG_FILE` is set. Install
|
||||||
config in a `gpu-turnstile.env` next to the binary (or pass
|
copies the binary to `/var/lib/gpu-turnstile/` and the config to
|
||||||
`-config /path` during install). The notify integration is a no-op in
|
`/etc/gpu-turnstile.env` (edit that one after installing). The service
|
||||||
|
runs sandboxed with `DynamicUser=yes` — a transient low-privilege UID,
|
||||||
|
read-only filesystem except its install dir (so self-update keeps
|
||||||
|
working), no capabilities, syscall-filtered: same least-privilege idea as
|
||||||
|
the Windows virtual account. The notify integration is a no-op in
|
||||||
containers and interactive shells.
|
containers and interactive shells.
|
||||||
|
|
||||||
**Auto-update is on by default**: the binary checks the repo's latest
|
**Auto-update is on by default**: the binary checks the repo's latest
|
||||||
release on startup and every `UPDATE_INTERVAL`, verifies the Ed25519
|
release on startup and every `UPDATE_INTERVAL`, verifies the Ed25519
|
||||||
signature of the download against the public key embedded at build time,
|
signature of the download against the public key embedded at build time,
|
||||||
and — once the GPU lock is idle — restarts the service onto the new
|
and — once the GPU lock is idle — restarts the service onto the new
|
||||||
version. Disable with `AUTO_UPDATE=false`. Releases are signed by CI with
|
version. `APP_VER` controls the target: `dev` disables updates, `stable`
|
||||||
|
(the default) tracks the latest release, and an exact `vX.Y.Z` pins that
|
||||||
|
release (even as a downgrade or to replace a dev build). Disable entirely
|
||||||
|
with `AUTO_UPDATE=false`. Releases are signed by CI with
|
||||||
OpenSSL; the matching public key lives in `internal/update/pubkey.go`
|
OpenSSL; the matching public key lives in `internal/update/pubkey.go`
|
||||||
(one-time setup: `openssl genpkey -algorithm ed25519 -out private.pem`,
|
(one-time setup: `openssl genpkey -algorithm ed25519 -out private.pem`,
|
||||||
`openssl pkey -in private.pem -pubout -out public.pem`; private key goes
|
`openssl pkey -in private.pem -pubout -out public.pem`; private key goes
|
||||||
to the `RELEASE_SIGNING_KEY` repo secret, public key is committed).
|
to the `RELEASE_SIGNING_KEY` repo secret, public key is committed).
|
||||||
|
`gpu-turnstile --force-update` checks immediately, stages the new binary
|
||||||
|
and restarts the running service (elevating via UAC only if needed).
|
||||||
|
|
||||||
### Docker
|
### Docker
|
||||||
|
|
||||||
|
|||||||
@@ -10,11 +10,12 @@ becomes very slow; on Linux it would OOM instead.
|
|||||||
## Goal
|
## Goal
|
||||||
|
|
||||||
A single Go binary that sits in front of **both** services and guarantees that
|
A single Go binary that sits in front of **both** services and guarantees that
|
||||||
at any moment the GPU is in exactly one of three states:
|
at any moment the GPU is in exactly one of four states:
|
||||||
|
|
||||||
- `idle` — nothing in flight
|
- `idle` — nothing in flight
|
||||||
- `llm` — N ≥ 1 Ollama inference requests in flight (concurrency allowed)
|
- `llm` — N ≥ 1 Ollama inference requests in flight (concurrency allowed)
|
||||||
- `image` — exactly one ComfyUI job in flight, Ollama models unloaded
|
- `image` — exactly one ComfyUI job in flight, Ollama models unloaded
|
||||||
|
- `external` — a foreign process (e.g. a game) holds the GPU; new work waits
|
||||||
|
|
||||||
Clients (LiteLLM, Open WebUI, n8n) point at gpu-turnstile instead of at the
|
Clients (LiteLLM, Open WebUI, n8n) point at gpu-turnstile instead of at the
|
||||||
services. gpu-turnstile is transparent for everything that does not touch the
|
services. gpu-turnstile is transparent for everything that does not touch the
|
||||||
@@ -53,9 +54,10 @@ consumer gets no listener, no startup probe, and no lock participation:
|
|||||||
blocks since no image jobs can arrive.
|
blocks since no image jobs can arrive.
|
||||||
- **Only `COMFY_URL`**: image jobs are tracked and ComfyUI's VRAM is freed
|
- **Only `COMFY_URL`**: image jobs are tracked and ComfyUI's VRAM is freed
|
||||||
afterwards, but the Ollama unload and warm-reload steps are skipped.
|
afterwards, but the Ollama unload and warm-reload steps are skipped.
|
||||||
- Future consumers (e.g. detecting a local game holding VRAM) plug into the
|
- **Game detection** is a third, optional consumer without a URL: enabled by
|
||||||
same lock the same way: enabled by their config knob, excluded when
|
`GAME_PROCS` and/or `GPU_FOREIGN_VRAM_MB` it watches for foreign processes
|
||||||
absent.
|
holding the GPU (see below) and plugs into the same lock the same way —
|
||||||
|
excluded when both knobs are unset.
|
||||||
|
|
||||||
### Lock semantics
|
### Lock semantics
|
||||||
|
|
||||||
@@ -75,6 +77,11 @@ are LLM requests and the single "writer" is an image job):
|
|||||||
requests start), waits until n == 0, sets state := `image`. Released after
|
requests start), waits until n == 0, sets state := `image`. Released after
|
||||||
the ComfyUI job finished and models were freed.
|
the ComfyUI job finished and models were freed.
|
||||||
- Concurrent image jobs queue FIFO behind each other.
|
- Concurrent image jobs queue FIFO behind each other.
|
||||||
|
- **External hold**: game detection calls `SetExternal(holder)` while a
|
||||||
|
foreign process holds the GPU. New LLM and image grants block (same busy
|
||||||
|
handling as above) until `ClearExternal()`; in-flight work is not
|
||||||
|
preempted, it drains. Snapshot reports state `external` once nothing else
|
||||||
|
is in flight.
|
||||||
- All waits are context-aware: a client that disconnects while waiting is
|
- All waits are context-aware: a client that disconnects while waiting is
|
||||||
removed from the queue.
|
removed from the queue.
|
||||||
|
|
||||||
@@ -121,6 +128,81 @@ state is `idle`, send `POST /api/generate {"model":WARM_MODEL,"keep_alive":-1}`
|
|||||||
with empty prompt to reload the chat model so the next chat doesn't pay the
|
with empty prompt to reload the chat model so the next chat doesn't pay the
|
||||||
load time. Off by default.
|
load time. Off by default.
|
||||||
|
|
||||||
|
### Managed ComfyUI (`COMFY_CMD` / `COMFY_DIR`)
|
||||||
|
|
||||||
|
When `COMFY_CMD` is set, gpu-turnstile runs ComfyUI as a supervised child
|
||||||
|
process instead of expecting an always-on server. Setting only `COMFY_DIR`
|
||||||
|
enables the same management with the launch command derived from the
|
||||||
|
standard venv layout under it (`.venv\Scripts\python.exe` on Windows,
|
||||||
|
`.venv/bin/python` on Linux; `ComfyUI\main.py`, or a flat `main.py` when
|
||||||
|
that is what exists; `--port` from the `COMFY_URL` port). Missing layout
|
||||||
|
files are flagged in the startup log.
|
||||||
|
|
||||||
|
- **Start on demand**: any ComfyUI request spawns it (double quotes in the
|
||||||
|
command line group arguments with spaces; `COMFY_DIR` sets the working
|
||||||
|
directory). `POST /prompt` additionally waits for readiness
|
||||||
|
(`/system_stats`) for up to `COMFY_START_TIMEOUT` *before* taking the GPU
|
||||||
|
lock, so LLM traffic flows while torch loads. Other requests are bridged
|
||||||
|
by the normal retry backoff. Spawn failure or a readiness timeout
|
||||||
|
answers 502.
|
||||||
|
- **Idle stop**: after `COMFY_IDLE_TIMEOUT` without requests or finished
|
||||||
|
jobs — and only while the GPU lock is idle — the process tree is killed,
|
||||||
|
freeing the VRAM ComfyUI holds. The next request restarts it.
|
||||||
|
- **Crash**: an unexpected exit is logged; the next request respawns.
|
||||||
|
gpu-turnstile's own shutdown stops the child too.
|
||||||
|
- **Coexistence**: before spawning, the URL is probed — if another server
|
||||||
|
already answers (e.g. the ComfyUI desktop app), it is used as-is and no
|
||||||
|
child is spawned; the idle watcher and shutdown only ever stop the
|
||||||
|
supervisor's own process, never the external one. The other direction —
|
||||||
|
starting the desktop app while the managed instance holds the port —
|
||||||
|
makes the *desktop* server fail to bind; gpu-turnstile is unaffected.
|
||||||
|
- Its stdout/stderr is forwarded to the log at INFO. The health check
|
||||||
|
skips the intentionally-stopped/starting states; a failed probe while
|
||||||
|
the process is alive and was previously ready is logged as DOWN.
|
||||||
|
- **Permissions**: the service account is sandboxed (Windows virtual
|
||||||
|
account, systemd `DynamicUser`), so a ComfyUI install inside a user
|
||||||
|
profile is off-limits by default. `--install-service` opens it up —
|
||||||
|
a recursive ACL grant for `NT SERVICE\gpu-turnstile` on Windows, a
|
||||||
|
`BindPaths=` in the unit on Linux — reading `COMFY_DIR` from the config
|
||||||
|
it installs. Re-run `--install-service` after changing `COMFY_DIR`, or
|
||||||
|
grant by hand (admin shell):
|
||||||
|
`icacls "<COMFY_DIR>" /grant "NT SERVICE\gpu-turnstile:(OI)(CI)M" /T`.
|
||||||
|
|
||||||
|
## Game detection (foreign GPU holders)
|
||||||
|
|
||||||
|
Games and other foreign GPU users sit outside the URL-based consumer model —
|
||||||
|
nothing proxies through gpu-turnstile for them. Two independent detection
|
||||||
|
paths, polled every `GAME_POLL_INTERVAL` (default 15 s); either one being
|
||||||
|
configured enables the feature:
|
||||||
|
|
||||||
|
- **Process watch list** (`GAME_PROCS`, comma-separated, case-insensitive,
|
||||||
|
`.exe` optional): while any listed process runs, the GPU counts as held.
|
||||||
|
This is the reliable path on Windows.
|
||||||
|
- **Foreign VRAM threshold** (`GPU_FOREIGN_VRAM_MB`): `nvidia-smi
|
||||||
|
--query-compute-apps` lists per-process GPU memory; any process not in
|
||||||
|
`GPU_IGNORE_PROCS` (default: Ollama and python — ComfyUI runs under python)
|
||||||
|
holding more than the threshold counts as a foreign holder. Needs
|
||||||
|
nvidia-smi on the PATH (absent: logged once, path disabled) and works best
|
||||||
|
on Linux — under Windows' WDDM driver, graphics-only games may not appear
|
||||||
|
in the per-process list.
|
||||||
|
|
||||||
|
While a holder is detected, gpu-turnstile:
|
||||||
|
|
||||||
|
1. takes the external hold on the lock (`SetExternal`), so new LLM/image
|
||||||
|
requests wait or are rejected per `LLM_BUSY_MODE`; busy responses name the
|
||||||
|
holder,
|
||||||
|
2. once in-flight work has drained, **frees VRAM for the foreign process**:
|
||||||
|
the managed ComfyUI is stopped (an external server on its port is never
|
||||||
|
touched) and Ollama's resident models are unloaded,
|
||||||
|
3. refuses to spawn the managed ComfyUI; ComfyUI requests that would need a
|
||||||
|
spawn are answered 503 + `Retry-After` (a running desktop instance keeps
|
||||||
|
being proxied),
|
||||||
|
4. logs the transitions at WARN ("GPU held by an external process …" /
|
||||||
|
"released the GPU; resuming") and reports state `external` in `/healthz`
|
||||||
|
and `/metrics`.
|
||||||
|
|
||||||
|
When the holder disappears, the hold is lifted and queued requests proceed.
|
||||||
|
|
||||||
## Configuration (env)
|
## Configuration (env)
|
||||||
|
|
||||||
Configuration comes from environment variables and/or an `.env`-style
|
Configuration comes from environment variables and/or an `.env`-style
|
||||||
@@ -131,8 +213,8 @@ override file values. A missing file is fine; a malformed one is fatal.
|
|||||||
|
|
||||||
| Var | Default | Meaning |
|
| Var | Default | Meaning |
|
||||||
|---|---|---|
|
|---|---|---|
|
||||||
| `LISTEN_OLLAMA` | `:11434` | Ollama-facing listener |
|
| `LISTEN_OLLAMA` | `:11434` | listener for Ollama-compatible clients |
|
||||||
| `LISTEN_COMFY` | `:8188` | ComfyUI-facing listener |
|
| `LISTEN_COMFY` | `:8188` | listener for ComfyUI clients |
|
||||||
| `OLLAMA_URL` | _(empty = disabled)_ | Ollama upstream; set to enable the Ollama consumer |
|
| `OLLAMA_URL` | _(empty = disabled)_ | Ollama upstream; set to enable the Ollama consumer |
|
||||||
| `COMFY_URL` | _(empty = disabled)_ | ComfyUI upstream; set to enable the ComfyUI consumer |
|
| `COMFY_URL` | _(empty = disabled)_ | ComfyUI upstream; set to enable the ComfyUI consumer |
|
||||||
| `UNLOAD_TIMEOUT` | `60s` | wait for Ollama to unload |
|
| `UNLOAD_TIMEOUT` | `60s` | wait for Ollama to unload |
|
||||||
@@ -142,12 +224,21 @@ override file values. A missing file is fine; a malformed one is fatal.
|
|||||||
| `LLM_BUSY_STATUS` | `503` | HTTP status for rejected LLM requests in reject mode (400–599, e.g. 429) |
|
| `LLM_BUSY_STATUS` | `503` | HTTP status for rejected LLM requests in reject mode (400–599, e.g. 429) |
|
||||||
| `BUSY_RETRY_AFTER` | `30` | seconds sent as `Retry-After` on busy responses (both modes) |
|
| `BUSY_RETRY_AFTER` | `30` | seconds sent as `Retry-After` on busy responses (both modes) |
|
||||||
| `WARM_MODEL` | `` | optional model to reload after an image job |
|
| `WARM_MODEL` | `` | optional model to reload after an image job |
|
||||||
|
| `COMFY_CMD` | _(derived from `COMFY_DIR`; both empty = unmanaged)_ | spawn and supervise ComfyUI on demand: first request starts it, idle stop after `COMFY_IDLE_TIMEOUT` frees its VRAM. Requires `COMFY_URL` |
|
||||||
|
| `COMFY_DIR` | `` | standard venv install root: set alone to supervise ComfyUI with the derived launch command (`.venv` + `main.py`, `--port` from `COMFY_URL`); also the working directory for `COMFY_CMD` |
|
||||||
|
| `COMFY_IDLE_TIMEOUT` | `5m` | stop the managed ComfyUI after this long without requests or jobs |
|
||||||
|
| `COMFY_START_TIMEOUT` | `2m` | how long a request waits for the managed ComfyUI to come up |
|
||||||
|
| `GAME_PROCS` | _(empty = disabled)_ | comma-separated process names (case-insensitive, `.exe` optional); while any runs, the GPU counts as held by it: requests wait, Ollama unloads, the managed ComfyUI stops |
|
||||||
|
| `GPU_FOREIGN_VRAM_MB` | `0` (disabled) | also treat the GPU as held when a process not in `GPU_IGNORE_PROCS` uses more VRAM than this; needs nvidia-smi |
|
||||||
|
| `GPU_IGNORE_PROCS` | `ollama,ollama app,ollama_llama_server,python,pythonw` | process names never counted as foreign GPU users |
|
||||||
|
| `GAME_POLL_INTERVAL` | `15s` | how often game/VRAM detection runs (nvidia-smi polls keep the GPU awake; don't go below ~10s) |
|
||||||
| `LOGLEVEL` | `warn` | `info` logs every request (colored arrows in text mode), `debug` adds lock transitions. `LOG_LEVEL` is accepted as an alias |
|
| `LOGLEVEL` | `warn` | `info` logs every request (colored arrows in text mode), `debug` adds lock transitions. `LOG_LEVEL` is accepted as an alias |
|
||||||
| `LOG_FORMAT` | `text` | `json` for structured JSON logs |
|
| `LOG_FORMAT` | `text` | `json` for structured JSON logs |
|
||||||
| `LOG_FILE` | `` | append logs to this file instead of stderr (useful as a service) |
|
| `LOG_FILE` | `` | append logs to this file instead of stderr (useful as a service) |
|
||||||
| `UNLOAD_POLL_INTERVAL` | `500ms` | `/api/ps` poll interval while unloading |
|
| `UNLOAD_POLL_INTERVAL` | `500ms` | `/api/ps` poll interval while unloading |
|
||||||
| `HISTORY_POLL_INTERVAL` | `1s` | `/history/<id>` poll interval while a job runs |
|
| `HISTORY_POLL_INTERVAL` | `1s` | `/history/<id>` poll interval while a job runs |
|
||||||
| `PROBE_TIMEOUT` | `5s` | startup probe of both upstreams |
|
| `PROBE_TIMEOUT` | `5s` | startup probe of both upstreams (also the per-probe health check timeout) |
|
||||||
|
| `HEALTH_INTERVAL` | `30s` | periodic probe of enabled upstreams; status changes (down/recovered) are logged |
|
||||||
| `FREE_TIMEOUT` | `30s` | `POST /free` call after an image job |
|
| `FREE_TIMEOUT` | `30s` | `POST /free` call after an image job |
|
||||||
| `WARM_TIMEOUT` | `2m` | warm-model reload after an image job |
|
| `WARM_TIMEOUT` | `2m` | warm-model reload after an image job |
|
||||||
| `SHUTDOWN_TIMEOUT` | `10s` | graceful shutdown on SIGINT/SIGTERM |
|
| `SHUTDOWN_TIMEOUT` | `10s` | graceful shutdown on SIGINT/SIGTERM |
|
||||||
@@ -158,6 +249,8 @@ override file values. A missing file is fine; a malformed one is fatal.
|
|||||||
| `UPDATE_INTERVAL` | `6h` | auto-update check interval |
|
| `UPDATE_INTERVAL` | `6h` | auto-update check interval |
|
||||||
| `UPDATE_REPO` | `https://git.rambossek.at/PUBLIC/gpu-turnstile` | repository to check for releases |
|
| `UPDATE_REPO` | `https://git.rambossek.at/PUBLIC/gpu-turnstile` | repository to check for releases |
|
||||||
| `UPDATE_ASSET` | `gpu-turnstile.exe` | release asset to download |
|
| `UPDATE_ASSET` | `gpu-turnstile.exe` | release asset to download |
|
||||||
|
| `APP_VER` | `stable` | version to run: `dev` disables updates, `stable` tracks the latest release, or an exact `vX.Y.Z` pin (up- or downgraded to) |
|
||||||
|
| `CFG_VER` | _(installer-managed)_ | config format reference written by `--install-service` (always a concrete `vX.Y.Z`; a dev build stamps `v0.0.0`); missing = the file is replaced with a fresh sample (backup `.bak`). New settings are appended (commented out) at install and at every startup after an update changed the version |
|
||||||
|
|
||||||
Startup fails fast on unparsable values and when neither consumer URL is
|
Startup fails fast on unparsable values and when neither consumer URL is
|
||||||
set. Enabled upstreams are probed once at start (`/api/version`,
|
set. Enabled upstreams are probed once at start (`/api/version`,
|
||||||
@@ -169,35 +262,78 @@ The binary runs natively on Windows (the current primary deployment) and on
|
|||||||
Linux with systemd (the future GPU server), as well as in Docker.
|
Linux with systemd (the future GPU server), as well as in Docker.
|
||||||
|
|
||||||
Service management is the same on both platforms:
|
Service management is the same on both platforms:
|
||||||
`gpu-turnstile --install-service [-config path]` registers and starts an
|
`gpu-turnstile --install-service [-config path]` installs, registers and
|
||||||
auto-start service; `--remove-service` stops and unregisters it (both need
|
starts an auto-start service; `--remove-service` stops and uninstalls it.
|
||||||
an elevated/root shell). The legacy form `gpu-turnstile service
|
Both need admin/root; on Windows a non-elevated shell triggers a UAC
|
||||||
install|remove` does the same thing.
|
prompt instead of failing — the command relaunches itself elevated, waits
|
||||||
|
for the child, and mirrors its exit code. The legacy form
|
||||||
|
`gpu-turnstile service install|remove` does the same thing.
|
||||||
|
|
||||||
|
Re-running install on an already-registered service converges instead of
|
||||||
|
failing: a running service is stopped first, the installed binary copy is
|
||||||
|
refreshed only when the content differs, the registration (Windows service
|
||||||
|
config / systemd unit) is updated only where it drifted, and the service is
|
||||||
|
started again only if it was running before.
|
||||||
|
|
||||||
|
By default install creates the canonical layout and copies the binary into
|
||||||
|
it (Windows: `%ProgramFiles%\gpu-turnstile\`, plus
|
||||||
|
`%ProgramData%\gpu-turnstile\` for logs; Linux: `/var/lib/gpu-turnstile/`
|
||||||
|
with the config at `/etc/gpu-turnstile.env`). If there is no config at all,
|
||||||
|
install writes a sample env file covering every setting — each with a
|
||||||
|
comment line, everything commented out — except the `CFG_VER`/`APP_VER`
|
||||||
|
header and `LOG_FILE`, which is active on Windows
|
||||||
|
(`%ProgramData%\gpu-turnstile\gpu-turnstile.log`) since a service has no
|
||||||
|
console; on Linux it stays commented because stderr goes to the journal.
|
||||||
|
The first line is `CFG_VER=vX.Y.Z`, recording the installer version. When a
|
||||||
|
later version's install finds an older `CFG_VER`, it appends every setting
|
||||||
|
the file does not mention (commented or not) at the end and updates
|
||||||
|
`CFG_VER`; a file without `CFG_VER` is invalid and gets replaced by a fresh
|
||||||
|
sample, with the old content kept as `<file>.bak`. `--no-copy` registers
|
||||||
|
the current executable location as-is and leaves the config untouched.
|
||||||
|
|
||||||
### Windows
|
### Windows
|
||||||
|
|
||||||
- `--install-service` registers a Windows service; recovery actions restart
|
- `--install-service` creates `%ProgramFiles%\gpu-turnstile\` and
|
||||||
it 5 s after any failure.
|
`%ProgramData%\gpu-turnstile\`, copies the exe and (if none exists there
|
||||||
- **Layout**: install to `C:\Program Files\gpu-turnstile\` (exe plus
|
yet) the `gpu-turnstile.env` into the Program Files directory, and
|
||||||
`gpu-turnstile.env`); logs belong in `C:\ProgramData\gpu-turnstile\` via
|
registers that copy as a Windows service; recovery actions restart it
|
||||||
`LOG_FILE`. The service must be able to write its install directory for
|
5 s after any failure. The install ensures the env file sets `LOG_FILE`
|
||||||
self-updates — Program Files is writable by LocalSystem and admins, which
|
to `%ProgramData%\gpu-turnstile\gpu-turnstile.log` since there is no
|
||||||
is why running as the default `LocalSystem` account is the simple choice.
|
console — an existing `LOG_FILE` setting is kept.
|
||||||
- **Account**: the default `LocalSystem` works out of the box. For least
|
- **Account**: the service always runs as the virtual account
|
||||||
privilege, create the service with the virtual account
|
`NT SERVICE\gpu-turnstile` — a per-service low-privilege identity the
|
||||||
`NT SERVICE\gpu-turnstile` and grant it write access to the install and
|
SCM manages (no password, automatic logon-as-a-service right, no admin
|
||||||
log directories only (no network logon, no user profile).
|
rights, gone when the service is removed). The installer grants it
|
||||||
- Use a config file (above) for the service — Windows services have no
|
modify access to the install and data directories (self-updates rewrite
|
||||||
convenient environment. Logs go to `LOG_FILE` since there is no console.
|
the exe) and the `LOG_FILE` directory (created if missing), plus read
|
||||||
|
access to the config file when it lives elsewhere. The grants happen
|
||||||
|
after service registration because the virtual account's SID only exists
|
||||||
|
from that point on; if a grant fails the service registration is rolled
|
||||||
|
back.
|
||||||
|
|
||||||
### Linux (systemd)
|
### Linux (systemd)
|
||||||
|
|
||||||
- `--install-service` writes `/etc/systemd/system/gpu-turnstile.service`
|
- `--install-service` copies the binary to `/var/lib/gpu-turnstile/`,
|
||||||
with `ExecStart` pointing at the current executable and the `-config`
|
copies the config to `/etc/gpu-turnstile.env` if none exists there yet,
|
||||||
file, then runs `systemctl daemon-reload` and `enable --now`. The unit
|
writes `/etc/systemd/system/gpu-turnstile.service`, then runs `systemctl
|
||||||
runs as root (it must be able to overwrite its own binary for
|
daemon-reload` and `enable --now`. `--remove-service` removes the unit
|
||||||
self-updates); harden with `ProtectSystem=strict` plus a writable
|
and the installed binary; the `/etc` config stays. The binary does not
|
||||||
`ReadWritePaths` if desired.
|
go to `/usr/local/sbin` on purpose: replacing a running binary needs
|
||||||
|
write access to its *directory*, and granting the sandboxed service
|
||||||
|
write access to a shared system directory would let a compromised
|
||||||
|
service overwrite other binaries — `/var/lib/gpu-turnstile` is
|
||||||
|
exclusively ours.
|
||||||
|
- **Sandboxing** mirrors the Windows virtual account: the unit runs with
|
||||||
|
`DynamicUser=yes` — a transient per-service UID with no login, no home
|
||||||
|
and no password, managed entirely by systemd. `ProtectSystem=strict`
|
||||||
|
makes the filesystem read-only except `StateDirectory=gpu-turnstile`
|
||||||
|
(the install dir, so self-updates can rewrite the binary), plus
|
||||||
|
`NoNewPrivileges`, `ProtectHome`, `PrivateTmp`, `ProtectKernel*`,
|
||||||
|
`ProtectControlGroups`, `RestrictNamespaces`, `RestrictSUIDSGID`,
|
||||||
|
`RestrictRealtime`, `LockPersonality`, `MemoryDenyWriteExecute`, empty
|
||||||
|
capability sets, `RestrictAddressFamilies=AF_UNIX AF_INET AF_INET6` and
|
||||||
|
`SystemCallFilter=@system-service`. The proxy needs only outbound
|
||||||
|
TCP/UDP and the notify socket, so it loses nothing.
|
||||||
- The unit is `Type=notify`: the binary sends `READY=1` via
|
- The unit is `Type=notify`: the binary sends `READY=1` via
|
||||||
`github.com/coreos/go-systemd` only after the listeners are bound, so
|
`github.com/coreos/go-systemd` only after the listeners are bound, so
|
||||||
`systemctl start` blocks until the proxy accepts connections. A 30 s
|
`systemctl start` blocks until the proxy accepts connections. A 30 s
|
||||||
@@ -211,14 +347,25 @@ install|remove` does the same thing.
|
|||||||
`RestartSec=5s` brings up the staged binary after the updater exits with
|
`RestartSec=5s` brings up the staged binary after the updater exits with
|
||||||
code 3.
|
code 3.
|
||||||
- **Auto-update**: on startup and every `UPDATE_INTERVAL`, the binary
|
- **Auto-update**: on startup and every `UPDATE_INTERVAL`, the binary
|
||||||
checks `UPDATE_REPO`'s latest release; if its tag is a newer `vX.Y.Z`,
|
consults `APP_VER`: `dev` disables updates; `stable` (the default)
|
||||||
it downloads `UPDATE_ASSET` plus its `.sig` (and `.sha256` when present)
|
fetches `UPDATE_REPO`'s latest release and applies it when its tag is a
|
||||||
and verifies an Ed25519 signature against the public key embedded in
|
newer `vX.Y.Z` (a `dev` binary cannot be compared and is replaced by the
|
||||||
|
latest release); a `vX.Y.Z` pin fetches that exact tag and stages it on
|
||||||
|
any difference, including downgrades. Applying means downloading
|
||||||
|
`UPDATE_ASSET` plus its `.sig` (and `.sha256` when present) and verifying
|
||||||
|
an Ed25519 signature against the public key embedded in
|
||||||
`internal/update/pubkey.go`. A verified binary is swapped in next to the
|
`internal/update/pubkey.go`. A verified binary is swapped in next to the
|
||||||
running exe (rename-aside, allowed on Windows), and once the GPU lock is
|
running exe (rename-aside, allowed on Windows), and once the GPU lock is
|
||||||
idle the process exits with code 3 so the service recovery restarts it
|
idle the process exits with code 3 so the service recovery restarts it
|
||||||
on the new version. Interactive runs only log "restart to apply".
|
on the new version. Interactive runs only log "restart to apply".
|
||||||
`dev` builds and builds without an embedded public key never update.
|
Builds without an embedded public key never update.
|
||||||
|
- **`--force-update`** runs the same check immediately, single-shot: one
|
||||||
|
attempt with a 30 s timeout, then exit — "up to date" (exit 0) or the
|
||||||
|
error (exit 1), no retries. When a newer release is found it downloads,
|
||||||
|
verifies and stages it, and if the service is running it restarts it
|
||||||
|
right away (otherwise the new version applies on next start). On Windows
|
||||||
|
it elevates via UAC only when the stage or restart needs permissions the
|
||||||
|
caller does not have.
|
||||||
- **Signing setup (one time)**: `openssl genpkey -algorithm ed25519 -out
|
- **Signing setup (one time)**: `openssl genpkey -algorithm ed25519 -out
|
||||||
private.pem`; `openssl pkey -in private.pem -pubout -out public.pem`.
|
private.pem`; `openssl pkey -in private.pem -pubout -out public.pem`.
|
||||||
Private key → repo secret `RELEASE_SIGNING_KEY`; public key → committed into
|
Private key → repo secret `RELEASE_SIGNING_KEY`; public key → committed into
|
||||||
@@ -302,7 +449,8 @@ are new.
|
|||||||
receives the first chunk before the last is sent (no buffering).
|
receives the first chunk before the last is sent (no buffering).
|
||||||
- `internal/config`: env-file parsing, precedence, fail-fast values.
|
- `internal/config`: env-file parsing, precedence, fail-fast values.
|
||||||
- `internal/update`: fake Gitea releases API; staged update happy path,
|
- `internal/update`: fake Gitea releases API; staged update happy path,
|
||||||
tampered signature rejected, older versions and dev builds skipped.
|
tampered signature rejected, older versions skipped, APP_VER=dev and
|
||||||
|
pinned releases honored.
|
||||||
|
|
||||||
## Build and CI
|
## Build and CI
|
||||||
|
|
||||||
|
|||||||
+495
-47
@@ -3,27 +3,32 @@
|
|||||||
package main
|
package main
|
||||||
|
|
||||||
import (
|
import (
|
||||||
|
"bufio"
|
||||||
"context"
|
"context"
|
||||||
"errors"
|
"errors"
|
||||||
"fmt"
|
"fmt"
|
||||||
"io"
|
"io"
|
||||||
|
"io/fs"
|
||||||
"log/slog"
|
"log/slog"
|
||||||
"net"
|
"net"
|
||||||
"net/http"
|
"net/http"
|
||||||
"os"
|
"os"
|
||||||
"os/signal"
|
"os/signal"
|
||||||
"path/filepath"
|
"path/filepath"
|
||||||
|
"runtime"
|
||||||
"strings"
|
"strings"
|
||||||
"syscall"
|
"syscall"
|
||||||
"time"
|
"time"
|
||||||
|
|
||||||
"gpu-turnstile/internal/comfy"
|
"gpu-turnstile/internal/comfy"
|
||||||
"gpu-turnstile/internal/config"
|
"gpu-turnstile/internal/config"
|
||||||
|
"gpu-turnstile/internal/game"
|
||||||
"gpu-turnstile/internal/lock"
|
"gpu-turnstile/internal/lock"
|
||||||
"gpu-turnstile/internal/metrics"
|
"gpu-turnstile/internal/metrics"
|
||||||
"gpu-turnstile/internal/ollama"
|
"gpu-turnstile/internal/ollama"
|
||||||
"gpu-turnstile/internal/proxy"
|
"gpu-turnstile/internal/proxy"
|
||||||
"gpu-turnstile/internal/service"
|
"gpu-turnstile/internal/service"
|
||||||
|
"gpu-turnstile/internal/supervise"
|
||||||
"gpu-turnstile/internal/update"
|
"gpu-turnstile/internal/update"
|
||||||
)
|
)
|
||||||
|
|
||||||
@@ -34,24 +39,127 @@ var version = "dev"
|
|||||||
// process: a signed update has been staged and the GPU lock is idle.
|
// process: a signed update has been staged and the GPU lock is idle.
|
||||||
const exitCodeUpdate = 3
|
const exitCodeUpdate = 3
|
||||||
|
|
||||||
func main() {
|
// stdoutIsTerminal reports whether stdout is a console (char device), as
|
||||||
configPath, install, remove, args := parseFlags(os.Args[1:])
|
// opposed to a pipe or file — which is what Docker containers and services
|
||||||
|
// see.
|
||||||
|
func stdoutIsTerminal() bool {
|
||||||
|
fi, err := os.Stdout.Stat()
|
||||||
|
return err == nil && fi.Mode()&os.ModeCharDevice != 0
|
||||||
|
}
|
||||||
|
|
||||||
|
// parseFlags extracts -config <path> (or -config=<path>), the
|
||||||
|
// --install-service / --remove-service switches, --no-copy, -h/--help,
|
||||||
|
// -v/--version, --force-update and the hidden --elevated-child marker from
|
||||||
|
// args.
|
||||||
|
func parseFlags(args []string) (configPath string, install, remove, noCopy, help, showVersion, forceUpdate, elevatedChild bool, rest []string) {
|
||||||
|
rest = args[:0]
|
||||||
|
for i := 0; i < len(args); i++ {
|
||||||
switch {
|
switch {
|
||||||
case install && remove:
|
case args[i] == "-config" && i+1 < len(args):
|
||||||
fmt.Fprintf(os.Stderr, "gpu-turnstile: --install-service and --remove-service are mutually exclusive\n")
|
configPath = args[i+1]
|
||||||
|
i++
|
||||||
|
case strings.HasPrefix(args[i], "-config="):
|
||||||
|
configPath = strings.TrimPrefix(args[i], "-config=")
|
||||||
|
case args[i] == "--install-service" || args[i] == "-install-service":
|
||||||
|
install = true
|
||||||
|
case args[i] == "--remove-service" || args[i] == "-remove-service":
|
||||||
|
remove = true
|
||||||
|
case args[i] == "--no-copy" || args[i] == "-no-copy":
|
||||||
|
noCopy = true
|
||||||
|
case args[i] == "-h" || args[i] == "--help" || args[i] == "-help":
|
||||||
|
help = true
|
||||||
|
case args[i] == "-v" || args[i] == "--version" || args[i] == "-version":
|
||||||
|
showVersion = true
|
||||||
|
case args[i] == "--force-update" || args[i] == "-force-update":
|
||||||
|
forceUpdate = true
|
||||||
|
case args[i] == "--elevated-child":
|
||||||
|
elevatedChild = true
|
||||||
|
default:
|
||||||
|
rest = append(rest, args[i])
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return configPath, install, remove, noCopy, help, showVersion, forceUpdate, elevatedChild, rest
|
||||||
|
}
|
||||||
|
|
||||||
|
// versionLine is printed at the top of every help and error screen.
|
||||||
|
func versionLine() string { return "gpu-turnstile " + version }
|
||||||
|
|
||||||
|
const usageText = `GPU arbitration proxy for Ollama + ComfyUI
|
||||||
|
|
||||||
|
Usage:
|
||||||
|
gpu-turnstile -config <path> run the proxy
|
||||||
|
gpu-turnstile --install-service [--no-copy] [-config path] install + start as a service
|
||||||
|
gpu-turnstile --remove-service stop + uninstall the service
|
||||||
|
gpu-turnstile -v | --version print just the version
|
||||||
|
gpu-turnstile --force-update check for a signed update now,
|
||||||
|
apply it and restart the service
|
||||||
|
gpu-turnstile -h | --help this help
|
||||||
|
|
||||||
|
Options:
|
||||||
|
-config <path> config file (default: gpu-turnstile.env next to the exe)
|
||||||
|
--install-service copies the binary into the canonical location
|
||||||
|
(%ProgramFiles%\gpu-turnstile or /var/lib/gpu-turnstile)
|
||||||
|
unless --no-copy; on Windows a UAC prompt appears when
|
||||||
|
the shell is not elevated
|
||||||
|
--no-copy with --install-service: register the current location as-is
|
||||||
|
|
||||||
|
All runtime settings are environment variables or KEY=VALUE lines in the
|
||||||
|
config file (OLLAMA_URL, COMFY_URL, LOGLEVEL, ...); see README.md.
|
||||||
|
`
|
||||||
|
|
||||||
|
// printHelp prints the version header plus the full help text.
|
||||||
|
func printHelp() {
|
||||||
|
fmt.Printf("%s — %s", versionLine(), usageText)
|
||||||
|
}
|
||||||
|
|
||||||
|
// fatalUsage prints the version header, an error message and the one-line
|
||||||
|
// usage summary, then exits with code 2.
|
||||||
|
func fatalUsage(format string, args ...any) {
|
||||||
|
fmt.Fprintf(os.Stderr, "%s\n\n", versionLine())
|
||||||
|
fmt.Fprintf(os.Stderr, format+"\n\n", args...)
|
||||||
|
fmt.Fprintln(os.Stderr, "usage: gpu-turnstile [-config path] [--install-service [--no-copy] | --remove-service]")
|
||||||
|
fmt.Fprintln(os.Stderr, " gpu-turnstile service install|remove [-config path]")
|
||||||
os.Exit(2)
|
os.Exit(2)
|
||||||
case install:
|
}
|
||||||
os.Exit(serviceCommand(configPath, []string{"install"}))
|
|
||||||
case remove:
|
func main() {
|
||||||
os.Exit(serviceCommand(configPath, []string{"remove"}))
|
configPath, install, remove, noCopy, help, showVersion, forceUpdate, elevatedChild, args := parseFlags(os.Args[1:])
|
||||||
|
if showVersion {
|
||||||
|
fmt.Println(version)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
bare := configPath == "" && !install && !remove && !forceUpdate && !elevatedChild && len(args) == 0
|
||||||
|
if help || (bare && stdoutIsTerminal()) {
|
||||||
|
// Bare invocation in a terminal (e.g. double-clicked on Windows)
|
||||||
|
// shows the help instead of starting a proxy window with no visible
|
||||||
|
// explanation. Without a terminal — Docker containers, services,
|
||||||
|
// pipes — a bare invocation starts the proxy as before.
|
||||||
|
printHelp()
|
||||||
|
return
|
||||||
}
|
}
|
||||||
if len(args) > 0 && args[0] == "service" {
|
if len(args) > 0 && args[0] == "service" {
|
||||||
os.Exit(serviceCommand(configPath, args[1:]))
|
// Legacy subcommand form: gpu-turnstile service install|remove.
|
||||||
|
if len(args) != 2 || (args[1] != "install" && args[1] != "remove") {
|
||||||
|
fatalUsage("error: expected 'service install' or 'service remove'")
|
||||||
|
}
|
||||||
|
install = args[1] == "install"
|
||||||
|
remove = !install
|
||||||
|
args = nil
|
||||||
|
}
|
||||||
|
switch {
|
||||||
|
case install && remove:
|
||||||
|
fatalUsage("error: --install-service and --remove-service are mutually exclusive")
|
||||||
|
case forceUpdate && (install || remove):
|
||||||
|
fatalUsage("error: --force-update cannot be combined with --install-service/--remove-service")
|
||||||
|
case install:
|
||||||
|
os.Exit(serviceCommand(configPath, true, noCopy, elevatedChild))
|
||||||
|
case remove:
|
||||||
|
os.Exit(serviceCommand(configPath, false, noCopy, elevatedChild))
|
||||||
|
case forceUpdate:
|
||||||
|
os.Exit(forceUpdateCommand(configPath, elevatedChild))
|
||||||
}
|
}
|
||||||
if len(args) > 0 {
|
if len(args) > 0 {
|
||||||
fmt.Fprintf(os.Stderr, "usage: gpu-turnstile [-config path] [--install-service | --remove-service]\n")
|
fatalUsage("error: unknown arguments: %s", strings.Join(args, " "))
|
||||||
fmt.Fprintf(os.Stderr, " gpu-turnstile service install|remove [-config path]\n")
|
|
||||||
os.Exit(2)
|
|
||||||
}
|
}
|
||||||
|
|
||||||
if exePath, err := os.Executable(); err == nil {
|
if exePath, err := os.Executable(); err == nil {
|
||||||
@@ -60,11 +168,12 @@ func main() {
|
|||||||
|
|
||||||
cfg, err := loadMergedConfig(configPath)
|
cfg, err := loadMergedConfig(configPath)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
fmt.Fprintf(os.Stderr, "gpu-turnstile: %v\n", err)
|
fmt.Fprintf(os.Stderr, "%s\n\ngpu-turnstile: %v\n", versionLine(), err)
|
||||||
os.Exit(1)
|
os.Exit(1)
|
||||||
}
|
}
|
||||||
log, logOut, logCloser := newLogger(cfg)
|
log, logOut, logCloser := newLogger(cfg)
|
||||||
defer logCloser.Close()
|
defer logCloser.Close()
|
||||||
|
syncEnvFile(resolveConfigPath(configPath), cfg.LogFile, log)
|
||||||
|
|
||||||
if service.IsService() {
|
if service.IsService() {
|
||||||
if err := service.Run(func(ctx context.Context) error { return run(ctx, cfg, log, logOut, true) }); err != nil {
|
if err := service.Run(func(ctx context.Context) error { return run(ctx, cfg, log, logOut, true) }); err != nil {
|
||||||
@@ -81,26 +190,25 @@ func main() {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
// parseFlags extracts -config <path> (or -config=<path>) and the
|
// syncEnvFile upgrades an installer-written config file after an update:
|
||||||
// --install-service / --remove-service switches from args.
|
// settings added since its CFG_VER are appended (commented out) and CFG_VER
|
||||||
func parseFlags(args []string) (configPath string, install, remove bool, rest []string) {
|
// is bumped. Files not written by the installer (no CFG_VER), up-to-date
|
||||||
rest = args[:0]
|
// files and dev builds are left untouched; a write failure is logged, not
|
||||||
for i := 0; i < len(args); i++ {
|
// fatal.
|
||||||
switch {
|
func syncEnvFile(path, logFile string, log *slog.Logger) {
|
||||||
case args[i] == "-config" && i+1 < len(args):
|
data, err := os.ReadFile(path)
|
||||||
configPath = args[i+1]
|
if err != nil {
|
||||||
i++
|
return // no config file; nothing to upgrade
|
||||||
case strings.HasPrefix(args[i], "-config="):
|
|
||||||
configPath = strings.TrimPrefix(args[i], "-config=")
|
|
||||||
case args[i] == "--install-service" || args[i] == "-install-service":
|
|
||||||
install = true
|
|
||||||
case args[i] == "--remove-service" || args[i] == "-remove-service":
|
|
||||||
remove = true
|
|
||||||
default:
|
|
||||||
rest = append(rest, args[i])
|
|
||||||
}
|
}
|
||||||
|
synced, changed := config.SyncSample(string(data), version, logFile)
|
||||||
|
if !changed {
|
||||||
|
return
|
||||||
}
|
}
|
||||||
return configPath, install, remove, rest
|
if err := os.WriteFile(path, []byte(synced), 0o644); err != nil {
|
||||||
|
log.Warn("could not append new settings to the config file", "path", path, "err", err)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
log.Warn("config file updated: new settings appended", "path", path, "version", version)
|
||||||
}
|
}
|
||||||
|
|
||||||
// defaultConfigPath returns gpu-turnstile.env next to the executable.
|
// defaultConfigPath returns gpu-turnstile.env next to the executable.
|
||||||
@@ -172,26 +280,150 @@ func newLogger(cfg config.Config) (*slog.Logger, io.Writer, io.Closer) {
|
|||||||
return log, out, closer
|
return log, out, closer
|
||||||
}
|
}
|
||||||
|
|
||||||
func serviceCommand(configPath string, args []string) int {
|
// waitForEnter keeps an elevated child's console window open until the
|
||||||
if len(args) != 1 || (args[0] != "install" && args[0] != "remove") {
|
// user has read the output.
|
||||||
fmt.Fprintf(os.Stderr, "usage: gpu-turnstile service install|remove [-config path]\n")
|
func waitForEnter() {
|
||||||
return 2
|
fmt.Print("\nPress Enter to close this window...")
|
||||||
|
bufio.NewReader(os.Stdin).ReadString('\n')
|
||||||
|
}
|
||||||
|
|
||||||
|
// elevateAndMirror relaunches the current command elevated (UAC) and
|
||||||
|
// mirrors the child's exit code. verb is used in messages.
|
||||||
|
func elevateAndMirror(verb string) (int, bool) {
|
||||||
|
args := append(append([]string{}, os.Args[1:]...), "--elevated-child")
|
||||||
|
code, err := service.RelaunchElevated(args)
|
||||||
|
if errors.Is(err, service.ErrUserCancelled) {
|
||||||
|
fmt.Fprintln(os.Stderr, "gpu-turnstile: UAC prompt declined")
|
||||||
|
return 1, true
|
||||||
|
}
|
||||||
|
if err != nil {
|
||||||
|
fmt.Fprintf(os.Stderr, "gpu-turnstile: could not elevate: %v\n", err)
|
||||||
|
return 1, true
|
||||||
|
}
|
||||||
|
if code != 0 {
|
||||||
|
fmt.Fprintf(os.Stderr, "gpu-turnstile %s failed in the elevated process (exit %d)\n", verb, code)
|
||||||
|
return code, true
|
||||||
|
}
|
||||||
|
return 0, true
|
||||||
|
}
|
||||||
|
|
||||||
|
// isPermission reports whether err is a permission problem (Windows
|
||||||
|
// ERROR_ACCESS_DENIED, POSIX EACCES/EPERM, possibly wrapped).
|
||||||
|
func isPermission(err error) bool {
|
||||||
|
return errors.Is(err, fs.ErrPermission) || strings.Contains(strings.ToLower(err.Error()), "access is denied")
|
||||||
|
}
|
||||||
|
|
||||||
|
// forceUpdateCommand checks for a signed update immediately, stages it if
|
||||||
|
// newer, and restarts the service when it is running so the new binary
|
||||||
|
// takes effect. Staging into a system directory and restarting a service
|
||||||
|
// need admin rights; instead of prompting unconditionally, permission
|
||||||
|
// failures trigger the UAC relaunch so a dev copy in a user-writable
|
||||||
|
// directory updates without a prompt.
|
||||||
|
func forceUpdateCommand(configPath string, elevatedChild bool) int {
|
||||||
|
if elevatedChild {
|
||||||
|
defer waitForEnter()
|
||||||
|
}
|
||||||
|
cfg, err := loadMergedConfig(configPath)
|
||||||
|
if errors.Is(err, config.ErrNoConsumer) {
|
||||||
|
err = nil // update settings do not depend on a consumer URL
|
||||||
|
}
|
||||||
|
if err != nil {
|
||||||
|
fmt.Fprintf(os.Stderr, "%s\n\ngpu-turnstile: %v\n", versionLine(), err)
|
||||||
|
return 1
|
||||||
|
}
|
||||||
|
exePath, err := os.Executable()
|
||||||
|
if err != nil {
|
||||||
|
fmt.Fprintf(os.Stderr, "gpu-turnstile: cannot locate executable: %v\n", err)
|
||||||
|
return 1
|
||||||
|
}
|
||||||
|
log, _, logCloser := newLogger(cfg)
|
||||||
|
defer logCloser.Close()
|
||||||
|
if cfg.AppVersion == "dev" {
|
||||||
|
fmt.Printf("%s: APP_VER=dev, updates disabled\n", versionLine())
|
||||||
|
return 0
|
||||||
|
}
|
||||||
|
u := &update.Updater{Repo: cfg.UpdateRepo, Asset: cfg.UpdateAsset, Version: version, Desired: cfg.AppVersion, Log: log}
|
||||||
|
|
||||||
|
// Single-shot: one attempt, fail fast when the server is unreachable
|
||||||
|
// instead of hanging in a TCP connect for minutes.
|
||||||
|
ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
|
||||||
|
defer cancel()
|
||||||
|
staged, err := u.Check(ctx, exePath)
|
||||||
|
if err != nil && isPermission(err) && !service.Elevated() {
|
||||||
|
code, _ := elevateAndMirror("--force-update")
|
||||||
|
if code == 0 {
|
||||||
|
fmt.Println("update applied (elevated)")
|
||||||
|
}
|
||||||
|
return code
|
||||||
|
}
|
||||||
|
if err != nil {
|
||||||
|
fmt.Fprintf(os.Stderr, "%s\n\ngpu-turnstile: update check failed: %v\n", versionLine(), err)
|
||||||
|
return 1
|
||||||
|
}
|
||||||
|
if !staged {
|
||||||
|
fmt.Printf("%s is up to date\n", versionLine())
|
||||||
|
return 0
|
||||||
|
}
|
||||||
|
fmt.Printf("%s: update staged\n", versionLine())
|
||||||
|
restarted, err := service.RestartIfRunning()
|
||||||
|
if err != nil && isPermission(err) && !service.Elevated() {
|
||||||
|
code, _ := elevateAndMirror("--force-update")
|
||||||
|
if code == 0 {
|
||||||
|
fmt.Println("update applied (elevated)")
|
||||||
|
}
|
||||||
|
return code
|
||||||
|
}
|
||||||
|
if err != nil {
|
||||||
|
fmt.Fprintf(os.Stderr, "gpu-turnstile: update staged but service restart failed: %v\n", err)
|
||||||
|
return 1
|
||||||
|
}
|
||||||
|
if restarted {
|
||||||
|
fmt.Println("service restarted on the new version")
|
||||||
|
} else {
|
||||||
|
fmt.Println("no running service; the new version applies on next start")
|
||||||
|
}
|
||||||
|
return 0
|
||||||
|
}
|
||||||
|
|
||||||
|
// serviceCommand installs (copyBin = register the canonical-layout copy)
|
||||||
|
// or removes the service and reports the result. On Windows, when the
|
||||||
|
// shell is not elevated, the command relaunches itself through a UAC
|
||||||
|
// prompt and mirrors the elevated child's exit code. An elevated child
|
||||||
|
// waits for a keypress so its console window does not flash closed before
|
||||||
|
// the output can be read.
|
||||||
|
func serviceCommand(configPath string, install, noCopy, elevatedChild bool) int {
|
||||||
|
verb, doneVerb := "remove", "removed"
|
||||||
|
if install {
|
||||||
|
verb, doneVerb = "install", "installed"
|
||||||
|
}
|
||||||
|
if elevatedChild {
|
||||||
|
defer waitForEnter()
|
||||||
|
}
|
||||||
|
if !service.Elevated() {
|
||||||
|
code, done := elevateAndMirror(verb)
|
||||||
|
if done && code != 0 {
|
||||||
|
return code
|
||||||
|
}
|
||||||
|
if done {
|
||||||
|
fmt.Printf("service %s: %s (elevated)\n", service.Name, doneVerb)
|
||||||
|
return 0
|
||||||
|
}
|
||||||
}
|
}
|
||||||
var err error
|
var err error
|
||||||
if args[0] == "install" {
|
if install {
|
||||||
path := resolveConfigPath(configPath)
|
path := resolveConfigPath(configPath)
|
||||||
if abs, absErr := filepath.Abs(path); absErr == nil {
|
if abs, absErr := filepath.Abs(path); absErr == nil {
|
||||||
path = abs
|
path = abs
|
||||||
}
|
}
|
||||||
err = service.Install(path)
|
err = service.Install(path, !noCopy, version)
|
||||||
} else {
|
} else {
|
||||||
err = service.Remove()
|
err = service.Remove()
|
||||||
}
|
}
|
||||||
if err != nil {
|
if err != nil {
|
||||||
fmt.Fprintf(os.Stderr, "gpu-turnstile service %s: %v\n", args[0], err)
|
fmt.Fprintf(os.Stderr, "gpu-turnstile service %s: %v\n", verb, err)
|
||||||
return 1
|
return 1
|
||||||
}
|
}
|
||||||
fmt.Printf("service %s: %sd\n", service.Name, args[0])
|
fmt.Printf("service %s: %s\n", service.Name, doneVerb)
|
||||||
return 0
|
return 0
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -203,7 +435,26 @@ func orDisabled(url string) string {
|
|||||||
return url
|
return url
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// managedComfyCommand resolves how ComfyUI is launched when it is managed:
|
||||||
|
// COMFY_CMD verbatim, or the standard venv layout under COMFY_DIR. Empty
|
||||||
|
// when neither is set (unmanaged).
|
||||||
|
func managedComfyCommand(cfg config.Config) string {
|
||||||
|
if cfg.ComfyCmd != "" {
|
||||||
|
return cfg.ComfyCmd
|
||||||
|
}
|
||||||
|
if cfg.ComfyDir != "" {
|
||||||
|
return supervise.DefaultComfyCommand(runtime.GOOS, cfg.ComfyDir, cfg.ComfyURL)
|
||||||
|
}
|
||||||
|
return ""
|
||||||
|
}
|
||||||
|
|
||||||
func run(ctx context.Context, cfg config.Config, log *slog.Logger, logOut io.Writer, isService bool) error {
|
func run(ctx context.Context, cfg config.Config, log *slog.Logger, logOut io.Writer, isService bool) error {
|
||||||
|
// ComfyUI can run as a managed child — COMFY_CMD verbatim, or the
|
||||||
|
// standard venv layout derived from COMFY_DIR alone: started on demand
|
||||||
|
// by the proxy, stopped after COMFY_IDLE_TIMEOUT idle (and on shutdown)
|
||||||
|
// so its VRAM is freed.
|
||||||
|
comfyCmdLine := managedComfyCommand(cfg)
|
||||||
|
|
||||||
// The startup line carries the version and every setting and is emitted
|
// The startup line carries the version and every setting and is emitted
|
||||||
// at WARN so it is visible even with the default (quiet) log level.
|
// at WARN so it is visible even with the default (quiet) log level.
|
||||||
log.Log(ctx, slog.LevelWarn, "starting gpu-turnstile",
|
log.Log(ctx, slog.LevelWarn, "starting gpu-turnstile",
|
||||||
@@ -221,6 +472,7 @@ func run(ctx context.Context, cfg config.Config, log *slog.Logger, logOut io.Wri
|
|||||||
"unload_poll_interval", cfg.UnloadPollInterval,
|
"unload_poll_interval", cfg.UnloadPollInterval,
|
||||||
"history_poll_interval", cfg.HistoryPollInterval,
|
"history_poll_interval", cfg.HistoryPollInterval,
|
||||||
"probe_timeout", cfg.ProbeTimeout,
|
"probe_timeout", cfg.ProbeTimeout,
|
||||||
|
"health_interval", cfg.HealthInterval,
|
||||||
"free_timeout", cfg.FreeTimeout,
|
"free_timeout", cfg.FreeTimeout,
|
||||||
"warm_timeout", cfg.WarmTimeout,
|
"warm_timeout", cfg.WarmTimeout,
|
||||||
"shutdown_timeout", cfg.ShutdownTimeout,
|
"shutdown_timeout", cfg.ShutdownTimeout,
|
||||||
@@ -228,6 +480,14 @@ func run(ctx context.Context, cfg config.Config, log *slog.Logger, logOut io.Wri
|
|||||||
"backoff_max", cfg.BackoffMax,
|
"backoff_max", cfg.BackoffMax,
|
||||||
"prompt_capture_limit", cfg.PromptCaptureLimit,
|
"prompt_capture_limit", cfg.PromptCaptureLimit,
|
||||||
"warm_model", cfg.WarmModel,
|
"warm_model", cfg.WarmModel,
|
||||||
|
"comfy_cmd", orDisabled(comfyCmdLine),
|
||||||
|
"comfy_dir", cfg.ComfyDir,
|
||||||
|
"comfy_idle_timeout", cfg.ComfyIdleTimeout,
|
||||||
|
"comfy_start_timeout", cfg.ComfyStartTimeout,
|
||||||
|
"game_procs", cfg.GameProcs,
|
||||||
|
"gpu_foreign_vram_mb", cfg.GPUForeignVRAMMB,
|
||||||
|
"gpu_ignore_procs", cfg.GPUIgnoreProcs,
|
||||||
|
"game_poll_interval", cfg.GamePollInterval,
|
||||||
"auto_update", cfg.AutoUpdate,
|
"auto_update", cfg.AutoUpdate,
|
||||||
"update_interval", cfg.UpdateInterval,
|
"update_interval", cfg.UpdateInterval,
|
||||||
"update_repo", cfg.UpdateRepo,
|
"update_repo", cfg.UpdateRepo,
|
||||||
@@ -254,12 +514,45 @@ func run(ctx context.Context, cfg config.Config, log *slog.Logger, logOut io.Wri
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
var comfySup *supervise.Process
|
||||||
|
if comfyCmdLine != "" {
|
||||||
|
if cfg.ComfyCmd == "" {
|
||||||
|
// Derived from COMFY_DIR: flag a wrong-looking layout early,
|
||||||
|
// while the operator is still watching the startup log. Inside
|
||||||
|
// a profile the service account may not enter, os.Stat fails
|
||||||
|
// with EACCES — that reads as "not found" but means "grant
|
||||||
|
// access", so say so.
|
||||||
|
python, script := supervise.ComfyLayout(runtime.GOOS, cfg.ComfyDir)
|
||||||
|
for _, p := range []string{python, script} {
|
||||||
|
if _, err := os.Stat(p); err != nil {
|
||||||
|
if isPermission(err) {
|
||||||
|
log.Warn("COMFY_DIR: not accessible to the service account; grant access or re-run --install-service", "path", p)
|
||||||
|
} else {
|
||||||
|
log.Warn("COMFY_DIR: file not found; ComfyUI requests will fail until it exists", "path", p)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
var err error
|
||||||
|
comfySup, err = supervise.New("comfy", comfyCmdLine, cfg.ComfyDir, comfyClient.Probe, cfg.ComfyStartTimeout, log)
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
defer comfySup.Stop()
|
||||||
|
gpuIdle := func() bool {
|
||||||
|
state, _, pending := lk.Snapshot()
|
||||||
|
return state == lock.StateIdle && !pending
|
||||||
|
}
|
||||||
|
go comfySup.WatchIdle(ctx, cfg.ComfyIdleTimeout, gpuIdle)
|
||||||
|
}
|
||||||
|
|
||||||
srv, err := proxy.New(proxy.Config{
|
srv, err := proxy.New(proxy.Config{
|
||||||
OllamaURL: cfg.OllamaURL,
|
OllamaURL: cfg.OllamaURL,
|
||||||
ComfyURL: cfg.ComfyURL,
|
ComfyURL: cfg.ComfyURL,
|
||||||
Lock: lk,
|
Lock: lk,
|
||||||
Ollama: ollamaClient,
|
Ollama: ollamaClient,
|
||||||
Comfy: comfyClient,
|
Comfy: comfyClient,
|
||||||
|
ComfySup: comfySup,
|
||||||
Metrics: metrics.New(),
|
Metrics: metrics.New(),
|
||||||
Log: log,
|
Log: log,
|
||||||
LogColor: !cfg.LogJSON && cfg.LogFile == "" && os.Getenv("NO_COLOR") == "",
|
LogColor: !cfg.LogJSON && cfg.LogFile == "" && os.Getenv("NO_COLOR") == "",
|
||||||
@@ -283,19 +576,52 @@ func run(ctx context.Context, cfg config.Config, log *slog.Logger, logOut io.Wri
|
|||||||
return err
|
return err
|
||||||
}
|
}
|
||||||
|
|
||||||
// Probe the enabled upstreams once; failure is logged, not fatal.
|
// Probe the enabled upstreams once; failure is logged, not fatal. A
|
||||||
probeCtx, probeCancel := context.WithTimeout(ctx, cfg.ProbeTimeout)
|
// managed ComfyUI is intentionally down at startup — the first request
|
||||||
|
// starts it — so neither the startup probe nor the health check treats
|
||||||
|
// that as an outage.
|
||||||
|
probes := map[string]func(context.Context) error{}
|
||||||
if ollamaClient != nil {
|
if ollamaClient != nil {
|
||||||
if err := ollamaClient.Probe(probeCtx); err != nil {
|
probes["ollama"] = ollamaClient.Probe
|
||||||
log.Warn("ollama probe failed", "url", cfg.OllamaURL, "err", err)
|
|
||||||
}
|
|
||||||
}
|
}
|
||||||
if comfyClient != nil {
|
if comfyClient != nil {
|
||||||
if err := comfyClient.Probe(probeCtx); err != nil {
|
if comfySup == nil {
|
||||||
log.Warn("comfy probe failed", "url", cfg.ComfyURL, "err", err)
|
probes["comfy"] = comfyClient.Probe
|
||||||
|
} else {
|
||||||
|
// Managed upstream: an idle-stopped or still-starting server is
|
||||||
|
// not an outage, so it is skipped until it has answered once
|
||||||
|
// (Ready resets on every spawn/stop). After that, a failed
|
||||||
|
// probe while the process lives is a real "DOWN".
|
||||||
|
probes["comfy"] = func(ctx context.Context) error {
|
||||||
|
if !comfySup.Ready() {
|
||||||
|
if comfySup.Running() {
|
||||||
|
if err := comfyClient.Probe(ctx); err == nil {
|
||||||
|
comfySup.MarkReady()
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return errManagedDown
|
||||||
|
}
|
||||||
|
return comfyClient.Probe(ctx)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
probeCtx, probeCancel := context.WithTimeout(ctx, cfg.ProbeTimeout)
|
||||||
|
for name, probe := range probes {
|
||||||
|
if err := probe(probeCtx); err != nil && !errors.Is(err, errManagedDown) {
|
||||||
|
log.Warn(name+" probe failed", "err", err)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
probeCancel()
|
probeCancel()
|
||||||
|
if cfg.HealthInterval > 0 {
|
||||||
|
go healthLoop(ctx, cfg.HealthInterval, cfg.ProbeTimeout, log, probes)
|
||||||
|
}
|
||||||
|
|
||||||
|
// Foreign GPU holders (games, other ML jobs) — enabled by GAME_PROCS
|
||||||
|
// and/or GPU_FOREIGN_VRAM_MB — hold the lock externally while they run.
|
||||||
|
if len(cfg.GameProcs) > 0 || cfg.GPUForeignVRAMMB > 0 {
|
||||||
|
det := game.New(cfg.GameProcs, cfg.GPUForeignVRAMMB, cfg.GPUIgnoreProcs, log)
|
||||||
|
go gameLoop(ctx, cfg, log, det, lk, ollamaClient, comfySup)
|
||||||
|
}
|
||||||
|
|
||||||
// Bind the listeners up front so a port conflict fails fast and the
|
// Bind the listeners up front so a port conflict fails fast and the
|
||||||
// readiness notification below really means "accepting connections".
|
// readiness notification below really means "accepting connections".
|
||||||
@@ -353,6 +679,128 @@ func run(ctx context.Context, cfg config.Config, log *slog.Logger, logOut io.Wri
|
|||||||
return nil
|
return nil
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// errManagedDown marks a managed upstream that is intentionally stopped
|
||||||
|
// (idle); health checks skip it instead of logging an outage.
|
||||||
|
var errManagedDown = errors.New("managed upstream intentionally stopped")
|
||||||
|
|
||||||
|
// gameLoop polls for foreign GPU holders (a game, another ML job). While one
|
||||||
|
// is detected it holds the lock externally so new LLM and image requests
|
||||||
|
// wait (or are rejected per LLM_BUSY_MODE), and — once in-flight work has
|
||||||
|
// drained — frees VRAM for it: the managed ComfyUI is stopped and Ollama's
|
||||||
|
// resident models are unloaded.
|
||||||
|
func gameLoop(ctx context.Context, cfg config.Config, log *slog.Logger, det *game.Detector, lk *lock.Lock, ollamaClient *ollama.Client, comfySup *supervise.Process) {
|
||||||
|
ticker := time.NewTicker(cfg.GamePollInterval)
|
||||||
|
defer ticker.Stop()
|
||||||
|
held, freed := false, false
|
||||||
|
for {
|
||||||
|
select {
|
||||||
|
case <-ctx.Done():
|
||||||
|
return
|
||||||
|
case <-ticker.C:
|
||||||
|
}
|
||||||
|
holders, err := det.Check(ctx)
|
||||||
|
if err != nil && ctx.Err() == nil {
|
||||||
|
log.Warn("game detection failed", "err", err)
|
||||||
|
}
|
||||||
|
switch {
|
||||||
|
case len(holders) > 0 && !held:
|
||||||
|
held = true
|
||||||
|
lk.SetExternal(summarizeHolders(holders))
|
||||||
|
log.Warn("GPU held by an external process; new LLM/image requests wait",
|
||||||
|
"holders", summarizeHolders(holders))
|
||||||
|
case len(holders) == 0 && held:
|
||||||
|
held, freed = false, false
|
||||||
|
lk.ClearExternal()
|
||||||
|
log.Warn("external process released the GPU; resuming")
|
||||||
|
}
|
||||||
|
if held && !freed {
|
||||||
|
if state, _, _ := lk.Snapshot(); state != lock.StateLLM && state != lock.StateImage {
|
||||||
|
freed = true
|
||||||
|
freeVRAM(ctx, cfg.UnloadTimeout, log, ollamaClient, comfySup)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// summarizeHolders joins holder descriptions for logs and busy responses,
|
||||||
|
// capping the list so a process name matching dozens of PIDs (system
|
||||||
|
// services) does not flood the log.
|
||||||
|
func summarizeHolders(holders []string) string {
|
||||||
|
const max = 5
|
||||||
|
if len(holders) > max {
|
||||||
|
return strings.Join(holders[:max], "; ") + fmt.Sprintf("; +%d more", len(holders)-max)
|
||||||
|
}
|
||||||
|
return strings.Join(holders, "; ")
|
||||||
|
}
|
||||||
|
|
||||||
|
// freeVRAM stops the managed ComfyUI (never an external server on its port)
|
||||||
|
// and unloads Ollama's resident models so the foreign process gets the GPU
|
||||||
|
// memory.
|
||||||
|
func freeVRAM(ctx context.Context, unloadTimeout time.Duration, log *slog.Logger, ollamaClient *ollama.Client, comfySup *supervise.Process) {
|
||||||
|
if comfySup != nil && comfySup.Running() {
|
||||||
|
log.Warn("stopping the managed ComfyUI to free VRAM")
|
||||||
|
comfySup.Stop()
|
||||||
|
}
|
||||||
|
if ollamaClient == nil {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
uctx, cancel := context.WithTimeout(ctx, unloadTimeout)
|
||||||
|
defer cancel()
|
||||||
|
if models, err := ollamaClient.LoadedModels(uctx); err != nil || len(models) == 0 {
|
||||||
|
return // nothing resident (or ollama unreachable); nothing to free
|
||||||
|
}
|
||||||
|
elapsed, err := ollamaClient.UnloadAll(uctx)
|
||||||
|
if err != nil {
|
||||||
|
log.Warn("ollama unload incomplete; continuing", "err", err)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
log.Warn("ollama models unloaded to free VRAM", "seconds", elapsed.Seconds())
|
||||||
|
}
|
||||||
|
|
||||||
|
// healthLoop probes the enabled upstreams every interval and logs status
|
||||||
|
// transitions — "is DOWN" when a previously healthy upstream stops
|
||||||
|
// answering, "recovered" when it comes back. The first round only
|
||||||
|
// establishes the baseline; the startup probe already reported that state.
|
||||||
|
func healthLoop(ctx context.Context, interval, probeTimeout time.Duration, log *slog.Logger, probes map[string]func(context.Context) error) {
|
||||||
|
ticker := time.NewTicker(interval)
|
||||||
|
defer ticker.Stop()
|
||||||
|
up := map[string]bool{}
|
||||||
|
managed := map[string]bool{}
|
||||||
|
for {
|
||||||
|
select {
|
||||||
|
case <-ctx.Done():
|
||||||
|
return
|
||||||
|
case <-ticker.C:
|
||||||
|
}
|
||||||
|
for name, probe := range probes {
|
||||||
|
pctx, cancel := context.WithTimeout(ctx, probeTimeout)
|
||||||
|
err := probe(pctx)
|
||||||
|
cancel()
|
||||||
|
if errors.Is(err, errManagedDown) {
|
||||||
|
managed[name] = true // intentionally stopped; not an outage
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
now := err == nil
|
||||||
|
if managed[name] {
|
||||||
|
// First real probe after an idle stop only re-baselines —
|
||||||
|
// an on-demand start is not a "recovery".
|
||||||
|
managed[name] = false
|
||||||
|
up[name] = now
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
was, seen := up[name]
|
||||||
|
if seen && now != was {
|
||||||
|
if now {
|
||||||
|
log.Warn(name + " upstream recovered")
|
||||||
|
} else {
|
||||||
|
log.Warn(name+" upstream is DOWN", "err", err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
up[name] = now
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
// updateLoop checks for signed updates on startup and every UPDATE_INTERVAL.
|
// updateLoop checks for signed updates on startup and every UPDATE_INTERVAL.
|
||||||
// In service mode a staged update is applied by exiting with exitCodeUpdate
|
// In service mode a staged update is applied by exiting with exitCodeUpdate
|
||||||
// once the GPU lock is idle; the service recovery configuration restarts the
|
// once the GPU lock is idle; the service recovery configuration restarts the
|
||||||
@@ -363,7 +811,7 @@ func updateLoop(ctx context.Context, cfg config.Config, log *slog.Logger, lk *lo
|
|||||||
log.Warn("auto-update disabled: cannot locate executable", "err", err)
|
log.Warn("auto-update disabled: cannot locate executable", "err", err)
|
||||||
return
|
return
|
||||||
}
|
}
|
||||||
u := &update.Updater{Repo: cfg.UpdateRepo, Asset: cfg.UpdateAsset, Version: version, Log: log}
|
u := &update.Updater{Repo: cfg.UpdateRepo, Asset: cfg.UpdateAsset, Version: version, Desired: cfg.AppVersion, Log: log}
|
||||||
for {
|
for {
|
||||||
staged, err := u.Check(ctx, exePath)
|
staged, err := u.Check(ctx, exePath)
|
||||||
if err != nil && ctx.Err() == nil {
|
if err != nil && ctx.Err() == nil {
|
||||||
|
|||||||
@@ -6,7 +6,7 @@
|
|||||||
# ComfyUI --listen 0.0.0.0 --port 8189).
|
# ComfyUI --listen 0.0.0.0 --port 8189).
|
||||||
services:
|
services:
|
||||||
gpu-turnstile:
|
gpu-turnstile:
|
||||||
image: git.rambossek.at/public/gpu-turnstile:v0.1.5
|
image: git.rambossek.at/public/gpu-turnstile:v0.2.0
|
||||||
restart: unless-stopped
|
restart: unless-stopped
|
||||||
environment:
|
environment:
|
||||||
# Each consumer is enabled by setting its URL; leave one unset to
|
# Each consumer is enabled by setting its URL; leave one unset to
|
||||||
|
|||||||
@@ -4,6 +4,7 @@ package config
|
|||||||
|
|
||||||
import (
|
import (
|
||||||
"bufio"
|
"bufio"
|
||||||
|
"errors"
|
||||||
"fmt"
|
"fmt"
|
||||||
"io"
|
"io"
|
||||||
"log/slog"
|
"log/slog"
|
||||||
@@ -25,6 +26,7 @@ type Config struct {
|
|||||||
UnloadPollInterval time.Duration
|
UnloadPollInterval time.Duration
|
||||||
HistoryPollInterval time.Duration
|
HistoryPollInterval time.Duration
|
||||||
ProbeTimeout time.Duration
|
ProbeTimeout time.Duration
|
||||||
|
HealthInterval time.Duration
|
||||||
FreeTimeout time.Duration
|
FreeTimeout time.Duration
|
||||||
WarmTimeout time.Duration
|
WarmTimeout time.Duration
|
||||||
ShutdownTimeout time.Duration
|
ShutdownTimeout time.Duration
|
||||||
@@ -37,6 +39,11 @@ type Config struct {
|
|||||||
UpdateRepo string
|
UpdateRepo string
|
||||||
UpdateAsset string
|
UpdateAsset string
|
||||||
|
|
||||||
|
// AppVersion is the version the user wants to run: "dev" disables
|
||||||
|
// updates, "stable" tracks the latest release, anything else is an
|
||||||
|
// exact vX.Y.Z release to pin. From APP_VER; defaults to "stable".
|
||||||
|
AppVersion string
|
||||||
|
|
||||||
// LLMBusyMode is "wait" (hold requests until the lock is free or
|
// LLMBusyMode is "wait" (hold requests until the lock is free or
|
||||||
// LLMWaitTimeout expires) or "reject" (immediately answer with
|
// LLMWaitTimeout expires) or "reject" (immediately answer with
|
||||||
// LLMBusyStatus + Retry-After when an image job is active or pending).
|
// LLMBusyStatus + Retry-After when an image job is active or pending).
|
||||||
@@ -44,6 +51,30 @@ type Config struct {
|
|||||||
LLMBusyStatus int
|
LLMBusyStatus int
|
||||||
BusyRetryAfter int
|
BusyRetryAfter int
|
||||||
|
|
||||||
|
// ComfyCmd spawns and supervises a ComfyUI server on demand. When
|
||||||
|
// ComfyCmd is empty but ComfyDir is set, management is enabled with the
|
||||||
|
// standard venv layout under ComfyDir (.venv + main.py or
|
||||||
|
// ComfyUI/main.py; --port from the COMFY_URL port) — ComfyCmd is the
|
||||||
|
// override for other layouts and doubles as the working directory when
|
||||||
|
// set explicitly. The managed server is stopped after ComfyIdleTimeout
|
||||||
|
// without requests, freeing its VRAM; ComfyStartTimeout bounds how long
|
||||||
|
// a request waits for it to come up.
|
||||||
|
ComfyCmd string
|
||||||
|
ComfyDir string
|
||||||
|
ComfyIdleTimeout time.Duration
|
||||||
|
ComfyStartTimeout time.Duration
|
||||||
|
|
||||||
|
// GameProcs (GAME_PROCS) is a watch list of process names; while any of
|
||||||
|
// them runs, the GPU is treated as held by a foreign process. The
|
||||||
|
// nvidia-smi path (GPUForeignVRAMMB, GPU_FOREIGN_VRAM_MB) does the same
|
||||||
|
// when a process not in GPUIgnoreProcs (GPU_IGNORE_PROCS) holds more than
|
||||||
|
// that many MiB of VRAM. GamePollInterval (GAME_POLL_INTERVAL) is how
|
||||||
|
// often both checks run.
|
||||||
|
GameProcs []string
|
||||||
|
GPUForeignVRAMMB int
|
||||||
|
GPUIgnoreProcs []string
|
||||||
|
GamePollInterval time.Duration
|
||||||
|
|
||||||
WarmModel string
|
WarmModel string
|
||||||
LogLevel slog.Level
|
LogLevel slog.Level
|
||||||
LogJSON bool
|
LogJSON bool
|
||||||
@@ -64,6 +95,7 @@ func Defaults() Config {
|
|||||||
UnloadPollInterval: 500 * time.Millisecond,
|
UnloadPollInterval: 500 * time.Millisecond,
|
||||||
HistoryPollInterval: time.Second,
|
HistoryPollInterval: time.Second,
|
||||||
ProbeTimeout: 5 * time.Second,
|
ProbeTimeout: 5 * time.Second,
|
||||||
|
HealthInterval: 30 * time.Second,
|
||||||
FreeTimeout: 30 * time.Second,
|
FreeTimeout: 30 * time.Second,
|
||||||
WarmTimeout: 2 * time.Minute,
|
WarmTimeout: 2 * time.Minute,
|
||||||
ShutdownTimeout: 10 * time.Second,
|
ShutdownTimeout: 10 * time.Second,
|
||||||
@@ -75,15 +107,29 @@ func Defaults() Config {
|
|||||||
UpdateInterval: 6 * time.Hour,
|
UpdateInterval: 6 * time.Hour,
|
||||||
UpdateRepo: "https://git.rambossek.at/PUBLIC/gpu-turnstile",
|
UpdateRepo: "https://git.rambossek.at/PUBLIC/gpu-turnstile",
|
||||||
UpdateAsset: "gpu-turnstile.exe",
|
UpdateAsset: "gpu-turnstile.exe",
|
||||||
|
AppVersion: "stable",
|
||||||
|
|
||||||
LLMBusyMode: "wait",
|
LLMBusyMode: "wait",
|
||||||
LLMBusyStatus: 503,
|
LLMBusyStatus: 503,
|
||||||
BusyRetryAfter: 30,
|
BusyRetryAfter: 30,
|
||||||
|
|
||||||
|
ComfyIdleTimeout: 5 * time.Minute,
|
||||||
|
ComfyStartTimeout: 2 * time.Minute,
|
||||||
|
|
||||||
|
// ComfyUI runs under python; excluding it (and Ollama) by name keeps
|
||||||
|
// our own consumers from tripping the foreign-VRAM check.
|
||||||
|
GPUIgnoreProcs: []string{"ollama", "ollama app", "ollama_llama_server", "python", "pythonw"},
|
||||||
|
GamePollInterval: 15 * time.Second,
|
||||||
|
|
||||||
LogLevel: slog.LevelWarn,
|
LogLevel: slog.LevelWarn,
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// ErrNoConsumer is returned by Load when neither OLLAMA_URL nor COMFY_URL
|
||||||
|
// is set. Commands that never talk to an upstream (--force-update) may
|
||||||
|
// ignore it and proceed with the rest of the configuration.
|
||||||
|
var ErrNoConsumer = errors.New("at least one of OLLAMA_URL or COMFY_URL must be set (each URL enables its consumer)")
|
||||||
|
|
||||||
// ParseEnvFile parses a .env-style file: KEY=VALUE lines, blank lines and
|
// ParseEnvFile parses a .env-style file: KEY=VALUE lines, blank lines and
|
||||||
// #-comments are ignored, no quoting. A line without '=' is an error.
|
// #-comments are ignored, no quoting. A line without '=' is an error.
|
||||||
func ParseEnvFile(r io.Reader) (map[string]string, error) {
|
func ParseEnvFile(r io.Reader) (map[string]string, error) {
|
||||||
@@ -110,6 +156,17 @@ func ParseEnvFile(r io.Reader) (map[string]string, error) {
|
|||||||
return values, scanner.Err()
|
return values, scanner.Err()
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// splitList parses a comma-separated setting into trimmed, non-empty items.
|
||||||
|
func splitList(v string) []string {
|
||||||
|
var out []string
|
||||||
|
for _, item := range strings.Split(v, ",") {
|
||||||
|
if item = strings.TrimSpace(item); item != "" {
|
||||||
|
out = append(out, item)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return out
|
||||||
|
}
|
||||||
|
|
||||||
func envDuration(getenv func(string) string, name string, dst *time.Duration) error {
|
func envDuration(getenv func(string) string, name string, dst *time.Duration) error {
|
||||||
v := getenv(name)
|
v := getenv(name)
|
||||||
if v == "" {
|
if v == "" {
|
||||||
@@ -136,6 +193,8 @@ func Load(getenv func(string) string) (Config, error) {
|
|||||||
{"OLLAMA_URL", &cfg.OllamaURL},
|
{"OLLAMA_URL", &cfg.OllamaURL},
|
||||||
{"COMFY_URL", &cfg.ComfyURL},
|
{"COMFY_URL", &cfg.ComfyURL},
|
||||||
{"WARM_MODEL", &cfg.WarmModel},
|
{"WARM_MODEL", &cfg.WarmModel},
|
||||||
|
{"COMFY_CMD", &cfg.ComfyCmd},
|
||||||
|
{"COMFY_DIR", &cfg.ComfyDir},
|
||||||
{"UPDATE_REPO", &cfg.UpdateRepo},
|
{"UPDATE_REPO", &cfg.UpdateRepo},
|
||||||
{"UPDATE_ASSET", &cfg.UpdateAsset},
|
{"UPDATE_ASSET", &cfg.UpdateAsset},
|
||||||
{"LOG_FILE", &cfg.LogFile},
|
{"LOG_FILE", &cfg.LogFile},
|
||||||
@@ -154,17 +213,34 @@ func Load(getenv func(string) string) (Config, error) {
|
|||||||
{"UNLOAD_POLL_INTERVAL", &cfg.UnloadPollInterval},
|
{"UNLOAD_POLL_INTERVAL", &cfg.UnloadPollInterval},
|
||||||
{"HISTORY_POLL_INTERVAL", &cfg.HistoryPollInterval},
|
{"HISTORY_POLL_INTERVAL", &cfg.HistoryPollInterval},
|
||||||
{"PROBE_TIMEOUT", &cfg.ProbeTimeout},
|
{"PROBE_TIMEOUT", &cfg.ProbeTimeout},
|
||||||
|
{"HEALTH_INTERVAL", &cfg.HealthInterval},
|
||||||
{"FREE_TIMEOUT", &cfg.FreeTimeout},
|
{"FREE_TIMEOUT", &cfg.FreeTimeout},
|
||||||
{"WARM_TIMEOUT", &cfg.WarmTimeout},
|
{"WARM_TIMEOUT", &cfg.WarmTimeout},
|
||||||
{"SHUTDOWN_TIMEOUT", &cfg.ShutdownTimeout},
|
{"SHUTDOWN_TIMEOUT", &cfg.ShutdownTimeout},
|
||||||
{"BACKOFF_INITIAL", &cfg.BackoffInitial},
|
{"BACKOFF_INITIAL", &cfg.BackoffInitial},
|
||||||
{"BACKOFF_MAX", &cfg.BackoffMax},
|
{"BACKOFF_MAX", &cfg.BackoffMax},
|
||||||
|
{"COMFY_IDLE_TIMEOUT", &cfg.ComfyIdleTimeout},
|
||||||
|
{"COMFY_START_TIMEOUT", &cfg.ComfyStartTimeout},
|
||||||
{"UPDATE_INTERVAL", &cfg.UpdateInterval},
|
{"UPDATE_INTERVAL", &cfg.UpdateInterval},
|
||||||
|
{"GAME_POLL_INTERVAL", &cfg.GamePollInterval},
|
||||||
} {
|
} {
|
||||||
if err := envDuration(getenv, e.name, e.dst); err != nil {
|
if err := envDuration(getenv, e.name, e.dst); err != nil {
|
||||||
return cfg, err
|
return cfg, err
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
if v := getenv("GAME_PROCS"); v != "" {
|
||||||
|
cfg.GameProcs = splitList(v)
|
||||||
|
}
|
||||||
|
if v := getenv("GPU_IGNORE_PROCS"); v != "" {
|
||||||
|
cfg.GPUIgnoreProcs = splitList(v)
|
||||||
|
}
|
||||||
|
if v := getenv("GPU_FOREIGN_VRAM_MB"); v != "" {
|
||||||
|
n, err := strconv.Atoi(v)
|
||||||
|
if err != nil || n < 0 {
|
||||||
|
return cfg, fmt.Errorf("GPU_FOREIGN_VRAM_MB: must be a non-negative integer (MiB, 0 = disabled)")
|
||||||
|
}
|
||||||
|
cfg.GPUForeignVRAMMB = n
|
||||||
|
}
|
||||||
if v := getenv("PROMPT_CAPTURE_LIMIT"); v != "" {
|
if v := getenv("PROMPT_CAPTURE_LIMIT"); v != "" {
|
||||||
n, err := strconv.ParseInt(v, 10, 64)
|
n, err := strconv.ParseInt(v, 10, 64)
|
||||||
if err != nil || n < 0 {
|
if err != nil || n < 0 {
|
||||||
@@ -199,6 +275,17 @@ func Load(getenv func(string) string) (Config, error) {
|
|||||||
}
|
}
|
||||||
cfg.BusyRetryAfter = n
|
cfg.BusyRetryAfter = n
|
||||||
}
|
}
|
||||||
|
if v := getenv("APP_VER"); v != "" {
|
||||||
|
switch {
|
||||||
|
case v == "dev" || v == "stable":
|
||||||
|
cfg.AppVersion = v
|
||||||
|
default:
|
||||||
|
if _, ok := parseVersion(v); !ok {
|
||||||
|
return cfg, fmt.Errorf("APP_VER: must be \"dev\", \"stable\" or a vX.Y.Z version")
|
||||||
|
}
|
||||||
|
cfg.AppVersion = "v" + strings.TrimPrefix(v, "v")
|
||||||
|
}
|
||||||
|
}
|
||||||
// LOGLEVEL is the canonical spelling; LOG_LEVEL is kept as an alias.
|
// LOGLEVEL is the canonical spelling; LOG_LEVEL is kept as an alias.
|
||||||
logLevelValue := getenv("LOGLEVEL")
|
logLevelValue := getenv("LOGLEVEL")
|
||||||
if logLevelValue == "" {
|
if logLevelValue == "" {
|
||||||
@@ -218,8 +305,14 @@ func Load(getenv func(string) string) (Config, error) {
|
|||||||
default:
|
default:
|
||||||
return cfg, fmt.Errorf("LOG_FORMAT: must be \"text\" or \"json\"")
|
return cfg, fmt.Errorf("LOG_FORMAT: must be \"text\" or \"json\"")
|
||||||
}
|
}
|
||||||
|
if cfg.ComfyCmd != "" && cfg.ComfyURL == "" {
|
||||||
|
return cfg, fmt.Errorf("COMFY_CMD requires COMFY_URL to be set (the proxy needs somewhere to forward)")
|
||||||
|
}
|
||||||
|
if cfg.ComfyCmd == "" && cfg.ComfyDir != "" && cfg.ComfyURL == "" {
|
||||||
|
return cfg, fmt.Errorf("COMFY_DIR without COMFY_CMD requires COMFY_URL to be set (it enables the managed ComfyUI)")
|
||||||
|
}
|
||||||
if cfg.OllamaURL == "" && cfg.ComfyURL == "" {
|
if cfg.OllamaURL == "" && cfg.ComfyURL == "" {
|
||||||
return cfg, fmt.Errorf("at least one of OLLAMA_URL or COMFY_URL must be set (each URL enables its consumer)")
|
return cfg, ErrNoConsumer
|
||||||
}
|
}
|
||||||
return cfg, nil
|
return cfg, nil
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -1,6 +1,7 @@
|
|||||||
package config
|
package config
|
||||||
|
|
||||||
import (
|
import (
|
||||||
|
"errors"
|
||||||
"log/slog"
|
"log/slog"
|
||||||
"strings"
|
"strings"
|
||||||
"testing"
|
"testing"
|
||||||
@@ -39,6 +40,61 @@ func TestLoadRequiresConsumer(t *testing.T) {
|
|||||||
if err == nil || !strings.Contains(err.Error(), "OLLAMA_URL") {
|
if err == nil || !strings.Contains(err.Error(), "OLLAMA_URL") {
|
||||||
t.Fatalf("err = %v, want missing-consumer error", err)
|
t.Fatalf("err = %v, want missing-consumer error", err)
|
||||||
}
|
}
|
||||||
|
if !errors.Is(err, ErrNoConsumer) {
|
||||||
|
t.Fatalf("err = %v, want errors.Is(err, ErrNoConsumer)", err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestAppVersion(t *testing.T) {
|
||||||
|
load := func(appVer string) (Config, error) {
|
||||||
|
return Load(func(k string) string {
|
||||||
|
switch k {
|
||||||
|
case "OLLAMA_URL":
|
||||||
|
return "http://127.0.0.1:11435"
|
||||||
|
case "APP_VER":
|
||||||
|
return appVer
|
||||||
|
}
|
||||||
|
return ""
|
||||||
|
})
|
||||||
|
}
|
||||||
|
cfg, err := load("")
|
||||||
|
if err != nil || cfg.AppVersion != "stable" {
|
||||||
|
t.Fatalf("default AppVersion = %q, err %v; want stable", cfg.AppVersion, err)
|
||||||
|
}
|
||||||
|
for _, v := range []string{"dev", "stable"} {
|
||||||
|
if cfg, err := load(v); err != nil || cfg.AppVersion != v {
|
||||||
|
t.Fatalf("APP_VER=%s: got %q, err %v", v, cfg.AppVersion, err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if cfg, err := load("1.2.3"); err != nil || cfg.AppVersion != "v1.2.3" {
|
||||||
|
t.Fatalf("APP_VER=1.2.3: got %q, err %v; want normalized v1.2.3", cfg.AppVersion, err)
|
||||||
|
}
|
||||||
|
if _, err := load("nightly"); err == nil {
|
||||||
|
t.Fatal("APP_VER=nightly: want validation error")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestComfyCmdRequiresURL(t *testing.T) {
|
||||||
|
_, err := Load(func(k string) string {
|
||||||
|
if k == "COMFY_CMD" {
|
||||||
|
return "python main.py"
|
||||||
|
}
|
||||||
|
return ""
|
||||||
|
})
|
||||||
|
if err == nil || !strings.Contains(err.Error(), "COMFY_CMD requires COMFY_URL") {
|
||||||
|
t.Fatalf("err = %v, want COMFY_CMD/COMFY_URL validation error", err)
|
||||||
|
}
|
||||||
|
if _, err := Load(func(k string) string {
|
||||||
|
switch k {
|
||||||
|
case "COMFY_CMD":
|
||||||
|
return "python main.py"
|
||||||
|
case "COMFY_URL":
|
||||||
|
return "http://127.0.0.1:8188"
|
||||||
|
}
|
||||||
|
return ""
|
||||||
|
}); err != nil {
|
||||||
|
t.Fatalf("COMFY_CMD with COMFY_URL must load: %v", err)
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
func TestParseEnvFile(t *testing.T) {
|
func TestParseEnvFile(t *testing.T) {
|
||||||
@@ -101,6 +157,8 @@ func TestLoadErrors(t *testing.T) {
|
|||||||
{"LLM_BUSY_MODE", "bogus"},
|
{"LLM_BUSY_MODE", "bogus"},
|
||||||
{"LLM_BUSY_STATUS", "200"},
|
{"LLM_BUSY_STATUS", "200"},
|
||||||
{"BUSY_RETRY_AFTER", "0"},
|
{"BUSY_RETRY_AFTER", "0"},
|
||||||
|
{"GPU_FOREIGN_VRAM_MB", "-1"},
|
||||||
|
{"GAME_POLL_INTERVAL", "bogus"},
|
||||||
} {
|
} {
|
||||||
_, err := Load(func(k string) string {
|
_, err := Load(func(k string) string {
|
||||||
if k == tc.key {
|
if k == tc.key {
|
||||||
@@ -116,3 +174,76 @@ func TestLoadErrors(t *testing.T) {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func TestGameDetectionSettings(t *testing.T) {
|
||||||
|
cfg, err := Load(func(k string) string {
|
||||||
|
switch k {
|
||||||
|
case "OLLAMA_URL":
|
||||||
|
return "http://127.0.0.1:11435"
|
||||||
|
case "GAME_PROCS":
|
||||||
|
return " cyberpunk2077.exe, hl2.exe ,, "
|
||||||
|
case "GPU_FOREIGN_VRAM_MB":
|
||||||
|
return "1024"
|
||||||
|
case "GPU_IGNORE_PROCS":
|
||||||
|
return "ollama, my-trainer"
|
||||||
|
}
|
||||||
|
return ""
|
||||||
|
})
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if len(cfg.GameProcs) != 2 || cfg.GameProcs[0] != "cyberpunk2077.exe" || cfg.GameProcs[1] != "hl2.exe" {
|
||||||
|
t.Fatalf("GameProcs = %v", cfg.GameProcs)
|
||||||
|
}
|
||||||
|
if cfg.GPUForeignVRAMMB != 1024 {
|
||||||
|
t.Fatalf("GPUForeignVRAMMB = %d", cfg.GPUForeignVRAMMB)
|
||||||
|
}
|
||||||
|
if len(cfg.GPUIgnoreProcs) != 2 || cfg.GPUIgnoreProcs[1] != "my-trainer" {
|
||||||
|
t.Fatalf("GPUIgnoreProcs = %v", cfg.GPUIgnoreProcs)
|
||||||
|
}
|
||||||
|
if cfg.GamePollInterval != 15*time.Second {
|
||||||
|
t.Fatalf("GamePollInterval = %v, want 15s default", cfg.GamePollInterval)
|
||||||
|
}
|
||||||
|
|
||||||
|
// Defaults: both detection paths off, ignore list covers our consumers.
|
||||||
|
cfg, err = Load(func(k string) string {
|
||||||
|
if k == "OLLAMA_URL" {
|
||||||
|
return "http://127.0.0.1:11435"
|
||||||
|
}
|
||||||
|
return ""
|
||||||
|
})
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if len(cfg.GameProcs) != 0 || cfg.GPUForeignVRAMMB != 0 {
|
||||||
|
t.Fatalf("detection must be off by default: %v %d", cfg.GameProcs, cfg.GPUForeignVRAMMB)
|
||||||
|
}
|
||||||
|
if len(cfg.GPUIgnoreProcs) == 0 {
|
||||||
|
t.Fatal("GPUIgnoreProcs default must not be empty")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestComfyDirOnlyEnablesManaged(t *testing.T) {
|
||||||
|
// COMFY_DIR without COMFY_CMD and without COMFY_URL is a mistake.
|
||||||
|
_, err := Load(func(k string) string {
|
||||||
|
if k == "COMFY_DIR" {
|
||||||
|
return `C:\ComfyUI`
|
||||||
|
}
|
||||||
|
return ""
|
||||||
|
})
|
||||||
|
if err == nil || !strings.Contains(err.Error(), "COMFY_DIR") {
|
||||||
|
t.Fatalf("err = %v, want COMFY_DIR/COMFY_URL validation error", err)
|
||||||
|
}
|
||||||
|
// With COMFY_URL it loads — the launch command is derived from the dir.
|
||||||
|
if _, err := Load(func(k string) string {
|
||||||
|
switch k {
|
||||||
|
case "COMFY_DIR":
|
||||||
|
return `C:\ComfyUI`
|
||||||
|
case "COMFY_URL":
|
||||||
|
return "http://127.0.0.1:8189"
|
||||||
|
}
|
||||||
|
return ""
|
||||||
|
}); err != nil {
|
||||||
|
t.Fatalf("COMFY_DIR with COMFY_URL must load: %v", err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|||||||
@@ -0,0 +1,193 @@
|
|||||||
|
package config
|
||||||
|
|
||||||
|
import (
|
||||||
|
"fmt"
|
||||||
|
"strconv"
|
||||||
|
"strings"
|
||||||
|
)
|
||||||
|
|
||||||
|
// sampleEntry is one line pair in the sample env file: a comment and the
|
||||||
|
// (usually commented-out) KEY=VALUE line.
|
||||||
|
type sampleEntry struct {
|
||||||
|
name string
|
||||||
|
value string
|
||||||
|
comment string
|
||||||
|
active bool // rendered uncommented
|
||||||
|
}
|
||||||
|
|
||||||
|
// appVerComment documents APP_VER wherever it is rendered.
|
||||||
|
const appVerComment = `Version to run: "dev" disables updates, "stable" tracks the latest release, or pin an exact release like v0.1.7`
|
||||||
|
|
||||||
|
// sampleEntries lists every setting in sample order. logFile activates the
|
||||||
|
// LOG_FILE line (Windows install); empty keeps it commented like the rest.
|
||||||
|
// CFG_VER and APP_VER are not entries — they head the file, always active.
|
||||||
|
func sampleEntries(logFile string) []sampleEntry {
|
||||||
|
return []sampleEntry{
|
||||||
|
{"LISTEN_OLLAMA", ":11434", "Listen address for Ollama-compatible clients (gpu-turnstile poses as Ollama here)", false},
|
||||||
|
{"LISTEN_COMFY", ":8188", "Listen address for ComfyUI clients (gpu-turnstile poses as ComfyUI here)", false},
|
||||||
|
{"OLLAMA_URL", "http://127.0.0.1:11434", "Ollama upstream URL; setting it enables the Ollama consumer (default: empty = disabled)", false},
|
||||||
|
{"COMFY_URL", "http://127.0.0.1:8188", "ComfyUI upstream URL; setting it enables the ComfyUI consumer (default: empty = disabled)", false},
|
||||||
|
{"WARM_MODEL", "", "Optional model to reload after an image job (default: empty = none)", false},
|
||||||
|
{"COMFY_CMD", `"C:\ComfyUI\.venv\Scripts\python.exe" ComfyUI\main.py --port 8188`, "Spawn and supervise ComfyUI on demand: the first request starts it, it stops after COMFY_IDLE_TIMEOUT to free VRAM (default: derived from COMFY_DIR; both empty = unmanaged)", false},
|
||||||
|
{"COMFY_DIR", `C:\ComfyUI`, "Root of a standard ComfyUI venv install (.venv + main.py): setting it alone supervises ComfyUI with the derived launch command; also the working directory for COMFY_CMD", false},
|
||||||
|
{"COMFY_IDLE_TIMEOUT", "5m", "Stop the managed ComfyUI after this long without requests or jobs (frees VRAM)", false},
|
||||||
|
{"COMFY_START_TIMEOUT", "2m", "How long a request waits for the managed ComfyUI to come up", false},
|
||||||
|
{"GAME_PROCS", "cyberpunk2077.exe,hl2.exe", "While a listed process runs, the GPU counts as held by it: requests wait, Ollama unloads, managed ComfyUI stops (default: empty = disabled)", false},
|
||||||
|
{"GPU_FOREIGN_VRAM_MB", "1024", "Also treat the GPU as held when a process not in GPU_IGNORE_PROCS uses more VRAM than this (needs nvidia-smi; 0/empty = disabled)", false},
|
||||||
|
{"GPU_IGNORE_PROCS", "ollama,ollama app,ollama_llama_server,python,pythonw", "Process names never counted as foreign GPU users (ComfyUI runs under python)", false},
|
||||||
|
{"GAME_POLL_INTERVAL", "15s", "How often game/VRAM detection runs (nvidia-smi polls keep the GPU awake; don't go below ~10s)", false},
|
||||||
|
{"UNLOAD_TIMEOUT", "60s", "How long to wait for Ollama to unload a model", false},
|
||||||
|
{"JOB_TIMEOUT", "15m", "Maximum time to wait for a ComfyUI job", false},
|
||||||
|
{"LLM_WAIT_TIMEOUT", "10m", "Max time an LLM request waits for the GPU before being answered 503 (wait mode)", false},
|
||||||
|
{"LLM_BUSY_MODE", "wait", `How blocked LLM requests are handled: "wait" holds them, "reject" fails them immediately`, false},
|
||||||
|
{"LLM_BUSY_STATUS", "503", "HTTP status for rejected LLM requests in reject mode (400-599, e.g. 429)", false},
|
||||||
|
{"BUSY_RETRY_AFTER", "30", "Seconds sent as Retry-After on busy responses (both modes)", false},
|
||||||
|
{"LOGLEVEL", "warn", `Log verbosity: debug, info, warn, error; "info" logs every request (LOG_LEVEL works too)`, false},
|
||||||
|
{"LOG_FORMAT", "text", `Log format: "text" or "json"`, false},
|
||||||
|
{"LOG_FILE", logFile, "Append logs to this file instead of stderr (a Windows service has no console)", logFile != ""},
|
||||||
|
{"UNLOAD_POLL_INTERVAL", "500ms", "/api/ps poll interval while unloading", false},
|
||||||
|
{"HISTORY_POLL_INTERVAL", "1s", "/history/<id> poll interval while a job runs", false},
|
||||||
|
{"PROBE_TIMEOUT", "5s", "Probe timeout for the startup probe and the periodic health check", false},
|
||||||
|
{"HEALTH_INTERVAL", "30s", "How often enabled upstreams are probed; status changes are logged", false},
|
||||||
|
{"FREE_TIMEOUT", "30s", "Timeout for the POST /free call after an image job", false},
|
||||||
|
{"WARM_TIMEOUT", "2m", "Timeout for the warm-model reload after an image job", false},
|
||||||
|
{"SHUTDOWN_TIMEOUT", "10s", "Graceful shutdown timeout on SIGINT/SIGTERM", false},
|
||||||
|
{"BACKOFF_INITIAL", "1s", "First retry wait when an upstream connection fails", false},
|
||||||
|
{"BACKOFF_MAX", "60s", "Cap for the exponential retry backoff", false},
|
||||||
|
{"PROMPT_CAPTURE_LIMIT", "65536", "Bytes of the /prompt response buffered to find prompt_id (pass-through unaffected)", false},
|
||||||
|
{"AUTO_UPDATE", "true", "Poll the releases API for signed updates", false},
|
||||||
|
{"UPDATE_INTERVAL", "6h", "Auto-update check interval", false},
|
||||||
|
{"UPDATE_REPO", "https://git.rambossek.at/PUBLIC/gpu-turnstile", "Repository to check for releases", false},
|
||||||
|
{"UPDATE_ASSET", "gpu-turnstile.exe", "Release asset to download", false},
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// SampleEnv renders a sample .env file covering every setting, each with a
|
||||||
|
// comment line. Everything is commented out — so all defaults apply —
|
||||||
|
// except the CFG_VER/APP_VER header and LOG_FILE when logFile is non-empty
|
||||||
|
// (a Windows service has no console). CFG_VER records the version that
|
||||||
|
// wrote the file so installs — and startups after an update — can upgrade
|
||||||
|
// it.
|
||||||
|
func SampleEnv(version, logFile string) string {
|
||||||
|
// CFG_VER is always a concrete vX.Y.Z — never "dev". A dev build
|
||||||
|
// stamps v0.0.0, which sorts older than any release, so the next
|
||||||
|
// release install upgrades the file and stamps a proper version.
|
||||||
|
cfgVer := version
|
||||||
|
if cfgVer == "" || cfgVer == "dev" {
|
||||||
|
cfgVer = "v0.0.0"
|
||||||
|
}
|
||||||
|
var b strings.Builder
|
||||||
|
fmt.Fprintf(&b, "CFG_VER=%s\n", cfgVer)
|
||||||
|
b.WriteString("# Config format reference, written by the installer — do not edit.\n")
|
||||||
|
b.WriteString("# The installer uses it to append newly added settings on updates.\n\n")
|
||||||
|
b.WriteString("# gpu-turnstile configuration\n")
|
||||||
|
b.WriteString("# KEY=VALUE lines; \"#\" starts a comment. Every setting below is at its\n")
|
||||||
|
b.WriteString("# default and commented out — remove the \"#\" to change it.\n")
|
||||||
|
b.WriteString("# At least one of OLLAMA_URL / COMFY_URL must be set for the proxy to start.\n\n")
|
||||||
|
writeEntry(&b, sampleEntry{"APP_VER", "stable", appVerComment, true})
|
||||||
|
for _, e := range sampleEntries(logFile) {
|
||||||
|
writeEntry(&b, e)
|
||||||
|
}
|
||||||
|
return b.String()
|
||||||
|
}
|
||||||
|
|
||||||
|
func writeEntry(b *strings.Builder, e sampleEntry) {
|
||||||
|
fmt.Fprintf(b, "# %s\n", e.comment)
|
||||||
|
if e.active {
|
||||||
|
fmt.Fprintf(b, "%s=%s\n\n", e.name, e.value)
|
||||||
|
} else {
|
||||||
|
fmt.Fprintf(b, "#%s=%s\n\n", e.name, e.value)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// SyncSample upgrades an installer-written env file: when its CFG_VER says
|
||||||
|
// it was written by an older gpu-turnstile, every setting the file does not
|
||||||
|
// mention — commented or not — is appended at the end, and CFG_VER is
|
||||||
|
// updated to version. Files without CFG_VER (hand-written or foreign),
|
||||||
|
// up-to-date files and "dev" builds are returned unchanged; changed reports
|
||||||
|
// whether the returned content differs.
|
||||||
|
func SyncSample(data, version, logFile string) (string, bool) {
|
||||||
|
if version == "" || version == "dev" {
|
||||||
|
return data, false
|
||||||
|
}
|
||||||
|
values, err := ParseEnvFile(strings.NewReader(data))
|
||||||
|
if err != nil || values["CFG_VER"] == "" {
|
||||||
|
return data, false // not installer-written; the caller decides
|
||||||
|
}
|
||||||
|
if compareVersions(values["CFG_VER"], version) >= 0 {
|
||||||
|
return data, false // same or newer
|
||||||
|
}
|
||||||
|
|
||||||
|
present := make(map[string]bool)
|
||||||
|
for _, line := range strings.Split(data, "\n") {
|
||||||
|
line = strings.TrimSpace(strings.TrimPrefix(strings.TrimSpace(line), "#"))
|
||||||
|
if name, _, ok := strings.Cut(line, "="); ok {
|
||||||
|
present[strings.TrimSpace(name)] = true
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
lines := strings.Split(data, "\n")
|
||||||
|
for i, l := range lines {
|
||||||
|
if strings.HasPrefix(strings.TrimSpace(l), "CFG_VER=") {
|
||||||
|
lines[i] = "CFG_VER=" + version
|
||||||
|
break
|
||||||
|
}
|
||||||
|
}
|
||||||
|
out := strings.Join(lines, "\n")
|
||||||
|
if !strings.HasSuffix(out, "\n") {
|
||||||
|
out += "\n"
|
||||||
|
}
|
||||||
|
|
||||||
|
var b strings.Builder
|
||||||
|
if !present["APP_VER"] {
|
||||||
|
fmt.Fprintf(&b, "\n# Added by gpu-turnstile %s:\n", version)
|
||||||
|
writeEntry(&b, sampleEntry{"APP_VER", "stable", appVerComment, true})
|
||||||
|
}
|
||||||
|
for _, e := range sampleEntries(logFile) {
|
||||||
|
if !present[e.name] {
|
||||||
|
fmt.Fprintf(&b, "\n# Added by gpu-turnstile %s:\n", version)
|
||||||
|
writeEntry(&b, e)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
out += b.String()
|
||||||
|
return out, out != data
|
||||||
|
}
|
||||||
|
|
||||||
|
// compareVersions orders two vX.Y.Z version strings. Unparseable versions
|
||||||
|
// (including "dev") sort before any release.
|
||||||
|
func compareVersions(a, b string) int {
|
||||||
|
pa, oka := parseVersion(a)
|
||||||
|
pb, okb := parseVersion(b)
|
||||||
|
if oka != okb {
|
||||||
|
if oka {
|
||||||
|
return 1
|
||||||
|
}
|
||||||
|
return -1
|
||||||
|
}
|
||||||
|
for i := range pa {
|
||||||
|
if pa[i] != pb[i] {
|
||||||
|
if pa[i] > pb[i] {
|
||||||
|
return 1
|
||||||
|
}
|
||||||
|
return -1
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return 0
|
||||||
|
}
|
||||||
|
|
||||||
|
func parseVersion(v string) ([3]int, bool) {
|
||||||
|
var out [3]int
|
||||||
|
v = strings.TrimPrefix(strings.TrimSpace(v), "v")
|
||||||
|
parts := strings.Split(v, ".")
|
||||||
|
if len(parts) != 3 {
|
||||||
|
return out, false
|
||||||
|
}
|
||||||
|
for i, p := range parts {
|
||||||
|
n, err := strconv.Atoi(p)
|
||||||
|
if err != nil || n < 0 {
|
||||||
|
return out, false
|
||||||
|
}
|
||||||
|
out[i] = n
|
||||||
|
}
|
||||||
|
return out, true
|
||||||
|
}
|
||||||
@@ -0,0 +1,156 @@
|
|||||||
|
package config
|
||||||
|
|
||||||
|
import (
|
||||||
|
"strings"
|
||||||
|
"testing"
|
||||||
|
)
|
||||||
|
|
||||||
|
const sampleLogPath = `C:\ProgramData\gpu-turnstile\gpu-turnstile.log`
|
||||||
|
|
||||||
|
var allSettingNames = []string{
|
||||||
|
"LISTEN_OLLAMA", "LISTEN_COMFY", "OLLAMA_URL", "COMFY_URL",
|
||||||
|
"WARM_MODEL", "UNLOAD_TIMEOUT", "JOB_TIMEOUT", "LLM_WAIT_TIMEOUT",
|
||||||
|
"LLM_BUSY_MODE", "LLM_BUSY_STATUS", "BUSY_RETRY_AFTER",
|
||||||
|
"LOGLEVEL", "LOG_FORMAT", "LOG_FILE",
|
||||||
|
"UNLOAD_POLL_INTERVAL", "HISTORY_POLL_INTERVAL", "PROBE_TIMEOUT",
|
||||||
|
"FREE_TIMEOUT", "WARM_TIMEOUT", "SHUTDOWN_TIMEOUT",
|
||||||
|
"BACKOFF_INITIAL", "BACKOFF_MAX", "PROMPT_CAPTURE_LIMIT",
|
||||||
|
"AUTO_UPDATE", "UPDATE_INTERVAL", "UPDATE_REPO", "UPDATE_ASSET",
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestSampleEnv(t *testing.T) {
|
||||||
|
sample := SampleEnv("v0.1.7", sampleLogPath)
|
||||||
|
|
||||||
|
if !strings.HasPrefix(sample, "CFG_VER=v0.1.7\n") {
|
||||||
|
t.Errorf("first line does not carry CFG_VER: %q", strings.SplitN(sample, "\n", 2)[0])
|
||||||
|
}
|
||||||
|
|
||||||
|
// Every setting known to Load must appear.
|
||||||
|
for _, name := range allSettingNames {
|
||||||
|
if !strings.Contains(sample, name+"=") {
|
||||||
|
t.Errorf("sample is missing %s", name)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// The sample must parse cleanly; active values are CFG_VER, APP_VER
|
||||||
|
// and LOG_FILE.
|
||||||
|
values, err := ParseEnvFile(strings.NewReader(sample))
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("sample does not parse: %v", err)
|
||||||
|
}
|
||||||
|
want := map[string]string{"CFG_VER": "v0.1.7", "APP_VER": "stable", "LOG_FILE": sampleLogPath}
|
||||||
|
if len(values) != len(want) {
|
||||||
|
t.Fatalf("active values = %v, want %v", values, want)
|
||||||
|
}
|
||||||
|
for k, v := range want {
|
||||||
|
if values[k] != v {
|
||||||
|
t.Errorf("%s = %q, want %q", k, values[k], v)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Without a log path LOG_FILE stays commented out.
|
||||||
|
values, err = ParseEnvFile(strings.NewReader(SampleEnv("v0.1.7", "")))
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("sample without log path does not parse: %v", err)
|
||||||
|
}
|
||||||
|
if len(values) != 2 || values["LOG_FILE"] != "" {
|
||||||
|
t.Fatalf("active values = %v, want only CFG_VER and APP_VER", values)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestSyncSample(t *testing.T) {
|
||||||
|
old := SampleEnv("v0.1.6", sampleLogPath)
|
||||||
|
// Simulate a setting that did not exist yet when the file was written.
|
||||||
|
old = strings.Replace(old, "#UPDATE_ASSET=gpu-turnstile.exe\n", "", 1)
|
||||||
|
|
||||||
|
out, changed := SyncSample(old, "v0.1.7", sampleLogPath)
|
||||||
|
if !changed {
|
||||||
|
t.Fatal("older installer file was not upgraded")
|
||||||
|
}
|
||||||
|
if !strings.Contains(out, "\nCFG_VER=v0.1.7\n") && !strings.HasPrefix(out, "CFG_VER=v0.1.7\n") {
|
||||||
|
t.Error("CFG_VER was not updated to the new version")
|
||||||
|
}
|
||||||
|
if !strings.Contains(out, "#UPDATE_ASSET=gpu-turnstile.exe") {
|
||||||
|
t.Error("missing setting was not appended")
|
||||||
|
}
|
||||||
|
if !strings.Contains(out, "LOG_FILE="+sampleLogPath) {
|
||||||
|
t.Error("existing active LOG_FILE was lost")
|
||||||
|
}
|
||||||
|
if !strings.Contains(out, "Added by gpu-turnstile v0.1.7") {
|
||||||
|
t.Error("appended section is not attributed")
|
||||||
|
}
|
||||||
|
if _, err := ParseEnvFile(strings.NewReader(out)); err != nil {
|
||||||
|
t.Fatalf("upgraded file does not parse: %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
// A file written by the same or a newer version is left alone.
|
||||||
|
if _, changed := SyncSample(SampleEnv("v0.1.7", sampleLogPath), "v0.1.7", sampleLogPath); changed {
|
||||||
|
t.Error("same-version file was modified")
|
||||||
|
}
|
||||||
|
if _, changed := SyncSample(SampleEnv("v0.2.0", sampleLogPath), "v0.1.7", sampleLogPath); changed {
|
||||||
|
t.Error("newer-version file was modified")
|
||||||
|
}
|
||||||
|
|
||||||
|
// Files without CFG_VER are not installer-written; the installer
|
||||||
|
// replaces them, SyncSample leaves them alone.
|
||||||
|
user := "OLLAMA_URL=http://host:11434\n"
|
||||||
|
if out, changed := SyncSample(user, "v0.1.7", sampleLogPath); changed || out != user {
|
||||||
|
t.Error("file without CFG_VER was modified")
|
||||||
|
}
|
||||||
|
|
||||||
|
// A dev build never upgrades.
|
||||||
|
if _, changed := SyncSample(old, "dev", sampleLogPath); changed {
|
||||||
|
t.Error("dev build modified the file")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestSyncSampleAppendsMissingAppVer(t *testing.T) {
|
||||||
|
old := SampleEnv("v0.1.6", sampleLogPath)
|
||||||
|
old = strings.Replace(old, "# "+appVerComment+"\n", "", 1)
|
||||||
|
old = strings.Replace(old, "APP_VER=stable\n", "", 1)
|
||||||
|
|
||||||
|
out, changed := SyncSample(old, "v0.1.7", sampleLogPath)
|
||||||
|
if !changed {
|
||||||
|
t.Fatal("file without APP_VER was not upgraded")
|
||||||
|
}
|
||||||
|
values, err := ParseEnvFile(strings.NewReader(out))
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("upgraded file does not parse: %v", err)
|
||||||
|
}
|
||||||
|
if values["APP_VER"] != "stable" {
|
||||||
|
t.Fatalf("APP_VER = %q, want appended default \"stable\"", values["APP_VER"])
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestSampleEnvDevStampsConcreteVersion(t *testing.T) {
|
||||||
|
// A dev build must never write CFG_VER=dev: it stamps v0.0.0, and the
|
||||||
|
// next release install upgrades the file to a proper version.
|
||||||
|
dev := SampleEnv("dev", sampleLogPath)
|
||||||
|
if !strings.HasPrefix(dev, "CFG_VER=v0.0.0\n") {
|
||||||
|
t.Errorf("dev sample first line: %q", strings.SplitN(dev, "\n", 2)[0])
|
||||||
|
}
|
||||||
|
out, changed := SyncSample(dev, "v0.1.7", sampleLogPath)
|
||||||
|
if !changed || !strings.HasPrefix(out, "CFG_VER=v0.1.7\n") {
|
||||||
|
t.Errorf("dev-stamped file was not upgraded to v0.1.7 (changed=%v)", changed)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestCompareVersions(t *testing.T) {
|
||||||
|
cases := []struct {
|
||||||
|
a, b string
|
||||||
|
want int
|
||||||
|
}{
|
||||||
|
{"v0.1.6", "v0.1.7", -1},
|
||||||
|
{"v0.1.7", "v0.1.7", 0},
|
||||||
|
{"v1.0.0", "v0.9.9", 1},
|
||||||
|
{"0.1.7", "v0.1.6", 1},
|
||||||
|
{"dev", "v0.1.7", -1},
|
||||||
|
{"v0.1.7", "dev", 1},
|
||||||
|
{"dev", "dev", 0},
|
||||||
|
}
|
||||||
|
for _, c := range cases {
|
||||||
|
if got := compareVersions(c.a, c.b); got != c.want {
|
||||||
|
t.Errorf("compareVersions(%q, %q) = %d, want %d", c.a, c.b, got, c.want)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,159 @@
|
|||||||
|
// Package game detects processes outside gpu-turnstile's control that hold
|
||||||
|
// the GPU — typically a game — so the proxy can block new GPU work and free
|
||||||
|
// VRAM while they run. Two detection paths: an explicit process watch list
|
||||||
|
// (GAME_PROCS) and a foreign-VRAM threshold via nvidia-smi
|
||||||
|
// (GPU_FOREIGN_VRAM_MB) that catches anything not on the ignore list.
|
||||||
|
package game
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"errors"
|
||||||
|
"fmt"
|
||||||
|
"log/slog"
|
||||||
|
"os/exec"
|
||||||
|
"strconv"
|
||||||
|
"strings"
|
||||||
|
)
|
||||||
|
|
||||||
|
// Process is one running OS process.
|
||||||
|
type Process struct {
|
||||||
|
PID int
|
||||||
|
Name string
|
||||||
|
}
|
||||||
|
|
||||||
|
// computeApp is one process holding GPU memory, as reported by nvidia-smi.
|
||||||
|
type computeApp struct {
|
||||||
|
PID int
|
||||||
|
UsedMB int
|
||||||
|
}
|
||||||
|
|
||||||
|
// Detector checks whether a foreign process holds the GPU. The zero value
|
||||||
|
// (no watch list, no threshold) never detects anything; main only starts the
|
||||||
|
// poll loop when at least one path is configured.
|
||||||
|
type Detector struct {
|
||||||
|
procs map[string]bool // normalized names from GAME_PROCS
|
||||||
|
vramMB int // foreign VRAM threshold; 0 = disabled
|
||||||
|
ignore map[string]bool // normalized names never counted as foreign
|
||||||
|
log *slog.Logger
|
||||||
|
noNvidia bool // nvidia-smi was not found; VRAM path disabled for good
|
||||||
|
}
|
||||||
|
|
||||||
|
// New builds a Detector from the configured watch list, VRAM threshold in
|
||||||
|
// MiB (0 disables the nvidia-smi path) and ignore list. Names are matched
|
||||||
|
// case-insensitively, with or without a trailing ".exe".
|
||||||
|
func New(procs []string, vramMB int, ignore []string, log *slog.Logger) *Detector {
|
||||||
|
if log == nil {
|
||||||
|
log = slog.Default()
|
||||||
|
}
|
||||||
|
return &Detector{
|
||||||
|
procs: nameSet(procs),
|
||||||
|
vramMB: vramMB,
|
||||||
|
ignore: nameSet(ignore),
|
||||||
|
log: log,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// normName lowercases a process name and strips a trailing ".exe" so the
|
||||||
|
// watch and ignore lists match on Windows and Linux spellings alike.
|
||||||
|
func normName(s string) string {
|
||||||
|
return strings.TrimSuffix(strings.ToLower(strings.TrimSpace(s)), ".exe")
|
||||||
|
}
|
||||||
|
|
||||||
|
func nameSet(names []string) map[string]bool {
|
||||||
|
set := make(map[string]bool, len(names))
|
||||||
|
for _, n := range names {
|
||||||
|
if n = normName(n); n != "" {
|
||||||
|
set[n] = true
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return set
|
||||||
|
}
|
||||||
|
|
||||||
|
// Check looks once for foreign GPU holders and returns a human-readable
|
||||||
|
// description of each (empty when the GPU is free for gpu-turnstile's
|
||||||
|
// consumers). A failing nvidia-smi call is returned as an error only when
|
||||||
|
// the process list found nothing; a missing nvidia-smi binary disables the
|
||||||
|
// VRAM path permanently (logged once).
|
||||||
|
func (d *Detector) Check(ctx context.Context) ([]string, error) {
|
||||||
|
ps, psErr := processes()
|
||||||
|
if d.vramMB <= 0 || d.noNvidia {
|
||||||
|
return d.detect(ps, nil), psErr
|
||||||
|
}
|
||||||
|
apps, err := queryComputeApps(ctx)
|
||||||
|
if errors.Is(err, exec.ErrNotFound) {
|
||||||
|
d.noNvidia = true
|
||||||
|
d.log.Warn("GPU_FOREIGN_VRAM_MB is set but nvidia-smi was not found; VRAM detection disabled")
|
||||||
|
return d.detect(ps, nil), nil
|
||||||
|
}
|
||||||
|
if err != nil {
|
||||||
|
return d.detect(ps, nil), err
|
||||||
|
}
|
||||||
|
return d.detect(ps, apps), nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// detect is the pure core of Check: given the process table and (optionally)
|
||||||
|
// the nvidia-smi compute-apps list, it returns the foreign holders.
|
||||||
|
func (d *Detector) detect(ps []Process, apps []computeApp) []string {
|
||||||
|
var holders []string
|
||||||
|
for _, p := range ps {
|
||||||
|
if d.procs[normName(p.Name)] {
|
||||||
|
holders = append(holders, fmt.Sprintf("%s (pid %d)", p.Name, p.PID))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if d.vramMB > 0 && apps != nil {
|
||||||
|
names := make(map[int]string, len(ps))
|
||||||
|
for _, p := range ps {
|
||||||
|
names[p.PID] = p.Name
|
||||||
|
}
|
||||||
|
for _, a := range apps {
|
||||||
|
name := names[a.PID]
|
||||||
|
if d.ignore[normName(name)] || a.UsedMB < d.vramMB {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
if name == "" {
|
||||||
|
name = "unknown process"
|
||||||
|
}
|
||||||
|
holders = append(holders, fmt.Sprintf("%s (pid %d) using %d MiB VRAM", name, a.PID, a.UsedMB))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return holders
|
||||||
|
}
|
||||||
|
|
||||||
|
// queryComputeApps runs nvidia-smi and parses the per-process VRAM list.
|
||||||
|
// Note: under Windows' WDDM driver, nvidia-smi only sees compute
|
||||||
|
// allocations, so graphics-only games may not appear there — GAME_PROCS is
|
||||||
|
// the reliable path on Windows; on Linux both work.
|
||||||
|
func queryComputeApps(ctx context.Context) ([]computeApp, error) {
|
||||||
|
out, err := exec.CommandContext(ctx, "nvidia-smi",
|
||||||
|
"--query-compute-apps=pid,used_memory", "--format=csv,noheader,nounits").Output()
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
return parseComputeApps(string(out))
|
||||||
|
}
|
||||||
|
|
||||||
|
// parseComputeApps parses "pid, used_memory" CSV lines (no header, MiB
|
||||||
|
// units). Unsupported rows ("N/A" on WDDM) are skipped.
|
||||||
|
func parseComputeApps(out string) ([]computeApp, error) {
|
||||||
|
var apps []computeApp
|
||||||
|
for _, line := range strings.Split(out, "\n") {
|
||||||
|
line = strings.TrimSpace(line)
|
||||||
|
if line == "" {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
pidStr, memStr, ok := strings.Cut(line, ",")
|
||||||
|
if !ok {
|
||||||
|
return nil, fmt.Errorf("nvidia-smi: unexpected line %q", line)
|
||||||
|
}
|
||||||
|
pid, err := strconv.Atoi(strings.TrimSpace(pidStr))
|
||||||
|
if err != nil {
|
||||||
|
return nil, fmt.Errorf("nvidia-smi: unexpected pid in %q", line)
|
||||||
|
}
|
||||||
|
mem, err := strconv.Atoi(strings.TrimSpace(memStr))
|
||||||
|
if err != nil {
|
||||||
|
continue // "N/A" and friends: unsupported under WDDM
|
||||||
|
}
|
||||||
|
apps = append(apps, computeApp{PID: pid, UsedMB: mem})
|
||||||
|
}
|
||||||
|
return apps, nil
|
||||||
|
}
|
||||||
@@ -0,0 +1,96 @@
|
|||||||
|
package game
|
||||||
|
|
||||||
|
import (
|
||||||
|
"runtime"
|
||||||
|
"slices"
|
||||||
|
"testing"
|
||||||
|
)
|
||||||
|
|
||||||
|
func TestNormName(t *testing.T) {
|
||||||
|
for in, want := range map[string]string{
|
||||||
|
"Cyberpunk2077.exe": "cyberpunk2077",
|
||||||
|
"ollama": "ollama",
|
||||||
|
"OLLAMA APP.EXE": "ollama app",
|
||||||
|
" python.exe ": "python",
|
||||||
|
"hl2": "hl2",
|
||||||
|
} {
|
||||||
|
if got := normName(in); got != want {
|
||||||
|
t.Errorf("normName(%q) = %q, want %q", in, got, want)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestParseComputeApps(t *testing.T) {
|
||||||
|
apps, err := parseComputeApps("1234, 512\n 42 , 8192 \n\n")
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
want := []computeApp{{PID: 1234, UsedMB: 512}, {PID: 42, UsedMB: 8192}}
|
||||||
|
if !slices.Equal(apps, want) {
|
||||||
|
t.Errorf("got %+v, want %+v", apps, want)
|
||||||
|
}
|
||||||
|
|
||||||
|
// WDDM reports "N/A" for memory; those rows are skipped, not fatal.
|
||||||
|
apps, err = parseComputeApps("1234, N/A\n42, 1024\n")
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if !slices.Equal(apps, []computeApp{{PID: 42, UsedMB: 1024}}) {
|
||||||
|
t.Errorf("got %+v", apps)
|
||||||
|
}
|
||||||
|
|
||||||
|
if _, err := parseComputeApps("garbage\n"); err == nil {
|
||||||
|
t.Error("expected an error for a malformed line")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestDetect(t *testing.T) {
|
||||||
|
d := New([]string{"Cyberpunk2077.exe", "hl2"}, 1024,
|
||||||
|
[]string{"ollama", "python", "pythonw"}, nil)
|
||||||
|
ps := []Process{
|
||||||
|
{PID: 10, Name: "ollama.exe"},
|
||||||
|
{PID: 20, Name: "python.exe"},
|
||||||
|
{PID: 30, Name: "cyberpunk2077.exe"},
|
||||||
|
}
|
||||||
|
apps := []computeApp{
|
||||||
|
{PID: 10, UsedMB: 8192}, // ignored: ollama
|
||||||
|
{PID: 20, UsedMB: 4096}, // ignored: python (ComfyUI)
|
||||||
|
{PID: 40, UsedMB: 2048}, // foreign, above threshold
|
||||||
|
{PID: 50, UsedMB: 100}, // foreign but below threshold
|
||||||
|
}
|
||||||
|
holders := d.detect(ps, apps)
|
||||||
|
if len(holders) != 2 {
|
||||||
|
t.Fatalf("got %v, want 2 holders", holders)
|
||||||
|
}
|
||||||
|
if holders[0] != "cyberpunk2077.exe (pid 30)" {
|
||||||
|
t.Errorf("holders[0] = %q", holders[0])
|
||||||
|
}
|
||||||
|
if holders[1] != "unknown process (pid 40) using 2048 MiB VRAM" {
|
||||||
|
t.Errorf("holders[1] = %q", holders[1])
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestDetectNothingConfigured(t *testing.T) {
|
||||||
|
d := New(nil, 0, nil, nil)
|
||||||
|
if got := d.detect([]Process{{PID: 1, Name: "game.exe"}}, nil); len(got) != 0 {
|
||||||
|
t.Errorf("got %v, want none", got)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestProcessesLive(t *testing.T) {
|
||||||
|
if runtime.GOOS != "windows" && runtime.GOOS != "linux" {
|
||||||
|
t.Skip("no process listing on this platform")
|
||||||
|
}
|
||||||
|
ps, err := processes()
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if len(ps) == 0 {
|
||||||
|
t.Fatal("no processes listed")
|
||||||
|
}
|
||||||
|
for _, p := range ps {
|
||||||
|
if p.Name == "" {
|
||||||
|
t.Errorf("pid %d has an empty name", p.PID)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,30 @@
|
|||||||
|
//go:build linux
|
||||||
|
|
||||||
|
package game
|
||||||
|
|
||||||
|
import (
|
||||||
|
"os"
|
||||||
|
"strconv"
|
||||||
|
"strings"
|
||||||
|
)
|
||||||
|
|
||||||
|
// processes lists the running processes from /proc/<pid>/comm.
|
||||||
|
func processes() ([]Process, error) {
|
||||||
|
entries, err := os.ReadDir("/proc")
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
var ps []Process
|
||||||
|
for _, e := range entries {
|
||||||
|
pid, err := strconv.Atoi(e.Name())
|
||||||
|
if err != nil {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
comm, err := os.ReadFile("/proc/" + e.Name() + "/comm")
|
||||||
|
if err != nil {
|
||||||
|
continue // process vanished mid-walk
|
||||||
|
}
|
||||||
|
ps = append(ps, Process{PID: pid, Name: strings.TrimSpace(string(comm))})
|
||||||
|
}
|
||||||
|
return ps, nil
|
||||||
|
}
|
||||||
@@ -0,0 +1,7 @@
|
|||||||
|
//go:build !windows && !linux
|
||||||
|
|
||||||
|
package game
|
||||||
|
|
||||||
|
// processes is unsupported on this platform; the process watch list never
|
||||||
|
// matches and the nvidia-smi path reports PIDs without names.
|
||||||
|
func processes() ([]Process, error) { return nil, nil }
|
||||||
@@ -0,0 +1,35 @@
|
|||||||
|
//go:build windows
|
||||||
|
|
||||||
|
package game
|
||||||
|
|
||||||
|
import (
|
||||||
|
"unsafe"
|
||||||
|
|
||||||
|
"golang.org/x/sys/windows"
|
||||||
|
)
|
||||||
|
|
||||||
|
// processes lists the running processes via the Toolhelp32 snapshot API.
|
||||||
|
func processes() ([]Process, error) {
|
||||||
|
h, err := windows.CreateToolhelp32Snapshot(windows.TH32CS_SNAPPROCESS, 0)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
defer windows.CloseHandle(h) //nolint:errcheck // best effort
|
||||||
|
|
||||||
|
var entry windows.ProcessEntry32
|
||||||
|
entry.Size = uint32(unsafe.Sizeof(entry))
|
||||||
|
if err := windows.Process32First(h, &entry); err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
var ps []Process
|
||||||
|
for {
|
||||||
|
ps = append(ps, Process{
|
||||||
|
PID: int(entry.ProcessID),
|
||||||
|
Name: windows.UTF16ToString(entry.ExeFile[:]),
|
||||||
|
})
|
||||||
|
if err := windows.Process32Next(h, &entry); err != nil {
|
||||||
|
break // ERROR_NO_MORE_FILES ends the walk
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return ps, nil
|
||||||
|
}
|
||||||
+40
-5
@@ -1,6 +1,8 @@
|
|||||||
// Package lock implements the two-mode GPU arbitration lock: any number of
|
// Package lock implements the two-mode GPU arbitration lock: any number of
|
||||||
// concurrent LLM requests ("readers") or exactly one image job ("writer"),
|
// concurrent LLM requests ("readers") or exactly one image job ("writer"),
|
||||||
// with image jobs taking priority over newly arriving LLM requests.
|
// with image jobs taking priority over newly arriving LLM requests. An
|
||||||
|
// external hold (SetExternal) blocks new grants of both kinds while a
|
||||||
|
// foreign process — e.g. a game — holds the GPU; in-flight work drains.
|
||||||
package lock
|
package lock
|
||||||
|
|
||||||
import (
|
import (
|
||||||
@@ -16,6 +18,7 @@ const (
|
|||||||
StateIdle State = "idle"
|
StateIdle State = "idle"
|
||||||
StateLLM State = "llm"
|
StateLLM State = "llm"
|
||||||
StateImage State = "image"
|
StateImage State = "image"
|
||||||
|
StateExternal State = "external"
|
||||||
)
|
)
|
||||||
|
|
||||||
type imageWaiter struct{ id uint64 }
|
type imageWaiter struct{ id uint64 }
|
||||||
@@ -30,6 +33,7 @@ type Lock struct {
|
|||||||
imageActive bool // an image job holds the GPU
|
imageActive bool // an image job holds the GPU
|
||||||
imageQ []imageWaiter
|
imageQ []imageWaiter
|
||||||
nextID uint64
|
nextID uint64
|
||||||
|
external string // non-empty: a foreign process (e.g. a game) holds the GPU
|
||||||
|
|
||||||
log *slog.Logger
|
log *slog.Logger
|
||||||
}
|
}
|
||||||
@@ -52,12 +56,41 @@ func (l *Lock) logTransition(msg string, args ...any) {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// SetExternal records that a process outside gpu-turnstile's control (a
|
||||||
|
// game, another ML job) holds the GPU: new LLM and image grants block until
|
||||||
|
// ClearExternal. In-flight work is not preempted. holder describes the
|
||||||
|
// process for logs and busy responses.
|
||||||
|
func (l *Lock) SetExternal(holder string) {
|
||||||
|
l.mu.Lock()
|
||||||
|
l.external = holder
|
||||||
|
l.broadcast()
|
||||||
|
l.mu.Unlock()
|
||||||
|
l.logTransition("lock transition", "state", StateExternal, "holder", holder)
|
||||||
|
}
|
||||||
|
|
||||||
|
// ClearExternal lifts the external hold; waiting LLM and image requests
|
||||||
|
// proceed.
|
||||||
|
func (l *Lock) ClearExternal() {
|
||||||
|
l.mu.Lock()
|
||||||
|
l.external = ""
|
||||||
|
l.broadcast()
|
||||||
|
l.mu.Unlock()
|
||||||
|
l.logTransition("lock transition", "state", StateIdle)
|
||||||
|
}
|
||||||
|
|
||||||
|
// External returns the current external holder, or "" when none.
|
||||||
|
func (l *Lock) External() string {
|
||||||
|
l.mu.Lock()
|
||||||
|
defer l.mu.Unlock()
|
||||||
|
return l.external
|
||||||
|
}
|
||||||
|
|
||||||
// AcquireLLM blocks until no image job is active or pending, then registers
|
// AcquireLLM blocks until no image job is active or pending, then registers
|
||||||
// one in-flight LLM request. Returns ctx.Err() if the context is cancelled
|
// one in-flight LLM request. Returns ctx.Err() if the context is cancelled
|
||||||
// while waiting; no state is changed in that case.
|
// while waiting; no state is changed in that case.
|
||||||
func (l *Lock) AcquireLLM(ctx context.Context) error {
|
func (l *Lock) AcquireLLM(ctx context.Context) error {
|
||||||
l.mu.Lock()
|
l.mu.Lock()
|
||||||
for l.imageActive || len(l.imageQ) > 0 {
|
for l.imageActive || len(l.imageQ) > 0 || l.external != "" {
|
||||||
ch := l.change
|
ch := l.change
|
||||||
l.mu.Unlock()
|
l.mu.Unlock()
|
||||||
select {
|
select {
|
||||||
@@ -76,10 +109,10 @@ func (l *Lock) AcquireLLM(ctx context.Context) error {
|
|||||||
|
|
||||||
// TryAcquireLLM acquires one in-flight LLM slot without waiting and
|
// TryAcquireLLM acquires one in-flight LLM slot without waiting and
|
||||||
// reports whether it succeeded. It fails when an image job is active or
|
// reports whether it succeeded. It fails when an image job is active or
|
||||||
// pending.
|
// pending or an external hold is set.
|
||||||
func (l *Lock) TryAcquireLLM() bool {
|
func (l *Lock) TryAcquireLLM() bool {
|
||||||
l.mu.Lock()
|
l.mu.Lock()
|
||||||
if l.imageActive || len(l.imageQ) > 0 {
|
if l.imageActive || len(l.imageQ) > 0 || l.external != "" {
|
||||||
l.mu.Unlock()
|
l.mu.Unlock()
|
||||||
return false
|
return false
|
||||||
}
|
}
|
||||||
@@ -120,7 +153,7 @@ func (l *Lock) AcquireImage(ctx context.Context) error {
|
|||||||
|
|
||||||
for {
|
for {
|
||||||
l.mu.Lock()
|
l.mu.Lock()
|
||||||
if l.imageQ[0].id == w.id && l.n == 0 && !l.imageActive {
|
if l.imageQ[0].id == w.id && l.n == 0 && !l.imageActive && l.external == "" {
|
||||||
l.imageQ = l.imageQ[1:]
|
l.imageQ = l.imageQ[1:]
|
||||||
l.imageActive = true
|
l.imageActive = true
|
||||||
l.mu.Unlock()
|
l.mu.Unlock()
|
||||||
@@ -166,6 +199,8 @@ func (l *Lock) Snapshot() (state State, llmInflight int, imagePending bool) {
|
|||||||
state = StateImage
|
state = StateImage
|
||||||
case l.n > 0:
|
case l.n > 0:
|
||||||
state = StateLLM
|
state = StateLLM
|
||||||
|
case l.external != "":
|
||||||
|
state = StateExternal
|
||||||
default:
|
default:
|
||||||
state = StateIdle
|
state = StateIdle
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -226,3 +226,70 @@ func TestRace(t *testing.T) {
|
|||||||
t.Fatalf("leaked lock state: state=%s n=%d pending=%v", state, n, pending)
|
t.Fatalf("leaked lock state: state=%s n=%d pending=%v", state, n, pending)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func TestExternalHoldBlocksBoth(t *testing.T) {
|
||||||
|
lk := New(nil)
|
||||||
|
ctx := context.Background()
|
||||||
|
|
||||||
|
lk.SetExternal("game.exe (pid 42)")
|
||||||
|
if lk.TryAcquireLLM() {
|
||||||
|
t.Fatal("TryAcquireLLM succeeded during external hold")
|
||||||
|
}
|
||||||
|
if got := lk.External(); got != "game.exe (pid 42)" {
|
||||||
|
t.Fatalf("External() = %q", got)
|
||||||
|
}
|
||||||
|
if state, _, _ := lk.Snapshot(); state != StateExternal {
|
||||||
|
t.Fatalf("state=%s, want external", state)
|
||||||
|
}
|
||||||
|
|
||||||
|
llmAcquired := make(chan struct{})
|
||||||
|
go func() {
|
||||||
|
if err := lk.AcquireLLM(ctx); err != nil {
|
||||||
|
t.Error(err)
|
||||||
|
}
|
||||||
|
close(llmAcquired)
|
||||||
|
}()
|
||||||
|
assertBlocked(t, llmAcquired, "LLM acquire during external hold")
|
||||||
|
|
||||||
|
imageAcquired := make(chan struct{})
|
||||||
|
go func() {
|
||||||
|
if err := lk.AcquireImage(ctx); err != nil {
|
||||||
|
t.Error(err)
|
||||||
|
}
|
||||||
|
close(imageAcquired)
|
||||||
|
}()
|
||||||
|
assertBlocked(t, imageAcquired, "image acquire during external hold")
|
||||||
|
|
||||||
|
lk.ClearExternal()
|
||||||
|
// The queued image job wins over the LLM waiter (image priority).
|
||||||
|
waitFor(t, imageAcquired, "image acquire after ClearExternal")
|
||||||
|
assertBlocked(t, llmAcquired, "LLM acquire while image active")
|
||||||
|
lk.ReleaseImage()
|
||||||
|
waitFor(t, llmAcquired, "LLM acquire after image release")
|
||||||
|
lk.ReleaseLLM()
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestExternalHoldDoesNotPreempt(t *testing.T) {
|
||||||
|
lk := New(nil)
|
||||||
|
ctx := context.Background()
|
||||||
|
if err := lk.AcquireLLM(ctx); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
|
||||||
|
lk.SetExternal("game.exe")
|
||||||
|
// In-flight LLM work keeps the llm state; the hold blocks new grants.
|
||||||
|
if state, _, _ := lk.Snapshot(); state != StateLLM {
|
||||||
|
t.Fatalf("state=%s, want llm while work in flight", state)
|
||||||
|
}
|
||||||
|
if lk.TryAcquireLLM() {
|
||||||
|
t.Fatal("TryAcquireLLM succeeded during external hold")
|
||||||
|
}
|
||||||
|
lk.ReleaseLLM()
|
||||||
|
if state, _, _ := lk.Snapshot(); state != StateExternal {
|
||||||
|
t.Fatalf("state=%s, want external after drain", state)
|
||||||
|
}
|
||||||
|
lk.ClearExternal()
|
||||||
|
if state, _, _ := lk.Snapshot(); state != StateIdle {
|
||||||
|
t.Fatalf("state=%s, want idle", state)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|||||||
@@ -89,7 +89,7 @@ func (m *Metrics) Render(w io.Writer, state string, llmInflight int, imagePendin
|
|||||||
fmt.Fprint(w, `# HELP gpu_turnstile_state Current GPU state (1 for the active state).
|
fmt.Fprint(w, `# HELP gpu_turnstile_state Current GPU state (1 for the active state).
|
||||||
# TYPE gpu_turnstile_state gauge
|
# TYPE gpu_turnstile_state gauge
|
||||||
`)
|
`)
|
||||||
for _, s := range []string{"idle", "llm", "image"} {
|
for _, s := range []string{"idle", "llm", "image", "external"} {
|
||||||
v := 0
|
v := 0
|
||||||
if s == state {
|
if s == state {
|
||||||
v = 1
|
v = 1
|
||||||
|
|||||||
+62
-3
@@ -26,6 +26,7 @@ import (
|
|||||||
"gpu-turnstile/internal/lock"
|
"gpu-turnstile/internal/lock"
|
||||||
"gpu-turnstile/internal/metrics"
|
"gpu-turnstile/internal/metrics"
|
||||||
"gpu-turnstile/internal/ollama"
|
"gpu-turnstile/internal/ollama"
|
||||||
|
"gpu-turnstile/internal/supervise"
|
||||||
)
|
)
|
||||||
|
|
||||||
// defaultCaptureLimit bounds how much of a /prompt response body is
|
// defaultCaptureLimit bounds how much of a /prompt response body is
|
||||||
@@ -44,6 +45,11 @@ type Config struct {
|
|||||||
Metrics *metrics.Metrics
|
Metrics *metrics.Metrics
|
||||||
Log *slog.Logger
|
Log *slog.Logger
|
||||||
|
|
||||||
|
// ComfySup, when non-nil, is the managed ComfyUI process: any ComfyUI
|
||||||
|
// request starts it on demand, /prompt additionally waits for
|
||||||
|
// readiness before taking the GPU lock. nil = unmanaged upstream.
|
||||||
|
ComfySup *supervise.Process
|
||||||
|
|
||||||
// LogColor enables ANSI colors in per-request log lines. Ignored when
|
// LogColor enables ANSI colors in per-request log lines. Ignored when
|
||||||
// the log level is above INFO (request lines are not emitted at all).
|
// the log level is above INFO (request lines are not emitted at all).
|
||||||
LogColor bool
|
LogColor bool
|
||||||
@@ -438,7 +444,7 @@ func isLLMRequest(r *http.Request) bool {
|
|||||||
return r.Method == http.MethodPost && llmPaths[r.URL.Path]
|
return r.Method == http.MethodPost && llmPaths[r.URL.Path]
|
||||||
}
|
}
|
||||||
|
|
||||||
// OllamaHandler serves the Ollama-facing listener.
|
// OllamaHandler serves the listener for Ollama-compatible clients.
|
||||||
func (s *Server) OllamaHandler() http.Handler {
|
func (s *Server) OllamaHandler() http.Handler {
|
||||||
return s.logRequests("ollama", http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
return s.logRequests("ollama", http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||||
switch r.URL.Path {
|
switch r.URL.Path {
|
||||||
@@ -458,10 +464,14 @@ func (s *Server) OllamaHandler() http.Handler {
|
|||||||
if s.busyMode == "reject" {
|
if s.busyMode == "reject" {
|
||||||
if !s.cfg.Lock.TryAcquireLLM() {
|
if !s.cfg.Lock.TryAcquireLLM() {
|
||||||
s.cfg.Metrics.ObserveLockWait("llm", time.Since(start).Seconds())
|
s.cfg.Metrics.ObserveLockWait("llm", time.Since(start).Seconds())
|
||||||
|
msg := "GPU busy: image job active or queued"
|
||||||
|
if holder := s.cfg.Lock.External(); holder != "" {
|
||||||
|
msg = "GPU busy: " + holder
|
||||||
|
}
|
||||||
s.log.Info("llm request rejected; GPU busy",
|
s.log.Info("llm request rejected; GPU busy",
|
||||||
"path", r.URL.Path, "status", s.busyStatus)
|
"path", r.URL.Path, "status", s.busyStatus)
|
||||||
w.Header().Set("Retry-After", strconv.Itoa(s.busyRetryAfter))
|
w.Header().Set("Retry-After", strconv.Itoa(s.busyRetryAfter))
|
||||||
http.Error(w, "GPU busy: image job active or queued", s.busyStatus)
|
http.Error(w, msg, s.busyStatus)
|
||||||
return
|
return
|
||||||
}
|
}
|
||||||
s.cfg.Metrics.ObserveLockWait("llm", time.Since(start).Seconds())
|
s.cfg.Metrics.ObserveLockWait("llm", time.Since(start).Seconds())
|
||||||
@@ -485,7 +495,7 @@ func (s *Server) OllamaHandler() http.Handler {
|
|||||||
}))
|
}))
|
||||||
}
|
}
|
||||||
|
|
||||||
// ComfyHandler serves the ComfyUI-facing listener.
|
// ComfyHandler serves the listener for ComfyUI clients.
|
||||||
func (s *Server) ComfyHandler() http.Handler {
|
func (s *Server) ComfyHandler() http.Handler {
|
||||||
return s.logRequests("comfy", http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
return s.logRequests("comfy", http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||||
switch r.URL.Path {
|
switch r.URL.Path {
|
||||||
@@ -500,6 +510,21 @@ func (s *Server) ComfyHandler() http.Handler {
|
|||||||
s.handlePrompt(w, r)
|
s.handlePrompt(w, r)
|
||||||
return
|
return
|
||||||
}
|
}
|
||||||
|
if s.cfg.ComfySup != nil {
|
||||||
|
// Any other ComfyUI request also wakes the managed server; the
|
||||||
|
// retry backoff bridges the time it needs to come up. While a
|
||||||
|
// foreign process holds the GPU we refuse to spawn it — the
|
||||||
|
// request gets the busy answer instead of fighting for VRAM.
|
||||||
|
if holder := s.cfg.Lock.External(); holder != "" && !s.cfg.ComfySup.Running() {
|
||||||
|
w.Header().Set("Retry-After", strconv.Itoa(s.busyRetryAfter))
|
||||||
|
http.Error(w, "GPU busy: "+holder, http.StatusServiceUnavailable)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
if err := s.cfg.ComfySup.EnsureRunning(); err != nil {
|
||||||
|
http.Error(w, fmt.Sprintf("cannot start ComfyUI: %v", err), http.StatusBadGateway)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
}
|
||||||
s.comfyProxy.ServeHTTP(w, r)
|
s.comfyProxy.ServeHTTP(w, r)
|
||||||
}))
|
}))
|
||||||
}
|
}
|
||||||
@@ -539,6 +564,23 @@ func (w *captureWriter) Unwrap() http.ResponseWriter { return w.ResponseWriter }
|
|||||||
func (s *Server) handlePrompt(w http.ResponseWriter, r *http.Request) {
|
func (s *Server) handlePrompt(w http.ResponseWriter, r *http.Request) {
|
||||||
log := s.log.With("op", "image")
|
log := s.log.With("op", "image")
|
||||||
|
|
||||||
|
// Normally the managed server is brought up *before* taking the GPU
|
||||||
|
// lock: torch can take a minute to load, and LLM traffic should keep
|
||||||
|
// flowing in the meantime. While a foreign process holds the GPU, LLM
|
||||||
|
// traffic is blocked anyway and a fresh ComfyUI would fight it for
|
||||||
|
// VRAM — so the lock comes first in that case.
|
||||||
|
comfyFirst := s.cfg.ComfySup != nil && s.cfg.Lock.External() == ""
|
||||||
|
if comfyFirst {
|
||||||
|
if err := s.cfg.ComfySup.EnsureRunning(); err != nil {
|
||||||
|
http.Error(w, fmt.Sprintf("cannot start ComfyUI: %v", err), http.StatusBadGateway)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
if err := s.cfg.ComfySup.WaitReady(r.Context()); err != nil {
|
||||||
|
http.Error(w, fmt.Sprintf("ComfyUI did not become ready: %v", err), http.StatusBadGateway)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
start := time.Now()
|
start := time.Now()
|
||||||
if err := s.cfg.Lock.AcquireImage(r.Context()); err != nil {
|
if err := s.cfg.Lock.AcquireImage(r.Context()); err != nil {
|
||||||
if errors.Is(err, context.DeadlineExceeded) {
|
if errors.Is(err, context.DeadlineExceeded) {
|
||||||
@@ -549,6 +591,19 @@ func (s *Server) handlePrompt(w http.ResponseWriter, r *http.Request) {
|
|||||||
s.cfg.Metrics.ObserveLockWait("image", time.Since(start).Seconds())
|
s.cfg.Metrics.ObserveLockWait("image", time.Since(start).Seconds())
|
||||||
log.Info("image lock acquired")
|
log.Info("image lock acquired")
|
||||||
|
|
||||||
|
if s.cfg.ComfySup != nil && !comfyFirst {
|
||||||
|
if err := s.cfg.ComfySup.EnsureRunning(); err != nil {
|
||||||
|
s.cfg.Lock.ReleaseImage()
|
||||||
|
http.Error(w, fmt.Sprintf("cannot start ComfyUI: %v", err), http.StatusBadGateway)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
if err := s.cfg.ComfySup.WaitReady(r.Context()); err != nil {
|
||||||
|
s.cfg.Lock.ReleaseImage()
|
||||||
|
http.Error(w, fmt.Sprintf("ComfyUI did not become ready: %v", err), http.StatusBadGateway)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
if s.cfg.Ollama != nil {
|
if s.cfg.Ollama != nil {
|
||||||
uctx, ucancel := context.WithTimeout(r.Context(), s.cfg.UnloadTimeout)
|
uctx, ucancel := context.WithTimeout(r.Context(), s.cfg.UnloadTimeout)
|
||||||
elapsed, uerr := s.cfg.Ollama.UnloadAll(uctx)
|
elapsed, uerr := s.cfg.Ollama.UnloadAll(uctx)
|
||||||
@@ -608,6 +663,10 @@ func (s *Server) finishImageJob(promptID string) {
|
|||||||
|
|
||||||
s.cfg.Lock.ReleaseImage()
|
s.cfg.Lock.ReleaseImage()
|
||||||
log.Info("image lock released")
|
log.Info("image lock released")
|
||||||
|
if s.cfg.ComfySup != nil {
|
||||||
|
// The idle clock starts when the job ends, not when it began.
|
||||||
|
s.cfg.ComfySup.NoteActivity()
|
||||||
|
}
|
||||||
|
|
||||||
if s.cfg.Ollama != nil && s.cfg.WarmModel != "" {
|
if s.cfg.Ollama != nil && s.cfg.WarmModel != "" {
|
||||||
if state, _, _ := s.cfg.Lock.Snapshot(); state == lock.StateIdle {
|
if state, _, _ := s.cfg.Lock.Snapshot(); state == lock.StateIdle {
|
||||||
|
|||||||
@@ -46,8 +46,8 @@ type fakes struct {
|
|||||||
rec *recorder
|
rec *recorder
|
||||||
ollama *httptest.Server
|
ollama *httptest.Server
|
||||||
comfy *httptest.Server
|
comfy *httptest.Server
|
||||||
server *httptest.Server // Ollama-facing gpu-turnstile listener
|
server *httptest.Server // gpu-turnstile listener for Ollama-compatible clients
|
||||||
comfySrv *httptest.Server // ComfyUI-facing gpu-turnstile listener
|
comfySrv *httptest.Server // gpu-turnstile listener for ComfyUI clients
|
||||||
freeCh chan struct{}
|
freeCh chan struct{}
|
||||||
chatCh chan struct{}
|
chatCh chan struct{}
|
||||||
historyMu sync.Mutex
|
historyMu sync.Mutex
|
||||||
|
|||||||
@@ -0,0 +1,63 @@
|
|||||||
|
// Shared env-file handling for the installers (Windows and Linux).
|
||||||
|
package service
|
||||||
|
|
||||||
|
import (
|
||||||
|
"fmt"
|
||||||
|
"os"
|
||||||
|
"strings"
|
||||||
|
|
||||||
|
"gpu-turnstile/internal/config"
|
||||||
|
)
|
||||||
|
|
||||||
|
// syncEnvFile ensures the installed env file at path is a current
|
||||||
|
// installer-written sample. A missing file is created; a file without a
|
||||||
|
// CFG_VER line (hand-written or from before versioning) is invalid and
|
||||||
|
// replaced by a fresh sample after a .bak backup; a file written by an
|
||||||
|
// older version gets newly added settings appended via config.SyncSample.
|
||||||
|
// logPath activates the LOG_FILE line (Windows); empty leaves it commented.
|
||||||
|
func syncEnvFile(path, version, logPath string) error {
|
||||||
|
data, err := os.ReadFile(path)
|
||||||
|
switch {
|
||||||
|
case os.IsNotExist(err):
|
||||||
|
return writeEnvFile(path, config.SampleEnv(version, logPath))
|
||||||
|
case err != nil:
|
||||||
|
return fmt.Errorf("read %s: %w", path, err)
|
||||||
|
}
|
||||||
|
values, perr := config.ParseEnvFile(strings.NewReader(string(data)))
|
||||||
|
if perr == nil && values["CFG_VER"] != "" {
|
||||||
|
synced, changed := config.SyncSample(string(data), version, logPath)
|
||||||
|
if !changed {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
return writeEnvFile(path, synced)
|
||||||
|
}
|
||||||
|
// No readable CFG_VER: the file is invalid — keep a backup and start
|
||||||
|
// from a fresh sample.
|
||||||
|
if err := os.WriteFile(path+".bak", data, 0o644); err != nil {
|
||||||
|
return fmt.Errorf("back up %s: %w", path, err)
|
||||||
|
}
|
||||||
|
return writeEnvFile(path, config.SampleEnv(version, logPath))
|
||||||
|
}
|
||||||
|
|
||||||
|
func writeEnvFile(path, content string) error {
|
||||||
|
if err := os.WriteFile(path, []byte(content), 0o644); err != nil {
|
||||||
|
return fmt.Errorf("write %s: %w", path, err)
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// configuredValue reads one key from the env file the service will load, so
|
||||||
|
// the installers can adapt the sandbox to it (ACL grants, unit directives).
|
||||||
|
// "" when unset or unreadable.
|
||||||
|
func configuredValue(configPath, key string) string {
|
||||||
|
f, err := os.Open(configPath)
|
||||||
|
if err != nil {
|
||||||
|
return ""
|
||||||
|
}
|
||||||
|
defer f.Close()
|
||||||
|
values, err := config.ParseEnvFile(f)
|
||||||
|
if err != nil {
|
||||||
|
return ""
|
||||||
|
}
|
||||||
|
return values[key]
|
||||||
|
}
|
||||||
@@ -0,0 +1,8 @@
|
|||||||
|
package service
|
||||||
|
|
||||||
|
import "errors"
|
||||||
|
|
||||||
|
// ErrUserCancelled is returned by RelaunchElevated when the user declines
|
||||||
|
// the UAC prompt. Windows-only in practice; defined here so cross-platform
|
||||||
|
// callers can compare against it.
|
||||||
|
var ErrUserCancelled = errors.New("UAC prompt declined")
|
||||||
@@ -2,12 +2,14 @@
|
|||||||
|
|
||||||
// Package service integrates gpu-turnstile with systemd on Linux: running
|
// Package service integrates gpu-turnstile with systemd on Linux: running
|
||||||
// under a unit with readiness notification and watchdog, plus
|
// under a unit with readiness notification and watchdog, plus
|
||||||
// install/remove helpers that manage a system unit.
|
// install/remove helpers that manage a hardened system unit.
|
||||||
package service
|
package service
|
||||||
|
|
||||||
import (
|
import (
|
||||||
|
"bytes"
|
||||||
"context"
|
"context"
|
||||||
"fmt"
|
"fmt"
|
||||||
|
"io"
|
||||||
"os"
|
"os"
|
||||||
"os/exec"
|
"os/exec"
|
||||||
"os/signal"
|
"os/signal"
|
||||||
@@ -21,9 +23,26 @@ const Name = "gpu-turnstile"
|
|||||||
// unitPath is where Install writes the unit file.
|
// unitPath is where Install writes the unit file.
|
||||||
const unitPath = "/etc/systemd/system/" + Name + ".service"
|
const unitPath = "/etc/systemd/system/" + Name + ".service"
|
||||||
|
|
||||||
|
// stateDir holds the installed binary (and staged updates); the unit's
|
||||||
|
// StateDirectory= directive makes systemd own it and grant the dynamic
|
||||||
|
// user write access. etcConfig is the config file the unit loads.
|
||||||
|
const (
|
||||||
|
stateDir = "/var/lib/" + Name
|
||||||
|
etcConfig = "/etc/" + Name + ".env"
|
||||||
|
)
|
||||||
|
|
||||||
// IsService reports whether the process was started by systemd.
|
// IsService reports whether the process was started by systemd.
|
||||||
func IsService() bool { return os.Getenv("INVOCATION_ID") != "" }
|
func IsService() bool { return os.Getenv("INVOCATION_ID") != "" }
|
||||||
|
|
||||||
|
// Elevated is always true on Linux: there is no UAC equivalent; privilege
|
||||||
|
// errors surface from the failing operation with a "run as root" hint.
|
||||||
|
func Elevated() bool { return true }
|
||||||
|
|
||||||
|
// RelaunchElevated is a Windows-only concept (UAC).
|
||||||
|
func RelaunchElevated([]string) (int, error) {
|
||||||
|
return 0, fmt.Errorf("elevated relaunch is only supported on Windows")
|
||||||
|
}
|
||||||
|
|
||||||
// Run executes run with SIGINT/SIGTERM cancellation (which is how systemctl
|
// Run executes run with SIGINT/SIGTERM cancellation (which is how systemctl
|
||||||
// stop signals the process) and tells systemd when the shutdown begins.
|
// stop signals the process) and tells systemd when the shutdown begins.
|
||||||
func Run(run func(ctx context.Context) error) error {
|
func Run(run func(ctx context.Context) error) error {
|
||||||
@@ -33,11 +52,24 @@ func Run(run func(ctx context.Context) error) error {
|
|||||||
return run(ctx)
|
return run(ctx)
|
||||||
}
|
}
|
||||||
|
|
||||||
// renderUnit builds the systemd unit: Type=notify so systemctl start blocks
|
// renderUnit builds the hardened systemd unit: Type=notify so systemctl
|
||||||
// until the listeners are bound, a 30s watchdog, and restart-on-failure
|
// start blocks until the listeners are bound, a 30s watchdog, and
|
||||||
// with a 5s delay — which is also what brings up a staged update after the
|
// restart-on-failure with a 5s delay — which is also what brings up a
|
||||||
// updater exits with a non-zero code.
|
// staged update after the updater exits with a non-zero code.
|
||||||
func renderUnit(exePath, configPath string) string {
|
//
|
||||||
|
// Sandboxing mirrors the Windows virtual account: DynamicUser=yes gives
|
||||||
|
// the service a transient per-service UID with no login and no home, the
|
||||||
|
// filesystem is read-only except StateDirectory (the install dir, so
|
||||||
|
// self-updates can rewrite the binary), and the usual no-privilege-escalation
|
||||||
|
// directives apply. The proxy needs nothing but outbound TCP/UDP and the
|
||||||
|
// notify socket, so it loses nothing. A managed ComfyUI (comfyDir) gets a
|
||||||
|
// BindPaths hole through ProtectHome/ProtectSystem: it reads its venv and
|
||||||
|
// writes output/temp/user data under COMFY_DIR.
|
||||||
|
func renderUnit(exePath, configPath, comfyDir string) string {
|
||||||
|
bind := ""
|
||||||
|
if comfyDir != "" {
|
||||||
|
bind = "BindPaths=" + comfyDir + "\n"
|
||||||
|
}
|
||||||
return fmt.Sprintf(`[Unit]
|
return fmt.Sprintf(`[Unit]
|
||||||
Description=gpu-turnstile GPU arbitration proxy for Ollama and ComfyUI
|
Description=gpu-turnstile GPU arbitration proxy for Ollama and ComfyUI
|
||||||
After=network-online.target
|
After=network-online.target
|
||||||
@@ -50,14 +82,71 @@ ExecStart=%q -config %q
|
|||||||
Restart=on-failure
|
Restart=on-failure
|
||||||
RestartSec=5s
|
RestartSec=5s
|
||||||
|
|
||||||
|
DynamicUser=yes
|
||||||
|
StateDirectory=%s
|
||||||
|
%sProtectSystem=strict
|
||||||
|
ProtectHome=yes
|
||||||
|
PrivateTmp=yes
|
||||||
|
NoNewPrivileges=yes
|
||||||
|
ProtectKernelTunables=yes
|
||||||
|
ProtectKernelModules=yes
|
||||||
|
ProtectKernelLogs=yes
|
||||||
|
ProtectControlGroups=yes
|
||||||
|
ProtectClock=yes
|
||||||
|
RestrictNamespaces=yes
|
||||||
|
RestrictSUIDSGID=yes
|
||||||
|
RestrictRealtime=yes
|
||||||
|
LockPersonality=yes
|
||||||
|
MemoryDenyWriteExecute=yes
|
||||||
|
CapabilityBoundingSet=
|
||||||
|
AmbientCapabilities=
|
||||||
|
RestrictAddressFamilies=AF_UNIX AF_INET AF_INET6
|
||||||
|
SystemCallFilter=@system-service
|
||||||
|
SystemCallErrorNumber=EPERM
|
||||||
|
|
||||||
[Install]
|
[Install]
|
||||||
WantedBy=multi-user.target
|
WantedBy=multi-user.target
|
||||||
`, exePath, configPath)
|
`, exePath, configPath, Name, bind)
|
||||||
}
|
}
|
||||||
|
|
||||||
// Install writes the unit for the current executable and the given config
|
// copyFile copies src to dst, creating dst with the given mode.
|
||||||
// file, then enables and starts it. Needs root.
|
func copyFile(src, dst string, mode os.FileMode) error {
|
||||||
func Install(configPath string) error {
|
in, err := os.Open(src)
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
defer in.Close()
|
||||||
|
out, err := os.OpenFile(dst, os.O_CREATE|os.O_TRUNC|os.O_WRONLY, mode)
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
defer out.Close()
|
||||||
|
if _, err := io.Copy(out, in); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
return out.Close()
|
||||||
|
}
|
||||||
|
|
||||||
|
// Install copies the current executable into /var/lib/gpu-turnstile, makes
|
||||||
|
// sure /etc/gpu-turnstile.env exists (copied from the given config file if
|
||||||
|
// provided), writes the hardened unit, then enables and starts it. With
|
||||||
|
// copyBin=false the current executable location and config path are
|
||||||
|
// registered as-is instead. When the config sets COMFY_DIR, the unit gets a
|
||||||
|
// BindPaths= for it so the sandboxed service can reach the managed ComfyUI
|
||||||
|
// even under /home. Needs root.
|
||||||
|
//
|
||||||
|
// Re-running install converges an existing unit instead of failing: it is
|
||||||
|
// stopped first if active, the installed binary is replaced only when the
|
||||||
|
// content differs, the unit file is rewritten (followed by daemon-reload)
|
||||||
|
// only when it changed, and the service is started again only if it was
|
||||||
|
// active before.
|
||||||
|
//
|
||||||
|
// The binary lives in the StateDirectory rather than /usr/local/sbin on
|
||||||
|
// purpose: replacing a running binary needs write access to its
|
||||||
|
// *directory*, and granting the sandboxed service user write access to a
|
||||||
|
// shared system directory would let a compromised service overwrite other
|
||||||
|
// binaries. /var/lib/gpu-turnstile is exclusively ours.
|
||||||
|
func Install(configPath string, copyBin bool, version string) error {
|
||||||
exe, err := os.Executable()
|
exe, err := os.Executable()
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return err
|
return err
|
||||||
@@ -65,26 +154,114 @@ func Install(configPath string) error {
|
|||||||
if abs, absErr := filepath.Abs(exe); absErr == nil {
|
if abs, absErr := filepath.Abs(exe); absErr == nil {
|
||||||
exe = abs
|
exe = abs
|
||||||
}
|
}
|
||||||
if err := os.WriteFile(unitPath, []byte(renderUnit(exe, configPath)), 0o644); err != nil {
|
unit := Name + ".service"
|
||||||
|
_, statErr := os.Stat(unitPath)
|
||||||
|
fresh := os.IsNotExist(statErr)
|
||||||
|
wasRunning := exec.Command("systemctl", "is-active", "--quiet", unit).Run() == nil
|
||||||
|
if wasRunning {
|
||||||
|
if out, err := exec.Command("systemctl", "stop", unit).CombinedOutput(); err != nil {
|
||||||
|
return fmt.Errorf("systemctl stop (run as root): %w (%s)", err, out)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
cfg := etcConfig
|
||||||
|
if copyBin {
|
||||||
|
if err := os.MkdirAll(stateDir, 0o755); err != nil {
|
||||||
|
return fmt.Errorf("create %s (run as root): %w", stateDir, err)
|
||||||
|
}
|
||||||
|
installedExe := filepath.Join(stateDir, Name)
|
||||||
|
if exe != installedExe {
|
||||||
|
if same, _ := sameFileContent(exe, installedExe); !same {
|
||||||
|
if err := copyFile(exe, installedExe, 0o755); err != nil {
|
||||||
|
return fmt.Errorf("install binary to %s: %w", installedExe, err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
exe = installedExe
|
||||||
|
if _, err := os.Stat(etcConfig); os.IsNotExist(err) && configPath != "" {
|
||||||
|
copyFile(configPath, etcConfig, 0o644) //nolint:errcheck // best effort
|
||||||
|
}
|
||||||
|
// Missing configs get a fully commented sample (LOG_FILE stays
|
||||||
|
// commented — stderr goes to the journal on Linux);
|
||||||
|
// installer-written ones from older versions get new settings
|
||||||
|
// appended; anything without CFG_VER is invalid and gets replaced
|
||||||
|
// (backup kept as .bak).
|
||||||
|
if err := syncEnvFile(etcConfig, version, ""); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
} else if configPath != "" {
|
||||||
|
if abs, absErr := filepath.Abs(configPath); absErr == nil {
|
||||||
|
cfg = abs
|
||||||
|
}
|
||||||
|
}
|
||||||
|
rendered := renderUnit(exe, cfg, configuredValue(cfg, "COMFY_DIR"))
|
||||||
|
if old, _ := os.ReadFile(unitPath); string(old) != rendered {
|
||||||
|
if err := os.WriteFile(unitPath, []byte(rendered), 0o644); err != nil {
|
||||||
return fmt.Errorf("write %s (run as root): %w", unitPath, err)
|
return fmt.Errorf("write %s (run as root): %w", unitPath, err)
|
||||||
}
|
}
|
||||||
if out, err := exec.Command("systemctl", "daemon-reload").CombinedOutput(); err != nil {
|
if out, err := exec.Command("systemctl", "daemon-reload").CombinedOutput(); err != nil {
|
||||||
return fmt.Errorf("systemctl daemon-reload: %w (%s)", err, out)
|
return fmt.Errorf("systemctl daemon-reload: %w (%s)", err, out)
|
||||||
}
|
}
|
||||||
if out, err := exec.Command("systemctl", "enable", "--now", Name+".service").CombinedOutput(); err != nil {
|
}
|
||||||
|
if fresh {
|
||||||
|
if out, err := exec.Command("systemctl", "enable", "--now", unit).CombinedOutput(); err != nil {
|
||||||
return fmt.Errorf("systemctl enable --now: %w (%s)", err, out)
|
return fmt.Errorf("systemctl enable --now: %w (%s)", err, out)
|
||||||
}
|
}
|
||||||
return nil
|
return nil
|
||||||
|
}
|
||||||
|
if exec.Command("systemctl", "is-enabled", "--quiet", unit).Run() != nil {
|
||||||
|
if out, err := exec.Command("systemctl", "enable", unit).CombinedOutput(); err != nil {
|
||||||
|
return fmt.Errorf("systemctl enable: %w (%s)", err, out)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if wasRunning {
|
||||||
|
if out, err := exec.Command("systemctl", "start", unit).CombinedOutput(); err != nil {
|
||||||
|
return fmt.Errorf("systemctl start: %w (%s)", err, out)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return nil
|
||||||
}
|
}
|
||||||
|
|
||||||
// Remove stops and disables the service and deletes the unit file.
|
// sameFileContent reports whether two files hold identical bytes. A missing
|
||||||
|
// destination is simply "different".
|
||||||
|
func sameFileContent(a, b string) (bool, error) {
|
||||||
|
ba, err := os.ReadFile(a)
|
||||||
|
if err != nil {
|
||||||
|
return false, err
|
||||||
|
}
|
||||||
|
bb, err := os.ReadFile(b)
|
||||||
|
if os.IsNotExist(err) {
|
||||||
|
return false, nil
|
||||||
|
}
|
||||||
|
if err != nil {
|
||||||
|
return false, err
|
||||||
|
}
|
||||||
|
return bytes.Equal(ba, bb), nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// Remove stops and disables the service and deletes the unit file and the
|
||||||
|
// installed binary. The config file in /etc is left in place (user data).
|
||||||
func Remove() error {
|
func Remove() error {
|
||||||
exec.Command("systemctl", "disable", "--now", Name+".service").Run() // ignore: may not exist
|
exec.Command("systemctl", "disable", "--now", Name+".service").Run() // ignore: may not exist
|
||||||
if err := os.Remove(unitPath); err != nil && !os.IsNotExist(err) {
|
if err := os.Remove(unitPath); err != nil && !os.IsNotExist(err) {
|
||||||
return fmt.Errorf("remove %s: %w", unitPath, err)
|
return fmt.Errorf("remove %s: %w", unitPath, err)
|
||||||
}
|
}
|
||||||
|
os.RemoveAll(stateDir) // installed binary + staged updates; ignore error
|
||||||
if out, err := exec.Command("systemctl", "daemon-reload").CombinedOutput(); err != nil {
|
if out, err := exec.Command("systemctl", "daemon-reload").CombinedOutput(); err != nil {
|
||||||
return fmt.Errorf("systemctl daemon-reload: %w (%s)", err, out)
|
return fmt.Errorf("systemctl daemon-reload: %w (%s)", err, out)
|
||||||
}
|
}
|
||||||
return nil
|
return nil
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// RestartIfRunning restarts the systemd unit when it is active (used after
|
||||||
|
// a forced update staged a new binary). Reports whether a restart
|
||||||
|
// happened. An inactive or missing unit is not an error. Needs root.
|
||||||
|
func RestartIfRunning() (bool, error) {
|
||||||
|
if err := exec.Command("systemctl", "is-active", "--quiet", Name+".service").Run(); err != nil {
|
||||||
|
return false, nil // inactive or not installed
|
||||||
|
}
|
||||||
|
if out, err := exec.Command("systemctl", "restart", Name+".service").CombinedOutput(); err != nil {
|
||||||
|
return false, fmt.Errorf("systemctl restart (run as root): %w (%s)", err, out)
|
||||||
|
}
|
||||||
|
return true, nil
|
||||||
|
}
|
||||||
|
|||||||
@@ -8,16 +8,30 @@ import (
|
|||||||
)
|
)
|
||||||
|
|
||||||
func TestRenderUnit(t *testing.T) {
|
func TestRenderUnit(t *testing.T) {
|
||||||
unit := renderUnit("/usr/local/bin/gpu-turnstile", "/etc/gpu-turnstile.env")
|
unit := renderUnit("/var/lib/gpu-turnstile/gpu-turnstile", "/etc/gpu-turnstile.env", "")
|
||||||
for _, want := range []string{
|
for _, want := range []string{
|
||||||
"Type=notify",
|
"Type=notify",
|
||||||
"WatchdogSec=30s",
|
"WatchdogSec=30s",
|
||||||
`ExecStart="/usr/local/bin/gpu-turnstile" -config "/etc/gpu-turnstile.env"`,
|
`ExecStart="/var/lib/gpu-turnstile/gpu-turnstile" -config "/etc/gpu-turnstile.env"`,
|
||||||
"Restart=on-failure",
|
"Restart=on-failure",
|
||||||
"WantedBy=multi-user.target",
|
"WantedBy=multi-user.target",
|
||||||
|
"DynamicUser=yes",
|
||||||
|
"StateDirectory=gpu-turnstile",
|
||||||
|
"ProtectSystem=strict",
|
||||||
|
"NoNewPrivileges=yes",
|
||||||
|
"RestrictAddressFamilies=AF_UNIX AF_INET AF_INET6",
|
||||||
|
"SystemCallFilter=@system-service",
|
||||||
} {
|
} {
|
||||||
if !strings.Contains(unit, want) {
|
if !strings.Contains(unit, want) {
|
||||||
t.Fatalf("unit missing %q:\n%s", want, unit)
|
t.Fatalf("unit missing %q:\n%s", want, unit)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
if strings.Contains(unit, "BindPaths") {
|
||||||
|
t.Fatalf("unit without COMFY_DIR must not bind anything:\n%s", unit)
|
||||||
|
}
|
||||||
|
|
||||||
|
unit = renderUnit("/var/lib/gpu-turnstile/gpu-turnstile", "/etc/gpu-turnstile.env", "/home/gpu/ComfyUI")
|
||||||
|
if !strings.Contains(unit, "BindPaths=/home/gpu/ComfyUI\n") {
|
||||||
|
t.Fatalf("unit with COMFY_DIR must bind it:\n%s", unit)
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -28,8 +28,18 @@ func Run(run func(ctx context.Context) error) error {
|
|||||||
return run(ctx)
|
return run(ctx)
|
||||||
}
|
}
|
||||||
|
|
||||||
// Install is unsupported on non-Windows platforms.
|
// Install is unsupported on non-Windows, non-Linux platforms.
|
||||||
func Install(string) error { return errUnsupported }
|
func Install(string, bool, string) error { return errUnsupported }
|
||||||
|
|
||||||
// Remove is unsupported on non-Windows platforms.
|
// Remove is unsupported on non-Windows, non-Linux platforms.
|
||||||
func Remove() error { return errUnsupported }
|
func Remove() error { return errUnsupported }
|
||||||
|
|
||||||
|
// Elevated is always true here: there is no UAC concept, and privilege
|
||||||
|
// errors surface from the failing operation with a "run as root" hint.
|
||||||
|
func Elevated() bool { return true }
|
||||||
|
|
||||||
|
// RelaunchElevated is unsupported on non-Windows, non-Linux platforms.
|
||||||
|
func RelaunchElevated([]string) (int, error) { return 0, errUnsupported }
|
||||||
|
|
||||||
|
// RestartIfRunning is a no-op on platforms without service integration.
|
||||||
|
func RestartIfRunning() (bool, error) { return false, nil }
|
||||||
|
|||||||
@@ -2,15 +2,26 @@
|
|||||||
|
|
||||||
// Package service integrates gpu-turnstile with the Windows Service
|
// Package service integrates gpu-turnstile with the Windows Service
|
||||||
// Control Manager: running as a service with graceful stop, plus
|
// Control Manager: running as a service with graceful stop, plus
|
||||||
// install/remove helpers.
|
// install/remove helpers. Installed services always run as the virtual
|
||||||
|
// account NT SERVICE\gpu-turnstile — a per-service low-privilege identity
|
||||||
|
// managed by the SCM, with no password and no admin rights.
|
||||||
package service
|
package service
|
||||||
|
|
||||||
import (
|
import (
|
||||||
|
"bytes"
|
||||||
"context"
|
"context"
|
||||||
|
"errors"
|
||||||
"fmt"
|
"fmt"
|
||||||
|
"io"
|
||||||
"os"
|
"os"
|
||||||
|
"os/exec"
|
||||||
|
"path/filepath"
|
||||||
|
"strings"
|
||||||
|
"syscall"
|
||||||
"time"
|
"time"
|
||||||
|
"unsafe"
|
||||||
|
|
||||||
|
"golang.org/x/sys/windows"
|
||||||
"golang.org/x/sys/windows/svc"
|
"golang.org/x/sys/windows/svc"
|
||||||
"golang.org/x/sys/windows/svc/mgr"
|
"golang.org/x/sys/windows/svc/mgr"
|
||||||
)
|
)
|
||||||
@@ -18,6 +29,25 @@ import (
|
|||||||
// Name is the Windows service name.
|
// Name is the Windows service name.
|
||||||
const Name = "gpu-turnstile"
|
const Name = "gpu-turnstile"
|
||||||
|
|
||||||
|
// virtualAccount is the per-service identity the service runs as. The SCM
|
||||||
|
// manages it: no password, automatic "log on as a service" right, gone
|
||||||
|
// when the service is removed.
|
||||||
|
const virtualAccount = `NT SERVICE\` + Name
|
||||||
|
|
||||||
|
// installDirs returns the canonical install (Program Files) and data
|
||||||
|
// (ProgramData) directories.
|
||||||
|
func installDirs() (install, data string) {
|
||||||
|
pf := os.Getenv("ProgramFiles")
|
||||||
|
if pf == "" {
|
||||||
|
pf = `C:\Program Files`
|
||||||
|
}
|
||||||
|
pd := os.Getenv("ProgramData")
|
||||||
|
if pd == "" {
|
||||||
|
pd = `C:\ProgramData`
|
||||||
|
}
|
||||||
|
return filepath.Join(pf, Name), filepath.Join(pd, Name)
|
||||||
|
}
|
||||||
|
|
||||||
// IsService reports whether the process is running as a Windows service.
|
// IsService reports whether the process is running as a Windows service.
|
||||||
func IsService() bool {
|
func IsService() bool {
|
||||||
isSvc, err := svc.IsWindowsService()
|
isSvc, err := svc.IsWindowsService()
|
||||||
@@ -64,34 +94,185 @@ func (h *handler) Execute(_ []string, requests <-chan svc.ChangeRequest, status
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
// Install registers gpu-turnstile as an auto-start Windows service whose
|
// Install registers gpu-turnstile as an auto-start Windows service running
|
||||||
// binPath loads the given config file. Recovery actions restart the
|
// as the NT SERVICE\gpu-turnstile virtual account, whose binPath loads the
|
||||||
// service after 5s on failure — this is also what brings up a staged
|
// given config file. With copyBin it first creates the canonical layout —
|
||||||
// update after the updater exits with a non-zero code.
|
// the binary is copied into %ProgramFiles%\gpu-turnstile and the config
|
||||||
func Install(configPath string) error {
|
// next to it (an existing config there is kept), %ProgramData%\gpu-turnstile
|
||||||
|
// is created for logs — and registers that copy; with copyBin=false the
|
||||||
|
// current executable location is registered as-is. Recovery actions restart
|
||||||
|
// the service after 5s on failure — this is also what brings up a staged
|
||||||
|
// update after the updater exits with a non-zero code. After registering,
|
||||||
|
// the virtual account is granted modify access to the install and data
|
||||||
|
// directories (self-updates rewrite the exe), read access to the
|
||||||
|
// config file if it lives elsewhere, and — when the config sets COMFY_DIR —
|
||||||
|
// recursive modify access to the managed ComfyUI's install tree, which may
|
||||||
|
// live inside a user profile the account otherwise cannot enter. The grants
|
||||||
|
// must come after CreateService: the virtual account's SID only exists once
|
||||||
|
// the service is registered.
|
||||||
|
//
|
||||||
|
// Re-running install on an existing service converges instead of failing:
|
||||||
|
// the service is stopped first if running (so the binary can be replaced),
|
||||||
|
// the installed binary is refreshed only when the content differs, the
|
||||||
|
// registration is updated only where it drifted, and the service is
|
||||||
|
// started again only if it was running before.
|
||||||
|
func Install(configPath string, copyBin bool, version string) error {
|
||||||
exe, err := os.Executable()
|
exe, err := os.Executable()
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return err
|
return err
|
||||||
}
|
}
|
||||||
|
if abs, absErr := filepath.Abs(exe); absErr == nil {
|
||||||
|
exe = abs
|
||||||
|
}
|
||||||
|
if configPath != "" {
|
||||||
|
if abs, absErr := filepath.Abs(configPath); absErr == nil {
|
||||||
|
configPath = abs
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
m, err := mgr.Connect()
|
m, err := mgr.Connect()
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return fmt.Errorf("connect to service manager (run as administrator): %w", err)
|
return fmt.Errorf("connect to service manager (run as administrator): %w", err)
|
||||||
}
|
}
|
||||||
defer m.Disconnect()
|
defer m.Disconnect()
|
||||||
|
|
||||||
|
// An existing service is converged, not an error. Stop it first so the
|
||||||
|
// binary copy can be replaced, and remember whether to start it again.
|
||||||
|
var s *mgr.Service
|
||||||
|
wasRunning := false
|
||||||
|
if existing, openErr := m.OpenService(Name); openErr == nil {
|
||||||
|
s = existing
|
||||||
|
defer s.Close()
|
||||||
|
if st, qErr := s.Query(); qErr == nil &&
|
||||||
|
(st.State == svc.Running || st.State == svc.StartPending) {
|
||||||
|
wasRunning = true
|
||||||
|
fmt.Println("stopping the running gpu-turnstile service")
|
||||||
|
if err := stopAndWait(s); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
installDir, dataDir := installDirs()
|
||||||
|
if copyBin {
|
||||||
|
targetCfg := filepath.Join(installDir, "gpu-turnstile.env")
|
||||||
|
if !strings.EqualFold(filepath.Dir(exe), installDir) {
|
||||||
|
if err := os.MkdirAll(installDir, 0o755); err != nil {
|
||||||
|
return fmt.Errorf("create %s: %w", installDir, err)
|
||||||
|
}
|
||||||
|
installedExe := filepath.Join(installDir, "gpu-turnstile.exe")
|
||||||
|
if same, _ := sameFileContent(exe, installedExe); !same {
|
||||||
|
fmt.Printf("installing %s\n", installedExe)
|
||||||
|
if err := copyFile(exe, installedExe); err != nil {
|
||||||
|
return fmt.Errorf("copy binary to %s: %w", installedExe, err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
exe = installedExe
|
||||||
|
if configPath != "" && !strings.EqualFold(configPath, targetCfg) {
|
||||||
|
if _, statErr := os.Stat(targetCfg); os.IsNotExist(statErr) {
|
||||||
|
copyFile(configPath, targetCfg) //nolint:errcheck // best effort
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
// A service has no console: without LOG_FILE the output vanishes, so
|
||||||
|
// the sample pre-wires it to ProgramData. Missing configs get a
|
||||||
|
// fresh sample; installer-written ones from older versions get new
|
||||||
|
// settings appended; anything without CFG_VER is invalid and gets
|
||||||
|
// replaced (backup kept as .bak).
|
||||||
|
logPath := filepath.Join(dataDir, "gpu-turnstile.log")
|
||||||
|
if err := syncEnvFile(targetCfg, version, logPath); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
configPath = targetCfg
|
||||||
|
}
|
||||||
binPath := fmt.Sprintf(`"%s" -config "%s"`, exe, configPath)
|
binPath := fmt.Sprintf(`"%s" -config "%s"`, exe, configPath)
|
||||||
s, err := m.CreateService(Name, binPath, mgr.Config{
|
|
||||||
|
if s == nil {
|
||||||
|
s, err = m.CreateService(Name, binPath, mgr.Config{
|
||||||
StartType: mgr.StartAutomatic,
|
StartType: mgr.StartAutomatic,
|
||||||
DisplayName: "gpu-turnstile",
|
DisplayName: "gpu-turnstile",
|
||||||
Description: "GPU arbitration proxy for Ollama and ComfyUI",
|
Description: "GPU arbitration proxy for Ollama and ComfyUI",
|
||||||
|
ServiceStartName: virtualAccount,
|
||||||
})
|
})
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return fmt.Errorf("create service: %w", err)
|
return fmt.Errorf("create service: %w", err)
|
||||||
}
|
}
|
||||||
defer s.Close()
|
defer s.Close()
|
||||||
|
|
||||||
restart := mgr.RecoveryAction{Type: mgr.ServiceRestart, Delay: 5 * time.Second}
|
if err := ensureRecovery(s); err != nil {
|
||||||
if err := s.SetRecoveryActions([]mgr.RecoveryAction{restart, restart, restart}, 24*60*60); err != nil {
|
s.Delete() // roll back so a retry starts clean
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
if err := grantAll(exe, configPath); err != nil {
|
||||||
|
s.Delete() // roll back so a retry starts clean
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
// Best effort: start now instead of waiting for the next boot. A
|
||||||
|
// missing config (no consumer URLs) fails the start; the service stays
|
||||||
|
// registered and can be started once the config exists.
|
||||||
|
fmt.Println("starting the gpu-turnstile service")
|
||||||
|
s.Start()
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// Existing service: update the registration only where it drifted.
|
||||||
|
cur, err := s.Config()
|
||||||
|
if err != nil {
|
||||||
|
return fmt.Errorf("query service config: %w", err)
|
||||||
|
}
|
||||||
|
const displayName = "gpu-turnstile"
|
||||||
|
const description = "GPU arbitration proxy for Ollama and ComfyUI"
|
||||||
|
if cur.BinaryPathName != binPath ||
|
||||||
|
cur.StartType != mgr.StartAutomatic ||
|
||||||
|
cur.ServiceStartName != virtualAccount ||
|
||||||
|
cur.DisplayName != displayName ||
|
||||||
|
cur.Description != description {
|
||||||
|
upd := cur
|
||||||
|
upd.BinaryPathName = binPath
|
||||||
|
upd.StartType = mgr.StartAutomatic
|
||||||
|
upd.ServiceStartName = virtualAccount
|
||||||
|
upd.DisplayName = displayName
|
||||||
|
upd.Description = description
|
||||||
|
if err := s.UpdateConfig(upd); err != nil {
|
||||||
|
return fmt.Errorf("update service config: %w", err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if err := ensureRecovery(s); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
// Idempotent: re-assert the virtual account's ACLs (granting an
|
||||||
|
// existing ACE is a no-op).
|
||||||
|
if err := grantAll(exe, configPath); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
if wasRunning {
|
||||||
|
fmt.Println("starting the gpu-turnstile service")
|
||||||
|
if err := s.Start(); err != nil {
|
||||||
|
return fmt.Errorf("start service: %w", err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// ensureRecovery makes sure the service restarts 5s after a failure — the
|
||||||
|
// mechanism that also brings up a staged update. The current actions are
|
||||||
|
// queried first so an up-to-date service is left untouched.
|
||||||
|
func ensureRecovery(s *mgr.Service) error {
|
||||||
|
want := mgr.RecoveryAction{Type: mgr.ServiceRestart, Delay: 5 * time.Second}
|
||||||
|
actions, err := s.RecoveryActions()
|
||||||
|
onFailure, flagErr := s.RecoveryActionsOnNonCrashFailures()
|
||||||
|
if err == nil && flagErr == nil && onFailure && len(actions) == 3 {
|
||||||
|
ok := true
|
||||||
|
for _, a := range actions {
|
||||||
|
if a.Type != want.Type || a.Delay != want.Delay {
|
||||||
|
ok = false
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if ok {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if err := s.SetRecoveryActions([]mgr.RecoveryAction{want, want, want}, 24*60*60); err != nil {
|
||||||
return fmt.Errorf("set recovery actions: %w", err)
|
return fmt.Errorf("set recovery actions: %w", err)
|
||||||
}
|
}
|
||||||
if err := s.SetRecoveryActionsOnNonCrashFailures(true); err != nil {
|
if err := s.SetRecoveryActionsOnNonCrashFailures(true); err != nil {
|
||||||
@@ -100,7 +281,108 @@ func Install(configPath string) error {
|
|||||||
return nil
|
return nil
|
||||||
}
|
}
|
||||||
|
|
||||||
// Remove stops (if running) and unregisters the service.
|
// stopAndWait stops the service and waits up to 30s for the stopped state.
|
||||||
|
func stopAndWait(s *mgr.Service) error {
|
||||||
|
if _, err := s.Control(svc.Stop); err != nil {
|
||||||
|
return fmt.Errorf("stop service: %w", err)
|
||||||
|
}
|
||||||
|
deadline := time.Now().Add(30 * time.Second)
|
||||||
|
for {
|
||||||
|
st, err := s.Query()
|
||||||
|
if err != nil {
|
||||||
|
return fmt.Errorf("query service: %w", err)
|
||||||
|
}
|
||||||
|
if st.State == svc.Stopped {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
if time.Now().After(deadline) {
|
||||||
|
return fmt.Errorf("service did not stop within 30s")
|
||||||
|
}
|
||||||
|
time.Sleep(300 * time.Millisecond)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// sameFileContent reports whether two files hold identical bytes. A missing
|
||||||
|
// destination is simply "different".
|
||||||
|
func sameFileContent(a, b string) (bool, error) {
|
||||||
|
ba, err := os.ReadFile(a)
|
||||||
|
if err != nil {
|
||||||
|
return false, err
|
||||||
|
}
|
||||||
|
bb, err := os.ReadFile(b)
|
||||||
|
if os.IsNotExist(err) {
|
||||||
|
return false, nil
|
||||||
|
}
|
||||||
|
if err != nil {
|
||||||
|
return false, err
|
||||||
|
}
|
||||||
|
return bytes.Equal(ba, bb), nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// copyFile copies src to dst (0755 on the new file).
|
||||||
|
func copyFile(src, dst string) error {
|
||||||
|
in, err := os.Open(src)
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
defer in.Close()
|
||||||
|
out, err := os.OpenFile(dst, os.O_CREATE|os.O_TRUNC|os.O_WRONLY, 0o755)
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
if _, err := io.Copy(out, in); err != nil {
|
||||||
|
out.Close()
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
return out.Close()
|
||||||
|
}
|
||||||
|
|
||||||
|
// grantAll gives the virtual account every ACL the service needs: modify
|
||||||
|
// on the install and ProgramData directories and the LOG_FILE directory
|
||||||
|
// (if configured elsewhere), read on a config file outside the install
|
||||||
|
// directory.
|
||||||
|
func grantAll(exe, configPath string) error {
|
||||||
|
_, dataDir := installDirs()
|
||||||
|
if err := os.MkdirAll(dataDir, 0o755); err != nil {
|
||||||
|
return fmt.Errorf("create %s: %w", dataDir, err)
|
||||||
|
}
|
||||||
|
if err := grantAccess(dataDir, "(OI)(CI)(M)"); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
exeDir := filepath.Dir(exe)
|
||||||
|
if err := grantAccess(exeDir, "(OI)(CI)(M)"); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
if configPath != "" && !strings.HasPrefix(strings.ToLower(configPath), strings.ToLower(exeDir)+`\`) {
|
||||||
|
if err := grantAccess(configPath, "(R)"); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if logFile := configuredValue(configPath, "LOG_FILE"); logFile != "" {
|
||||||
|
dir := filepath.Dir(logFile)
|
||||||
|
if err := os.MkdirAll(dir, 0o755); err == nil {
|
||||||
|
if err := grantAccess(dir, "(OI)(CI)(M)"); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
// A managed ComfyUI whose install lives somewhere the virtual account
|
||||||
|
// may not go (a user profile) needs an explicit grant — recursively,
|
||||||
|
// since ComfyUI also writes output/temp/user data next to its code. A
|
||||||
|
// missing directory is skipped: the startup warning covers it.
|
||||||
|
if comfyDir := configuredValue(configPath, "COMFY_DIR"); comfyDir != "" {
|
||||||
|
if _, err := os.Stat(comfyDir); err == nil {
|
||||||
|
if err := grantAccessTree(comfyDir, "(OI)(CI)(M)"); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// Remove stops (if running) and unregisters the service. The virtual
|
||||||
|
// account ceases to exist with it; the ACL grants on the install and log
|
||||||
|
// directories are left in place (harmless without the account).
|
||||||
func Remove() error {
|
func Remove() error {
|
||||||
m, err := mgr.Connect()
|
m, err := mgr.Connect()
|
||||||
if err != nil {
|
if err != nil {
|
||||||
@@ -118,3 +400,143 @@ func Remove() error {
|
|||||||
}
|
}
|
||||||
return nil
|
return nil
|
||||||
}
|
}
|
||||||
|
|
||||||
|
var procShellExecuteExW = windows.NewLazySystemDLL("shell32.dll").NewProc("ShellExecuteExW")
|
||||||
|
|
||||||
|
const seeMaskNoCloseProcess = 0x40
|
||||||
|
|
||||||
|
// shellExecuteInfo mirrors SHELLEXECUTEINFOW (64-bit layout).
|
||||||
|
type shellExecuteInfo struct {
|
||||||
|
cbSize uint32
|
||||||
|
fMask uint32
|
||||||
|
hwnd uintptr
|
||||||
|
lpVerb *uint16
|
||||||
|
lpFile *uint16
|
||||||
|
lpParameters *uint16
|
||||||
|
lpDirectory *uint16
|
||||||
|
nShow int32
|
||||||
|
_ int32
|
||||||
|
hInstApp uintptr
|
||||||
|
lpIDList unsafe.Pointer
|
||||||
|
lpClass *uint16
|
||||||
|
hkeyClass uintptr
|
||||||
|
dwHotKey uint32
|
||||||
|
_ uint32
|
||||||
|
hIcon uintptr
|
||||||
|
hProcess windows.Handle
|
||||||
|
}
|
||||||
|
|
||||||
|
// Elevated reports whether the current process token is UAC-elevated.
|
||||||
|
func Elevated() bool {
|
||||||
|
var token windows.Token
|
||||||
|
if err := windows.OpenProcessToken(windows.CurrentProcess(), windows.TOKEN_QUERY, &token); err != nil {
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
defer token.Close()
|
||||||
|
return token.IsElevated()
|
||||||
|
}
|
||||||
|
|
||||||
|
// RelaunchElevated re-runs the current executable elevated via the UAC
|
||||||
|
// "runas" verb with the given arguments, waits for the child, and returns
|
||||||
|
// its exit code. The child gets a fresh console window for its output.
|
||||||
|
func RelaunchElevated(args []string) (int, error) {
|
||||||
|
exe, err := os.Executable()
|
||||||
|
if err != nil {
|
||||||
|
return 0, err
|
||||||
|
}
|
||||||
|
quoted := make([]string, len(args))
|
||||||
|
for i, a := range args {
|
||||||
|
quoted[i] = syscall.EscapeArg(a)
|
||||||
|
}
|
||||||
|
cwd, _ := os.Getwd()
|
||||||
|
verb, _ := windows.UTF16PtrFromString("runas")
|
||||||
|
exeP, _ := windows.UTF16PtrFromString(exe)
|
||||||
|
params, _ := windows.UTF16PtrFromString(strings.Join(quoted, " "))
|
||||||
|
dir, _ := windows.UTF16PtrFromString(cwd)
|
||||||
|
info := shellExecuteInfo{
|
||||||
|
fMask: seeMaskNoCloseProcess,
|
||||||
|
lpVerb: verb,
|
||||||
|
lpFile: exeP,
|
||||||
|
lpParameters: params,
|
||||||
|
lpDirectory: dir,
|
||||||
|
nShow: windows.SW_NORMAL,
|
||||||
|
}
|
||||||
|
info.cbSize = uint32(unsafe.Sizeof(info))
|
||||||
|
r, _, callErr := procShellExecuteExW.Call(uintptr(unsafe.Pointer(&info)))
|
||||||
|
if r == 0 {
|
||||||
|
if errors.Is(callErr, syscall.Errno(1223)) { // ERROR_CANCELLED
|
||||||
|
return 0, ErrUserCancelled
|
||||||
|
}
|
||||||
|
return 0, fmt.Errorf("ShellExecuteEx: %w", callErr)
|
||||||
|
}
|
||||||
|
defer windows.CloseHandle(windows.Handle(info.hProcess))
|
||||||
|
windows.WaitForSingleObject(windows.Handle(info.hProcess), windows.INFINITE)
|
||||||
|
var code uint32
|
||||||
|
if err := windows.GetExitCodeProcess(windows.Handle(info.hProcess), &code); err != nil {
|
||||||
|
return 0, err
|
||||||
|
}
|
||||||
|
return int(code), nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// grantAccess gives the virtual account the icacls permission set (e.g.
|
||||||
|
// "(OI)(CI)(M)") on path.
|
||||||
|
func grantAccess(path, perms string) error {
|
||||||
|
return runIcacls(path, perms, false)
|
||||||
|
}
|
||||||
|
|
||||||
|
// grantAccessTree is grantAccess with /T: the ACE is applied to the
|
||||||
|
// existing tree, not just inherited by children created later. Needed when
|
||||||
|
// the tree already exists, e.g. a ComfyUI install in a user profile. On a
|
||||||
|
// large tree (a venv has tens of thousands of files) this takes minutes,
|
||||||
|
// so it says what it is doing instead of looking hung.
|
||||||
|
func grantAccessTree(path, perms string) error {
|
||||||
|
return runIcacls(path, perms, true)
|
||||||
|
}
|
||||||
|
|
||||||
|
func runIcacls(path, perms string, recursive bool) error {
|
||||||
|
args := []string{path, "/grant", virtualAccount + ":" + perms}
|
||||||
|
if recursive {
|
||||||
|
fmt.Printf("granting %s modify access to %s (large trees can take minutes)\n", virtualAccount, path)
|
||||||
|
args = append(args, "/T")
|
||||||
|
}
|
||||||
|
start := time.Now()
|
||||||
|
out, err := exec.Command("icacls", args...).CombinedOutput()
|
||||||
|
if err != nil {
|
||||||
|
return fmt.Errorf("grant %s access to %s: %w (%s)", virtualAccount, path, err, strings.TrimSpace(string(out)))
|
||||||
|
}
|
||||||
|
if recursive {
|
||||||
|
fmt.Printf("access granted in %s\n", time.Since(start).Round(time.Second))
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// RestartIfRunning restarts the service when it is installed and running
|
||||||
|
// (used after a forced update staged a new binary). Reports whether a
|
||||||
|
// restart happened. A service that is not installed or not running is not
|
||||||
|
// an error. Needs elevation.
|
||||||
|
func RestartIfRunning() (bool, error) {
|
||||||
|
m, err := mgr.Connect()
|
||||||
|
if err != nil {
|
||||||
|
return false, fmt.Errorf("connect to service manager (run as administrator): %w", err)
|
||||||
|
}
|
||||||
|
defer m.Disconnect()
|
||||||
|
s, err := m.OpenService(Name)
|
||||||
|
if err != nil {
|
||||||
|
return false, nil // not installed
|
||||||
|
}
|
||||||
|
defer s.Close()
|
||||||
|
st, err := s.Query()
|
||||||
|
if err != nil {
|
||||||
|
return false, fmt.Errorf("query service: %w", err)
|
||||||
|
}
|
||||||
|
if st.State != svc.Running && st.State != svc.StartPending {
|
||||||
|
return false, nil
|
||||||
|
}
|
||||||
|
if err := stopAndWait(s); err != nil {
|
||||||
|
return false, err
|
||||||
|
}
|
||||||
|
if err := s.Start(); err != nil {
|
||||||
|
return false, fmt.Errorf("start service: %w", err)
|
||||||
|
}
|
||||||
|
return true, nil
|
||||||
|
}
|
||||||
|
|||||||
@@ -0,0 +1,314 @@
|
|||||||
|
// Package supervise runs an upstream server (ComfyUI) as a managed child
|
||||||
|
// process: started on demand when a request needs it, stopped after an
|
||||||
|
// idle timeout so the GPU memory it holds is freed, and stopped with the
|
||||||
|
// parent. Crashes are logged; the next request respawns it.
|
||||||
|
package supervise
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"fmt"
|
||||||
|
"io"
|
||||||
|
"log/slog"
|
||||||
|
"net/url"
|
||||||
|
"os"
|
||||||
|
"os/exec"
|
||||||
|
"path/filepath"
|
||||||
|
"runtime"
|
||||||
|
"strings"
|
||||||
|
"sync"
|
||||||
|
"time"
|
||||||
|
)
|
||||||
|
|
||||||
|
// ComfyLayout returns the interpreter and script path of a standard ComfyUI
|
||||||
|
// venv install rooted at dir for the given GOOS: .venv\Scripts\python.exe
|
||||||
|
// on Windows, .venv/bin/python elsewhere. The script is main.py — either in
|
||||||
|
// a ComfyUI subdirectory or directly under dir, whichever exists (the
|
||||||
|
// subdirectory form wins ties and is the default when neither exists yet,
|
||||||
|
// so the caller's missing-file warning points at the documented layout).
|
||||||
|
func ComfyLayout(goos, dir string) (python, script string) {
|
||||||
|
if goos == "windows" {
|
||||||
|
python = filepath.Join(dir, ".venv", "Scripts", "python.exe")
|
||||||
|
} else {
|
||||||
|
python = filepath.Join(dir, ".venv", "bin", "python")
|
||||||
|
}
|
||||||
|
script = filepath.Join(dir, "ComfyUI", "main.py")
|
||||||
|
if _, err := os.Stat(script); err != nil {
|
||||||
|
if _, err := os.Stat(filepath.Join(dir, "main.py")); err == nil {
|
||||||
|
script = filepath.Join(dir, "main.py")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return python, script
|
||||||
|
}
|
||||||
|
|
||||||
|
// DefaultComfyCommand builds the launch command for the standard venv
|
||||||
|
// layout (see ComfyLayout): the script is passed relative to dir so dir
|
||||||
|
// stays the working directory, and --port is taken from comfyURL when the
|
||||||
|
// URL carries one.
|
||||||
|
func DefaultComfyCommand(goos, dir, comfyURL string) string {
|
||||||
|
python, script := ComfyLayout(goos, dir)
|
||||||
|
rel, err := filepath.Rel(dir, script)
|
||||||
|
if err != nil {
|
||||||
|
rel = script
|
||||||
|
}
|
||||||
|
cmd := `"` + python + `" ` + rel
|
||||||
|
if u, err := url.Parse(comfyURL); err == nil && u.Port() != "" {
|
||||||
|
cmd += " --port " + u.Port()
|
||||||
|
}
|
||||||
|
return cmd
|
||||||
|
}
|
||||||
|
|
||||||
|
// Process is one managed child process.
|
||||||
|
type Process struct {
|
||||||
|
name string
|
||||||
|
argv []string
|
||||||
|
dir string
|
||||||
|
probe func(context.Context) error
|
||||||
|
startTimeout time.Duration
|
||||||
|
log *slog.Logger
|
||||||
|
|
||||||
|
mu sync.Mutex
|
||||||
|
cmd *exec.Cmd
|
||||||
|
stopping bool
|
||||||
|
ready bool
|
||||||
|
external bool // someone else serves the port; not our process
|
||||||
|
lastActivity time.Time
|
||||||
|
}
|
||||||
|
|
||||||
|
// New parses cmdLine (double quotes group arguments containing spaces) and
|
||||||
|
// prepares a managed process. probe reports whether the server answers
|
||||||
|
// (e.g. the comfy client's Probe); startTimeout bounds WaitReady. dir is
|
||||||
|
// the child's working directory; empty inherits ours.
|
||||||
|
func New(name, cmdLine, dir string, probe func(context.Context) error, startTimeout time.Duration, log *slog.Logger) (*Process, error) {
|
||||||
|
argv, err := splitCommandLine(cmdLine)
|
||||||
|
if err != nil {
|
||||||
|
return nil, fmt.Errorf("%s command: %w", name, err)
|
||||||
|
}
|
||||||
|
if len(argv) == 0 {
|
||||||
|
return nil, fmt.Errorf("%s command is empty", name)
|
||||||
|
}
|
||||||
|
if log == nil {
|
||||||
|
log = slog.Default()
|
||||||
|
}
|
||||||
|
if startTimeout <= 0 {
|
||||||
|
startTimeout = 2 * time.Minute
|
||||||
|
}
|
||||||
|
return &Process{name: name, argv: argv, dir: dir, probe: probe, startTimeout: startTimeout, log: log}, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// splitCommandLine splits a command line on whitespace, treating
|
||||||
|
// double-quoted sections as one argument (quotes removed). Backslashes are
|
||||||
|
// literal — this matches Windows paths.
|
||||||
|
func splitCommandLine(s string) ([]string, error) {
|
||||||
|
var argv []string
|
||||||
|
var cur strings.Builder
|
||||||
|
inQuote := false
|
||||||
|
have := false
|
||||||
|
flush := func() {
|
||||||
|
if have {
|
||||||
|
argv = append(argv, cur.String())
|
||||||
|
cur.Reset()
|
||||||
|
have = false
|
||||||
|
}
|
||||||
|
}
|
||||||
|
for _, r := range s {
|
||||||
|
switch {
|
||||||
|
case r == '"':
|
||||||
|
inQuote = !inQuote
|
||||||
|
have = true
|
||||||
|
case (r == ' ' || r == '\t') && !inQuote:
|
||||||
|
flush()
|
||||||
|
default:
|
||||||
|
cur.WriteRune(r)
|
||||||
|
have = true
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if inQuote {
|
||||||
|
return nil, fmt.Errorf("unterminated quote in %q", s)
|
||||||
|
}
|
||||||
|
flush()
|
||||||
|
return argv, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// Running reports whether the child process is currently alive.
|
||||||
|
func (p *Process) Running() bool {
|
||||||
|
p.mu.Lock()
|
||||||
|
defer p.mu.Unlock()
|
||||||
|
return p.cmd != nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// Ready reports whether the server has answered a probe since its last
|
||||||
|
// (re)start. Health checks use it to tell "starting up" from "outage".
|
||||||
|
func (p *Process) Ready() bool {
|
||||||
|
p.mu.Lock()
|
||||||
|
defer p.mu.Unlock()
|
||||||
|
return p.ready
|
||||||
|
}
|
||||||
|
|
||||||
|
// MarkReady records that the server answered.
|
||||||
|
func (p *Process) MarkReady() {
|
||||||
|
p.mu.Lock()
|
||||||
|
p.ready = true
|
||||||
|
p.mu.Unlock()
|
||||||
|
}
|
||||||
|
|
||||||
|
// NoteActivity resets the idle clock; called for every request served.
|
||||||
|
func (p *Process) NoteActivity() {
|
||||||
|
p.mu.Lock()
|
||||||
|
p.lastActivity = time.Now()
|
||||||
|
p.mu.Unlock()
|
||||||
|
}
|
||||||
|
|
||||||
|
// EnsureRunning starts the child if it is not running. It returns as soon
|
||||||
|
// as the process is spawned; readiness is WaitReady's job (and the proxy's
|
||||||
|
// retry backoff bridges the gap for plain proxied requests). When the URL
|
||||||
|
// already answers — e.g. the ComfyUI desktop app grabbed the port — no
|
||||||
|
// child is spawned: the external server is used as-is, and the idle
|
||||||
|
// watcher never touches it (it only kills its own child).
|
||||||
|
func (p *Process) EnsureRunning() error {
|
||||||
|
p.mu.Lock()
|
||||||
|
defer p.mu.Unlock()
|
||||||
|
p.lastActivity = time.Now()
|
||||||
|
if p.cmd != nil {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
pctx, cancel := context.WithTimeout(context.Background(), 2*time.Second)
|
||||||
|
err := p.probe(pctx)
|
||||||
|
cancel()
|
||||||
|
if err == nil {
|
||||||
|
p.ready = true
|
||||||
|
if !p.external {
|
||||||
|
p.external = true
|
||||||
|
p.log.Info(p.name + " is already served externally; not spawning a managed instance")
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
p.external = false
|
||||||
|
cmd := exec.Command(p.argv[0], p.argv[1:]...)
|
||||||
|
cmd.Dir = p.dir
|
||||||
|
stdout, err := cmd.StdoutPipe()
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
stderr, err := cmd.StderrPipe()
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
if err := cmd.Start(); err != nil {
|
||||||
|
return fmt.Errorf("start %s: %w", p.name, err)
|
||||||
|
}
|
||||||
|
p.cmd = cmd
|
||||||
|
p.stopping = false
|
||||||
|
p.ready = false
|
||||||
|
go p.pipeLog(stdout)
|
||||||
|
go p.pipeLog(stderr)
|
||||||
|
go func() {
|
||||||
|
err := cmd.Wait()
|
||||||
|
p.mu.Lock()
|
||||||
|
p.cmd = nil
|
||||||
|
p.ready = false
|
||||||
|
intentional := p.stopping
|
||||||
|
p.mu.Unlock()
|
||||||
|
if intentional {
|
||||||
|
p.log.Info(p.name + " stopped")
|
||||||
|
} else {
|
||||||
|
p.log.Warn(p.name+" exited unexpectedly; the next request restarts it", "err", err)
|
||||||
|
}
|
||||||
|
}()
|
||||||
|
p.log.Info(p.name+" starting", "pid", cmd.Process.Pid, "cmd", strings.Join(p.argv, " "))
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// WaitReady blocks until the probe succeeds, ctx ends, or the start
|
||||||
|
// timeout passes.
|
||||||
|
func (p *Process) WaitReady(ctx context.Context) error {
|
||||||
|
ctx, cancel := context.WithTimeout(ctx, p.startTimeout)
|
||||||
|
defer cancel()
|
||||||
|
for {
|
||||||
|
pctx, pcancel := context.WithTimeout(ctx, 5*time.Second)
|
||||||
|
err := p.probe(pctx)
|
||||||
|
pcancel()
|
||||||
|
if err == nil {
|
||||||
|
p.NoteActivity()
|
||||||
|
p.MarkReady()
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
select {
|
||||||
|
case <-ctx.Done():
|
||||||
|
return fmt.Errorf("%s did not become ready: %w", p.name, ctx.Err())
|
||||||
|
case <-time.After(500 * time.Millisecond):
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Stop kills the child process (the whole tree on Windows) if running.
|
||||||
|
func (p *Process) Stop() {
|
||||||
|
p.mu.Lock()
|
||||||
|
cmd := p.cmd
|
||||||
|
if cmd == nil {
|
||||||
|
p.mu.Unlock()
|
||||||
|
return
|
||||||
|
}
|
||||||
|
p.stopping = true
|
||||||
|
p.ready = false
|
||||||
|
p.mu.Unlock()
|
||||||
|
stopTree(cmd)
|
||||||
|
}
|
||||||
|
|
||||||
|
// WatchIdle stops the child after idleTimeout without activity, but only
|
||||||
|
// when gpuIdle reports the GPU lock is free (no active or pending work).
|
||||||
|
// Returns when ctx ends.
|
||||||
|
func (p *Process) WatchIdle(ctx context.Context, idleTimeout time.Duration, gpuIdle func() bool) {
|
||||||
|
ticker := time.NewTicker(5 * time.Second)
|
||||||
|
defer ticker.Stop()
|
||||||
|
for {
|
||||||
|
select {
|
||||||
|
case <-ctx.Done():
|
||||||
|
return
|
||||||
|
case <-ticker.C:
|
||||||
|
}
|
||||||
|
p.mu.Lock()
|
||||||
|
idleFor := time.Since(p.lastActivity)
|
||||||
|
running := p.cmd != nil
|
||||||
|
p.mu.Unlock()
|
||||||
|
if running && idleFor > idleTimeout && gpuIdle() {
|
||||||
|
p.log.Info(p.name+" idle; stopping to free the GPU", "idle_for", idleFor.Round(time.Second))
|
||||||
|
p.Stop()
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// pipeLog forwards one child output stream to the log at INFO, line by
|
||||||
|
// line, prefixed with the process name.
|
||||||
|
func (p *Process) pipeLog(r io.Reader) {
|
||||||
|
buf := make([]byte, 4096)
|
||||||
|
var line string
|
||||||
|
for {
|
||||||
|
n, err := r.Read(buf)
|
||||||
|
line += string(buf[:n])
|
||||||
|
for {
|
||||||
|
i := strings.IndexByte(line, '\n')
|
||||||
|
if i < 0 {
|
||||||
|
break
|
||||||
|
}
|
||||||
|
p.log.Info(p.name + ": " + strings.TrimRight(line[:i], "\r"))
|
||||||
|
line = line[i+1:]
|
||||||
|
}
|
||||||
|
if err != nil {
|
||||||
|
if strings.TrimSpace(line) != "" {
|
||||||
|
p.log.Info(p.name + ": " + line)
|
||||||
|
}
|
||||||
|
return
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// stopTree kills cmd's process, including its children on Windows (python
|
||||||
|
// launchers tend to spawn some). The Wait goroutine reaps it.
|
||||||
|
func stopTree(cmd *exec.Cmd) {
|
||||||
|
if runtime.GOOS == "windows" {
|
||||||
|
exec.Command("taskkill", "/T", "/F", "/PID",
|
||||||
|
fmt.Sprint(cmd.Process.Pid)).Run() //nolint:errcheck // best effort
|
||||||
|
return
|
||||||
|
}
|
||||||
|
cmd.Process.Kill() //nolint:errcheck // best effort
|
||||||
|
}
|
||||||
@@ -0,0 +1,241 @@
|
|||||||
|
package supervise
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"errors"
|
||||||
|
"log/slog"
|
||||||
|
"net/http"
|
||||||
|
"net/http/httptest"
|
||||||
|
"os"
|
||||||
|
"path/filepath"
|
||||||
|
"strings"
|
||||||
|
"testing"
|
||||||
|
"time"
|
||||||
|
)
|
||||||
|
|
||||||
|
func TestSplitCommandLine(t *testing.T) {
|
||||||
|
cases := []struct {
|
||||||
|
in string
|
||||||
|
want []string
|
||||||
|
}{
|
||||||
|
{`python main.py --port 8188`, []string{"python", "main.py", "--port", "8188"}},
|
||||||
|
{`"C:\Program Files\py\python.exe" main.py`, []string{`C:\Program Files\py\python.exe`, "main.py"}},
|
||||||
|
{` spaced out `, []string{"spaced", "out"}},
|
||||||
|
{`a "b c" d`, []string{"a", "b c", "d"}},
|
||||||
|
{"", nil},
|
||||||
|
}
|
||||||
|
for _, c := range cases {
|
||||||
|
got, err := splitCommandLine(c.in)
|
||||||
|
if err != nil {
|
||||||
|
t.Errorf("splitCommandLine(%q): %v", c.in, err)
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
if len(got) != len(c.want) {
|
||||||
|
t.Errorf("splitCommandLine(%q) = %v, want %v", c.in, got, c.want)
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
for i := range got {
|
||||||
|
if got[i] != c.want[i] {
|
||||||
|
t.Errorf("splitCommandLine(%q) = %v, want %v", c.in, got, c.want)
|
||||||
|
break
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if _, err := splitCommandLine(`"unterminated`); err == nil {
|
||||||
|
t.Error("expected error for unterminated quote")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// TestHelperProcess is the child executed by the lifecycle tests: it just
|
||||||
|
// sleeps. The env marker is set only around the spawn, so in the normal
|
||||||
|
// test run this returns immediately.
|
||||||
|
func TestHelperProcess(t *testing.T) {
|
||||||
|
if os.Getenv("GO_HELPER_PROCESS") != "1" {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
time.Sleep(30 * time.Second)
|
||||||
|
os.Exit(0)
|
||||||
|
}
|
||||||
|
|
||||||
|
func newHelper(t *testing.T, name string) *Process {
|
||||||
|
t.Helper()
|
||||||
|
// The probe always fails: nothing external serves the port, so
|
||||||
|
// EnsureRunning spawns the helper child.
|
||||||
|
p, err := New(name, `"`+os.Args[0]+`" -test.run=TestHelperProcess`, "",
|
||||||
|
func(context.Context) error { return errors.New("nothing there") }, 5*time.Second, slog.Default())
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
return p
|
||||||
|
}
|
||||||
|
|
||||||
|
// startHelper spawns the child with the marker set; exec.Command inherits
|
||||||
|
// the environment at spawn time, so it can be unset right after.
|
||||||
|
func startHelper(t *testing.T, p *Process) {
|
||||||
|
t.Helper()
|
||||||
|
os.Setenv("GO_HELPER_PROCESS", "1")
|
||||||
|
defer os.Unsetenv("GO_HELPER_PROCESS")
|
||||||
|
if err := p.EnsureRunning(); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func waitStopped(t *testing.T, p *Process, timeout time.Duration) {
|
||||||
|
t.Helper()
|
||||||
|
deadline := time.Now().Add(timeout)
|
||||||
|
for p.Running() && time.Now().Before(deadline) {
|
||||||
|
time.Sleep(50 * time.Millisecond)
|
||||||
|
}
|
||||||
|
if p.Running() {
|
||||||
|
t.Fatal("process still running")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestEnsureRunningAndStop(t *testing.T) {
|
||||||
|
p := newHelper(t, "helper")
|
||||||
|
if p.Running() {
|
||||||
|
t.Fatal("Running before start")
|
||||||
|
}
|
||||||
|
startHelper(t, p)
|
||||||
|
if !p.Running() {
|
||||||
|
t.Fatal("not Running after EnsureRunning")
|
||||||
|
}
|
||||||
|
if err := p.EnsureRunning(); err != nil {
|
||||||
|
t.Fatal("second EnsureRunning must be a no-op")
|
||||||
|
}
|
||||||
|
p.Stop()
|
||||||
|
waitStopped(t, p, 5*time.Second)
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestEnsureRunningPrefersExternalServer(t *testing.T) {
|
||||||
|
// The port is already served (e.g. the ComfyUI desktop app): no child
|
||||||
|
// is spawned, the supervisor reports ready, and Stop is a no-op.
|
||||||
|
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||||
|
w.WriteHeader(http.StatusOK)
|
||||||
|
}))
|
||||||
|
defer srv.Close()
|
||||||
|
probe := func(ctx context.Context) error {
|
||||||
|
req, _ := http.NewRequestWithContext(ctx, http.MethodGet, srv.URL, nil)
|
||||||
|
resp, err := http.DefaultClient.Do(req)
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
resp.Body.Close()
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
p, err := New("external", `"`+os.Args[0]+`"`, "", probe, 5*time.Second, slog.Default())
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if err := p.EnsureRunning(); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if p.Running() {
|
||||||
|
t.Fatal("spawned a child even though the port is already served")
|
||||||
|
}
|
||||||
|
if !p.Ready() {
|
||||||
|
t.Fatal("external server should count as ready")
|
||||||
|
}
|
||||||
|
p.Stop() // must not touch the external server
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestWaitReady(t *testing.T) {
|
||||||
|
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||||
|
w.WriteHeader(http.StatusOK)
|
||||||
|
}))
|
||||||
|
defer srv.Close()
|
||||||
|
probe := func(ctx context.Context) error {
|
||||||
|
req, _ := http.NewRequestWithContext(ctx, http.MethodGet, srv.URL, nil)
|
||||||
|
resp, err := http.DefaultClient.Do(req)
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
resp.Body.Close()
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
p, err := New("ready", `"`+os.Args[0]+`"`, "", probe, 5*time.Second, slog.Default())
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if err := p.WaitReady(context.Background()); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
|
||||||
|
failing, err := New("failing", `"`+os.Args[0]+`"`, "",
|
||||||
|
func(context.Context) error { return errors.New("no") }, 500*time.Millisecond, slog.Default())
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if err := failing.WaitReady(context.Background()); err == nil {
|
||||||
|
t.Fatal("expected timeout error from WaitReady")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestWatchIdleStops(t *testing.T) {
|
||||||
|
p := newHelper(t, "idle")
|
||||||
|
startHelper(t, p)
|
||||||
|
ctx, cancel := context.WithCancel(context.Background())
|
||||||
|
defer cancel()
|
||||||
|
go p.WatchIdle(ctx, 200*time.Millisecond, func() bool { return true })
|
||||||
|
waitStopped(t, p, 10*time.Second)
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestWatchIdleRespectsBusyGPU(t *testing.T) {
|
||||||
|
p := newHelper(t, "busy")
|
||||||
|
startHelper(t, p)
|
||||||
|
defer p.Stop()
|
||||||
|
ctx, cancel := context.WithCancel(context.Background())
|
||||||
|
go p.WatchIdle(ctx, 100*time.Millisecond, func() bool { return false })
|
||||||
|
time.Sleep(600 * time.Millisecond)
|
||||||
|
cancel()
|
||||||
|
if !p.Running() {
|
||||||
|
t.Fatal("process was stopped while the GPU was busy")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestComfyLayoutAndDefaultCommand(t *testing.T) {
|
||||||
|
// Nested layout (ComfyUI/main.py under dir) wins.
|
||||||
|
dir := t.TempDir()
|
||||||
|
nested := filepath.Join(dir, "ComfyUI", "main.py")
|
||||||
|
if err := os.MkdirAll(filepath.Dir(nested), 0o755); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if err := os.WriteFile(nested, []byte("x"), 0o644); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
python, script := ComfyLayout("windows", dir)
|
||||||
|
if want := filepath.Join(dir, ".venv", "Scripts", "python.exe"); python != want {
|
||||||
|
t.Errorf("python = %s, want %s", python, want)
|
||||||
|
}
|
||||||
|
if script != nested {
|
||||||
|
t.Errorf("script = %s, want %s", script, nested)
|
||||||
|
}
|
||||||
|
cmd := DefaultComfyCommand("windows", dir, "http://127.0.0.1:8189")
|
||||||
|
want := `"` + filepath.Join(dir, ".venv", "Scripts", "python.exe") + `" ` + filepath.Join("ComfyUI", "main.py") + " --port 8189"
|
||||||
|
if cmd != want {
|
||||||
|
t.Errorf("cmd = %q, want %q", cmd, want)
|
||||||
|
}
|
||||||
|
|
||||||
|
// Flat layout (main.py directly under dir) is found too.
|
||||||
|
flat := t.TempDir()
|
||||||
|
if err := os.WriteFile(filepath.Join(flat, "main.py"), []byte("x"), 0o644); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if _, script := ComfyLayout("linux", flat); script != filepath.Join(flat, "main.py") {
|
||||||
|
t.Errorf("flat script = %s", script)
|
||||||
|
}
|
||||||
|
cmd = DefaultComfyCommand("linux", flat, "http://comfy.internal")
|
||||||
|
if strings.Contains(cmd, "--port") {
|
||||||
|
t.Errorf("cmd = %q, want no --port for a port-less URL", cmd)
|
||||||
|
}
|
||||||
|
if !strings.HasSuffix(cmd, `" main.py`) {
|
||||||
|
t.Errorf("cmd = %q, want quoted python + relative main.py", cmd)
|
||||||
|
}
|
||||||
|
|
||||||
|
// Neither exists yet: default to the documented nested form so the
|
||||||
|
// startup warning points there.
|
||||||
|
empty := t.TempDir()
|
||||||
|
if _, script := ComfyLayout("windows", empty); script != filepath.Join(empty, "ComfyUI", "main.py") {
|
||||||
|
t.Errorf("missing-layout script = %s", script)
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -26,11 +26,15 @@ import (
|
|||||||
// maxAssetSize bounds release asset downloads.
|
// maxAssetSize bounds release asset downloads.
|
||||||
const maxAssetSize = 512 << 20
|
const maxAssetSize = 512 << 20
|
||||||
|
|
||||||
// Updater checks one Gitea repository for newer releases.
|
// Updater checks one Gitea repository for releases.
|
||||||
type Updater struct {
|
type Updater struct {
|
||||||
Repo string // e.g. https://git.rambossek.at/PUBLIC/gpu-turnstile
|
Repo string // e.g. https://git.rambossek.at/PUBLIC/gpu-turnstile
|
||||||
Asset string // e.g. gpu-turnstile.exe
|
Asset string // e.g. gpu-turnstile.exe
|
||||||
Version string // current version, e.g. v0.1.2 ("dev" disables updates)
|
Version string // current binary version, e.g. v0.1.2 ("dev" cannot be compared)
|
||||||
|
// Desired is the APP_VER policy: "dev" disables updates, "stable" (or
|
||||||
|
// empty) tracks the latest release, anything else is an exact vX.Y.Z
|
||||||
|
// release tag to pin — staging it even when that means a downgrade.
|
||||||
|
Desired string
|
||||||
Log *slog.Logger
|
Log *slog.Logger
|
||||||
Client *http.Client
|
Client *http.Client
|
||||||
}
|
}
|
||||||
@@ -167,15 +171,20 @@ func CleanupOld(exePath string) {
|
|||||||
os.Remove(exePath + ".new")
|
os.Remove(exePath + ".new")
|
||||||
}
|
}
|
||||||
|
|
||||||
// Check performs a single update check. staged is true when a newer,
|
// Check performs a single update check. staged is true when a
|
||||||
// signature-verified binary has been swapped into place at exePath; the
|
// signature-verified binary has been swapped into place at exePath; the
|
||||||
// caller should then restart the process. A nil error with staged=false
|
// caller should then restart the process. A nil error with staged=false
|
||||||
// means "no action" (up to date, disabled, or dev build); a non-nil error
|
// means "no action" (up to date, APP_VER=dev, or no embedded public key);
|
||||||
// means the check failed and the running binary is untouched.
|
// a non-nil error means the check failed and the running binary is
|
||||||
|
// untouched.
|
||||||
func (u *Updater) Check(ctx context.Context, exePath string) (staged bool, err error) {
|
func (u *Updater) Check(ctx context.Context, exePath string) (staged bool, err error) {
|
||||||
log := u.logger()
|
log := u.logger()
|
||||||
if u.Version == "" || u.Version == "dev" {
|
desired := u.Desired
|
||||||
log.Debug("auto-update: dev build, skipping")
|
if desired == "" {
|
||||||
|
desired = "stable"
|
||||||
|
}
|
||||||
|
if desired == "dev" {
|
||||||
|
log.Debug("auto-update: APP_VER=dev, skipping")
|
||||||
return false, nil
|
return false, nil
|
||||||
}
|
}
|
||||||
if publicKeyPEM == "" {
|
if publicKeyPEM == "" {
|
||||||
@@ -187,14 +196,29 @@ func (u *Updater) Check(ctx context.Context, exePath string) (staged bool, err e
|
|||||||
return false, err
|
return false, err
|
||||||
}
|
}
|
||||||
|
|
||||||
body, err := u.get(ctx, api+"/releases/latest")
|
pinned := desired != "stable"
|
||||||
|
endpoint := api + "/releases/latest"
|
||||||
|
if pinned {
|
||||||
|
endpoint = api + "/releases/tags/" + desired
|
||||||
|
}
|
||||||
|
body, err := u.get(ctx, endpoint)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return false, fmt.Errorf("fetch latest release: %w", err)
|
return false, fmt.Errorf("fetch release: %w", err)
|
||||||
}
|
}
|
||||||
var rel release
|
var rel release
|
||||||
if err := json.Unmarshal(body, &rel); err != nil {
|
if err := json.Unmarshal(body, &rel); err != nil {
|
||||||
return false, fmt.Errorf("parse release: %w", err)
|
return false, fmt.Errorf("parse release: %w", err)
|
||||||
}
|
}
|
||||||
|
if pinned {
|
||||||
|
// Pin mode: any difference from the target tag means stage it —
|
||||||
|
// including downgrades and replacing a dev binary.
|
||||||
|
if u.Version == rel.TagName {
|
||||||
|
log.Debug("auto-update: already on pinned version", "version", u.Version)
|
||||||
|
return false, nil
|
||||||
|
}
|
||||||
|
} else if u.Version != "" && u.Version != "dev" {
|
||||||
|
// Stable mode: only strictly newer releases count; a dev binary
|
||||||
|
// cannot be compared and is always replaced by the latest release.
|
||||||
newer, err := newerVersion(u.Version, rel.TagName)
|
newer, err := newerVersion(u.Version, rel.TagName)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return false, err
|
return false, err
|
||||||
@@ -203,6 +227,7 @@ func (u *Updater) Check(ctx context.Context, exePath string) (staged bool, err e
|
|||||||
log.Debug("auto-update: up to date", "version", u.Version, "latest", rel.TagName)
|
log.Debug("auto-update: up to date", "version", u.Version, "latest", rel.TagName)
|
||||||
return false, nil
|
return false, nil
|
||||||
}
|
}
|
||||||
|
}
|
||||||
|
|
||||||
urls := make(map[string]string, len(rel.Assets))
|
urls := make(map[string]string, len(rel.Assets))
|
||||||
for _, a := range rel.Assets {
|
for _, a := range rel.Assets {
|
||||||
|
|||||||
@@ -43,7 +43,7 @@ func newFakeGitea(t *testing.T, tag string, assetContent []byte) *fakeGitea {
|
|||||||
sign := func() []byte { return ed25519.Sign(priv, f.asset) }
|
sign := func() []byte { return ed25519.Sign(priv, f.asset) }
|
||||||
|
|
||||||
mux := http.NewServeMux()
|
mux := http.NewServeMux()
|
||||||
mux.HandleFunc("/api/v1/repos/o/r/releases/latest", func(w http.ResponseWriter, r *http.Request) {
|
serveRelease := func(w http.ResponseWriter, r *http.Request) {
|
||||||
assets := []map[string]string{
|
assets := []map[string]string{
|
||||||
{"name": "gpu-turnstile.exe", "browser_download_url": f.srv.URL + "/dl/exe"},
|
{"name": "gpu-turnstile.exe", "browser_download_url": f.srv.URL + "/dl/exe"},
|
||||||
{"name": "gpu-turnstile.exe.sha256", "browser_download_url": f.srv.URL + "/dl/sha"},
|
{"name": "gpu-turnstile.exe.sha256", "browser_download_url": f.srv.URL + "/dl/sha"},
|
||||||
@@ -52,7 +52,9 @@ func newFakeGitea(t *testing.T, tag string, assetContent []byte) *fakeGitea {
|
|||||||
assets = append(assets, map[string]string{"name": "gpu-turnstile.exe.sig", "browser_download_url": f.srv.URL + "/dl/sig"})
|
assets = append(assets, map[string]string{"name": "gpu-turnstile.exe.sig", "browser_download_url": f.srv.URL + "/dl/sig"})
|
||||||
}
|
}
|
||||||
json.NewEncoder(w).Encode(map[string]any{"tag_name": f.tag, "assets": assets})
|
json.NewEncoder(w).Encode(map[string]any{"tag_name": f.tag, "assets": assets})
|
||||||
})
|
}
|
||||||
|
mux.HandleFunc("/api/v1/repos/o/r/releases/latest", serveRelease)
|
||||||
|
mux.HandleFunc("/api/v1/repos/o/r/releases/tags/"+f.tag, serveRelease)
|
||||||
mux.HandleFunc("/dl/exe", func(w http.ResponseWriter, r *http.Request) { w.Write(f.asset) })
|
mux.HandleFunc("/dl/exe", func(w http.ResponseWriter, r *http.Request) { w.Write(f.asset) })
|
||||||
mux.HandleFunc("/dl/sig", func(w http.ResponseWriter, r *http.Request) {
|
mux.HandleFunc("/dl/sig", func(w http.ResponseWriter, r *http.Request) {
|
||||||
sig := sign()
|
sig := sign()
|
||||||
@@ -73,6 +75,12 @@ func (f *fakeGitea) updater(version string) *Updater {
|
|||||||
return &Updater{Repo: f.srv.URL + "/o/r", Asset: "gpu-turnstile.exe", Version: version}
|
return &Updater{Repo: f.srv.URL + "/o/r", Asset: "gpu-turnstile.exe", Version: version}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func (f *fakeGitea) updaterDesired(version, desired string) *Updater {
|
||||||
|
u := f.updater(version)
|
||||||
|
u.Desired = desired
|
||||||
|
return u
|
||||||
|
}
|
||||||
|
|
||||||
func fakeExe(t *testing.T) string {
|
func fakeExe(t *testing.T) string {
|
||||||
t.Helper()
|
t.Helper()
|
||||||
exe := filepath.Join(t.TempDir(), "gpu-turnstile.exe")
|
exe := filepath.Join(t.TempDir(), "gpu-turnstile.exe")
|
||||||
@@ -153,12 +161,61 @@ func TestCheckSkipsWithoutPublicKey(t *testing.T) {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
func TestCheckSkipsDevBuild(t *testing.T) {
|
func TestCheckDevBuildGetsStable(t *testing.T) {
|
||||||
|
// A dev binary cannot be compared; APP_VER=stable replaces it with the
|
||||||
|
// latest release.
|
||||||
f := newFakeGitea(t, "v9.9.9", []byte("new-binary"))
|
f := newFakeGitea(t, "v9.9.9", []byte("new-binary"))
|
||||||
withPublicKey(t, f.pubPEM)
|
withPublicKey(t, f.pubPEM)
|
||||||
staged, err := f.updater("dev").Check(context.Background(), fakeExe(t))
|
exe := fakeExe(t)
|
||||||
|
staged, err := f.updater("dev").Check(context.Background(), exe)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if !staged {
|
||||||
|
t.Fatal("expected dev binary to be replaced by the latest release")
|
||||||
|
}
|
||||||
|
content, _ := os.ReadFile(exe)
|
||||||
|
if string(content) != "new-binary" {
|
||||||
|
t.Fatalf("exe content = %q", content)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestCheckDesiredDevDisables(t *testing.T) {
|
||||||
|
f := newFakeGitea(t, "v9.9.9", []byte("new-binary"))
|
||||||
|
withPublicKey(t, f.pubPEM)
|
||||||
|
for _, version := range []string{"dev", "v0.1.2"} {
|
||||||
|
staged, err := f.updaterDesired(version, "dev").Check(context.Background(), fakeExe(t))
|
||||||
if err != nil || staged {
|
if err != nil || staged {
|
||||||
t.Fatalf("staged=%v err=%v, want no action for dev build", staged, err)
|
t.Fatalf("version %s: staged=%v err=%v, want no action with APP_VER=dev", version, staged, err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestCheckPinned(t *testing.T) {
|
||||||
|
// Pin mode stages the exact tag — up or down — and skips when the
|
||||||
|
// binary already matches.
|
||||||
|
for _, version := range []string{"v0.1.2", "v9.9.9", "dev"} {
|
||||||
|
f := newFakeGitea(t, "v0.5.0", []byte("pinned-binary"))
|
||||||
|
withPublicKey(t, f.pubPEM)
|
||||||
|
exe := fakeExe(t)
|
||||||
|
staged, err := f.updaterDesired(version, "v0.5.0").Check(context.Background(), exe)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("version %s: %v", version, err)
|
||||||
|
}
|
||||||
|
if !staged {
|
||||||
|
t.Fatalf("version %s: expected pinned v0.5.0 to be staged", version)
|
||||||
|
}
|
||||||
|
content, _ := os.ReadFile(exe)
|
||||||
|
if string(content) != "pinned-binary" {
|
||||||
|
t.Fatalf("version %s: exe content = %q", version, content)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
f := newFakeGitea(t, "v0.5.0", []byte("pinned-binary"))
|
||||||
|
withPublicKey(t, f.pubPEM)
|
||||||
|
staged, err := f.updaterDesired("v0.5.0", "v0.5.0").Check(context.Background(), fakeExe(t))
|
||||||
|
if err != nil || staged {
|
||||||
|
t.Fatalf("staged=%v err=%v, want no action when already on the pinned version", staged, err)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
Reference in New Issue
Block a user