Game detection: foreign GPU holders take an external lock hold (GAME_PROCS, GPU_FOREIGN_VRAM_MB)

This commit is contained in:
mram
2026-09-21 17:16:56 +02:00
parent 14120bf4a4
commit e4bdc92ece
16 changed files with 736 additions and 25 deletions
+33 -6
View File
@@ -2,9 +2,10 @@
GPU arbitration proxy for Ollama + ComfyUI. One consumer GPU is shared by an
LLM server (Ollama) and an image generator (ComfyUI); gpu-turnstile sits in
front of both and guarantees the GPU is always in exactly one of three states:
`idle`, `llm` (N ≥ 1 Ollama requests in flight), or `image` (exactly one
ComfyUI job, Ollama models unloaded). See [SPEC.md](SPEC.md) for the full
front of both and guarantees the GPU is always in exactly one of four states:
`idle`, `llm` (N ≥ 1 Ollama requests in flight), `image` (exactly one
ComfyUI job, Ollama models unloaded), or `external` (a foreign process such
as a game holds the GPU). See [SPEC.md](SPEC.md) for the full
design.
gpu-turnstile listens on the ports the services normally use; the actual
@@ -32,8 +33,9 @@ Open WebUI / n8n ────► :8188 ───┘
Each consumer is enabled by setting its URL (`OLLAMA_URL`, `COMFY_URL`) and
disabled by leaving it empty — at least one is required. With only Ollama
the proxy is a pass-through (no image jobs can arrive); with only ComfyUI
the Ollama unload/warm steps are skipped. Future consumers (e.g. local game
detection) plug into the same lock the same way.
the Ollama unload/warm steps are skipped. A third, URL-less consumer —
detection of foreign GPU holders such as games — is enabled by `GAME_PROCS`
and/or `GPU_FOREIGN_VRAM_MB` (see below).
## Configuration
@@ -60,6 +62,10 @@ override file values. Invalid values fail at startup.
| `COMFY_DIR` | _(empty)_ | Working directory for `COMFY_CMD` |
| `COMFY_IDLE_TIMEOUT` | `5m` | Stop the managed ComfyUI after this long idle |
| `COMFY_START_TIMEOUT` | `2m` | Max wait for the managed ComfyUI to come up |
| `GAME_PROCS` | _(empty = disabled)_ | Process names (comma-separated); while any runs, the GPU counts as held: requests wait, Ollama unloads, managed ComfyUI stops |
| `GPU_FOREIGN_VRAM_MB` | `0` (disabled) | Also treat the GPU as held when a non-ignored process uses more VRAM than this (needs nvidia-smi) |
| `GPU_IGNORE_PROCS` | `ollama,ollama app,ollama_llama_server,python,pythonw` | Process names never counted as foreign GPU users |
| `GAME_POLL_INTERVAL` | `5s` | How often game/VRAM detection runs |
| `LOGLEVEL` | `warn` | `info` logs every request (colored arrows in text mode), `debug` adds lock transitions. `LOG_LEVEL` works as an alias |
| `LOG_FORMAT` | `text` | `json` for structured JSON logs |
| `LOG_FILE` | _(empty)_ | Append logs to this file instead of stderr |
@@ -81,7 +87,7 @@ override file values. Invalid values fail at startup.
## Observability
- `GET /healthz` (both listeners): `{"state":"idle|llm|image","llm_inflight":N,"image_pending":B}`
- `GET /healthz` (both listeners): `{"state":"idle|llm|image|external","llm_inflight":N,"image_pending":B}`
- `GET /metrics` (both listeners): Prometheus text format — `gpu_turnstile_state`,
`gpu_turnstile_llm_inflight`, `gpu_turnstile_image_pending`,
`gpu_turnstile_image_jobs_total`, `gpu_turnstile_lock_wait_seconds`
@@ -118,6 +124,27 @@ answers on the port, gpu-turnstile just uses it instead of spawning
instance already holds the port when you open the desktop app, the
desktop's server is the one that fails to bind.
## Game detection
Want to game on the same GPU without Ollama/ComfyUI squatting on the VRAM?
gpu-turnstile can watch for foreign GPU holders and, while one is active,
make LLM/image requests wait (or 503, per `LLM_BUSY_MODE`), unload Ollama's
models and stop the managed ComfyUI so the game gets the memory. Two
detection paths, each optional, polled every `GAME_POLL_INTERVAL` (5s):
```
GAME_PROCS=cyberpunk2077.exe,bg3.exe # the reliable way on Windows
GPU_FOREIGN_VRAM_MB=1024 # catch-all via nvidia-smi
```
`GAME_PROCS` matches running process names (case-insensitive, `.exe`
optional). `GPU_FOREIGN_VRAM_MB` asks nvidia-smi which processes hold GPU
memory and treats anything not in `GPU_IGNORE_PROCS` above the threshold as
foreign — handy as a catch-all, but note that under Windows' WDDM driver
graphics-only games may not show up in nvidia-smi's per-process list, so
name your games in `GAME_PROCS` there; on Linux both paths work. When the
game exits, requests resume automatically.
## Build and run
```sh