Game detection: foreign GPU holders take an external lock hold (GAME_PROCS, GPU_FOREIGN_VRAM_MB)
This commit is contained in:
@@ -2,9 +2,10 @@
|
||||
|
||||
GPU arbitration proxy for Ollama + ComfyUI. One consumer GPU is shared by an
|
||||
LLM server (Ollama) and an image generator (ComfyUI); gpu-turnstile sits in
|
||||
front of both and guarantees the GPU is always in exactly one of three states:
|
||||
`idle`, `llm` (N ≥ 1 Ollama requests in flight), or `image` (exactly one
|
||||
ComfyUI job, Ollama models unloaded). See [SPEC.md](SPEC.md) for the full
|
||||
front of both and guarantees the GPU is always in exactly one of four states:
|
||||
`idle`, `llm` (N ≥ 1 Ollama requests in flight), `image` (exactly one
|
||||
ComfyUI job, Ollama models unloaded), or `external` (a foreign process such
|
||||
as a game holds the GPU). See [SPEC.md](SPEC.md) for the full
|
||||
design.
|
||||
|
||||
gpu-turnstile listens on the ports the services normally use; the actual
|
||||
@@ -32,8 +33,9 @@ Open WebUI / n8n ────► :8188 ───┘
|
||||
Each consumer is enabled by setting its URL (`OLLAMA_URL`, `COMFY_URL`) and
|
||||
disabled by leaving it empty — at least one is required. With only Ollama
|
||||
the proxy is a pass-through (no image jobs can arrive); with only ComfyUI
|
||||
the Ollama unload/warm steps are skipped. Future consumers (e.g. local game
|
||||
detection) plug into the same lock the same way.
|
||||
the Ollama unload/warm steps are skipped. A third, URL-less consumer —
|
||||
detection of foreign GPU holders such as games — is enabled by `GAME_PROCS`
|
||||
and/or `GPU_FOREIGN_VRAM_MB` (see below).
|
||||
|
||||
## Configuration
|
||||
|
||||
@@ -60,6 +62,10 @@ override file values. Invalid values fail at startup.
|
||||
| `COMFY_DIR` | _(empty)_ | Working directory for `COMFY_CMD` |
|
||||
| `COMFY_IDLE_TIMEOUT` | `5m` | Stop the managed ComfyUI after this long idle |
|
||||
| `COMFY_START_TIMEOUT` | `2m` | Max wait for the managed ComfyUI to come up |
|
||||
| `GAME_PROCS` | _(empty = disabled)_ | Process names (comma-separated); while any runs, the GPU counts as held: requests wait, Ollama unloads, managed ComfyUI stops |
|
||||
| `GPU_FOREIGN_VRAM_MB` | `0` (disabled) | Also treat the GPU as held when a non-ignored process uses more VRAM than this (needs nvidia-smi) |
|
||||
| `GPU_IGNORE_PROCS` | `ollama,ollama app,ollama_llama_server,python,pythonw` | Process names never counted as foreign GPU users |
|
||||
| `GAME_POLL_INTERVAL` | `5s` | How often game/VRAM detection runs |
|
||||
| `LOGLEVEL` | `warn` | `info` logs every request (colored arrows in text mode), `debug` adds lock transitions. `LOG_LEVEL` works as an alias |
|
||||
| `LOG_FORMAT` | `text` | `json` for structured JSON logs |
|
||||
| `LOG_FILE` | _(empty)_ | Append logs to this file instead of stderr |
|
||||
@@ -81,7 +87,7 @@ override file values. Invalid values fail at startup.
|
||||
|
||||
## Observability
|
||||
|
||||
- `GET /healthz` (both listeners): `{"state":"idle|llm|image","llm_inflight":N,"image_pending":B}`
|
||||
- `GET /healthz` (both listeners): `{"state":"idle|llm|image|external","llm_inflight":N,"image_pending":B}`
|
||||
- `GET /metrics` (both listeners): Prometheus text format — `gpu_turnstile_state`,
|
||||
`gpu_turnstile_llm_inflight`, `gpu_turnstile_image_pending`,
|
||||
`gpu_turnstile_image_jobs_total`, `gpu_turnstile_lock_wait_seconds`
|
||||
@@ -118,6 +124,27 @@ answers on the port, gpu-turnstile just uses it instead of spawning
|
||||
instance already holds the port when you open the desktop app, the
|
||||
desktop's server is the one that fails to bind.
|
||||
|
||||
## Game detection
|
||||
|
||||
Want to game on the same GPU without Ollama/ComfyUI squatting on the VRAM?
|
||||
gpu-turnstile can watch for foreign GPU holders and, while one is active,
|
||||
make LLM/image requests wait (or 503, per `LLM_BUSY_MODE`), unload Ollama's
|
||||
models and stop the managed ComfyUI so the game gets the memory. Two
|
||||
detection paths, each optional, polled every `GAME_POLL_INTERVAL` (5s):
|
||||
|
||||
```
|
||||
GAME_PROCS=cyberpunk2077.exe,bg3.exe # the reliable way on Windows
|
||||
GPU_FOREIGN_VRAM_MB=1024 # catch-all via nvidia-smi
|
||||
```
|
||||
|
||||
`GAME_PROCS` matches running process names (case-insensitive, `.exe`
|
||||
optional). `GPU_FOREIGN_VRAM_MB` asks nvidia-smi which processes hold GPU
|
||||
memory and treats anything not in `GPU_IGNORE_PROCS` above the threshold as
|
||||
foreign — handy as a catch-all, but note that under Windows' WDDM driver
|
||||
graphics-only games may not show up in nvidia-smi's per-process list, so
|
||||
name your games in `GAME_PROCS` there; on Linux both paths work. When the
|
||||
game exits, requests resume automatically.
|
||||
|
||||
## Build and run
|
||||
|
||||
```sh
|
||||
|
||||
Reference in New Issue
Block a user