5 Commits
Author SHA1 Message Date
mram 14c2e30478 Clarify listener wording: Ollama-compatible clients, not Ollama-facing 2026-09-21 12:38:03 +02:00
mram 30a1f55aae Make consumers optional: a URL enables its mode, empty disables it
OLLAMA_URL and COMFY_URL no longer have defaults; each consumer
(listener, client, startup probe, lock participation) is enabled by
setting its URL and disabled by leaving it empty. At least one must be
set. Ollama-only mode is a pure pass-through; ComfyUI-only mode skips
the unload and warm-reload steps. /metrics is now served on both
listeners. This is the extension pattern for future consumers such as
local game detection.
2026-09-20 22:29:16 +02:00
mram bacb26772a Add LLM busy modes: wait (hang) or reject with Retry-After
LLM_BUSY_MODE=reject answers blocked LLM requests immediately with
LLM_BUSY_STATUS (default 503, 429 works) and Retry-After, so routers
like LiteLLM can cool down and retry instead of holding a hung
connection. The default wait mode now also sends Retry-After when
LLM_WAIT_TIMEOUT expires. Document the service account (LocalSystem
default, NT SERVICE virtual-account hardening) and the Program
Files / ProgramData install layout.
2026-09-20 21:43:03 +02:00
mram db11da8307 Rename to gpu-turnstile; swap port roles
The proxy now listens on the standard service ports (Ollama :11434,
ComfyUI :8188) and the actual services move one port up (:11435, :8189).
Metric prefix is now gpu_turnstile_.
2026-09-20 18:10:22 +02:00
mram 065c294a96 gpulock: GPU arbitration proxy for Ollama + ComfyUI
Implements SPEC.md: two listeners, one writer-preferring two-mode lock,
Ollama unload before image jobs, ComfyUI history polling + VRAM free,
optional model warm-up, healthz/metrics endpoints, streaming-safe
reverse proxies, Dockerfile and Gitea Actions CI.
2026-09-20 18:05:20 +02:00