gpu-turnstile

16 Commits 1 Branch 9 Tags

3 Commits

Include renames

Author	SHA1	Message	Date
mram	bacb26772a	Add LLM busy modes: wait (hang) or reject with Retry-After LLM_BUSY_MODE=reject answers blocked LLM requests immediately with LLM_BUSY_STATUS (default 503, 429 works) and Retry-After, so routers like LiteLLM can cool down and retry instead of holding a hung connection. The default wait mode now also sends Retry-After when LLM_WAIT_TIMEOUT expires. Document the service account (LocalSystem default, NT SERVICE virtual-account hardening) and the Program Files / ProgramData install layout.	2026-09-20 21:43:03 +02:00
mram	db11da8307	Rename to gpu-turnstile; swap port roles The proxy now listens on the standard service ports (Ollama :11434, ComfyUI :8188) and the actual services move one port up (:11435, :8189). Metric prefix is now gpu_turnstile_.	2026-09-20 18:10:22 +02:00
mram	065c294a96	gpulock: GPU arbitration proxy for Ollama + ComfyUI Implements SPEC.md: two listeners, one writer-preferring two-mode lock, Ollama unload before image jobs, ComfyUI history polling + VRAM free, optional model warm-up, healthz/metrics endpoints, streaming-safe reverse proxies, Dockerfile and Gitea Actions CI.	2026-09-20 18:05:20 +02:00

Author

SHA1

Message

Date

mram

bacb26772a

Add LLM busy modes: wait (hang) or reject with Retry-After

LLM_BUSY_MODE=reject answers blocked LLM requests immediately with
LLM_BUSY_STATUS (default 503, 429 works) and Retry-After, so routers
like LiteLLM can cool down and retry instead of holding a hung
connection. The default wait mode now also sends Retry-After when
LLM_WAIT_TIMEOUT expires. Document the service account (LocalSystem
default, NT SERVICE virtual-account hardening) and the Program
Files / ProgramData install layout.

2026-09-20 21:43:03 +02:00

mram

db11da8307

Rename to gpu-turnstile; swap port roles

The proxy now listens on the standard service ports (Ollama :11434,
ComfyUI :8188) and the actual services move one port up (:11435, :8189).
Metric prefix is now gpu_turnstile_.

2026-09-20 18:10:22 +02:00

mram

065c294a96

gpulock: GPU arbitration proxy for Ollama + ComfyUI

Implements SPEC.md: two listeners, one writer-preferring two-mode lock,
Ollama unload before image jobs, ComfyUI history polling + VRAM free,
optional model warm-up, healthz/metrics endpoints, streaming-safe
reverse proxies, Dockerfile and Gitea Actions CI.

2026-09-20 18:05:20 +02:00