Rename to gpu-turnstile; swap port roles

The proxy now listens on the standard service ports (Ollama :11434,
ComfyUI :8188) and the actual services move one port up (:11435, :8189).
Metric prefix is now gpu_turnstile_.
This commit is contained in:
mram
2026-09-20 18:10:22 +02:00
parent 065c294a96
commit db11da8307
10 changed files with 113 additions and 105 deletions
+29 -25
View File
@@ -1,15 +1,19 @@
# gpulock
# gpu-turnstile
GPU arbitration proxy for Ollama + ComfyUI. One consumer GPU is shared by an
LLM server (Ollama) and an image generator (ComfyUI); gpulock sits in front of
both and guarantees the GPU is always in exactly one of three states: `idle`,
`llm` (N ≥ 1 Ollama requests in flight), or `image` (exactly one ComfyUI job,
Ollama models unloaded). See [SPEC.md](SPEC.md) for the full design.
LLM server (Ollama) and an image generator (ComfyUI); gpu-turnstile sits in
front of both and guarantees the GPU is always in exactly one of three states:
`idle`, `llm` (N ≥ 1 Ollama requests in flight), or `image` (exactly one
ComfyUI job, Ollama models unloaded). See [SPEC.md](SPEC.md) for the full
design.
gpu-turnstile listens on the ports the services normally use; the actual
services run one port higher (Ollama on 11435, ComfyUI on 8189).
```
LiteLLM / Open WebUI ──► :11435 ─┐ ┌─► Ollama :11434
├── gpulock (1 lock) ──┤
Open WebUI / n8n ────► :8189 ───┘ └─► ComfyUI :8188
LiteLLM / Open WebUI ──► :11434 ─┐ ┌─► Ollama :11435
├── gpu-turnstile (1 lock) ──┤
Open WebUI / n8n ────► :8188 ───┘ └─► ComfyUI :8189
```
- LLM endpoints (`/api/generate`, `/api/chat`, `/api/embed`, `/v1/*`) take the
@@ -29,10 +33,10 @@ startup.
| Var | Default | Meaning |
|---|---|---|
| `LISTEN_OLLAMA` | `:11435` | Ollama-facing listener |
| `LISTEN_COMFY` | `:8189` | ComfyUI-facing listener |
| `OLLAMA_URL` | `http://127.0.0.1:11434` | Ollama upstream |
| `COMFY_URL` | `http://127.0.0.1:8188` | ComfyUI upstream |
| `LISTEN_OLLAMA` | `:11434` | Ollama-facing listener |
| `LISTEN_COMFY` | `:8188` | ComfyUI-facing listener |
| `OLLAMA_URL` | `http://127.0.0.1:11435` | Ollama upstream |
| `COMFY_URL` | `http://127.0.0.1:8189` | ComfyUI upstream |
| `UNLOAD_TIMEOUT` | `60s` | Wait for Ollama to unload before an image job |
| `JOB_TIMEOUT` | `15m` | Wait for a ComfyUI job to finish |
| `LLM_WAIT_TIMEOUT` | `10m` | Max lock wait for an LLM request before 503 |
@@ -43,29 +47,29 @@ startup.
## Observability
- `GET /healthz` (both listeners): `{"state":"idle|llm|image","llm_inflight":N,"image_pending":B}`
- `GET /metrics` (Ollama listener): Prometheus text format — `gpulock_state`,
`gpulock_llm_inflight`, `gpulock_image_pending`, `gpulock_image_jobs_total`,
`gpulock_lock_wait_seconds` (histogram, `kind="llm|image"`),
`gpulock_unload_seconds`.
- `GET /metrics` (Ollama listener): Prometheus text format — `gpu_turnstile_state`,
`gpu_turnstile_llm_inflight`, `gpu_turnstile_image_pending`,
`gpu_turnstile_image_jobs_total`, `gpu_turnstile_lock_wait_seconds`
(histogram, `kind="llm|image"`), `gpu_turnstile_unload_seconds`.
## Build and run
```sh
go build ./cmd/gpulock
./gpulock
go build ./cmd/gpu-turnstile
./gpu-turnstile
```
```sh
docker build -t gpulock .
docker run --rm -p 11435:11435 -p 8189:8189 \
-e OLLAMA_URL=http://<workstation-ip>:11434 \
-e COMFY_URL=http://<workstation-ip>:8188 \
gpulock
docker build -t gpu-turnstile .
docker run --rm -p 11434:11434 -p 8188:8188 \
-e OLLAMA_URL=http://<workstation-ip>:11435 \
-e COMFY_URL=http://<workstation-ip>:8189 \
gpu-turnstile
```
Releases are built by Gitea Actions (`.gitea/workflows/ci.yml`): pushes run
`go vet` and `go test -race` and publish
`git.rambossek.at/<owner>/gpulock:sha-<short>`; `main` additionally gets
`git.rambossek.at/<owner>/gpu-turnstile:sha-<short>`; `main` additionally gets
`:latest`, and a git tag `vX.Y.Z` produces the versioned image.
## Development
@@ -78,7 +82,7 @@ go test -race ./...
Stdlib only, Go 1.23+. Layout:
```
cmd/gpulock/main.go wiring, config, listeners
cmd/gpu-turnstile/main.go wiring, config, listeners
internal/lock/ two-mode lock (LLM readers / image writer, FIFO)
internal/ollama/ ps / unload / warm client
internal/comfy/ history poll / free client