Rename to gpu-turnstile; swap port roles
The proxy now listens on the standard service ports (Ollama :11434, ComfyUI :8188) and the actual services move one port up (:11435, :8189). Metric prefix is now gpu_turnstile_.
This commit is contained in:
@@ -1,15 +1,19 @@
|
||||
# gpulock
|
||||
# gpu-turnstile
|
||||
|
||||
GPU arbitration proxy for Ollama + ComfyUI. One consumer GPU is shared by an
|
||||
LLM server (Ollama) and an image generator (ComfyUI); gpulock sits in front of
|
||||
both and guarantees the GPU is always in exactly one of three states: `idle`,
|
||||
`llm` (N ≥ 1 Ollama requests in flight), or `image` (exactly one ComfyUI job,
|
||||
Ollama models unloaded). See [SPEC.md](SPEC.md) for the full design.
|
||||
LLM server (Ollama) and an image generator (ComfyUI); gpu-turnstile sits in
|
||||
front of both and guarantees the GPU is always in exactly one of three states:
|
||||
`idle`, `llm` (N ≥ 1 Ollama requests in flight), or `image` (exactly one
|
||||
ComfyUI job, Ollama models unloaded). See [SPEC.md](SPEC.md) for the full
|
||||
design.
|
||||
|
||||
gpu-turnstile listens on the ports the services normally use; the actual
|
||||
services run one port higher (Ollama on 11435, ComfyUI on 8189).
|
||||
|
||||
```
|
||||
LiteLLM / Open WebUI ──► :11435 ─┐ ┌─► Ollama :11434
|
||||
├── gpulock (1 lock) ──┤
|
||||
Open WebUI / n8n ────► :8189 ───┘ └─► ComfyUI :8188
|
||||
LiteLLM / Open WebUI ──► :11434 ─┐ ┌─► Ollama :11435
|
||||
├── gpu-turnstile (1 lock) ──┤
|
||||
Open WebUI / n8n ────► :8188 ───┘ └─► ComfyUI :8189
|
||||
```
|
||||
|
||||
- LLM endpoints (`/api/generate`, `/api/chat`, `/api/embed`, `/v1/*`) take the
|
||||
@@ -29,10 +33,10 @@ startup.
|
||||
|
||||
| Var | Default | Meaning |
|
||||
|---|---|---|
|
||||
| `LISTEN_OLLAMA` | `:11435` | Ollama-facing listener |
|
||||
| `LISTEN_COMFY` | `:8189` | ComfyUI-facing listener |
|
||||
| `OLLAMA_URL` | `http://127.0.0.1:11434` | Ollama upstream |
|
||||
| `COMFY_URL` | `http://127.0.0.1:8188` | ComfyUI upstream |
|
||||
| `LISTEN_OLLAMA` | `:11434` | Ollama-facing listener |
|
||||
| `LISTEN_COMFY` | `:8188` | ComfyUI-facing listener |
|
||||
| `OLLAMA_URL` | `http://127.0.0.1:11435` | Ollama upstream |
|
||||
| `COMFY_URL` | `http://127.0.0.1:8189` | ComfyUI upstream |
|
||||
| `UNLOAD_TIMEOUT` | `60s` | Wait for Ollama to unload before an image job |
|
||||
| `JOB_TIMEOUT` | `15m` | Wait for a ComfyUI job to finish |
|
||||
| `LLM_WAIT_TIMEOUT` | `10m` | Max lock wait for an LLM request before 503 |
|
||||
@@ -43,29 +47,29 @@ startup.
|
||||
## Observability
|
||||
|
||||
- `GET /healthz` (both listeners): `{"state":"idle|llm|image","llm_inflight":N,"image_pending":B}`
|
||||
- `GET /metrics` (Ollama listener): Prometheus text format — `gpulock_state`,
|
||||
`gpulock_llm_inflight`, `gpulock_image_pending`, `gpulock_image_jobs_total`,
|
||||
`gpulock_lock_wait_seconds` (histogram, `kind="llm|image"`),
|
||||
`gpulock_unload_seconds`.
|
||||
- `GET /metrics` (Ollama listener): Prometheus text format — `gpu_turnstile_state`,
|
||||
`gpu_turnstile_llm_inflight`, `gpu_turnstile_image_pending`,
|
||||
`gpu_turnstile_image_jobs_total`, `gpu_turnstile_lock_wait_seconds`
|
||||
(histogram, `kind="llm|image"`), `gpu_turnstile_unload_seconds`.
|
||||
|
||||
## Build and run
|
||||
|
||||
```sh
|
||||
go build ./cmd/gpulock
|
||||
./gpulock
|
||||
go build ./cmd/gpu-turnstile
|
||||
./gpu-turnstile
|
||||
```
|
||||
|
||||
```sh
|
||||
docker build -t gpulock .
|
||||
docker run --rm -p 11435:11435 -p 8189:8189 \
|
||||
-e OLLAMA_URL=http://<workstation-ip>:11434 \
|
||||
-e COMFY_URL=http://<workstation-ip>:8188 \
|
||||
gpulock
|
||||
docker build -t gpu-turnstile .
|
||||
docker run --rm -p 11434:11434 -p 8188:8188 \
|
||||
-e OLLAMA_URL=http://<workstation-ip>:11435 \
|
||||
-e COMFY_URL=http://<workstation-ip>:8189 \
|
||||
gpu-turnstile
|
||||
```
|
||||
|
||||
Releases are built by Gitea Actions (`.gitea/workflows/ci.yml`): pushes run
|
||||
`go vet` and `go test -race` and publish
|
||||
`git.rambossek.at/<owner>/gpulock:sha-<short>`; `main` additionally gets
|
||||
`git.rambossek.at/<owner>/gpu-turnstile:sha-<short>`; `main` additionally gets
|
||||
`:latest`, and a git tag `vX.Y.Z` produces the versioned image.
|
||||
|
||||
## Development
|
||||
@@ -78,7 +82,7 @@ go test -race ./...
|
||||
Stdlib only, Go 1.23+. Layout:
|
||||
|
||||
```
|
||||
cmd/gpulock/main.go wiring, config, listeners
|
||||
cmd/gpu-turnstile/main.go wiring, config, listeners
|
||||
internal/lock/ two-mode lock (LLM readers / image writer, FIFO)
|
||||
internal/ollama/ ps / unload / warm client
|
||||
internal/comfy/ history poll / free client
|
||||
|
||||
Reference in New Issue
Block a user