Rename to gpu-turnstile; swap port roles

The proxy now listens on the standard service ports (Ollama :11434,
ComfyUI :8188) and the actual services move one port up (:11435, :8189).
Metric prefix is now gpu_turnstile_.
This commit is contained in:
mram
2026-09-20 18:10:22 +02:00
parent 065c294a96
commit db11da8307
10 changed files with 113 additions and 105 deletions
+2 -2
View File
@@ -1,2 +1,2 @@
/gpulock /gpu-turnstile
/gpulock.exe /gpu-turnstile.exe
+4 -4
View File
@@ -4,10 +4,10 @@ COPY go.mod ./
COPY cmd ./cmd COPY cmd ./cmd
COPY internal ./internal COPY internal ./internal
ARG VERSION=dev ARG VERSION=dev
RUN CGO_ENABLED=0 go build -ldflags="-s -w -X main.version=${VERSION}" -o /gpulock ./cmd/gpulock RUN CGO_ENABLED=0 go build -ldflags="-s -w -X main.version=${VERSION}" -o /gpu-turnstile ./cmd/gpu-turnstile
FROM gcr.io/distroless/static:nonroot FROM gcr.io/distroless/static:nonroot
COPY --from=build /gpulock /gpulock COPY --from=build /gpu-turnstile /gpu-turnstile
EXPOSE 8189 11435 EXPOSE 8188 11434
USER nonroot USER nonroot
ENTRYPOINT ["/gpulock"] ENTRYPOINT ["/gpu-turnstile"]
+29 -25
View File
@@ -1,15 +1,19 @@
# gpulock # gpu-turnstile
GPU arbitration proxy for Ollama + ComfyUI. One consumer GPU is shared by an GPU arbitration proxy for Ollama + ComfyUI. One consumer GPU is shared by an
LLM server (Ollama) and an image generator (ComfyUI); gpulock sits in front of LLM server (Ollama) and an image generator (ComfyUI); gpu-turnstile sits in
both and guarantees the GPU is always in exactly one of three states: `idle`, front of both and guarantees the GPU is always in exactly one of three states:
`llm` (N ≥ 1 Ollama requests in flight), or `image` (exactly one ComfyUI job, `idle`, `llm` (N ≥ 1 Ollama requests in flight), or `image` (exactly one
Ollama models unloaded). See [SPEC.md](SPEC.md) for the full design. ComfyUI job, Ollama models unloaded). See [SPEC.md](SPEC.md) for the full
design.
gpu-turnstile listens on the ports the services normally use; the actual
services run one port higher (Ollama on 11435, ComfyUI on 8189).
``` ```
LiteLLM / Open WebUI ──► :11435 ─┐ ┌─► Ollama :11434 LiteLLM / Open WebUI ──► :11434 ─┐ ┌─► Ollama :11435
├── gpulock (1 lock) ──┤ ├── gpu-turnstile (1 lock) ──┤
Open WebUI / n8n ────► :8189 ───┘ └─► ComfyUI :8188 Open WebUI / n8n ────► :8188 ───┘ └─► ComfyUI :8189
``` ```
- LLM endpoints (`/api/generate`, `/api/chat`, `/api/embed`, `/v1/*`) take the - LLM endpoints (`/api/generate`, `/api/chat`, `/api/embed`, `/v1/*`) take the
@@ -29,10 +33,10 @@ startup.
| Var | Default | Meaning | | Var | Default | Meaning |
|---|---|---| |---|---|---|
| `LISTEN_OLLAMA` | `:11435` | Ollama-facing listener | | `LISTEN_OLLAMA` | `:11434` | Ollama-facing listener |
| `LISTEN_COMFY` | `:8189` | ComfyUI-facing listener | | `LISTEN_COMFY` | `:8188` | ComfyUI-facing listener |
| `OLLAMA_URL` | `http://127.0.0.1:11434` | Ollama upstream | | `OLLAMA_URL` | `http://127.0.0.1:11435` | Ollama upstream |
| `COMFY_URL` | `http://127.0.0.1:8188` | ComfyUI upstream | | `COMFY_URL` | `http://127.0.0.1:8189` | ComfyUI upstream |
| `UNLOAD_TIMEOUT` | `60s` | Wait for Ollama to unload before an image job | | `UNLOAD_TIMEOUT` | `60s` | Wait for Ollama to unload before an image job |
| `JOB_TIMEOUT` | `15m` | Wait for a ComfyUI job to finish | | `JOB_TIMEOUT` | `15m` | Wait for a ComfyUI job to finish |
| `LLM_WAIT_TIMEOUT` | `10m` | Max lock wait for an LLM request before 503 | | `LLM_WAIT_TIMEOUT` | `10m` | Max lock wait for an LLM request before 503 |
@@ -43,29 +47,29 @@ startup.
## Observability ## Observability
- `GET /healthz` (both listeners): `{"state":"idle|llm|image","llm_inflight":N,"image_pending":B}` - `GET /healthz` (both listeners): `{"state":"idle|llm|image","llm_inflight":N,"image_pending":B}`
- `GET /metrics` (Ollama listener): Prometheus text format — `gpulock_state`, - `GET /metrics` (Ollama listener): Prometheus text format — `gpu_turnstile_state`,
`gpulock_llm_inflight`, `gpulock_image_pending`, `gpulock_image_jobs_total`, `gpu_turnstile_llm_inflight`, `gpu_turnstile_image_pending`,
`gpulock_lock_wait_seconds` (histogram, `kind="llm|image"`), `gpu_turnstile_image_jobs_total`, `gpu_turnstile_lock_wait_seconds`
`gpulock_unload_seconds`. (histogram, `kind="llm|image"`), `gpu_turnstile_unload_seconds`.
## Build and run ## Build and run
```sh ```sh
go build ./cmd/gpulock go build ./cmd/gpu-turnstile
./gpulock ./gpu-turnstile
``` ```
```sh ```sh
docker build -t gpulock . docker build -t gpu-turnstile .
docker run --rm -p 11435:11435 -p 8189:8189 \ docker run --rm -p 11434:11434 -p 8188:8188 \
-e OLLAMA_URL=http://<workstation-ip>:11434 \ -e OLLAMA_URL=http://<workstation-ip>:11435 \
-e COMFY_URL=http://<workstation-ip>:8188 \ -e COMFY_URL=http://<workstation-ip>:8189 \
gpulock gpu-turnstile
``` ```
Releases are built by Gitea Actions (`.gitea/workflows/ci.yml`): pushes run Releases are built by Gitea Actions (`.gitea/workflows/ci.yml`): pushes run
`go vet` and `go test -race` and publish `go vet` and `go test -race` and publish
`git.rambossek.at/<owner>/gpulock:sha-<short>`; `main` additionally gets `git.rambossek.at/<owner>/gpu-turnstile:sha-<short>`; `main` additionally gets
`:latest`, and a git tag `vX.Y.Z` produces the versioned image. `:latest`, and a git tag `vX.Y.Z` produces the versioned image.
## Development ## Development
@@ -78,7 +82,7 @@ go test -race ./...
Stdlib only, Go 1.23+. Layout: Stdlib only, Go 1.23+. Layout:
``` ```
cmd/gpulock/main.go wiring, config, listeners cmd/gpu-turnstile/main.go wiring, config, listeners
internal/lock/ two-mode lock (LLM readers / image writer, FIFO) internal/lock/ two-mode lock (LLM readers / image writer, FIFO)
internal/ollama/ ps / unload / warm client internal/ollama/ ps / unload / warm client
internal/comfy/ history poll / free client internal/comfy/ history poll / free client
+35 -32
View File
@@ -1,4 +1,4 @@
# gpulock — GPU arbitration proxy for Ollama + ComfyUI # gpu-turnstile — GPU arbitration proxy for Ollama + ComfyUI
## Problem ## Problem
@@ -16,8 +16,9 @@ at any moment the GPU is in exactly one of three states:
- `llm` — N ≥ 1 Ollama inference requests in flight (concurrency allowed) - `llm` — N ≥ 1 Ollama inference requests in flight (concurrency allowed)
- `image` — exactly one ComfyUI job in flight, Ollama models unloaded - `image` — exactly one ComfyUI job in flight, Ollama models unloaded
Clients (LiteLLM, Open WebUI, n8n) point at gpulock instead of at the services. Clients (LiteLLM, Open WebUI, n8n) point at gpu-turnstile instead of at the
gpulock is transparent for everything that does not touch the GPU. services. gpu-turnstile is transparent for everything that does not touch the
GPU.
## Non-goals ## Non-goals
@@ -30,15 +31,16 @@ gpulock is transparent for everything that does not touch the GPU.
## Architecture ## Architecture
``` ```
LiteLLM / Open WebUI ──► :11435 ─┐ ┌─► Ollama :11434 LiteLLM / Open WebUI ──► :11434 ─┐ ┌─► Ollama :11435
├── gpulock (1 lock) ──┤ ├── gpu-turnstile (1 lock) ──┤
Open WebUI / n8n ────► :8189 ───┘ └─► ComfyUI :8188 Open WebUI / n8n ────► :8188 ───┘ └─► ComfyUI :8189
``` ```
Two listeners, one process, one lock. Each listener is an gpu-turnstile listens on the ports the services normally use; the actual
`httputil.ReverseProxy` to its upstream. Websocket upgrades (ComfyUI `/ws`) services run one port higher. Two listeners, one process, one lock. Each
and streaming bodies (Ollama NDJSON / SSE) must pass through unbuffered listener is an `httputil.ReverseProxy` to its upstream. Websocket upgrades
(`FlushInterval = -1`). (ComfyUI `/ws`) and streaming bodies (Ollama NDJSON / SSE) must pass through
unbuffered (`FlushInterval = -1`).
### Lock semantics ### Lock semantics
@@ -58,7 +60,7 @@ are LLM requests and the single "writer" is an image job):
### Endpoint classification ### Endpoint classification
Ollama listener (`:11435``OLLAMA_URL`): Ollama listener (`:11434``OLLAMA_URL`):
| Path | Handling | | Path | Handling |
|---|---| |---|---|
@@ -66,7 +68,7 @@ Ollama listener (`:11435` → `OLLAMA_URL`):
| `POST /v1/chat/completions`, `/v1/completions`, `/v1/embeddings` | LLM lock | | `POST /v1/chat/completions`, `/v1/completions`, `/v1/embeddings` | LLM lock |
| everything else (`/api/tags`, `/api/ps`, `/api/show`, `/api/version`, `/v1/models`, `/api/pull`, …) | pass-through, no lock | | everything else (`/api/tags`, `/api/ps`, `/api/show`, `/api/version`, `/v1/models`, `/api/pull`, …) | pass-through, no lock |
ComfyUI listener (`:8189``COMFY_URL`): ComfyUI listener (`:8188``COMFY_URL`):
| Path | Handling | | Path | Handling |
|---|---| |---|---|
@@ -100,10 +102,10 @@ load time. Off by default.
| Var | Default | Meaning | | Var | Default | Meaning |
|---|---|---| |---|---|---|
| `LISTEN_OLLAMA` | `:11435` | Ollama-facing listener | | `LISTEN_OLLAMA` | `:11434` | Ollama-facing listener |
| `LISTEN_COMFY` | `:8189` | ComfyUI-facing listener | | `LISTEN_COMFY` | `:8188` | ComfyUI-facing listener |
| `OLLAMA_URL` | `http://127.0.0.1:11434` | upstream | | `OLLAMA_URL` | `http://127.0.0.1:11435` | upstream |
| `COMFY_URL` | `http://127.0.0.1:8188` | upstream | | `COMFY_URL` | `http://127.0.0.1:8189` | upstream |
| `UNLOAD_TIMEOUT` | `60s` | wait for Ollama to unload | | `UNLOAD_TIMEOUT` | `60s` | wait for Ollama to unload |
| `JOB_TIMEOUT` | `15m` | wait for ComfyUI job | | `JOB_TIMEOUT` | `15m` | wait for ComfyUI job |
| `LLM_WAIT_TIMEOUT` | `10m` | max time an LLM request waits for the lock before 503 | | `LLM_WAIT_TIMEOUT` | `10m` | max time an LLM request waits for the lock before 503 |
@@ -119,9 +121,9 @@ start (`/api/version`, `/system_stats`); failure is logged, not fatal.
`{"state":"idle|llm|image","llm_inflight":N,"image_pending":B}`. `{"state":"idle|llm|image","llm_inflight":N,"image_pending":B}`.
- `GET /metrics` on the Ollama listener: Prometheus text format, no external - `GET /metrics` on the Ollama listener: Prometheus text format, no external
dependency needed: dependency needed:
`gpulock_state{state="…"} 1`, `gpulock_llm_inflight`, `gpu_turnstile_state{state="…"} 1`, `gpu_turnstile_llm_inflight`,
`gpulock_image_jobs_total`, `gpulock_lock_wait_seconds` (histogram, label `gpu_turnstile_image_jobs_total`, `gpu_turnstile_lock_wait_seconds`
`kind="llm|image"`), `gpulock_unload_seconds`. (histogram, label `kind="llm|image"`), `gpu_turnstile_unload_seconds`.
- Structured logs (`log/slog`, JSON when `LOG_FORMAT=json`), one line per - Structured logs (`log/slog`, JSON when `LOG_FORMAT=json`), one line per
state transition and per image job phase with `prompt_id`. state transition and per image job phase with `prompt_id`.
@@ -143,8 +145,8 @@ start (`/api/version`, `/system_stats`); failure is logged, not fatal.
## Repository layout ## Repository layout
``` ```
gpulock/ gpu-turnstile/
cmd/gpulock/main.go # wiring, config, listeners cmd/gpu-turnstile/main.go # wiring, config, listeners
internal/lock/lock.go # two-mode lock + tests internal/lock/lock.go # two-mode lock + tests
internal/ollama/client.go # ps / unload / warm internal/ollama/client.go # ps / unload / warm
internal/comfy/client.go # history poll / free internal/comfy/client.go # history poll / free
@@ -176,30 +178,31 @@ are new.
- Go 1.23+, stdlib only. `CGO_ENABLED=0`, `-ldflags="-s -w"`, version from - Go 1.23+, stdlib only. `CGO_ENABLED=0`, `-ldflags="-s -w"`, version from
`git describe` injected via `-X main.version=`. `git describe` injected via `-X main.version=`.
- Dockerfile: multi-stage, final image `gcr.io/distroless/static` (or - Dockerfile: multi-stage, final image `gcr.io/distroless/static` (or
`scratch`), non-root user, `EXPOSE 8189 11435`, `ENTRYPOINT ["/gpulock"]`. `scratch`), non-root user, `EXPOSE 8188 11434`,
`ENTRYPOINT ["/gpu-turnstile"]`.
- `.gitea/workflows/ci.yml` (Gitea Actions): - `.gitea/workflows/ci.yml` (Gitea Actions):
1. on push and tag: `go vet`, `go test -race ./...`, `golangci-lint` if 1. on push and tag: `go vet`, `go test -race ./...`, `golangci-lint` if
available in the runner image available in the runner image
2. build image with buildx, tags `:sha-<short>` and `:latest` on main, 2. build image with buildx, tags `:sha-<short>` and `:latest` on main,
`:<tag>` on tags `:<tag>` on tags
3. push to the Gitea registry `git.rambossek.at/<owner>/gpulock` using the 3. push to the Gitea registry `git.rambossek.at/<owner>/gpu-turnstile`
workflow token (`${{ secrets.GITEA_TOKEN }}` / `gitea.actor`) using the workflow token (`${{ secrets.GITEA_TOKEN }}` / `gitea.actor`)
- Release: a git tag `vX.Y.Z` produces the versioned image; the Open WebUI - Release: a git tag `vX.Y.Z` produces the versioned image; the Open WebUI
compose pins that tag. compose pins that tag.
## Deployment (target) ## Deployment (target)
```yaml ```yaml
gpulock: gpu-turnstile:
image: git.rambossek.at/<owner>/gpulock:v0.1.0 image: git.rambossek.at/<owner>/gpu-turnstile:v0.1.0
environment: environment:
OLLAMA_URL: http://<workstation-ip>:11434 OLLAMA_URL: http://<workstation-ip>:11435
COMFY_URL: http://<workstation-ip>:8188 COMFY_URL: http://<workstation-ip>:8189
networks: [internal] networks: [internal]
``` ```
LiteLLM `api_base` → `http://gpulock:11435`; Open WebUI LiteLLM `api_base` → `http://gpu-turnstile:11434`; Open WebUI
`COMFYUI_BASE_URL` → `http://gpulock:8189`. Nothing else talks to the `COMFYUI_BASE_URL` → `http://gpu-turnstile:8188`. Nothing else talks to the
workstation directly. workstation directly.
## Open questions ## Open questions
@@ -207,5 +210,5 @@ workstation directly.
- Should embedding requests (`/api/embed`, `/v1/embeddings`) count as LLM - Should embedding requests (`/api/embed`, `/v1/embeddings`) count as LLM
traffic for the lock? They do in this spec (they hold VRAM); reconsider if traffic for the lock? They do in this spec (they hold VRAM); reconsider if
RAG indexing starves image jobs for too long. RAG indexing starves image jobs for too long.
- Whether to add a `POST /gpulock/release` admin endpoint to force-reset the - Whether to add a `POST /gpu-turnstile/release` admin endpoint to force-reset
lock without restarting. Cheap to add; decide once it's been stuck once. the lock without restarting. Cheap to add; decide once it's been stuck once.
@@ -1,4 +1,4 @@
// gpulock is a GPU arbitration proxy that sits in front of Ollama and // gpu-turnstile is a GPU arbitration proxy that sits in front of Ollama and
// ComfyUI and guarantees only one of them uses the GPU at a time. // ComfyUI and guarantees only one of them uses the GPU at a time.
package main package main
@@ -52,10 +52,10 @@ func envDuration(getenv func(string) string, name string, dst *time.Duration) er
func loadConfig(getenv func(string) string) (config, error) { func loadConfig(getenv func(string) string) (config, error) {
cfg := config{ cfg := config{
listenOllama: ":11435", listenOllama: ":11434",
listenComfy: ":8189", listenComfy: ":8188",
ollamaURL: "http://127.0.0.1:11434", ollamaURL: "http://127.0.0.1:11435",
comfyURL: "http://127.0.0.1:8188", comfyURL: "http://127.0.0.1:8189",
unloadTimeout: time.Minute, unloadTimeout: time.Minute,
jobTimeout: 15 * time.Minute, jobTimeout: 15 * time.Minute,
llmWaitTimeout: 10 * time.Minute, llmWaitTimeout: 10 * time.Minute,
@@ -107,7 +107,7 @@ func loadConfig(getenv func(string) string) (config, error) {
func main() { func main() {
cfg, err := loadConfig(os.Getenv) cfg, err := loadConfig(os.Getenv)
if err != nil { if err != nil {
fmt.Fprintf(os.Stderr, "gpulock: %v\n", err) fmt.Fprintf(os.Stderr, "gpu-turnstile: %v\n", err)
os.Exit(1) os.Exit(1)
} }
@@ -119,7 +119,7 @@ func main() {
log := slog.New(handler) log := slog.New(handler)
slog.SetDefault(log) slog.SetDefault(log)
log.Info("starting gpulock", log.Info("starting gpu-turnstile",
"version", version, "version", version,
"listen_ollama", cfg.listenOllama, "listen_ollama", cfg.listenOllama,
"listen_comfy", cfg.listenComfy, "listen_comfy", cfg.listenComfy,
+2 -2
View File
@@ -1,5 +1,5 @@
// Package comfy is a minimal client for the ComfyUI endpoints gpulock needs // Package comfy is a minimal client for the ComfyUI endpoints gpu-turnstile
// after a prompt has been accepted: polling /history and freeing VRAM. // needs after a prompt has been accepted: polling /history and freeing VRAM.
package comfy package comfy
import ( import (
+21 -21
View File
@@ -1,4 +1,4 @@
// Package metrics provides the Prometheus text exposition for gpulock // Package metrics provides the Prometheus text exposition for gpu-turnstile
// without any external dependencies. // without any external dependencies.
package metrics package metrics
@@ -51,7 +51,7 @@ func trimLabel(label string) string {
return label return label
} }
// Metrics holds all gpulock metric values. // Metrics holds all gpu-turnstile metric values.
type Metrics struct { type Metrics struct {
llmWait *histogram llmWait *histogram
imageWait *histogram imageWait *histogram
@@ -86,36 +86,36 @@ func (m *Metrics) IncImageJobs() { m.imageJobs.Add(1) }
// Render writes the Prometheus text exposition for the given lock snapshot. // Render writes the Prometheus text exposition for the given lock snapshot.
func (m *Metrics) Render(w io.Writer, state string, llmInflight int, imagePending bool) { func (m *Metrics) Render(w io.Writer, state string, llmInflight int, imagePending bool) {
fmt.Fprint(w, `# HELP gpulock_state Current GPU state (1 for the active state). fmt.Fprint(w, `# HELP gpu_turnstile_state Current GPU state (1 for the active state).
# TYPE gpulock_state gauge # TYPE gpu_turnstile_state gauge
`) `)
for _, s := range []string{"idle", "llm", "image"} { for _, s := range []string{"idle", "llm", "image"} {
v := 0 v := 0
if s == state { if s == state {
v = 1 v = 1
} }
fmt.Fprintf(w, "gpulock_state{state=%q} %d\n", s, v) fmt.Fprintf(w, "gpu_turnstile_state{state=%q} %d\n", s, v)
} }
pending := 0 pending := 0
if imagePending { if imagePending {
pending = 1 pending = 1
} }
fmt.Fprintf(w, `# HELP gpulock_llm_inflight LLM requests currently in flight. fmt.Fprintf(w, `# HELP gpu_turnstile_llm_inflight LLM requests currently in flight.
# TYPE gpulock_llm_inflight gauge # TYPE gpu_turnstile_llm_inflight gauge
gpulock_llm_inflight %d gpu_turnstile_llm_inflight %d
# HELP gpulock_image_pending Whether an image job is active or waiting. # HELP gpu_turnstile_image_pending Whether an image job is active or waiting.
# TYPE gpulock_image_pending gauge # TYPE gpu_turnstile_image_pending gauge
gpulock_image_pending %d gpu_turnstile_image_pending %d
# HELP gpulock_image_jobs_total Image jobs accepted by ComfyUI. # HELP gpu_turnstile_image_jobs_total Image jobs accepted by ComfyUI.
# TYPE gpulock_image_jobs_total counter # TYPE gpu_turnstile_image_jobs_total counter
gpulock_image_jobs_total %d gpu_turnstile_image_jobs_total %d
# HELP gpulock_lock_wait_seconds Time spent waiting to acquire the GPU lock. # HELP gpu_turnstile_lock_wait_seconds Time spent waiting to acquire the GPU lock.
# TYPE gpulock_lock_wait_seconds histogram # TYPE gpu_turnstile_lock_wait_seconds histogram
`, llmInflight, pending, m.imageJobs.Load()) `, llmInflight, pending, m.imageJobs.Load())
m.llmWait.write(w, "gpulock_lock_wait_seconds", `kind="llm",`) m.llmWait.write(w, "gpu_turnstile_lock_wait_seconds", `kind="llm",`)
m.imageWait.write(w, "gpulock_lock_wait_seconds", `kind="image",`) m.imageWait.write(w, "gpu_turnstile_lock_wait_seconds", `kind="image",`)
fmt.Fprint(w, `# HELP gpulock_unload_seconds Time spent unloading Ollama models before an image job. fmt.Fprint(w, `# HELP gpu_turnstile_unload_seconds Time spent unloading Ollama models before an image job.
# TYPE gpulock_unload_seconds histogram # TYPE gpu_turnstile_unload_seconds histogram
`) `)
m.unload.write(w, "gpulock_unload_seconds", "") m.unload.write(w, "gpu_turnstile_unload_seconds", "")
} }
+2 -1
View File
@@ -1,5 +1,6 @@
// Package ollama is a minimal client for the Ollama management endpoints // Package ollama is a minimal client for the Ollama management endpoints
// gpulock needs: listing loaded models, unloading them, and warming a model. // gpu-turnstile needs: listing loaded models, unloading them, and warming a
// model.
package ollama package ollama
import ( import (
+2 -2
View File
@@ -1,4 +1,4 @@
// Package proxy contains the HTTP handlers for both gpulock listeners: // Package proxy contains the HTTP handlers for both gpu-turnstile listeners:
// reverse proxies to Ollama and ComfyUI with GPU lock arbitration in front // reverse proxies to Ollama and ComfyUI with GPU lock arbitration in front
// of the endpoints that load models. // of the endpoints that load models.
package proxy package proxy
@@ -43,7 +43,7 @@ type Config struct {
WarmModel string WarmModel string
} }
// Server serves both gpulock listeners. // Server serves both gpu-turnstile listeners.
type Server struct { type Server struct {
cfg Config cfg Config
log *slog.Logger log *slog.Logger
+9 -9
View File
@@ -40,13 +40,13 @@ func (r *recorder) index(e string) int {
return -1 return -1
} }
// fakes wires up fake Ollama and ComfyUI upstreams plus the gpulock server. // fakes wires up fake Ollama and ComfyUI upstreams plus the gpu-turnstile server.
type fakes struct { type fakes struct {
rec *recorder rec *recorder
ollama *httptest.Server ollama *httptest.Server
comfy *httptest.Server comfy *httptest.Server
server *httptest.Server // Ollama-facing gpulock listener server *httptest.Server // Ollama-facing gpu-turnstile listener
comfySrv *httptest.Server // ComfyUI-facing gpulock listener comfySrv *httptest.Server // ComfyUI-facing gpu-turnstile listener
freeCh chan struct{} freeCh chan struct{}
chatCh chan struct{} chatCh chan struct{}
historyMu sync.Mutex historyMu sync.Mutex
@@ -279,7 +279,7 @@ func TestStreamingNotBuffered(t *testing.T) {
} }
defer resp.Body.Close() defer resp.Body.Close()
// If gpulock buffered the stream, this read would never complete while // If gpu-turnstile buffered the stream, this read would never complete while
// the gate is closed. // the gate is closed.
line, err := bufio.NewReader(resp.Body).ReadString('\n') line, err := bufio.NewReader(resp.Body).ReadString('\n')
if err != nil { if err != nil {
@@ -321,11 +321,11 @@ func TestHealthzAndMetrics(t *testing.T) {
resp.Body.Close() resp.Body.Close()
text := string(body) text := string(body)
for _, want := range []string{ for _, want := range []string{
`gpulock_state{state="idle"} 1`, `gpu_turnstile_state{state="idle"} 1`,
"gpulock_llm_inflight 0", "gpu_turnstile_llm_inflight 0",
"gpulock_image_jobs_total 0", "gpu_turnstile_image_jobs_total 0",
`gpulock_lock_wait_seconds_bucket{kind="llm",le="+Inf"} 0`, `gpu_turnstile_lock_wait_seconds_bucket{kind="llm",le="+Inf"} 0`,
"gpulock_unload_seconds_count{} 0", "gpu_turnstile_unload_seconds_count{} 0",
} { } {
if !strings.Contains(text, want) { if !strings.Contains(text, want) {
t.Fatalf("metrics missing %q", want) t.Fatalf("metrics missing %q", want)