PUBLIC/gpu-turnstile
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 6 Wiki Activity

Managed ComfyUI: COMFY_CMD starts it on demand, idle stop frees VRAM (internal/supervise)

Browse Source
This commit is contained in:
mram
2026-09-21 13:48:07 +02:00
parent e43ad02fc4
commit d7566329ae
9 changed files with 607 additions and 4 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+23
View File
@@ -56,6 +56,10 @@ override file values. Invalid values fail at startup.
| `LLM_BUSY_STATUS` | `503` | HTTP status for rejected LLM requests in reject mode (400–599, e.g. 429) |
| `BUSY_RETRY_AFTER` | `30` | Seconds sent as `Retry-After` on busy responses (both modes) |
| `WARM_MODEL` | _(empty)_ | Model to reload after an image job (off by default) |
| `COMFY_CMD` | _(empty = unmanaged)_ | Supervise ComfyUI: start on demand, stop when idle to free VRAM. Requires `COMFY_URL` |
| `COMFY_DIR` | _(empty)_ | Working directory for `COMFY_CMD` |
| `COMFY_IDLE_TIMEOUT` | `5m` | Stop the managed ComfyUI after this long idle |
| `COMFY_START_TIMEOUT` | `2m` | Max wait for the managed ComfyUI to come up |
| `LOGLEVEL` | `warn` | `info` logs every request (colored arrows in text mode), `debug` adds lock transitions. `LOG_LEVEL` works as an alias |
| `LOG_FORMAT` | `text` | `json` for structured JSON logs |
| `LOG_FILE` | _(empty)_ | Append logs to this file instead of stderr |
@@ -89,6 +93,25 @@ override file values. Invalid values fail at startup.
class) in text mode, which renders in `docker compose logs` on Windows
Terminal. Set `NO_COLOR` to disable colors.
## Managed ComfyUI (`COMFY_CMD`)
Don't want ComfyUI running 24/7 (it holds VRAM even when idle — and the
Desktop app kills its server when you close it)? Point `COMFY_CMD` at a
standalone launch command and gpu-turnstile supervises it: the first
request starts it, it stops again after `COMFY_IDLE_TIMEOUT` (default 5m)
without work, freeing the GPU for games or the LLM. Example for a Desktop
install (run it once manually to confirm it works):
```
COMFY_URL=http://127.0.0.1:8188
COMFY_CMD="C:\ComfyUI\.venv\Scripts\python.exe ComfyUI\main.py --port 8188"
COMFY_DIR=C:\ComfyUI
```
`POST /prompt` waits for the server to answer before taking the GPU lock
(LLM traffic keeps flowing while torch loads); crashes are logged and the
next request respawns. Shutting gpu-turnstile down stops the child too.
## Build and run
```sh