Managed ComfyUI: COMFY_CMD starts it on demand, idle stop frees VRAM (internal/supervise)
This commit is contained in:
@@ -121,6 +121,27 @@ state is `idle`, send `POST /api/generate {"model":WARM_MODEL,"keep_alive":-1}`
|
||||
with empty prompt to reload the chat model so the next chat doesn't pay the
|
||||
load time. Off by default.
|
||||
|
||||
### Managed ComfyUI (`COMFY_CMD`)
|
||||
|
||||
When `COMFY_CMD` is set, gpu-turnstile runs ComfyUI as a supervised child
|
||||
process instead of expecting an always-on server:
|
||||
|
||||
- **Start on demand**: any ComfyUI request spawns it (double quotes in the
|
||||
command line group arguments with spaces; `COMFY_DIR` sets the working
|
||||
directory). `POST /prompt` additionally waits for readiness
|
||||
(`/system_stats`) for up to `COMFY_START_TIMEOUT` *before* taking the GPU
|
||||
lock, so LLM traffic flows while torch loads. Other requests are bridged
|
||||
by the normal retry backoff. Spawn failure or a readiness timeout
|
||||
answers 502.
|
||||
- **Idle stop**: after `COMFY_IDLE_TIMEOUT` without requests or finished
|
||||
jobs — and only while the GPU lock is idle — the process tree is killed,
|
||||
freeing the VRAM ComfyUI holds. The next request restarts it.
|
||||
- **Crash**: an unexpected exit is logged; the next request respawns.
|
||||
gpu-turnstile's own shutdown stops the child too.
|
||||
- Its stdout/stderr is forwarded to the log at INFO. The health check
|
||||
skips the intentionally-stopped/starting states; a failed probe while
|
||||
the process is alive and was previously ready is logged as DOWN.
|
||||
|
||||
## Configuration (env)
|
||||
|
||||
Configuration comes from environment variables and/or an `.env`-style
|
||||
@@ -142,6 +163,10 @@ override file values. A missing file is fine; a malformed one is fatal.
|
||||
| `LLM_BUSY_STATUS` | `503` | HTTP status for rejected LLM requests in reject mode (400–599, e.g. 429) |
|
||||
| `BUSY_RETRY_AFTER` | `30` | seconds sent as `Retry-After` on busy responses (both modes) |
|
||||
| `WARM_MODEL` | `` | optional model to reload after an image job |
|
||||
| `COMFY_CMD` | _(empty = unmanaged)_ | spawn and supervise ComfyUI on demand: first request starts it, idle stop after `COMFY_IDLE_TIMEOUT` frees its VRAM. Requires `COMFY_URL` |
|
||||
| `COMFY_DIR` | `` | working directory for `COMFY_CMD` |
|
||||
| `COMFY_IDLE_TIMEOUT` | `5m` | stop the managed ComfyUI after this long without requests or jobs |
|
||||
| `COMFY_START_TIMEOUT` | `2m` | how long a request waits for the managed ComfyUI to come up |
|
||||
| `LOGLEVEL` | `warn` | `info` logs every request (colored arrows in text mode), `debug` adds lock transitions. `LOG_LEVEL` is accepted as an alias |
|
||||
| `LOG_FORMAT` | `text` | `json` for structured JSON logs |
|
||||
| `LOG_FILE` | `` | append logs to this file instead of stderr (useful as a service) |
|
||||
|
||||
Reference in New Issue
Block a user