Extend retry backoff; rework logging (LOGLEVEL, request lines, colors)

Backoff now covers all dial-phase errors (refused, timeout, DNS), TLS
handshake failures, and 5xx responses with replayable bodies; streamed
POSTs are never replayed to avoid duplicate work. First retry of an
episode logs at WARN, subsequent attempts at INFO.

LOGLEVEL (LOG_LEVEL kept as alias) now defaults to warn: startup logs
version plus every setting; INFO adds one line per incoming request and
per response with status/duration, ANSI-colored in text mode (bypasses
slog's escaping so colors render in docker compose logs); NO_COLOR or
LOG_FORMAT=json disables colors.
This commit is contained in:
mram
2026-09-20 20:00:57 +02:00
parent 20d5439b5f
commit 42e1386811
5 changed files with 312 additions and 42 deletions
+7 -1
View File
@@ -41,7 +41,7 @@ startup.
| `JOB_TIMEOUT` | `15m` | Wait for a ComfyUI job to finish |
| `LLM_WAIT_TIMEOUT` | `10m` | Max lock wait for an LLM request before 503 |
| `WARM_MODEL` | _(empty)_ | Model to reload after an image job (off by default) |
| `LOG_LEVEL` | `info` | `debug` logs every lock transition |
| `LOGLEVEL` | `warn` | `info` logs every request (colored arrows in text mode), `debug` adds lock transitions. `LOG_LEVEL` works as an alias |
| `LOG_FORMAT` | `text` | `json` for structured JSON logs |
| `UNLOAD_POLL_INTERVAL` | `500ms` | `/api/ps` poll interval while unloading |
| `HISTORY_POLL_INTERVAL` | `1s` | `/history/<id>` poll interval while a job runs |
@@ -60,6 +60,12 @@ startup.
`gpu_turnstile_llm_inflight`, `gpu_turnstile_image_pending`,
`gpu_turnstile_image_jobs_total`, `gpu_turnstile_lock_wait_seconds`
(histogram, `kind="llm|image"`), `gpu_turnstile_unload_seconds`.
- Logs: startup logs the version and every setting (visible even at the
default `warn` level). With `LOGLEVEL=info` or `debug`, every request
logs a `-->` incoming line and a `<--` response line with status and
duration — ANSI-colored (cyan incoming; green/yellow/red by status
class) in text mode, which renders in `docker compose logs` on Windows
Terminal. Set `NO_COLOR` to disable colors.
## Build and run