Retry upstream connection-refused with exponential backoff

A refused dial (service down/restarting) is retried with a wait that
doubles from BACKOFF_INITIAL (1s) up to BACKOFF_MAX (60s) until the
upstream answers or the client disconnects. Handles the Windows WSA
errno (10061) as well as POSIX ECONNREFUSED. compose.yaml.example now
uses host.docker.internal like the working local deployment.
This commit is contained in:
mram
2026-09-20 19:45:48 +02:00
parent 3aaa5d80a9
commit 20d5439b5f
6 changed files with 241 additions and 16 deletions
+7
View File
@@ -119,6 +119,8 @@ load time. Off by default.
| `FREE_TIMEOUT` | `30s` | `POST /free` call after an image job |
| `WARM_TIMEOUT` | `2m` | warm-model reload after an image job |
| `SHUTDOWN_TIMEOUT` | `10s` | graceful shutdown on SIGINT/SIGTERM |
| `BACKOFF_INITIAL` | `1s` | first retry wait when an upstream refuses a connection |
| `BACKOFF_MAX` | `60s` | cap for the exponential retry backoff |
| `PROMPT_CAPTURE_LIMIT` | `65536` | bytes of the `/prompt` response buffered to find `prompt_id` (pass-through is unaffected) |
Startup fails fast on unparsable values. Both upstreams are probed once at
@@ -146,6 +148,11 @@ start (`/api/version`, `/system_stats`); failure is logged, not fatal.
`JOB_TIMEOUT` releases the lock; log at warn.
- Ollama unreachable during unload: continue with the image job; the whole
point is not to block users on a misbehaving neighbour.
- Upstream connection refused while proxying (service down or restarting):
retry with exponential backoff — `BACKOFF_INITIAL`, doubling per attempt,
capped at `BACKOFF_MAX` — until the upstream answers or the client
disconnects. Retrying a refused connection is safe: no request bytes were
sent. Other upstream errors are not retried.
- `POST /prompt` with a body that ComfyUI rejects (400): lock released
immediately, body passed back.
- Websocket `/ws` connections are long-lived and never take the lock.