mrambossekandClaude Opus 5
082a2314ef
Sharing port 443 between the admin UI and the control plane forces two hostnames — one port and one name is one certificate, and the two need different ones. That difference had been leaking into every enrollment link, so an operator handed out fmr-1.echo-lot.app when the thing they and their users know is fmr.echo-lot.app. The link now carries the public name and the app asks GET /v1/discover where to actually connect. The endpoint is plumbing: it exists to select a certificate, and nobody needs to see it. Discovery hands out an address and never a pin. The pin stays in the link. Fetching it over an ordinary TLS connection would make pinning worth exactly what the certificate authorities are worth, and pinning is there to survive one the operator does not control — a root injected by corporate device management, say, which is unremarkable on the networks this tool gets pointed at. With the pin pre-shared, an intercepted discovery can only send a device somewhere the pin will not match: an outage, not a compromise. Optional on both sides. A server that does not answer, or a link that already names the control endpoint, works unchanged — enrollment must not start failing because a lookup did. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
echolot-server
The probe server (spec). Pure Go, stdlib only, GPL-3.0-or-later.
Skeleton status: control plane (enroll / profile / sessions with the spec's HKDF key schedule), UDP data plane (ECHO with observation block, TIMESYNC, HMAC gate, anti-replay, anti-amplification — wire format covered by tests). Not yet: TCP/TLS echo, STUN, canary DNS, actions, observations API, admin UI beyond token minting.
Deployment requirements
No reverse proxy, no 80/443 — by design. Traefik/nginx on the same host are fine; this server never touches their ports, and putting it behind them would break two protocol properties:
- Clients trust the control plane only via the SPKI pin from enrollment (self-signed is first-class). A proxy terminates TLS with its own rotating ACME cert → pins break. The pin model exists so no real certificate is ever needed.
- On the data plane, the observed source address/port/TTL/DSCP is the measurement. Any proxy or NAT layer (including Docker's) substitutes its own — hence host networking.
What a target host actually needs:
| Port | Proto | Purpose | Notes |
|---|---|---|---|
| 8443 | tcp | control plane (pinned HTTPS) | any port — it travels in the enrollment QR + profile |
| 8442 | udp | UDP probe data plane | any port, profile-driven |
| 8441 | tcp | TCP/TLS echo | any port (not yet implemented) |
| 3478 | udp | STUN | keep standard: vanilla RFC 5389 for tool interop; rarely contended |
| 8444 | tcp | admin | loopback-only by design — reach via SSH tunnel |
All configurable via ECHOLOT_*_LISTEN. Plus:
- A public IP on the host (v4, ideally also v6 — v6 topology issues are half of what clients want to measure). Behind NAT, plain port-forwards work.
- Second IP (optional): full RFC 5780 NAT-behavior discovery (
stun-5780) needs an alternate reply address; without it the profile advertisesstun-basicand clients degrade gracefully. - Delegated DNS subzone (for
canary-dns): setECHOLOT_DNS_LISTEN(udp+tcp/53 on the service IPs) andECHOLOT_CANARY_ZONE(e.g.c.echo-lot.app), then delegate the zone to this host in your DNS provider:The server is authoritative for that zone only, serving the spec §6.1 reference records (frozen inc.echo-lot.app. NS fmr-1.echo-lot.app. c.echo-lot.app. NS fmr-2.echo-lot.app.internal/canarydns/dns_reference.go) plus per-query<nonce>.<session>.<zone>lookups it logs. Binding :53 on the public IPs is fine even with systemd-resolved (it only claims 127.0.0.53). Absent config → capability simply not advertised. - Outbound freedom for connect-back / delayed-echo actions — no extra inbound ports; generated traffic goes only to the session's observed source.
Deliberately out of scope here: an echo listener on 443 (to detect port-based egress filtering) — that genuinely needs 443 and belongs on a dedicated IP, not on a host running a reverse proxy.
Host tuning (measurement fidelity)
A measurement server must not let the kernel distort what clients observe. Apply the recommended sysctls and the daemon will confirm the host is clean:
sudo cp deploy/99-echolot-sysctl.conf /etc/sysctl.d/ && sudo sysctl --system
The daemon self-tests at startup and via GET /admin/selftest (localhost):
- sysctl audit — flags settings that would distort results (RA acceptance on a static host, ICMP redirects, ICMP rate-limiting of the server's own errors, disabled TCP options).
- egress-MTU self-proof — DF-probes external anchors (
ECHOLOT_MTU_PROBE_TARGETS, default 1.1.1.1 + a v6 anchor) and reads the discovered path MTU. If the server's own uplink can't carry 1500, client MTU results would measure this server, not the client — so the profile exposesserver_selftest.mtu_okand the log warns loudly.
Both signals ride in GET /v1/profile as server_selftest so a client can trust — or skip —
MTU testing accordingly.
Run in Docker (config via env)
docker compose up -d # see compose.yaml — network_mode: host is required
Host networking is not negotiable: behind Docker NAT the server would observe the proxy's
source addresses and TTLs instead of the client's — falsifying exactly what it measures.
Container mode is autodetected (/.dockerenv etc.); --docker / ECHOLOT_DOCKER=1 forces it.
In this mode systemd install and self-update are refused — update by pulling a new image tag.
Run native (systemd)
go build -o /usr/local/bin/echolot-server ./cmd/echolot-server
sudo /usr/local/bin/echolot-server --install-systemd # writes unit, enables, starts
sudo /usr/local/bin/echolot-server --uninstall-systemd
Config precedence: flags > ECHOLOT_* env > defaults. Every flag has an env twin
(--udp-listen ↔ ECHOLOT_UDP_LISTEN). Host config lives in /etc/echolot-server.env
(seeded by --install-systemd, never overwritten).
Multi-IP hosts: listen specs are comma-separated, and you should bind explicit addresses — a wildcard bind would also claim management-only IPs:
ECHOLOT_CONTROL_LISTEN=203.0.113.10:8443,[2001:db8::10]:8443
ECHOLOT_UDP_LISTEN=203.0.113.10:8442,203.0.113.11:8442,[2001:db8::10]:8442,[2001:db8::11]:8442
Passing --self-update-api to --install-systemd additionally installs a daily randomized
self-update timer (echolot-server-update.timer) that restarts the service after a successful
update. Updates are checksum-verified against the release's SHA256SUMS (integrity, not
authenticity — signature verification remains TODO before treating the update source as
untrusted).
Self-update (opt-in, native only)
echolot-server --self-update \
--self-update-api https://git.example.net/api/v1/repos/owner/repo
Fetches the newest server-v* release asset for this OS/arch and atomically replaces the
binary; systemd's Restart= brings up the new version. Run it from a systemd timer for
unattended updates. TODO before enabling anywhere untrusted: signature verification of the
downloaded asset.
First contact
# 1. mint an enrollment token (admin listener is loopback-only)
curl -s -X POST 'http://127.0.0.1:8444/admin/enroll-tokens?note=phone'
# 2. device enrolls with it (normally via the echolot:// QR code)
curl -sk -X POST https://<host>:8443/v1/enroll -H 'Authorization: Bearer <token>'
# 3. device fetches its profile
curl -sk https://<host>:8443/v1/profile -H 'Authorization: Bearer <credential>'
The SPKI pin clients must verify is logged at startup (pin-sha256).
Development
go test ./... # includes wire-format tests for the UDP data plane
go vet ./...
CI (.gitea/workflows/build-server.yml): tests on every push touching server/;
tagging server-v1.2.3 builds + pushes the container image to the Gitea registry and
attaches static linux amd64/arm64 binaries (+ SHA256SUMS) to a release — the same
artifacts --self-update consumes.
TLS for the admin UI
The binary terminates TLS itself; there is no reverse proxy in the design. It already serves TLS for the control plane, so this is reuse rather than new machinery, and it keeps the "one process, one config file" property. A proxy would also invite someone to eventually front the control plane too — which would break SPKI pinning, because clients pin that certificate's key.
ECHOLOT_ADMIN_LISTEN=[2001:db8::2]:443
ECHOLOT_ADMIN_TLS_CERT=/etc/echolot/admin.pem
ECHOLOT_ADMIN_TLS_KEY=/etc/echolot/admin.key
ECHOLOT_ADMIN_BASE_URL=https://admin.example.net
Certificates come from any ACME client. DNS-01 is the one to use here: it needs no inbound port 80, which matters on a host where 80 is awkward or already spoken for.
acme.sh --issue --dns dns_cf -d admin.example.net \
--key-file /etc/echolot/admin.key \
--fullchain-file /etc/echolot/admin.pem
No reload hook is needed. The certificate is re-read when the files change, so a renewal that drops new files in place is picked up on the next handshake. That is deliberate: a reload hook is the part of a renewal setup that quietly stops working, months later, and is noticed only once the certificate has already expired. A torn write — renewal tools write cert and key separately — keeps the previous certificate rather than failing the listener.
Serving the admin UI in plaintext on a non-loopback address is refused: the session cookie is a
bearer credential for everything the server can do, and the OIDC authorization code arrives in a
URL. Bind to loopback and use an SSH tunnel (ssh -L 8444:localhost:8444 host), supply a
certificate, or set ECHOLOT_ADMIN_INSECURE=1 if you mean it.
The control-plane certificate is deliberately not hot-reloaded. Clients pin its public key, so replacing it is a rotation an operator should have to think about, not something that happens because a file changed.