Letting the kernel fragment an oversized datagram answers one question — do
fragments get through. It cannot answer the more interesting one, because the
kernel always emits them in order, first one first.
The classic middlebox fault is exactly about that ordering. Only the first
fragment carries the UDP header, and therefore the ports; a stateful firewall
or NAT that has not seen it has no flow to match the rest against, and many
drop them. That is invisible to any in-order test and shows up in the field as
"large DNS answers fail on this network" or "the tunnel breaks when the MTU
drops" — it works until the network reorders, then fails intermittently, which
is the hardest kind of fault to chase.
So the server now builds the fragments itself (raw socket, IP_HDRINCL) and
controls their order: in_order as a baseline, reversed, and first-fragment-last.
The datagram is assembled and signed whole before being cut up, so what the
client reassembles is indistinguishable from an ordinary packet — otherwise it
would be measuring our sender rather than the path.
Two details that would silently produce wrong answers:
- The UDP checksum is computed rather than left zero. A zero-checksum datagram
is dropped by some middleboxes, and that drop would be recorded as a
fragmentation failure, which is the wrong conclusion entirely.
- Fragment offsets are in 8-byte units, so non-final fragments are rounded to
a multiple of 8. A 100-byte fragment is not an error, it is a datagram no
host will ever reassemble.
frag-send is advertised only when a raw socket can actually be opened — checked
by opening one, since a permission model has more ways to say no than a
capability bit has to say yes.
Fragment header arithmetic is unit-tested (reassembly coverage, MF flags, shared
IP ID, 8-byte offsets, checksum verification), cross-compiled and run on Linux
since the code is build-tagged.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Both sides now declare what they will talk to, and enforce it. Two axes kept
deliberately separate, because conflating them is the trap:
protocol_version — CAN these builds talk. The correctness axis. Below 1.0.0
the minor is the breaking axis, per SemVer §4.
release window — MAY they, per policy. [min, max), advertised in the
profile, overridable by the operator.
The server refuses out-of-window apps with 426 and a body naming both versions
and the accepted range; the app checks the profile in both directions before a
run rather than discovering mid-measurement that it will be refused.
Three rules that shape the rest:
- GET /v1/profile is never gated. It is where a refused client learns which
version it needs; gating it leaves the user with a network error instead of
an answer, which is precisely the confusion this exists to remove.
- An unparseable or absent version is "unknown", and is allowed. Development
builds report "dev", and a client too old to send the header cannot be
identified anyway.
- Bounds sit at breaking boundaries, not at releases, so shipping a patch
never requires editing a range. The app's server minimum is 0.4.2 for a
stated reason: earlier multi-homed servers mis-addressed granted sends and
the client measured 100% downstream loss that never happened.
The app's versionCode is now derived from its SemVer instead of being a second
number someone has to remember to bump.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
fmr binds two IPv4 addresses. connFor picked whichever socket of the right
family came first in the bind list, so a downtrain for a session established on
.150 went out from .151 — and every packet was dropped by the client's NAT,
which has no mapping for that pair. tcpdump on the server showed all 50 leaving;
the client saw none. Read as "100% downstream loss", which is the worst kind of
wrong: a confident measurement of something that never happened.
Sessions now record which of our own bound addresses received their traffic, and
granted sends (and delayed echo) go back out through that socket. The fallback
to a family match is kept for the case where nothing has been received yet, and
the test pins both paths — a single-homed lab can never reproduce this.
Also: the client-side halves of the same work — anonymizer (core-privacy), local
run archive with retention (core-archive), upload client, and the app's settings
and history screens.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
big_send now forces the Don't-Fragment bit for the whole burst by default, so
the largest size that arrives IS the downstream path MTU rather than "fragments
got through" — two different measurements the schema already separates. Sizes
above our own egress MTU (from the startup self-test) are refused up front and
reported as max_df_bytes, because absence caused by our kernel must not be read
as a limit of the client's path.
Uploads: one JSON file per run under the state dir, with the policy the operator
actually cares about — who may upload (off / anonymous / account), how large,
how long to keep, and the least anonymization accepted. The profile advertises
all of it so the app can present the switch honestly instead of discovering the
rules by failing. `account` refuses today rather than falling back to anonymous:
picking the strict setting before OIDC lands must not silently mean the loose one.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The grant is the keystone that makes server->client sends safe: created
only by an authenticated control-plane action, bound at creation to the
session's OBSERVED data-plane source (so it can never be aimed at a third
party), and bounded by bytes, average rate and expiry. Sends stop the
moment the budget runs out, so a buggy action cannot become a flood.
Two granted actions on top of it:
- downtrain: N packets at a given size/interval toward the client, with
seq + send-timestamp in the payload — downstream loss/reorder/jitter,
which an upstream-only train cannot measure.
- big_send: one datagram per requested size, echoing the intended size in
the payload — downstream MTU / black-hole evidence the client cannot
produce for itself (only the far end can emit a large packet toward it).
Tests cover the security properties: no grant without a verified
destination, client requests clamped to server limits, byte budget stops
sending exactly, expiry refuses, and the rate ceiling throttles a burst.
Capabilities gain downtrain + big-send.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Server ACKs each DF-flagged probe with a tiny MTU_ACK carrying the size it
received; the client binary-searches the path MTU. Non-amplifying by
construction. Tested.
Also records: v0.3.2 (http-echo + tls-reference) verified live on fmr, and
the finding that upstream trains are already observable via the
observations API (dedicated TRAIN_REPORT deferred — needs an
anti-amplification grant + columnar encoding).
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
- stun: RFC 5389 binding responder + RFC 5780 attributes (OTHER-ADDRESS,
RESPONSE-ORIGIN, CHANGE-REQUEST) on a primary/alt-port socket grid per
address; advertises stun-5780 with >=2 same-family addrs, else
stun-basic. Unmodified framing for tooling interop. Tested.
- tcpecho: JSON greeting with observed src + TCP_INFO MSS/options
(Linux getsockopt; zeroed elsewhere via build tags), then byte echo.
- session: per-packet UDP observations + connect-back results, ByID lookup.
- control: GET /v1/sessions/{id}/observations, POST .../actions
(delayed_echo → DELAYED_ECHO at the observed data-plane source;
connect_back → dial the control-plane source, record connected/refused/
timeout+rtt). Capabilities computed from what is actually wired.
- config/main: comma-separated STUN listeners; all planes bind explicit
addresses; graceful shutdown of the new listeners.
Full flow smoke-tested; go test green (stun binding/change-port,
dataplane wire format).
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Pure stdlib. Implements the spec's core: enrollment (single-use tokens),
profile (SPKI pin, only real capabilities advertised), sessions with the
§2.4 HKDF-SHA256 key schedule; UDP data plane with the 32-byte ELT1
header, 4-byte HMAC gate, 1024-wide anti-replay window, ECHO_RESP with
observation block, TIMESYNC, and the §3.4 anti-amplification cap. Wire
format has tests (roundtrip + silent-drop cases); enroll→profile→session
smoke-tested live.
Modes: container (autodetect /.dockerenv|/run/.containerenv|cgroup, or
--docker/ECHOLOT_DOCKER=1; config via ECHOLOT_* env; distroless image;
network_mode host required — Docker NAT would falsify observed sources)
and native (--install-systemd/--uninstall-systemd with a hardened unit,
opt-in --self-update from Gitea releases; refused in containers).
CI: tests on any server/ push; server-v* tags build+push the image to the
Gitea registry and attach linux amd64/arm64 binaries + SHA256SUMS to a
release — the artifact self-update consumes.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
