Compare commits

...
Author SHA1 Message Date
mrambossekandClaude Opus 5 d574c76630 docs: log the prober fold and server v0.9.2 feature work
server-release / image (push) Successful in 16s
server-test / test (push) Successful in 48s
server-release / release (push) Successful in 1m2s
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-02 13:06:09 +02:00
mrambossekandClaude Opus 5 8118e213ae server: upstream trains, observed TTL/DSCP/ECN, rate limits, action ids
Types 0x03/0x04/0x05 land with a bounded columnar train buffer (head kept,
truncation declared) and grant-free multi-part reports - a report row is
smaller than the packet it answers, so $3.4 holds without a grant. The
read loop now collects TTL/TOS cmsgs on Linux, replacing the 0xFF stubs in
the observation block with what the kernel saw; downtrain gained a dscp
parameter, so DSCP survival is measurable in both directions.

Rate limiting ($2.5) exists now: per-credential AND per-source buckets,
429 on the control plane, silent drop on the data plane after the HMAC
gate and before the replay window. UDP ceilings default above the largest
legitimate run - a limit that clips a real measurement produces a
confidently wrong number.

Every granted packet carries its action_id at payload[8:16]; overlapping
actions were unattributable before. Canary DNS logs now honor the stated
24h privacy default. /admin/enroll-tokens answers the spec's JSON shape.
protocol_version 1.0.1 (additive).

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-02 13:04:54 +02:00
mrambossekandClaude Opus 5 f6849f8e6a app: fold the prober's validated capabilities into core-probe
traceroute.udp4 lands as TracerouteProbe: ICMP time-exceeded read off the
socket error queue via Os.recvmsg(MSG_ERRQUEUE) through the reflection
facade the prober validated on both devices - no root, no raw socket, and
the C-over-JNI shim stays dead. Emits the schema's TracerouteEvidence with
rtt_ns. OsAbi carries the hardcoded sockopt ABI numbers across, including
the measured fact that Os.getsockoptInt exists on neither device, so PMTU
must come from the errqueue, never getsockopt(IP_MTU).

local.mdns_inventory lands as MdnsInventoryProbe with both hardware-bought
lessons intact: the meta-query lies (0 results beside live services on both
devices), and 4s of listening misses what 10s catches.

App version 0.2.2.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-02 12:39:52 +02:00
mrambossekandClaude Opus 5 7d98a43866 docs: close the VPN, v6, reserved-probe and signing items in the build log
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-02 12:32:02 +02:00
mrambossekandClaude Opus 5 a49bef5821 server: refuse unsigned releases and polluted reserved addresses
Self-update now verifies SHA256SUMS.sig (ed25519, relsign package) against
a public key baked into the binary; the private key exists only in the CI
secret store, so a compromised release host can withhold updates but not
inject one. CI signs on every server-v* tag and hard-fails without the
secret. Operators with their own pipeline override the key via
ECHOLOT_SELF_UPDATE_PUBKEY (mint a pair with release-sign -gen).

Startup also now proves 80/443 are actually free on the reserved
measurement addresses by asking the OS (throwaway bind), not the config -
CheckReserved could never see a stray process, and the adb-beacon receiver
on 0.0.0.0:443 was exactly that.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-02 12:32:02 +02:00
mrambossekandClaude Opus 5 20cfecf566 app: name what a VPN blocked, and prove v6 broken before saying so
Constraints are detected up front (one throwaway bind per network) and land
in run.constraints, a measurement.vpn_constrained finding, the $7.3 verdict
(INCONCLUSIVE outright) and a banner on the run screen - a VPN'd run looked
exactly like a clean run of a healthy network before this.

v6.broken returns to the registry now that it can be earned: V6ConnectProbe
(v6.brokenness) makes a real TCP connection over IPv6 to the enrolled
server, and only both transports failing on a network that advertises IPv6
justifies the claim. TCP succeeding turns the finding into 'ICMPv6 is
filtered, IPv6 works' at high confidence instead.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-02 12:31:44 +02:00
mrambossekandClaude Opus 5 987b2ceb47 server: read the verdict from where the schema puts it
Every uploaded run showed "not recorded" in the web UI because the meta
extractor read summary.verdict. Schema §7.3 calls that field
summary.overall; "verdict" is the per-category field one level down. So
the verdict was never stored, and the UI faithfully reported a gap that
was this parser's doing rather than the document's.

The test encoded the same mistake — its fixture posted
summary.verdict:"warn" — so it passed throughout against a parser that
read a field nothing writes. Corrected to summary.overall, and to a
verdict that exists: §7.3 defines green|yellow|red|inconclusive, and
"warn" was never one of them.

The eleven runs already stored had their meta backfilled from the
documents, which are kept byte-for-byte and still carry the real value.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-02 10:52:25 +02:00
mrambossekandClaude Opus 5 d5b1bab577 app: stop the autorun upload pointing at a listener that is gone
"upload failed: HTTP 400 client sent an HTTP request to an HTTPS server"
is a TLS listener rejecting cleartext, and the cleartext was ours:
REPORT_UPLOAD_URL was http://89.185.109.150:443/report, which used to be
the adb-beacon receiver holding 0.0.0.0:443 in plaintext. Disabling that
receiver and giving 443 to echolot-server left this posting plain HTTP at
a TLS port.

Blanked rather than repointed. The endpoint existed so an unattended run
could be collected without adb, and autorun reports are now read straight
off the device with `run-as cat` — so it was buying nothing and emitting
an alarming error for a debugging convenience. Deliberately not aimed at
/v1/runs either: that is the consent-gated upload, and a debugging
shortcut must not be able to satisfy it by accident.

The message says what happened instead of implying something broke.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-02 10:36:49 +02:00
mrambossekandClaude Opus 5 a17c3fd9e6 app: catch a search domain that swallows DNS queries
A tablet on a healthy network could not resolve anything. The DNS server
answered the bare name correctly — NOERROR, two records, A and AAAA, with
and without EDNS0 — so the earlier finding blamed the device's resolver.
It was wrong. The network advertised hudelist.local as a search domain and
the server silently dropped every query under it: not NXDOMAIN, nothing at
all. Resolvers append search domains, so they waited for a reply that was
never coming.

Silence is the part that makes this vicious. A negative answer moves a
resolver on; no answer looks like packet loss, so it retries, and some
give up on the lookup entirely. It also explains how two devices on one
network can disagree about whether DNS works — the phone tried the plain
name first and never noticed.

The probe now asks about a nonce name under each advertised search domain,
where the wanted answer is NXDOMAIN and only silence is a fault. The
finding is ordered ahead of dns.system_resolver_broken so the two cannot
both fire: without that, this exact network gets told its device is
broken.

Severity follows the harm rather than the shape. HIGH when resolution is
actually failing, MEDIUM when the domain is a black hole but this resolver
happens to try the plain name first — calling that HIGH would be crying
wolf on a network that works. The message names the fix and notes that
.local is reserved for mDNS by RFC 6762 and widely dropped by design,
while home.arpa (RFC 8375) is the name reserved for this.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-02 10:29:04 +02:00
mrambossekandClaude Opus 5 cfa58e8d60 app: a DNS reply is not a DNS answer
The probe counted any well-formed packet from the server as "the server
answers", checking only the transaction id and a minimum length. REFUSED
and SERVFAIL are well-formed packets. So a server actively refusing this
client would have been reported as healthy, and the finding — whose whole
output is "the network is fine, your device is not" — would have pointed
confidently at the wrong component.

It now requires rcode 0 and at least one record, and reports a refusal as
what it is: a working server saying no, which points back at the network.
The rcode is named rather than numbered, because "REFUSED" is a fact an
operator can act on and "rcode 5" is a lookup.

Caught by decoding what fmr's router actually replied — ab cd 81 80 00 01
00 02, NOERROR with two answers — after realising the earlier check only
counted bytes. The reply was genuinely good, so the finding on the tablet
stands; the check was wrong regardless.

The advice is broader too. That tablet's fault survived a reboot, which
makes "toggle wifi and it clears" wrong as a flat claim: it now says what
to look at when a restart does not fix it — something on the device
filtering DNS, or a per-client rule on the router.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-02 10:10:36 +02:00
mrambossekandClaude Opus 5 2ed4d1f478 Reach the server by address when its name will not resolve
A measurement tool that cannot report from a broken network is useless
exactly when it matters, and a wedged resolver is one of the faults this
app is built to find — it should not also be the thing that stops the
finding being delivered. The profile already carries the server's
addresses; they are now kept and used when the name fails.

Safe because the pin is the trust and the name is not part of it: the
server presents the same certificate however it was reached, and a wrong
address fails the pin like anything else. Only the primaries are cached —
the alternate pair exists for NAT behaviour discovery and does not carry
the control plane, so falling back to one would fail for a second,
unrelated reason.

Substituted only when the name genuinely does not resolve, and only after
checking the candidate answers on the port: on a v4-only network a v6
address would otherwise be chosen and fail slowly, which is the wrong
answer delivered late.

The server had to meet it halfway. Sharing 443 by SNI meant a client
arriving by IP sent no server name and got the Let's Encrypt certificate,
failing the pin. A numeric host — or no SNI at all — now selects the
pinned certificate and routes to the control plane. That is sound because
the admin UI is only ever reached by name: browsers always send SNI, and
nobody bookmarks an IP for a site with a CA-issued certificate.

Verified against fmr: by IP on both families the served pin is the
control one and /v1/profile answers 401, while fmr.echo-lot.app still
serves the Let's Encrypt certificate and the admin UI.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-02 10:03:14 +02:00
mrambossekandClaude Opus 5 d65dbbc75a app: tell a broken device resolver apart from a broken network
Diagnosing a tablet that claimed "no internet" took twenty adb commands
to establish something the app should have said in one run: ping to
1.1.1.1 worked, the configured DNS server answered a raw UDP query in
65 bytes, and Android still could not resolve a hostname. The network was
fine; netd had wedged.

Those two failures look identical to a user and want opposite responses —
"look at your router" against "toggle your wifi" — so dns.resolver asks
the network's own servers directly and compares the answer against what
the platform returns for the same name. The query is hand-rolled over a
plain DatagramSocket on purpose: anything routed through a resolver API
would inherit the very fault being looked for.

dns.system_resolver_broken fires only on the pairing that is otherwise
unattributable: server answered, platform did not. Per network, because a
phone can have wedged wifi and working cellular at once.

Also records Android's own verdict per network — validated, captive
portal, partial connectivity — which the app reproduced with its own HTTP
probes but never stored. It is free, it is what the user sees in the
status bar, and its disagreement with our measurements is exactly what
identified the tablet. NET_CAPABILITY_PARTIAL_CONNECTIVITY is @SystemApi
so the constant is inlined with its rationale, in the manner of OsAbi.kt,
and read defensively enough to report unknown rather than false.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-02 09:44:49 +02:00
mrambossekandClaude Opus 5 40e76c52ca adminui: show the enrolment link as a QR code
Enrolling a device that cannot reach the admin UI meant transcribing a
200-character link with a base64 pin in it — the step the link format
exists to avoid, and the one where a pin wrong by one character fails
later as an inscrutable TLS error.

Rendered as inline SVG rather than a PNG data: URI, because the page's
CSP is default-src 'none' and means it: a data: image would need img-src
opened, markup needs nothing. One path rather than a rect per module,
since a link this long encodes to about 60x60 and two thousand elements
is a lot of DOM for a picture of a square. It is generated from the same
validated value as the href, so a rejected link produces neither.

This relaxes the stdlib-only rule, deliberately and recorded in CLAUDE.md.
The rule bought one self-contained binary with no supply chain to audit,
which one small pure-Go package barely dents; F-Droid never applied to
the server, only the app ships there. A correct QR encoder is ~500 lines
of Reed-Solomon that nobody should be hand-writing.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-02 08:48:02 +02:00
mrambossekandClaude Opus 5 5b02d40802 app: notice when Shizuku is authorised
The banner watched only for the binder arriving or dying, and granting
permission does neither. So after the user tapped the banner, answered
the dialog and came back, it still read "running but not authorised" —
wrong at precisely the moment they were looking for confirmation that it
had worked.

Two listeners were missing, because there are two ways this changes.
Answering our own request now fires OnRequestPermissionResultListener.
That is not enough on its own: permission can equally be granted inside
Shizuku's own app, and Shizuku started or stopped there, none of which
calls back into this process — so the state is re-read whenever the
screen comes forward, which is the only thing that covers every route.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-02 08:37:45 +02:00
mrambossekandClaude Opus 5 416b783647 app: lay the server facts out in real columns
The block was rendered as one padded string, which only lines up in a
monospaced font — and the monospace never took, so the values sat at
ragged offsets and the point of the list was lost. Padding text to fake a
table makes the layout depend on a typeface decision made elsewhere.

It is a label column of fixed width and a value column that takes the
rest, so the addresses align whatever the font does and a long value wraps
inside its own column instead of under the labels. The capability list
goes back to plain comma-separated text: hand-wrapping it at three per
line was working around the same missing alignment.

Placement too. The facts now sit under the "Check server" button that
fetches them, rather than among the input fields, and the sentence
explaining the endpoint sits directly under the URL field it describes —
it had ended up orphaned between the two, reading as a comment on nothing.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-02 08:24:00 +02:00
mrambossekandClaude Opus 5 c4f2a10790 app: show what the server reports as facts, not as inputs
The settings card offered three editable boxes and said nothing about the
server itself — which addresses a test will actually use, on which ports,
what it can measure. That is the part a person checks before trusting a
result, and "which address did this come from" is precisely the question
a report leaves open.

The server now publishes it. The profile's targets carried one IPv4 and a
TODO; it reports both families and both alternates, derived from the UDP
listen spec rather than configured separately, so the list cannot drift
from what is actually bound. No reservation means no alternate is
claimed: announcing a second address as the RFC 5780 alternate when none
was set aside would promise a redirect the server will not send.

The app renders them read-only, in a panel visibly distinct from the
fields above. An editable box that changes nothing is worse than no box,
and these are facts to read rather than settings to apply.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-02 08:14:42 +02:00
mrambossekandClaude Opus 5 fe4ec23ba1 app: show the server by the name its operator handed out
The Server URL field showed the endpoint the app dials, which after
discovery is not the name anyone was given — enrolling against
fmr.echo-lot.app left the settings reading fmr-1.echo-lot.app, with no
explanation of where the -1 came from.

It shows the public name now. The endpoint is not hidden, just demoted to
a line beneath that says where the connection actually goes and why the
two differ: a network engineer debugging a failed connection wants that,
and burying it would trade one confusion for another.

Typing a URL by hand sets both, since there is no discovery to consult in
that case — setting only the public one would leave the app still dialling
the previous server, which is the kind of half-applied change that fails
much later and somewhere else.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-02 08:07:55 +02:00
mrambossekandClaude Opus 5 33799b8135 adminui: the enrolment link was rendered as a dead anchor
html/template rewrites an href whose scheme it does not recognise to
"#ZgotmplZ". echolot:// is not on its list, so "Open in the Echolot app"
was not a link at all — tapping it did nothing, and nothing showed it:
the markup reads correctly, the app resolves the scheme, and only the
sanitised attribute in the served HTML gives it away.

Marking the value template.URL opts out of that sanitising, which is only
safe because the shape is now checked first. The link arrives in a query
parameter, so without the check a crafted /devices?link=javascript:… would
put a script URL into the page for an admin to click.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-02 08:04:56 +02:00
mrambossekandClaude Opus 5 f6e093944c build: document the poisoned build cache; compare like with like
An entire module was missing from the APK. The app died with
ClassNotFoundException for app.echo_lot.protocol.EnrollmentLink while the
build was green, the module's jar was correct, and :app:dependencies
listed it on debugRuntimeClasspath — its code simply never reached AGP's
intermediates.

The cause was a poisoned Gradle build cache entry, which is why nothing
obvious fixed it: clean, rm -rf */build and --rerun-tasks all leave the
build cache alone. Only --no-build-cache did. Every app build made in this
session shipped without core-protocol, so enrolment, sign-in and upload
would all have crashed identically; several hours of "the tap does
nothing" were this, misread as a UI problem.

CLAUDE.md now carries the symptom, the fix, and the verification —
grepping the dex for a string literal only that module defines, because
grepping for a class *name* proves nothing: callers carry the name as a
reference whether or not the class is packaged. That false check is what
let me believe an earlier rebuild had fixed it.

Also: the enrolment dialog compared the stored endpoint against the link's
public URL, so re-enrolling with the same server announced itself as a
move to a different one. Those are deliberately different strings now that
discovery exists; the comparison uses the public name on both sides.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-02 08:01:58 +02:00
mrambossekandClaude Opus 5 bc384531e6 app: enrollment feedback beside its own button
The enrollment result was written to the same status line as everything
else, which renders at the far end of the server card below three text
fields — and on a fresh install renders nowhere at all, because that line
only appears once a run exists. So enrolling looked identical whether it
worked or not.

It has its own line now, directly under the Enroll button that caused it,
and its own state rather than sharing one with "Check server": two
actions, two results.

The server fields were also re-read the instant the button was pressed,
before the enrollment coroutine had done anything, so they showed the
previous server's values. They now refresh when the result lands, which
is the point at which there is something new to show.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-02 00:48:36 +02:00
mrambossekandClaude Opus 5 082a2314ef protocol: enrollment links carry the public name, not the endpoint
Sharing port 443 between the admin UI and the control plane forces two
hostnames — one port and one name is one certificate, and the two need
different ones. That difference had been leaking into every enrollment
link, so an operator handed out fmr-1.echo-lot.app when the thing they
and their users know is fmr.echo-lot.app.

The link now carries the public name and the app asks GET /v1/discover
where to actually connect. The endpoint is plumbing: it exists to select
a certificate, and nobody needs to see it.

Discovery hands out an address and never a pin. The pin stays in the
link. Fetching it over an ordinary TLS connection would make pinning
worth exactly what the certificate authorities are worth, and pinning is
there to survive one the operator does not control — a root injected by
corporate device management, say, which is unremarkable on the networks
this tool gets pointed at. With the pin pre-shared, an intercepted
discovery can only send a device somewhere the pin will not match: an
outage, not a compromise.

Optional on both sides. A server that does not answer, or a link that
already names the control endpoint, works unchanged — enrollment must not
start failing because a lookup did.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-02 00:13:27 +02:00
mrambossekandClaude Opus 5 a720e84411 app: one activity for deep links, and say what re-enrolling actually does
MainActivity had no launchMode, so every echolot:// link stacked a fresh
activity with its own ViewModel. The enrolment then ran in a throwaway
copy and pressing back returned to the original screen showing none of
it — silent, and indistinguishable from the link not working at all.
singleTask plus onNewIntent means the link reaches the screen already in
front of the user.

The confirmation dialog also read as nonsense when re-enrolling with the
server already configured: "already enrolled with X ... enrolling with X
replaces that". Naming one URL twice looks like a bug and buries the
consequence that does apply — the credential is replaced, the old one
stops working at once, and the device appears on the server as a second
entry next to the first, which is worth revoking afterwards.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-01 23:54:08 +02:00
mrambossekandClaude Opus 5 30d9501507 app: confirm before an enrollment link replaces an existing one
The scheme was already registered and the deep link already worked — it
enrolled on arrival, with no confirmation. Now that the web UI offers the
link as something to follow, that is one tap between a working enrollment
and a replaced one, from a page that might be showing a link minted for a
different device entirely.

Enrolling is not additive: the new credential replaces the old, and on
the previous server this device simply stops reporting. So the link is
held and the user is asked, with both server URLs named — the question is
"which server", and it cannot be answered without seeing both.

The dialog says what survives, because that is the part someone hesitates
over: uploads already on the old server stay there, runs stored on the
phone are untouched, and the device reappears on the new server as a new
device rather than carrying its history across.

Enrolling also clears the cached canary zone. It describes the old
server's deployment, and querying it against the new one would measure
somebody else's zone and file the answer under this network.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-01 23:42:37 +02:00
mrambossekandClaude Opus 5 c639d58043 adminui: make the enrolment link followable on the phone being enrolled
The link was printed as text next to an adb command, so enrolling a phone
meant getting a 200-character string from one device onto another —
which is the step that goes wrong, and it does not have to happen at all.
The app registers the echolot:// scheme, so on the phone being enrolled
following the link is the entire procedure.

The raw string and the adb form stay for every other case: reading it on
a laptop, or enrolling a device that is not the one holding the browser.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-01 23:39:55 +02:00
mrambossekandClaude Opus 5 6d042a9d89 server: control plane shares port 443 with the admin UI
Captive portals, hotel wifi and corporate firewalls routinely permit only
80 and 443 — exactly the networks this tool exists to diagnose. A control
plane on 8443 is unreachable precisely when it matters most, and it fails
as "cannot reach server", which tells the user nothing about why.

The two cannot share a certificate, so sharing the port needs two names.
The control plane is trusted by SPKI pin and uses a long-lived
self-signed certificate; a browser needs one a CA vouches for. One name
on one port is one certificate. Pinning the Let's Encrypt key instead was
considered and rejected: it survives renewal only while key reuse holds,
so a routine key rotation would brick the fleet.

One listener now picks the certificate by SNI and the handler by Host.
Both have to agree, or a client gets the pinned certificate with the
admin UI behind it.

8443 stays open. Devices enrolled before this carry that URL in their
settings, and closing it for the sake of a port number would strand every
one of them; it can go once nothing points at it.

Verified per SNI on 443: fmr.echo-lot.app serves the Let's Encrypt cert,
fmr-1.echo-lot.app serves the self-signed one whose pin is unchanged, and
/v1/profile answers 401 on the control name against 303 to the login page
on the UI name.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-01 23:25:23 +02:00
mrambossekandClaude Opus 5 4aaaa5f5d4 app: probe the server this device is enrolled with, not ours
The canary-DNS zone and the STUN host were compiled in as c.echo-lot.app
and fmr-1.echo-lot.app, so every copy of the app measured against this
particular deployment whatever server its owner had enrolled with. On
someone else's install those two tests describe our infrastructure and
report the result as a fact about their network.

The zone comes from the server's own profile, which has advertised
canary_zone all along — the app simply never read it. It is cached in
settings because the canary probe runs at device tier, before anything
has contacted the control plane, and a probe that had to make a call
first would fail on exactly the networks worth measuring. The STUN host
is derived from the configured server URL rather than stored, since a
second copy of the server's name goes stale the moment someone
re-enrolls elsewhere.

With no server configured both now report SKIPPED. StunProbe previously
would have reported FAILED on a blank host, which reads as a finding
about the network when the truth is that no packet was ever sent — the
same conflation between "measured nothing" and "measured a fault" that
the ICMPv6 finding had.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-01 23:20:52 +02:00
mrambossekandClaude Opus 5 e0428b4c84 server: enroll against fmr.echo-lot.app; mint links from the CLI
The control plane now advertises the same name the web UI answers on.
That is safe because the client authenticates by SPKI pin and explicitly
does not verify the hostname — "pin is the trust, not the name" — so no
certificate covers or needs to cover either name.

The per-host name still means something, though, and the rule it encodes
has to survive: pinning binds a client to one server's key, so fmr may be
a CNAME to exactly one host and never a multi-address service record. A
second server gets enrolled as fmr-2 explicitly, because a client that
reaches a different key does not fail over, it fails.

Minting a link was broken and had been since the authenticated admin UI
replaced the old admin API: enroll-link.sh still posted to
127.0.0.1:8444/admin/enroll-tokens, an endpoint that no longer exists on
a listener that no longer binds loopback. Rather than add a second
unauthenticated door — which is how the old one ended up briefly reachable
from the network — the binary mints its own link. Whoever can run it
against the state directory already holds every privilege the server has,
so authenticating them to themselves would be theatre.

EnrollmentURI is shared with the running server's EnrollmentLink rather
than reimplemented. Two copies of that encoding would eventually disagree,
and the failure mode is a pin that looks right and surfaces as an
inscrutable TLS error rather than as a bad pin.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-01 23:16:23 +02:00
mrambossekandClaude Opus 5 8001234e8b adminui: render the self-test instead of dumping it
The box under "Self-test" was `%+v` of a Go struct on one line, read
through a horizontal scrollbar — on a phone you could see about six words
of it, from the middle. The design pass had polished the frame around it
and left the contents a debug dump.

The report was structured the whole time: each sysctl check carries the
name, what was found, what was wanted, a severity, and a sentence
explaining why the setting matters to measurement. All of that was being
flattened into one string. It now renders as records like everything
else, with the explanation set as prose across the full row, because it
is a sentence and not a fourth column.

Two faults the render caught: the desktop row grid applied to every
readout, so the standalone summary panel was chopped into four narrow
columns and "full 1500" broke into "ful/l/150/0"; and a fixed first
column wrapped `net.ipv6.conf.all.accept_ra` mid-word. The grid is now
scoped to readouts inside a row, and the label column may grow to 18rem
before it wraps.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-01 23:04:55 +02:00
mrambossekandClaude Opus 5 332b6f3589 adminui: give the admin UI the instrument's own visual language
Echolot is the German word for an echo sounder — an instrument that emits
a ping and reads what comes back — and the UI now looks like one instead
of like a dashboard. The three big-number stat cards went first: that
layout is the stock answer for any admin page, and it told a network
engineer nothing they could act on.

Palette is a water column rather than a neutral near-black, with a
desaturated sea-green return for the accent. Verdict colours come from
the domain, so they carry meaning rather than decorate. No web fonts —
the CSP forbids loading anything and shipping font files would trade
what makes this a single pleasant binary for a typeface — so the
character comes from treatment: machine-set headings, tracked small
caps, hairlines.

The one ornament is a trace of returns across time on the runs page, one
bar per run coloured by verdict, oldest to newest. It is real data, pure
CSS, and it is what an echo sounder actually draws.

The mobile fix is the same idea rather than a fallback. Tables become
label-and-value records with dotted leaders, which is how a sounding log
prints and is easier to read on a phone than any table that scrolls
sideways. Above 46rem every row shares one grid so the columns agree by
construction; the first attempt used table-cell and each row wrapped
independently, which produces a table that does not align — a list
paying for borders.

Found by rendering it rather than reading the CSS: the trace stranded
itself against the right edge when runs were few, "Open run" broke across
two lines, equal columns wrapped device names while a one-digit count
kept a quarter of the row, and the sign-in page carried no wordmark at
all, so you arrived somewhere that never said what it was.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-01 22:45:33 +02:00
mrambossekandClaude Opus 5 621ee99b77 adminui: make the web UI usable on a phone
The header was a rigid flex row, so on a narrow screen the account name
and the sign-out button were pushed off the side of the viewport where
they could not be reached at all — not merely ugly, unusable. It wraps
now, and below 40rem the account block takes its own full-width row so a
long display name cannot crowd out the navigation.

Wide content scrolls inside its own box rather than dragging the page
sideways with it. Tables sit in an overflow-x container and <pre> is
capped at the viewport width; without that, one long self-test line or
one device table makes every other column of text unreadable, and on a
phone it is not obvious that the page has moved at all. Long opaque
strings — device ids, enrolment links — wrap anywhere rather than
insisting on a width nothing has.

Also: box-sizing on everything, stat cards that share a row instead of
each claiming the full width, and larger touch targets on buttons, where
.4rem is comfortable with a mouse and fiddly with a thumb.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-01 22:33:41 +02:00
mrambossekandClaude Opus 5 89b084338f docs: fmr is on .150/::150 only; ::2 removed and verified across a reboot
Listeners came off ::2 before the address did — the other order fails to
bind on restart — and the address stayed up until the CNAME to fmr-1 had
landed, since dropping it earlier would have broken ACME renewal for the
name the certificate is issued to.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-01 22:27:57 +02:00
mrambossekandClaude Opus 5 23ed835e00 docs: record that a revived beacon belongs in the web UI, not its own listener
Its Python service wildcard-bound 0.0.0.0:443, which is what silently
compromised the reserved measurement address. Two routes on the admin UI
would inherit the TLS and certificate already in place, need no extra
port, and pick up authentication the standalone receiver never had.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-01 22:17:55 +02:00
mrambossekandClaude Opus 5 5b5a54db7d server: reserve measurement addresses; serve the UI on both families
fmr keeps .151/::151 for measurement. Their diagnostic value is entirely
in their listening state being known: a TLS handshake that completes on a
port nothing listens on proves interception, with no competing
explanation. One stray bind turns that proof into a shrug, and nothing
about the failure is visible — the run still says the network is clean.

CheckReserved refuses to start when a listener would take one. Wildcards
are refused outright, because that is how this actually happens: every
listener defaults to ":port" and the next one added gets copied from an
existing default, claiming every address without anyone deciding to.

Reserved is not silent, though. The first version of the guard would have
refused the live config's UDP and canary-DNS binds on .151, which are
deliberate — as is STUN's RFC 5780 alternate. Reserving an address and
then forbidding the measurements that need it defeats the purpose. The
rule is narrower: no services, and never ports 80 or 443.

The adb-beacon receiver was wildcard-bound to 0.0.0.0:443, holding port
443 on every IPv4 address including the reserved one, so the IPv4
interception test had been compromised for as long as it had run. It is
disabled; restore with systemctl enable --now echolot-adb-beacon. This
also marks the guard's limit: it governs this server's listeners, and a
process outside its config can still pollute a reserved address.

The admin UI and ACME responder were single-address, which is why the UI
could only live on ::2 and why the server was reachable over IPv6 alone —
the thing that made it look nonexistent from a phone without working
IPv6. Both now take address lists like every other listener.

Verified from outside: .150/::150/::2 answer on 443 with a valid cert for
fmr.echo-lot.app, .151/::151 are closed on 80 and 443, and canary DNS is
still up on .151.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-01 22:17:29 +02:00
mrambossekandClaude Opus 5 d9ee8bc2ae app: don't report ICMPv6 silence that was never measured
The per-network attribution fix worked — the finding named rmnet_data1
instead of the IPv4-only wifi — and immediately exposed a worse problem
underneath. Cellular's result was `ok: false` because binding a socket to
it failed with EPERM, so no echo request was ever sent; the finding then
reported "IPv6 is configured, but ICMPv6 gets no reply" about a network
the app had never pinged. That is an assertion about the user's carrier
with nothing behind it.

`attempted` now travels beside `ok`, set only once sendto has returned,
and the finding requires both. Failing to bind is a fact about this app's
permissions on this device; it says nothing about the network, and the
two must not share a boolean.

Verified on hardware with a VPN active: every network fails to bind with
EPERM, nothing is sent, and no ICMPv6 finding is emitted — where the
previous build would have blamed the carrier. Recorded in build-status:
Android blocks per-network binding entirely while a VPN holds the default
route, so per-network measurement is unavailable to anyone with one
connected. That needs a deliberate answer rather than a silently green run.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-01 21:32:34 +02:00
mrambossekandClaude Opus 5 7a5004f293 server: a non-admin account can manage its own uploads
Signing in and being allowed to administer the server were the same
question: the OIDC callback refused a session outright to anyone outside
the admin group. A legitimate user could authenticate, be told what they
could not do, and be left with no way to see or delete the data their own
devices had uploaded.

They are separate questions now. Everyone who authenticates gets a
session; the admin flag rides inside the MAC'd payload, so promoting
yourself means forging a signature rather than editing a cookie, and a
role that does not parse fails closed to "user".

Pages scope themselves through visibleDevices/mayTouchRun rather than
filtering individually — per-page scoping is what the next page added
will be missing, and that failure is silent, since a listing that leaks
other people's uploads looks exactly like one that does not. Someone
else's run answers 404, not 403: a distinguishable refusal would confirm
the run exists. Revoking devices and minting enrolment tokens affect the
whole server and stay behind adminOnly at the route table, where someone
looking for who-may-do-what will actually find it.

Ownership is re-read per request instead of captured at sign-in, so
unlinking an account takes effect immediately rather than at session
expiry. Tests cover that, plus the degenerate case of an empty subject,
which must own nothing rather than everything with an empty account id.

Also: attribute the ICMPv6 finding per network. It compared "is IPv6
configured anywhere on this device" against "did any network answer",
which on a phone reports IPv6-is-broken about a network where IPv6 was
never configured. network_ref is null on every test, so the probe now
records per-network outcomes structurally rather than as prose a finding
would have to parse.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-01 21:10:38 +02:00
mrambossekandClaude Opus 5 7eaf0c4190 app: name the two half-configured IPv6 shapes, per network
v6.no_icmp_reply infers trouble from silence, which is ambiguous by
construction: a firewall dropping echo requests looks the same as a
network that cannot carry IPv6 at all. Two much stronger signals were
already sitting unread in the link snapshot, and a test device on a
Netbird tunnel surfaced both at once.

v6.route_without_address — a ::/0 route with no global address. The
router advertises itself as an IPv6 gateway while SLAAC produces nothing
usable. Hosts believe IPv6 is available and pay a connection timeout on
every dual-stack destination before falling back, which is felt as
general slowness with no packet loss to explain it.

v6.no_default_route — the mirror: a global address with nothing to route
it. A VPN installing host routes to specific destinations produces this
deliberately and it works, so a VPN transport reports it as INFO rather
than as a fault; without one it means the network handed out an address
it does not carry traffic for.

Both are read from the routing table, so neither is inferred from
silence, and both are reported per interface — "IPv6 is broken" is
useless advice when wifi is the broken one and cellular is fine.

Classification lives in core-measurement rather than the ViewModel so it
can be tested without a device; the fixtures are a real dumpsys table
(wifi advertising a route it cannot source from, working cellular, a
tunnel with two host routes) because the risk here is not bad boolean
logic but imagining shapes real networks do not produce.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-01 20:42:03 +02:00
mrambossekandClaude Fable 5 fec374abf5 findings: v6.broken claimed a cause it had no evidence for
A phone reported "IPv6 is configured but not working" while loading an IPv6-only
site over TCP perfectly well. The finding fired on one signal - ICMPv6 echo
getting no reply - at HIGH confidence. ICMPv6 echo is widely filtered on
networks where IPv6 works, so the two cases are indistinguishable from where the
app stands, and it was picking one.

Same class of error as the multi-homed downstream-loss bug: a confident
measurement of something that was not happening. Now v6.no_icmp_reply, low
severity, medium confidence, naming both explanations. Still reported, because
filtered ICMPv6 breaks Path MTU Discovery - large packets vanish instead of
being reported as too big - which is a fault in its own right.

Corroborating with a real IPv6 connection would separate the two properly, but
needs a target, which runs into the hardcoded-deployment issue already open.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-01 20:16:37 +02:00
mrambossekandClaude Fable 5 b5c8dda9a2 app: sign in to the server's identity provider
Authorization code with PKCE, a Sign in card in settings, and the echolot://auth
redirect handled next to the enrolment one - told apart by host, because one
spends a token and the other completes an authorization, and confusing them
would fail obscurely.

The detail that decides whether this survives a real phone: the PKCE verifier is
written to storage before the browser opens rather than held in memory. Handing
control to a browser backgrounds the process and Android may kill it, so the
callback arrives at a fresh one. An in-memory verifier works on a developer's
device and fails under memory pressure.

Pending state is cleared before the exchange is attempted, whatever the outcome:
it is single-use, and leaving it behind would let a later callback complete a
flow nobody started.

Also records an open issue the question about server requirements surfaced: two
probes hardcode the reference deployment, so a user with no server still sends
DNS and STUN traffic to fmr without being told. For a tool this careful about
what leaves the device, that is the wrong default.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-01 19:53:38 +02:00
mrambossekandClaude Fable 5 4e6f2da3fb runs: scope by account; app: the PKCE half of signing in
server-release / image (push) Successful in 15s
server-test / test (push) Successful in 36s
server-release / release (push) Successful in 38s
Three phones on one account now produce one history, which is the main reason to
have accounts beyond upload permission. GET /v1/runs returns the account's runs
and says how many devices contributed; fetching and deleting resolve a run id
against the caller's own devices, so an id from another account is not found
rather than fetched from wherever it happens to live.

The rule that needed stating: the empty account is never a group. Devices nobody
has signed in on are unrelated devices that share the absence of an owner, and
matching on "" would let any anonymous device read every other one's runs.
Tested, along with sibling-device access working and cross-account access not.

App side: authorization code with PKCE. The app is a public client - anything
compiled into an APK can be read out with unzip and strings - and the redirect
returns through a custom URI scheme that any app on the device may register, so
an intercepted code is a real risk. PKCE makes a stolen code worthless: it can
only be exchanged by presenting a verifier that never left the process.

A callback whose state does not match is refused before the code is spent and
before any network call, since that is exactly how someone gets a victim to
complete the attacker's sign-in.

Nothing from the IdP is retained. The ID token is used once to prove who is
signing in and then discarded; the device credential authenticates everything
afterwards. No access tokens to store, no refresh tokens to rotate.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-01 19:46:32 +02:00
mrambossekandClaude Fable 5 0eaba6150b adminui: an admin interface, behind authentication without exception
server-release / image (push) Successful in 15s
server-test / test (push) Successful in 36s
server-release / release (push) Successful in 38s
Replaces the unauthenticated admin mux. Everything but /healthz requires a
session, and that is the point: the previous arrangement relied on binding to
loopback, which worked exactly until the address changed and then failed
silently and publicly. A binding address is a deployment detail, not an access
control, and this package does not treat it as one.

Two ways in. OIDC through the confidential client, with state and PKCE - PKCE
even here, because it costs one hash and closes code interception independently
of the secret. And the break-glass password, throttled, for when the IdP is the
thing that is broken. Signing in without the admin group is refused with the
group named, because "you are not an admin" is a different problem from "your
password is wrong" and the remedy is elsewhere.

Sessions are MAC-checked cookies: HttpOnly, SameSite=Lax, Secure when TLS is on.
CSRF tokens are derived from the session rather than stored, so there is no
server-side table to keep in sync, and they are required on every state-changing
POST - SameSite already blocks cross-site posts in current browsers, but this is
the control that does not depend on the browser being current.

Server-rendered with html/template and no JavaScript: the pages are lists and
forms, and a framework would add a build step, a dependency tree and an update
treadmill to a program that has none of those. The CSP is default-src 'none'
accordingly.

Pages: overview, devices (with revocation and enrolment-link minting), uploaded
runs and a run viewer. Revocations and deletions are logged with who did them.
Runs are shown exactly as uploaded, at the privacy level their uploader chose -
nothing in the UI can un-redact one.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-01 19:31:06 +02:00
mrambossekandClaude Fable 5 7bb54e1ec8 docs: record the admin-listener exposure, and the encrypted-upload design
The incident is written down with its cause rather than just its fix: the admin
listener was built localhost-only, and that assumption travelled with it when I
changed the address. The compounding error is the one worth remembering -
checkAdminExposure verifies encryption and says nothing about authentication, so
it passed and gave false confidence. A green light on an adjacent property is
worse than no check.

Also records the encrypted-upload idea while the reasoning is fresh, including
the four consequences that decide whether it is worth building: what metadata
must stay readable (and what the UI loses if it does not), that losing the
passphrase loses the data by design, that metadata is not hidden regardless, and
that it makes a server-side anonymization floor unenforceable - which is fine,
since encryption serves the same purpose better.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-01 19:04:09 +02:00
mrambossekandClaude Fable 5 c7750fbf0b oidc: one verifier per issuer, because IdPs mint one per application
server-release / image (push) Successful in 15s
server-test / test (push) Successful in 34s
server-release / release (push) Successful in 35s
Authentik derives the issuer from the application slug, so two applications mean
two issuers - and a token's `iss` must match whoever signed it. A single pinned
issuer could therefore only ever serve one of the two clients.

So there is a verifier per issuer, and each accepts only the client belonging to
it. That is tighter than the previous arrangement as well as more general: a
token minted for the phone cannot be replayed at the admin login, and vice
versa, because they arrive at different verifiers with different audiences.

ECHOLOT_OIDC_APP_ISSUER is optional - empty means both clients share
ECHOLOT_OIDC_ISSUER, which is what IdPs with one global issuer do.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-01 18:47:34 +02:00
mrambossekandClaude Fable 5 5d7f59a66a acme: answer HTTP-01 from the server itself, on port 80
server-release / image (push) Successful in 15s
server-test / test (push) Successful in 34s
server-release / release (push) Successful in 35s
HTTP-01 always arrives on port 80 - the CA chooses the port, not the operator -
so it never collides with an admin UI on 443. The conflict only exists for
TLS-ALPN-01, which is the challenge type that does use 443.

Given that, the server keeps a permanent listener on 80 that answers challenges
from a webroot and redirects everything else to the admin UI. Same arrangement
as the webroot plugins for Apache and nginx, and better than letting the ACME
client bind 80 per renewal: nothing binds and unbinds, so a renewal cannot fail
because the port was briefly busy, and the client needs only write access to a
directory instead of the privilege to bind a low port. Port 80 also gets a use
it would want anyway.

The ACME client stays an external program. lego is also a Go library, but
importing it would put a large dependency tree into a server that deliberately
has none, and the CLI does the same job from a timer.

Tokens are validated by *shape* before any filesystem call, so traversal never
reaches the disk - a stronger guarantee than sanitising a path and trusting the
sanitiser.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-01 18:16:09 +02:00
mrambossekandClaude Fable 5 6afcb131ef admin: terminate TLS in the binary, with a certificate that reloads itself
server-test / test (push) Successful in 33s
Direct rather than behind Caddy or nginx. This binary already serves TLS for the
control plane, so it is reuse rather than new machinery; one process with one
config file is most of what makes this thing pleasant to run; and a proxy on the
box would invite someone to eventually front the control plane too, which would
break SPKI pinning because clients pin that certificate's key.

The hard part of TLS is not termination, it is renewal - so the certificate is
re-read when the files change. No reload hook to write, and none to quietly stop
working months later and be noticed only after the certificate has expired. A
torn write (renewal tools write cert and key separately) keeps the previous
certificate rather than taking the listener down.

Not applied to the control plane, on purpose: clients pin that key, so replacing
it should cost an operator a moment's thought and a restart, not happen because
a file changed. Two listeners, two different right answers.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-01 17:51:58 +02:00
mrambossekandClaude Fable 5 cd187f9ef5 config: the OIDC client secret, admin TLS, and a stop on plaintext admin
server-test / test (push) Successful in 34s
Two gaps found while answering where configuration lives.

The confidential admin client needs a secret and there was nowhere to put one -
I had added the issuer and both client ids but not the secret the admin login
actually needs. It now reads from ECHOLOT_OIDC_CLIENT_SECRET, and preferably
from ECHOLOT_OIDC_CLIENT_SECRET_FILE: a secret in the environment is readable by
anything that can see /proc/<pid>/environ and lands in every dump of the unit's
config, whereas a path is one file whose permissions an operator can reason
about. (/etc/echolot-server.env was also 0644; now 0600 on fmr.)

And the server now refuses to serve the admin UI in plaintext on a non-loopback
address. The session cookie is a bearer credential for everything the server can
do, and the OIDC authorization code arrives in a URL; in the clear, both belong
to anyone on the path - and on a globally routable address that is the internet.
A hard stop rather than a warning, because a warning in a log is not read by the
person who most needs it, and because the safe answers are cheap: bind to
loopback and tunnel, or supply a certificate. ECHOLOT_ADMIN_INSECURE=1 overrides
it, so the decision is made rather than stumbled into.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-01 17:49:26 +02:00
mrambossekandClaude Fable 5 3a4cb1c327 cli: survive the --serve transition when nobody is watching
server-release / image (push) Successful in 16s
server-test / test (push) Successful in 33s
server-release / release (push) Successful in 34s
Deploying v0.8.0 broke fmr, and the reason is a flaw I should have seen:
self-update is executed by the OLD binary, so the unit repair I put in the new
binary's updater cannot fix the very update that installs it. The unit kept its
argument-less ExecStart, the new binary answered that with usage and exit 2, and
the service went into a restart loop.

Fixed on fmr by hand, but that is not a fix for anyone else - and the whole
premise of an unattended self-update is that nobody is watching when it happens.

So: when started with no verb *and* systemd started us, the server repairs the
unit and serves anyway, loudly. systemd sets INVOCATION_ID for every service
invocation and nothing else does, so a person at a terminal still gets usage and
a non-zero exit. Marked as a one-release shim to remove once no deployment
predates --serve.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-01 17:44:33 +02:00
mrambossekandClaude Fable 5 3cdbccee18 cli: serving is an explicit verb; no arguments prints usage
server-release / image (push) Successful in 15s
server-test / test (push) Successful in 33s
server-release / release (push) Successful in 34s
Running an unfamiliar binary by name should tell you what it does, not bind a
dozen ports and start answering the internet. --serve (or --daemon) now does
that, and a bare invocation prints usage and exits 2 - non-zero on purpose, so a
service manager sees a failure rather than concluding the server ran and
finished cleanly.

The hazard this creates is worth spelling out, because it bites once and
silently: three places started the binary with no arguments - the systemd unit,
the unit template, and the Dockerfile - and --self-update replaces the binary
but never the unit. A routine update would therefore leave a service that cannot
start, discovered whenever the host next rebooted.

So the updater repairs it: after replacing the binary it appends --serve to an
ExecStart that has no flags, but only in a unit this program wrote (identified
by its description). Editing an operator's hand-written unit would be overreach;
leaving ours broken would be negligence.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-01 17:42:20 +02:00
mrambossekandClaude Fable 5 80d2092f1b oidc: accept both the app's public client and the server's confidential one
server-test / test (push) Successful in 37s
Explaining public vs confidential clients surfaced a gap in my own design: I had
assumed a single client id, but there are two clients here with genuinely
different properties.

  the Android app     public + PKCE, because an APK cannot keep a secret
  the admin UI        confidential, because the server can keep one in
                      /etc/echolot-server.env and weakening it to public buys
                      nothing

So the audience check now accepts either registered client id - and only those
two. "Any client of this issuer" would let every other application registered
with the same IdP authenticate here, which is the entire reason the check
exists. Either id alone is enough to enable sign-in, since an operator may
register only the app or only the admin UI.

The profile advertises the *app's* client id, since that is what a phone should
authorize as.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-01 17:31:33 +02:00
mrambossekandClaude Fable 5 89a5ff9139 adminauth: a break-glass local admin alongside OIDC
server-test / test (push) Successful in 44s
If the IdP is misconfigured, unreachable, or the admin group is a typo, the
operator is locked out of their own server with no way back short of editing
JSON on disk. A fallback that only matters when everything else is broken is
exactly the thing you cannot add later - by then you cannot get in to add it.

Stored as PBKDF2-HMAC-SHA256 from the standard library (Go 1.24+ has it, so no
dependency), 600k iterations, per-credential salt. A password rather than a
bearer token on purpose: a break-glass credential is the one most likely to end
up in a backup or a config-management repo, and a hash survives that where a
token does not. There is no email reset flow and should not be -
--set-admin-password on the host is the reset, and whoever can run it already
has the machine.

The password is read from stdin, never a flag, so it stays out of shell history
and the process list; piping still works for automation.

Details the tests pin, each for a reason:
  - the username is compared in constant time too, or a fast rejection is a
    timing oracle for which usernames exist;
  - the *stored* iteration count is used, so raising the constant later does not
    lock out existing passwords;
  - the throttle grows with consecutive failures but stays bounded and forgives
    after a quiet minute - a break-glass credential an attacker can lock out is
    a denial of service against the one person who needs it;
  - sessions are MAC-checked before anything in them is read, and rotating the
    secret invalidates every one at once, which is how they are revoked.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-01 17:12:54 +02:00
mrambossekandClaude Fable 5 ce6d0c2f64 oidc: the server becomes a relying party, and devices can carry an account
server-release / image (push) Successful in 15s
server-test / test (push) Successful in 35s
server-release / release (push) Successful in 34s
Echolot delegates identity to whatever IdP the operator already runs and stores
no passwords - no hashing, no reset flow, no lockout policy, and no credential
database to lose. For a tool people self-host next to other services, that is
the difference between one more service and one more thing that can leak
someone's password.

Verification is stdlib-only, matching the server's no-dependency rule. Longer
than jwt.Parse, and auditable in one sitting. The part that matters is the
algorithm allow-list: taking `alg` from the token is the classic forgery, so it
is fixed in code. Tests cover the real attacks against a genuine signer - a
self-contained IdP with real keys, because a mock that returns success proves
nothing about a verifier:

  alg=none, HS256/RS256 confusion, a payload swapped under a valid signature,
  a token addressed to another client, a token from another issuer, expired
  and future-dated tokens, and discovery that renames the issuer (which would
  otherwise have us fetch a stranger's keys believing they were the provider's).

With no admin group configured nobody is an admin. An operator who has not said
who may administer the server has not thereby said "anyone who can log in".

Device and account stay separate concepts: enrollment admits a device (operator's
token), signing in attributes it to a person (POST /v1/account/link, device
credential plus ID token - both required, neither substitutes). uploads=account
now means what it says instead of refusing everyone, and signing in does not
override uploads=off.

The profile advertises the sign-in configuration so the app can offer the button
only when there is something behind it, and drive PKCE without anyone typing an
issuer URL. A discovery failure is reported rather than hidden, so "configured
but the provider is not answering" is distinguishable from "not configured".

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-01 16:52:56 +02:00
mrambossekandClaude Fable 5 57a5ef8796 app: the stable-pseudonym switch was live at a level that pseudonymizes nothing
At `full` the anonymizer returns the document unchanged, so the salt has nothing
to act on - but the switch was enabled and looked like it did something. A
control that silently does nothing is the same class of fault as the preview
button and the archived-level label: the screen implying more than is true.

Shown disabled with the reason rather than hidden. The setting is still stored
and applies the moment the level changes, so making it vanish would hide state
that is still there; and a settings screen whose controls appear and disappear
as you touch other controls is harder to trust, not easier. The label dims with
the switch so "not active right now" reads at a glance.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-01 16:16:36 +02:00
mrambossekandClaude Fable 5 c19f382640 privacy: scrub identifiers inside raw shell output
Running the Shizuku tier for the first time uploaded every MAC address on the
local network to the server at the balanced level - fourteen of them, router and
all. The probes embed raw command output verbatim (ip neigh, ip route), which is
good evidence and also a complete household device inventory, and the anonymizer
could not see it: classification is by field name and whole-value shape, and
ip_neigh is one long string that is itself neither a MAC nor an address.

measurement-schema.md flagged raw dumps as hard to anonymize and proposed
dropping them from exports. Scrubbing is better: identifiers inside unclassified
strings are replaced in place with the same pseudonyms used elsewhere, so a MAC
appearing in both a parsed field and a raw dump still reads as one device, and
the dump stays readable - neighbour-table shape, host count, RFC1918 addresses
and vendor prefixes all survive. Dropping it would have protected the same data
by destroying the reason for collecting it.

One pass, not three: sequential passes re-process their own output. Once a MAC
became 78:9a:18:xx:yy:zz the IPv6 pattern matched it - six hex groups separated
by colons is an address - and destroyed the vendor prefix the MAC rule had just
preserved. Ordered alternation resolves each position once, MAC first.

RealDocumentTest runs the anonymizer over a captured run when ECHOLOT_REAL_RUN
points at one and fails on any surviving MAC; it self-skips otherwise so no
one's network lands in the repo. Against the document that leaked: 14 in, 0 out.

Also: the Settings preview button did nothing, reading UiState.history which is
empty until the History screen has been opened - same root cause as the "0
run(s)" count. It reads the archive now, and says when there is nothing to show.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-01 16:12:13 +02:00
mrambossekandClaude Fable 5 d04babff51 engine: live test for upstream throughput
3125 sent, 3125 counted by the server, 0% loss. The assertion that earns its
keep is received <= sent: that is what catches a counter that was never reset
between runs, which would otherwise look like a suspiciously good result.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-01 15:56:51 +02:00
mrambossekandClaude Fable 5 892e952a8e throughput: the upstream direction, counted by the only party that can
server-release / image (push) Successful in 15s
server-test / test (push) Successful in 33s
server-release / release (push) Successful in 33s
The client generates the traffic and the server counts it. No grant is involved
- the client is sending its own packets, so there is nothing to amplify - but it
does need the server's tally, because only the far end knows how much arrived.
Without that number a sender measures how fast it can transmit, which is usually
just the speed of the local NIC and is not the question being asked.

A new wire type the server counts and deliberately never answers: a reply would
double the traffic and drag the return path into a measurement that is
specifically about the outbound one.

The tally is a counter, not a list, and short-circuits before the observation
log. A five-second run at 20 Mbps is around ten thousand packets; one struct
each would turn a measurement into an allocation storm on a shared server, and
nothing needs the per-packet detail since the client holds the send-side record.
The gap between the two counts is the loss.

direction=up on the throughput action sends nothing - it zeroes the counter, so
a second run in one session measures itself instead of inheriting the first.

Same honesty rule as downstream: measures_network is false when what arrived
matches what was offered, because then the path was never the constraint.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-01 15:54:09 +02:00
mrambossekandClaude Fable 5 8646bab52d findings: adopt the registry in the app module; rename ipv6.* to v6.*
The registry was only used in core-engine. The app still emitted seven codes as
raw strings, so the registry test passed while codes lived outside it - among
them ipv6.broken, which fired on a real network and was in no registry at all.

All seven now take their code, category and severity from a registry entry, so
those three cannot disagree at a call site. Grepping for code = "..." across the
app, engine and probe modules now returns nothing.

ipv6.* -> v6.* is the third instance of the same rule being broken: they
declared Category.IPV6 while the prefix map only knows "v6", so
TestType.category("ipv6.broken") fell through to connectivity and the finding
rolled up under the wrong verdict light. The test-type registry already used v6.

Two severities reconciled rather than assumed:

  connectivity.captive_portal is medium, not high. The registry had guessed
  high; the probe emitting it had always said medium, and the probe was the
  considered value - a captive portal on hotel wifi is what should be there.
  no_internet keeps high, since nothing local fixes that.

  v6.not_offered stays info, and the registry now says why it must. Most
  networks still do not offer IPv6; a warning there lights a yellow verdict on a
  healthy network and teaches people to ignore the light.

Plus a BackHandler: the screen was a plain state variable with nothing tying it
to the back stack, so Back left the app from Settings/History instead of
returning to the run screen.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-01 15:47:52 +02:00
mrambossekandClaude Fable 5 6bba420845 app: the history row was naming the wrong document's privacy level
A row read "22 kB · full" directly beneath "uploaded to fmr", while the status
line above said the upload went as BALANCED. Both were true and they described
different documents: the row showed ArchivedRun.anonymization, which describes
the *archived* copy - deliberately unredacted, so always "full" - and the status
line described the *uploaded* copy.

Read together, that says the complete data was uploaded when a redacted copy was
sent. A privacy display that overstates what left the device is worse than none,
and telling the user what left the device is the one thing this screen is for.

The level a run was uploaded at is now recorded separately (uploaded_as) and the
row says "kept complete on this device" / "uploaded to fmr as balanced" - each
label naming the copy it belongs to.

Two more from the same screenshot:

  - Every row showed no verdict. The archive read summary.verdict; the schema
    calls it summary.overall. Silently null on every run, so the list's most
    prominent element was blank while everything else looked fine. The test
    fixture had the same wrong field name, which is why it passed.
  - The status line rendered the server's raw JSON index entry into the UI.

Verified on device: a fresh run archives with verdict "yellow" and
uploaded_as "balanced" beside anonymization "full".

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-01 15:37:05 +02:00
mrambossekandClaude Fable 5 ac6c653115 privacy: pseudonymize the whole ULA prefix, not just its tail
Found in a real uploaded run from the phone: the server held
fda1:3fb1:ff92:6696::2662 for a DNS server. The general IPv6 path keeps the
leading two groups on purpose - for a global address that preserves the ISP
allocation, which is the useful part - but for a ULA that passes through 32 of
the 40 random bits of the global ID.

A ULA looks like the v6 RFC1918 and the instinct is to treat it the same. It is
not analogous, and the difference is the point: an RFC1918 prefix is shared by
millions of networks and identifies none of them, while a ULA global ID is
random and unique to one network by construction (RFC 4193). The prefix IS the
identifier, so it was a network fingerprint surviving redaction.

Pseudonymized as a unit now, so two addresses on one ULA subnet still share a
pseudonymous prefix - "these hosts are on one network" survives, "this is that
network" does not. RFC1918 stays readable, and the contrast is what justifies
it; a test pins both halves.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-01 15:26:09 +02:00
mrambossekandClaude Fable 5 305d21f8a7 app: insets on the two newer screens, and one source for the run count
On-device verification found both.

safeDrawingPadding() was on the run screen but not on Settings or History -
they were added later and never got it - so "< Back  Settings" sat under the
status-bar clock. The same fault the run screen had already fixed, reintroduced
by new code that did not know about it.

Settings also read "0 run(s), 23 kB stored": the count came from
UiState.history, which stays empty until the History screen has been opened,
while the size read the archive directly. Two sources for one fact; the count
now reads the archive too.

Verified on a OnePlus 15 (A16): header clears the status bar, count reads
"1 run(s), 23 kB stored".

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-01 15:20:57 +02:00
mrambossekandClaude Fable 5 172afb421d privacy: fix a real leak - global IPv6 addresses were uploaded verbatim
Setting out to build the machine-readable schema, the first step was checking
whether the anonymizer covers the fields the schema declares sensitive. It did
not, and five identifying values were going out at the `balanced` level:

  networks[].link.addresses[].addr   the device's own global IPv6 address
  networks[].link.routes[].gateway   the ISP allocation
  networks[].link.dns.servers[]      the configured resolver
  private_dns_hostname               an internal hostname
  search_domains[]                   the internal domain

The settings screen describes that level as pseudonymizing addresses.

Root cause: classification keyed on field names, and the schema's actual names
were never added to the table. Every existing test passed, because each checked
a field somebody had remembered to write a case for - an unfalsifiable design
for a privacy control.

So beyond adding the names, classification now falls back to the *value* when
the name is unknown: anything shaped like an IPv4/IPv6 address or a MAC is
treated as one. Hostnames deliberately are not inferred by shape, since
train.udp_updown is indistinguishable from a domain and mangling a test type
would corrupt the document to protect nothing.

LeakTest is the guard, and is written to fail for fields nobody thought of: it
plants identifying values wherever one can occur and asserts none survive. It
also pins that RFC1918 addresses stay readable, so it cannot pass by
over-redacting. Route prefixes and :: needed care - 0.0.0.0/0 must stay itself
or a routing table becomes unreadable for no privacy gain.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-01 14:45:52 +02:00
mrambossekandClaude Fable 5 e7afc2210f findings: a registry, because the codes had already drifted
A finding code is the stable half of a result - what a dashboard groups by and
what someone greps a year of archived runs for. That only holds if a code means
exactly one thing forever, which fifteen ad-hoc string literals cannot promise.

By the time this was written the failure had happened twice:

  - Two emitters independently produced connectivity.downstream_loss and
    connectivity.loss_downstream for the same claim. Nothing objected. Anyone
    aggregating either would have silently seen half their data.
  - Two codes sat under nat.* while being declared Category.CONNECTIVITY.
    nat.udp_unreachable is not about NAT, and the prefix decides the category,
    which decides which verdict light the finding rolls up into. Renamed while
    that is still cheap.

Codes are now typed FindingSpecs carrying category and default severity;
emitters reference the spec rather than retyping the string, so a typo is a
compile error and two call sites cannot disagree about a finding's category.

docs/findings-registry.md is the contract and a test reads it, failing when the
document and the code disagree on which codes exist or how severe they are.
Documentation that drifts from its implementation is worse than none, because it
still looks authoritative. The check reads table rows only, so the prose can go
on explaining which codes were retired and why.

Closes open item 1 of measurement-schema.md section 9.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-01 14:25:54 +02:00
mrambossekandClaude Fable 5 f7701c2d2f engine: throughput is opt-in in the run config; document the work
A 5-second run at 50 Mbps moves ~30 MB. On a metered connection that is the
user's money, and a measurement tool that spends it unasked is not one people
keep installed.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-01 14:14:20 +02:00
mrambossekandClaude Fable 5 3c9af04e6f grant: replace the rate check with a token bucket
server-release / image (push) Successful in 15s
server-test / test (push) Successful in 32s
server-release / release (push) Successful in 33s
The live throughput test found it: a 3-second run delivered 104 packets and
stopped after 50 milliseconds.

The rate check exempted the first 50 ms entirely, meaning to be lenient at
startup. The effect was the opposite. A sender could dump an unbounded burst
into that free window, and the instant the check switched on it compared those
bytes against 50 ms worth of allowance and refused everything until real time
caught up. Every short test passed — downtrain sends 50 packets, big_send seven
— and every sustained send died about fifty milliseconds in.

A token bucket (allowance = burst + rate x elapsed) has no such cliff; it is
smooth from t=0. The burst is 100 ms of the allowed rate, floored at one
ordinary datagram so a single packet is never refused outright. The floor is
deliberately one datagram: at 8 kbps a 64 KB floor would be sixty-four seconds'
worth, which is precisely the instant dump the ceiling exists to prevent. The
existing rate test caught that when I first tried it, and it was right.

Second half of the same bug: callers treated any refusal as terminal. TryAllow
now says why, so a sender can pace through a transient "too fast just now" and
still stop dead on a spent budget or an expired grant.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-01 14:10:30 +02:00
mrambossekandClaude Fable 5 3333788d9e throughput: paced downstream rate, with the qualifier that makes it honest
server-release / image (push) Successful in 15s
server-test / test (push) Successful in 32s
server-release / release (push) Successful in 32s
A throughput number reports the smallest limit on the path, and the sender's own
ceiling is one of the candidates. If the server was asked for 50 Mbps and 50
Mbps arrived, the network was never the constraint and "50 Mbps" says nothing
about it. So the result always carries limited_by and measures_network, and a
finding is raised only when the path is actually implicated.

Loss is computed against the *sender's* count, not the requested rate: the
server reports what it put on the wire, and the gap is the loss. A receiver
alone cannot tell "the network dropped it" from "the sender never sent it", and
guessing turns a healthy server-side limit into a phantom network fault. The
count is stored per action, not per packet — half a million packets of structs
would turn a measurement into memory exhaustion.

Sending is paced rather than flat out. An unpaced burst measures the server's
NIC and the first queue it meets, then collapses into loss that reads as a
network fault. The schedule is absolute rather than sleep-per-packet, which
would accumulate scheduler error and drift the rate down over a ten-second run.

Throughput gets its own grant budget sized from the request, so every other
action stays bounded at 8 MiB. When the byte cap binds before the clock does,
the *duration* is shortened and reported, rather than the run being truncated
halfway: promising thirty seconds and delivering twenty-one is the same
information with a surprise attached, and it keeps "the clock ended the run" as
the normal case — the only case where the rate is a clean property of the path.

That last behaviour came out of a test that failed honestly: 30 s at 100 Mbps
needs 375 MB against a 256 MB cap.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-01 14:05:23 +02:00
mrambossekandClaude Fable 5 35744c609e docs: record frag_send and the current testing state
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-01 13:49:43 +02:00
117 changed files with 12030 additions and 307 deletions
+24 -6
View File
@@ -10,12 +10,18 @@
# releases don't trigger each other's pipelines. # releases don't trigger each other's pipelines.
# #
# Required secrets: # Required secrets:
# REGISTRY_TOKEN personal access token with read+write package scope — # REGISTRY_TOKEN personal access token with read+write package scope —
# the built-in Actions token is NOT accepted by the # the built-in Actions token is NOT accepted by the
# container registry (docker login → unauthorized). # container registry (docker login → unauthorized).
# Create: user Settings → Applications → Generate token. # Create: user Settings → Applications → Generate token.
# REGISTRY_USER optional; defaults to the pushing actor's username. # REGISTRY_USER optional; defaults to the pushing actor's username.
# The release job needs only the built-in GITHUB_TOKEN. # RELEASE_SIGNING_KEY base64 ed25519 seed that signs SHA256SUMS. Self-updating
# servers verify the signature against the public key baked
# into the binary (selfupdate.DefaultPublicKeyB64) and REFUSE
# unsigned releases, so this job hard-fails without it —
# a release nobody can install is better failed loudly here.
# Mint a pair with: go run ./cmd/release-sign -gen
# The release job otherwise needs only the built-in GITHUB_TOKEN.
name: server-release name: server-release
on: on:
@@ -44,6 +50,18 @@ jobs:
done done
(cd ../dist && sha256sum * > SHA256SUMS) (cd ../dist && sha256sum * > SHA256SUMS)
- name: Sign SHA256SUMS
working-directory: server
env:
RELEASE_SIGNING_KEY: ${{ secrets.RELEASE_SIGNING_KEY }}
run: |
[ -n "$RELEASE_SIGNING_KEY" ] || { echo "::error::secret RELEASE_SIGNING_KEY is missing — self-updating servers refuse unsigned releases, so publishing one would strand the fleet. Add it under Settings → Actions → Secrets."; exit 1; }
go run ./cmd/release-sign ../dist/SHA256SUMS
# Verify with the key baked into the binary we just built — catches a
# secret that does not match DefaultPublicKeyB64 before it ships.
PUB=$(grep -o 'DefaultPublicKeyB64 = "[^"]*"' internal/selfupdate/selfupdate.go | cut -d'"' -f2)
go run ./cmd/release-sign -verify -pub "$PUB" ../dist/SHA256SUMS
- name: Create release + attach binaries - name: Create release + attach binaries
env: env:
TOKEN: ${{ secrets.GITHUB_TOKEN }} TOKEN: ${{ secrets.GITHUB_TOKEN }}
+11
View File
@@ -145,6 +145,17 @@ First build downloads AGP/Compose/Shizuku from Google Maven + Maven Central.
Shizuku, **toggle Wireless debugging off/on** — Shizuku keeps running (separate process), a fresh Shizuku, **toggle Wireless debugging off/on** — Shizuku keeps running (separate process), a fresh
port + mDNS record appear, and the beacon/connector recover. Plan the Shizuku-tier dev loop port + mDNS record appear, and the beacon/connector recover. Plan the Shizuku-tier dev loop
around this (or USB, if ever available). around this (or USB, if ever available).
- **A poisoned Gradle *build cache* entry can silently drop a whole module from the APK.**
Symptom: the app dies with `ClassNotFoundException` for a class that plainly exists, while the
build is green and `./gradlew :app:dependencies` lists the module on `debugRuntimeClasspath`.
The module's own jar is correct; its code simply never reaches AGP's intermediates. `clean`,
`rm -rf */build` and `--rerun-tasks` all fail to fix it, because **none of them touch the build
cache** — look for `compileKotlin FROM-CACHE` in the log. Fix: rebuild with `--no-build-cache`.
Verify by grepping the APK's dex for a string literal that only that module defines; grepping for
a *class name* proves nothing, because callers carry the name as a reference whether or not the
class is packaged:
`unzip -o -q app-debug.apk "classes*.dex" && grep -a "pin-sha256:" *.dex`
Suspect this whenever a runtime failure contradicts a successful build.
- **Empty-jar race with the IDE.** VSCodium's Java/Kotlin extension runs its own Gradle daemon on - **Empty-jar race with the IDE.** VSCodium's Java/Kotlin extension runs its own Gradle daemon on
the same project; when it overlaps a CLI build, a module's `build/libs/*.jar` can end up the same project; when it overlaps a CLI build, a module's `build/libs/*.jar` can end up
containing only a manifest, and Gradle then considers `jar` up-to-date. Dependent modules fail containing only a manifest, and Gradle then considers `jar` up-to-date. Dependent modules fail
+621 -3
View File
@@ -211,8 +211,8 @@ Two collection-loop gotchas found while driving the phone over USB:
2. ~~If `trace.errqueue_reachable` = PARTIAL, add a C-over-JNI errqueue shim.~~ **Retired** — 2. ~~If `trace.errqueue_reachable` = PARTIAL, add a C-over-JNI errqueue shim.~~ **Retired** —
SUPPORTED on both known devices; `traceroute.udp4` reads real hops via `Os.recvmsg` + SUPPORTED on both known devices; `traceroute.udp4` reads real hops via `Os.recvmsg` +
`StructMsghdr` reflection, so no `:native` module is needed. `StructMsghdr` reflection, so no `:native` module is needed.
3. Start the Go server skeleton (enrollment + profile + sessions + UDP echo with observation 3. ~~Start the Go server skeleton per probe-protocol.md.~~ **Shipped** — live on fmr since
blocks + canary-DNS reference records) per probe-protocol.md. v0.2.0 (2026-07-31); see the server sections below.
4. Fold confirmed capabilities into the production `core-probe` / `core-shizuku` modules. 4. Fold confirmed capabilities into the production `core-probe` / `core-shizuku` modules.
## Production probe server — LIVE on dedicated VM "fmr" (2026-07-31) ## Production probe server — LIVE on dedicated VM "fmr" (2026-07-31)
@@ -222,7 +222,7 @@ SSH only — verified untouched by the daemon (explicit multi-address binds, no
Control: fmr-1:8443 (SPKI pin `zRV9qkiLnRexAeh4RrSfJzbPWO+U/2Oj2/NVM/KfXlg=`, verified Control: fmr-1:8443 (SPKI pin `zRV9qkiLnRexAeh4RrSfJzbPWO+U/2Oj2/NVM/KfXlg=`, verified
externally over v4+v6). UDP data plane on all four service addresses :8442 — the second IP is externally over v4+v6). UDP data plane on all four service addresses :8442 — the second IP is
the stun-5780 substrate. Daily randomized self-update timer installed (checksum-verified the stun-5780 substrate. Daily randomized self-update timer installed (checksum-verified
against SHA256SUMS; signature verification still TODO before treating the source as untrusted). against SHA256SUMS; signature verification landed 2026-08-02 — see "Release signing" below).
Host config in `/etc/echolot-server.env`. SSH access for sessions: `ssh claude-echolot`. Host config in `/etc/echolot-server.env`. SSH access for sessions: `ssh claude-echolot`.
## Server v0.3.0 — STUN + TCP echo + observations + actions (2026-07-31) ## Server v0.3.0 — STUN + TCP echo + observations + actions (2026-07-31)
@@ -728,3 +728,621 @@ tenfold asymmetry that a round-trip measurement cannot see at all.
10 unit tests on the arithmetic (a wrong denominator here does not crash, it produces a plausible 10 unit tests on the arithmetic (a wrong denominator here does not crash, it produces a plausible
number pointing at the wrong half of the network) plus the live correlation check. number pointing at the wrong half of the network) plus the live correlation check.
### frag_send: crafted IP fragments, so *ordering* is testable (server-v0.6.0, 2026-08-01)
`big_send` with `df=false` answers one question — do fragments get through. It cannot answer the
more interesting one, because the kernel always emits fragments in order, first one first.
The classic middlebox fault is exactly about that ordering. Only the **first** fragment carries the
UDP header, and therefore the ports; a stateful firewall or NAT that has not seen it has no flow to
match the rest against, and many simply drop them. That is invisible to every in-order test, and in
the field it looks like "large DNS answers fail on this network" or "the tunnel breaks when the MTU
drops" — it works until the network reorders, then fails intermittently, which is the hardest kind
of fault to chase.
So the server builds the fragments itself (raw socket, `IP_HDRINCL`) and controls their order:
`in_order` (baseline), `reversed` (last fragment first), `first_last` (first fragment held back
250 ms). The datagram is assembled and **signed whole** before being cut up, so what the client
reassembles is indistinguishable from an ordinary packet — otherwise the test would be measuring
our sender rather than the path. New test type `mtu.frag_ordering`; findings
`mtu.fragments_blocked` and `mtu.fragment_reorder_sensitive`.
Two details that would otherwise produce confidently wrong answers:
- **The UDP checksum is computed, not left zero.** Zero is legal in IPv4 and would be less code,
but zero-checksum datagrams are dropped by some middleboxes — and that drop would be recorded as
a fragmentation failure, which is the wrong conclusion entirely.
- **Fragment offsets are in 8-byte units**, so non-final fragments are rounded down to a multiple
of 8. A 100-byte fragment is not an error; it is a datagram no host will ever reassemble.
`frag-send` is advertised only when a raw socket can actually be opened — checked by opening one,
because a permission model has more ways to say no (userns, seccomp, LSM) than a capability bit has
to say yes. fmr runs as root with `cap_net_raw` in its bounding set, so it is available there.
Fragment ordering runs only after `mtu.frag_delivery` shows fragments arrive at all; otherwise the
three orderings would each report "not delivered" and read as three faults instead of one.
The header arithmetic is unit-tested (reassembly coverage with no gaps or double-delivery, MF
flags, shared IP ID, 8-byte offsets, checksum verification over odd and even lengths). Because the
code is `//go:build linux`, the tests are **cross-compiled and run on fmr** — there is no Go
toolchain there, so `go test -c` plus scp is the loop.
Live against fmr: 4 fragments per burst, and all three orderings reassembled — a healthy path, and
the baseline against which a mobile network will be interesting.
### Testing state (2026-08-01)
Six live tests against fmr, all green, no device involved: `LiveServerTest`, `LiveMeasurement`,
`LiveGranted`, `LiveDownstream`, `LiveUpload`, `LiveCompat`, `LiveEnrollment`. Plus 74 client unit
tests and the full Go suite. Everything in the last several entries is verified from the PC; the
app's UI (settings, history, deep-link enrollment) and `mtu.pmtud_up` remain device-only.
### throughput: a rate, plus the qualifier that makes it a measurement (server-v0.6.1 … v0.6.2)
A throughput test reports the *smallest* limit on the path — and the sender's own ceiling is one of
the candidates. If the server is asked for 50 Mbps and 50 Mbps arrives, the network was never the
constraint and "50 Mbps" says nothing about it. So `perf.throughput_udp` always carries
`limited_by` (duration | budget | rate | send_error) and `measures_network`, and a finding is
raised only when the path is actually implicated. The live run against fmr reports 20 Mbit/s with
`measures_network: false`, which is the correct and useful answer.
Loss is computed against the **sender's own count**, fetched from the observations API, not against
the requested rate. A receiver alone cannot tell "the network dropped it" from "the sender never
sent it", and guessing turns a healthy server-side limit into a phantom network fault. The server
keeps one summary per action rather than per-packet records — a ten-second run at 50 Mbps is half a
million packets, and a struct each would turn a measurement into memory exhaustion.
Sending is **paced**, on an absolute schedule. Unpaced would measure the server's NIC and the first
queue it meets, then collapse into loss that reads as a network fault; sleep-per-packet would
accumulate scheduler error and drift the rate down over a ten-second run.
Throughput gets its own grant budget sized from the request, so every *other* action stays bounded
at 8 MiB. When the byte cap binds before the clock does, the **duration is shortened and reported**
rather than the run being truncated: promising thirty seconds and delivering twenty-one is the same
information with a surprise attached, and it keeps "the clock ended the run" as the normal case —
the only case where the rate is a clean property of the path. That behaviour came out of a test
that failed honestly (30 s at 100 Mbps needs 375 MB against a 256 MB cap).
It is **opt-in** in the run config, default off. A 5-second run at 50 Mbps moves ~30 MB; on a
metered mobile connection that is the user's money, and a tool that spends it without being asked
is not one people keep installed.
#### The bug the live test found
The first live run delivered 104 packets and stopped after 50 ms. The grant's rate check exempted
the first 50 ms entirely, meaning to be lenient at startup — the effect was the opposite. A sender
could dump an unbounded burst into that free window, and the instant the check switched on it
compared those bytes against 50 ms worth of allowance and refused everything until real time caught
up. **Every short test passed** (downtrain sends 50 packets, big_send seven); every sustained send
died fifty milliseconds in.
Replaced with a token bucket (`allowance = burst + rate × elapsed`), which is smooth from t=0.
The burst is 100 ms of the allowed rate, floored at one ordinary datagram — deliberately one, since
at 8 kbps a 64 KB floor is sixty-four seconds' worth, exactly the instant dump the ceiling exists to
prevent. The pre-existing rate test caught that when I first tried the generous floor, and it was
right to. Second half of the same bug: callers treated *any* refusal as terminal, so `TryAllow` now
says why — a sender paces through a transient "too fast just now" and still stops dead on a spent
budget or an expired grant. Both halves are pinned by regression tests.
### Findings registry (2026-08-01)
Closes open item 1 of measurement-schema.md §9. A finding code is the stable, machine-readable half
of a result — what a dashboard groups by and what someone greps a year of archived runs for — and
that only holds if a code means exactly one thing forever. Ad-hoc string literals at fifteen call
sites cannot promise that, and by the time the registry was written the failure had already
happened.
**Two emitters had independently produced `connectivity.downstream_loss` and
`connectivity.loss_downstream` for the same claim**, and nothing anywhere objected. Anyone
aggregating either one would have silently seen half their data. Merged into
`connectivity.loss_downstream`, paired with `loss_upstream` so the two directions read as a set.
**Two codes were also renamed out of `nat.*`.** `nat.udp_unreachable` is not about NAT — it means
no replies came back — but the prefix determines the category, and the category determines which
verdict light the finding rolls up into (§7.3). A `nat.*` code landing under *connectivity* is not
a naming quibble; it changes which light turns red. Cheap to fix now, a breaking change later.
Codes are now declared as typed `FindingSpec`s carrying their category and default severity, and
emitters reference the spec instead of retyping the string — so a typo is a compile error and two
call sites cannot disagree about a finding's category.
`docs/findings-registry.md` is the contract, and a test reads it: it fails when the document and
the registry have codes the other lacks, or when a severity differs. Documentation that drifts from
its implementation is worse than none, because it still looks authoritative. The check scopes
itself to table rows, so the prose can keep explaining which codes were retired and why.
Six tests: uniqueness, declared-vs-listed, prefix↔category agreement, naming convention, a
word-order-anagram check (the shape the duplication actually took), and the document agreement.
### A real privacy leak, found by starting on the machine-readable schema (2026-08-01)
The intent was `measurement.schema.json` (§8's promised companion). The first step — checking
whether the anonymizer actually covers the fields the schema declares as sensitive — found that it
did not, so that became the work.
**At the `balanced` level, five identifying values were being uploaded verbatim:**
| value | field | why it matters |
|---|---|---|
| `2001:…::150` | `networks[].link.addresses[].addr` | the device's own global IPv6 address — a strong, geolocatable device identifier |
| `2a02:…::1` | `networks[].link.routes[].gateway` | identifies the ISP allocation |
| `203.0.113.77` | `networks[].link.dns.servers[]` | the configured resolver |
| `nas.example.lan` | `private_dns_hostname` | an internal hostname |
| `example.lan` | `search_domains[]` | the internal domain |
The settings screen describes that level as pseudonymizing addresses. It was not.
**Root cause:** classification keyed on field *names*, and the schema's actual names (`addr`,
`gateway`, `dst`, `servers`, `search_domains`, `private_dns_hostname`) had never been added to the
table. Not a subtle bug — just an unfalsifiable design. The existing tests all passed, because each
one checked a field somebody had remembered to write a case for.
**Two fixes, one of them structural:**
1. The missing names were added.
2. More importantly, a **shape-based backstop**: when a field name is unrecognised, the *value* is
inspected, and anything shaped like an IPv4/IPv6 address or a MAC is treated as one. A name
table can only protect fields someone thought of, which is precisely the wrong property for a
privacy control. Hostnames are deliberately *not* inferred by shape — `train.udp_updown` is
indistinguishable from a domain, and mangling a test type would corrupt the document to protect
nothing.
`LeakTest` is the new guard and is written to fail for fields nobody has considered: it plants
identifying values wherever one can actually occur and asserts none survive, rather than checking
a list of known cases. It also pins that RFC1918 addresses still come through readable, so the
test cannot pass by over-redacting everything.
Route prefixes and the unspecified address needed care in the transform: `0.0.0.0/0` and `::/0`
must stay themselves, or a routing table becomes unreadable for no privacy gain.
**Still outstanding:** `measurement.schema.json` itself. Worth noting what this episode implies for
it — much of a document's payload lives in `evidence`/`metrics`/`params`, which are per-test-type
`JsonObject` by design and therefore *outside* any schema. A schema-driven anonymizer would have
less coverage there than the name-plus-shape one now does, so the schema should be built for
validation and external tooling, not as a replacement for the classifier.
### ULA prefixes are pseudonymized whole (2026-08-01)
Spotted in a real uploaded run from the phone: the server had
`fda1:3fb1:ff92:6696::2662` for a DNS server. The general IPv6 path preserves the leading two
groups (deliberately — for a global address that keeps the ISP allocation, which is the
diagnostically useful part), and for a ULA that passed through **32 of the 40 random bits** of the
global ID.
ULA looks like the v6 equivalent of RFC1918 and the instinct is to treat it the same. That
reasoning does not carry over, and the difference is the whole point: an RFC1918 prefix is shared
by millions of networks and identifies none of them, while a ULA global ID is random and unique to
one network by construction (RFC 4193). The prefix *is* the identifier — it is a network
fingerprint that was surviving redaction.
Now pseudonymized as a unit, so two addresses on the same ULA subnet still land on the same
pseudonymous prefix: "these hosts are on one network" survives, "this is *that* network" does not.
Three tests, one of which uses the exact value observed on the wire.
Worth recording as a reasoning trap: I had originally raised this as "ULA should probably be kept
verbatim, like RFC1918, for consistency". The surface analogy pointed the wrong way, and the
correct answer was the opposite.
### Registry adopted everywhere; v6 findings renamed; Back works (2026-08-01)
The findings registry was only adopted in `core-engine`. The app module still emitted seven codes
as raw strings, so the registry test passed while codes existed outside it — including
`ipv6.broken`, which fired on a real network and was in no registry at all.
All seven now reference registry entries for code, category and severity, so those three cannot
disagree at a call site. A grep for `code = "…"` across the app, engine and probe modules returns
nothing.
**`ipv6.*` → `v6.*`.** The third instance of rule 1: they declared `Category.IPV6` while the prefix
map only knows `v6`, so `TestType.category("ipv6.broken")` fell through to *connectivity* and the
finding rolled up under the wrong verdict light. The test-type registry already used `v6.`.
Two severities reconciled while merging:
- `connectivity.captive_portal` is **medium**, not high. The registry had guessed high; the probe
that emits it had always said medium, and the probe was the considered value — a captive portal
on hotel wifi is what should be there, and logging in clears it. `connectivity.no_internet` is
the high one, because nothing the user does locally fixes that.
- `v6.not_offered` is **info, and the registry says it must stay info**. Most networks still do not
offer IPv6 and that is not a fault; a warning here lights a yellow verdict on a healthy network,
which teaches people to ignore the light.
Also: a `BackHandler` now returns from Settings/History to the run screen. The screen was a plain
state variable with nothing connecting it to the back stack, so the system Back gesture left the
app entirely. Enabled only when there is somewhere to go back to, so Back still exits from the run
screen.
### Upstream throughput (server-v0.6.3, 2026-08-01)
The mirror of the downstream case: the client generates the traffic and the server counts it. No
grant is involved — the client is sending its own packets, so there is nothing to amplify — but it
does need the server's tally, because **only the far end knows how much arrived**. Without that
number a sender measures how fast it can *transmit*, which is usually just the speed of the local
NIC and is a different question from the one being asked.
`TYPE_THROUGHPUT_UP` (0x0F) is counted and deliberately **never answered**: a reply would double
the traffic and drag the return path into a measurement that is specifically about the outbound
one.
The tally is a counter, not a list, and short-circuits **before** the observation log. A
five-second run at 20 Mbps is around ten thousand packets; one struct each would turn a
measurement into an allocation storm on a shared server, and nothing needs the per-packet detail
since the client holds the send-side record. The gap between the two counts is the loss.
`direction=up` on the throughput action sends nothing — it zeroes the counter, so a second run in
one session measures itself rather than inheriting the first one's packets. The live test asserts
`received <= sent`, which is what catches a counter that was never reset.
Live against fmr: **3125 sent, 3125 counted, 0 % loss, 10.0 Mbit/s** at a 10 Mbit/s request, with
`measures_network: false` — correct, since what arrived matched what was offered, so the path was
never the constraint.
### Raw shell dumps leaked the whole LAN (2026-08-01)
Found by running the Shizuku shell tier for the first time. The tier works — `tiers.shizuku: true`,
`exec_path: UserService` (so the UserService binds on the OnePlus, as recorded), `runs_as
shell(2000)`, 7/7 commands — and the run promptly uploaded **every MAC address on the local
network** to fmr at the `balanced` level: router, phones, whatever else was on the wifi. Fourteen
of them.
The probes embed raw command output verbatim (`ip neigh`, `ip route`, `id`), which is genuinely
good evidence and also a complete household device inventory. The anonymizer could not see it:
classification is by field name and by whole-value shape, and `ip_neigh` is one long string that is
itself neither a MAC nor an address. measurement-schema.md §9 item 2 had flagged raw dumps as "hard
to anonymize" and proposed dropping them from exports; nothing enforced either.
**Scrubbing beats dropping.** Identifiers inside any unclassified string are now replaced in place,
using the same pseudonyms as everywhere else — so a MAC that appears both in a parsed field and in
a raw dump still reads as one device. The dump stays readable and auditable: you can still see the
neighbour table's shape, the host count, RFC1918 addresses and vendor prefixes. Dropping the
evidence would have protected the same data while destroying the reason for collecting it.
Two implementation notes worth keeping:
- **One pass, not three.** Sequential passes re-process their own output: once a MAC became
`78:9a:18:xx:yy:zz`, the IPv6 pattern matched it — six hex groups separated by colons *is* an
address — and destroyed the vendor prefix the MAC rule had just preserved. Ordered alternation
resolves each position once, MAC first.
- The patterns are conservative on purpose. A missed address gets caught by another rule or not at
all; an over-eager one mangles timestamps and version strings, corrupting evidence to protect
nothing.
`RealDocumentTest` runs the anonymizer over a captured run when `ECHOLOT_REAL_RUN` points at one,
and fails on any MAC that survives. It self-skips otherwise, so no one's network is committed to the
repo. Against the actual leaked document: **14 MACs in, 0 surviving.**
Also fixed: the Settings *Preview what an upload would send* button did nothing. It read
`UiState.history`, which is empty until the History screen has been opened — the same root cause as
the "0 run(s)" count. It now reads the archive directly, and says so when there is nothing to
preview rather than silently ignoring the tap.
### Security: the admin listener was publicly exposed for ~15 minutes (2026-08-01)
Moving the admin listener to `[::2]:443` for the UI exposed `/admin/enroll-tokens` and
`/admin/selftest` to the internet **with no authentication**. Anyone who could reach
`fmr.echo-lot.app` could mint enrolment tokens.
The listener was designed localhost-only — its own flag help says *"keep localhost"* — and that
assumption travelled with it when the address changed. The compounding error: `checkAdminExposure`,
added the same day, verifies **encryption** and says nothing about **authentication**. It passed,
and a green light on an adjacent property is worse than no check, because it invites you to stop
looking.
Closed by returning to loopback (the TLS and ACME work is retained, just not exposed). All 68 device
enrolments matched the timestamps of test runs, so there is no evidence of abuse — but the window
existed on a freshly published hostname and absence cannot be proven. 39 unused enrolment tokens
were purged, since any could have been minted by someone else and they cost nothing to replace, and
63 test devices removed.
**The admin listener does not become reachable again until it authenticates.** That reorders the UI
work: auth on the listener first, everything else after.
### Open: encrypted uploads, where the operator cannot read the data
Not built. Recorded because the shape is decided by a few early choices, and the current design
happens to leave the door open.
The goal: hand someone an account, let them upload, and be unable to read what they uploaded.
Sketch: a random per-account **master key**, generated on the first device and wrapped under a
key derived from a passphrase (PBKDF2-HMAC-SHA256 — stdlib on both sides). The wrapped key is
stored server-side as an opaque blob, so a new device signs in, fetches it, and unwraps locally;
the server never sees either key. Runs are encrypted client-side with AES-256-GCM, fresh nonce per
run. All of this is stdlib in Go and `javax.crypto` in Kotlin — no dependency either side.
Four consequences that decide whether it is worth it:
1. **What stays readable determines what the UI can do.** The server builds its index by *parsing*
the document — verdict, finding count, started_at. An opaque payload means the client supplies
that metadata or the index disappears, and with it retention-by-verdict and any "runs with
findings" view. The honest version supplies only run id, timestamp and size, and moves the rest
client-side.
2. **Lose the passphrase, lose the data.** That is the feature working, and also the support
burden. It needs a recovery code printed at setup, not a reset flow — there is nothing to reset.
3. **Metadata is not hidden.** The operator still sees which account uploaded, when, how often and
how large. "Cannot see it" is about content, not existence, and saying otherwise would oversell.
4. **It makes `min_anonymization` unenforceable** — a server cannot check a level it cannot read.
That is not a conflict so much as a redundancy: the anonymization floor exists to protect the
user from the operator, and encryption does that better. The two should not both be demanded of
one upload.
What keeps this possible: uploads are already stored byte-for-byte as received, and every index
field is derived in one function (`runs.Put`). The thing to avoid is admin features that *require*
reading content — those would have to be unbuilt later.
### App sign-in, and an undisclosed dependency it surfaced (2026-08-01)
The app can now sign in to the server's identity provider: authorization code with PKCE, a
`Sign in` card in settings, and the `echolot://auth` redirect handled alongside the enrolment one
(told apart by host, since one spends a token and the other completes an authorization).
The detail that decides whether this works on a real phone: **the PKCE verifier is written to
storage before the browser opens**, not held in memory. Handing control to a browser backgrounds
the process and Android may kill it; the callback then arrives at a fresh process. An in-memory
verifier works on a developer's device and fails under memory pressure, which is the worst way for
a sign-in to break.
Nothing from the IdP is retained. The ID token proves who is signing in, once, and the device
credential authenticates everything after — no access tokens stored, no refresh tokens rotated.
**A server remains entirely optional.** All eight probes are device-tier; `serverConfigured` gates
only upload and the account. But answering that question exposed something worth fixing: two probes
hardcode the reference deployment —
```kotlin
DnsCanaryProbe(canaryZone = "c.echo-lot.app", ...) // "Hardcoded to the reference deployment"
StunProbe(serverHost = "fmr-1.echo-lot.app")
```
so a user with no server of their own still sends DNS and STUN traffic to fmr without being told.
For a tool that goes to this much trouble over what leaves the device, an undisclosed dependency on
a third party's infrastructure is the wrong default. It should prefer the configured server, and be
explicit when there is none. **Closed** — both probes take the enrolled server from settings
(canary zone learned from the profile, cleared on re-enroll) and report themselves SKIPPED with
the reason when none is configured.
### v6.broken was a false positive waiting to happen (2026-08-01)
A phone could not open `https://fmr.echo-lot.app` while loading the same server by IP literal
perfectly well. Two things came out of chasing it.
**The admin UI is IPv6-only, by consequence rather than intent.** `fmr.echo-lot.app` has an AAAA
and no A record — verified identical at Cloudflare, Google and Quad9, so DNS itself is healthy.
That follows from reserving all four measurement addresses for testing, which left only `::2` for
management, and `::2` has no IPv4 counterpart. Any client without working IPv6 sees an unreachable
admin interface — a poor property for the interface you reach *from the networks you are debugging*.
**And the app's own `v6.broken` finding was unsound.** It fired on exactly one signal — ICMPv6 echo
getting no reply — with `Confidence.HIGH`. ICMPv6 echo is widely filtered on networks where IPv6
works fine, which is precisely what that phone demonstrated: no ICMPv6 replies, working IPv6 TCP.
The finding asserted a cause it had no evidence for, which is the same class of error as the
multi-homed `100 % downstream loss` earlier: a confident measurement of something that was not
happening.
Now `v6.no_icmp_reply`, severity low, confidence medium, and the text names *both* explanations
instead of choosing one. It is still worth reporting, because filtered ICMPv6 breaks Path MTU
Discovery — large packets vanish rather than being reported as too big — which is a real fault even
when IPv6 works.
The proper fix is corroboration: attempt a real IPv6 connection and only call it broken when that
fails too. That needs a target, which runs into the hardcoded-reference-deployment issue already
open above. **Both closed 2026-08-02** — see "Corroborated IPv6 findings" below.
## Per-network probing is blocked while a VPN is up (2026-08-01)
`Network.bindSocket()` fails with `EPERM` for every underlying network when a VPN holds the
default route — verified on the OnePlus 15 with Netbird active: `Binding socket to network 101
failed: EPERM` for both cellular and wifi. This is Android preventing VPN leaks, not a bug to work
around, and it means the whole per-network measurement approach is unavailable to any user with a
VPN connected. Worth deciding deliberately rather than discovering per report:
- The run currently succeeds and simply measures nothing per network. Honest, but silent — the
document records `attempted: false` and the UI says green.
- A user with a corporate VPN permanently on would get a green run that measured almost nothing.
Options are to detect the VPN and say so plainly ("this network cannot be measured while a VPN is
active"), to measure the tunnel itself as the network under test, or both. **Decided and built
2026-08-02**: say so plainly, everywhere the run is read — see "Constrained runs" below.
Related: `icmp.ping6` now records `attempted` alongside `ok` per network, because collapsing them
made the app report "IPv6 is configured, but ICMPv6 gets no reply" about an interface it had never
succeeded in sending on — a claim about the user's carrier with no evidence behind it.
## Reserved measurement addresses, and the web UI on both families (2026-08-01)
fmr has two IPv4 (.150/.151) and three IPv6 (::150/::151/::2) addresses. `.150`/`::150` now carry
the services; `.151`/`::151` are reserved for measurement, declared in `ECHOLOT_RESERVED_ADDRS`.
Reserved does **not** mean silent. The UDP data plane, the canary DNS and STUN's RFC 5780 alternate
all belong there — reserving an address and then forbidding the measurements that need it would
defeat the purpose. What must never appear is a service, and above all not ports 80 or 443: a
handshake completing on a port known not to be listening is what proves interception, and that
proof survives exactly as long as nothing binds those ports. `config.CheckReserved` enforces it at
startup, refusing wildcard binds outright (every listener defaults to `:port`, so the next one added
will claim reserved addresses without anyone deciding to).
The first version of the guard was too strict and the live config caught it: it would have refused
the existing UDP and DNS binds on `.151`. The rule is about services and web ports, not about
listening at all.
**The adb-beacon receiver was wildcard-bound to `0.0.0.0:443`**, occupying port 443 on every IPv4
address including the reserved one — so the IPv4 interception test had been compromised for as long
as it had been running, silently. It is now `systemctl disable --now echolot-adb-beacon`; restore
with `systemctl enable --now`. Note what this implies: the guard covers this server's own listeners,
and a stray process outside its config can still pollute a reserved address. A startup probe that
*verifies* 80/443 are actually free on the reserved addresses would be a stronger guarantee than
checking our own configuration — built 2026-08-02 (`selftest.ReservedWebPortsFree`, fatal at
startup when anything is listening there).
The admin UI and the ACME responder now take comma-separated addresses like every other listener;
they were single-address, which is why the UI could only ever live on `::2`. It serves on `.150:443` and
`[::150]:443`; sshd on `.150:2322` and `[::150]:2322`.
`::2` is gone entirely — unbound, then removed from `/etc/systemd/network/ext.network`. The
transition kept it bound throughout and dropped it only after the CNAME landed, because removing it
first would have broken both the UI and ACME renewal for the very name the certificate is issued
to. Listeners came off before the address did, in that order, or the services would have failed to
bind on restart.
Verified after a full reboot: `fmr.echo-lot.app` answers 200 over both families, `.151`/`::151` are
closed on 80 and 443, canary DNS is still up on `.151`, and neither `::2` nor the beacon returns.
(`echolot-server` is `After=network-online.target` with `Restart=on-failure`, which is what makes
binding specific addresses safe across a boot — a wildcard bind would not have needed it, and that
is the trade for the reserved addresses being meaningful.)
The point of all this: `fmr.echo-lot.app` gained an A record, so the server stopped being reachable
only over IPv6 — which is what made it unreachable from a phone with no working IPv6, presenting as
"this host does not exist" in two different browsers.
### If the beacon comes back, it belongs in the web UI
Not as a separate listener. The receiver being its own Python service on `0.0.0.0:443` is exactly
what silently compromised the reserved address, and a second process racing for a port is a
recurring problem rather than a one-off: whoever loses the race simply fails to start, and on a
reboot which one that is comes down to unit ordering.
Folding it in costs little and settles several things at once. It would be two routes on the admin
UI (`POST` the observed adb port, `GET /apk` for the staged build), behind the TLS the UI already
terminates and the certificate it already renews, with no extra port and no wildcard. It also gets
authentication for free — the current receiver accepts a port report from anyone who can reach it,
which is tolerable for a dev tool on a trusted network and not something to keep once it lives
beside an admin session.
The one thing that changes on the device side is that the POST becomes HTTPS. That is a real
certificate rather than a self-signed one, so it costs a URL scheme rather than any trust plumbing.
## The control plane shares port 443 (2026-08-01)
`fmr-1.echo-lot.app:443` is the control plane, `fmr.echo-lot.app:443` the admin UI, both on
`.150`/`::150`, one listener, selected by SNI for the certificate and by `Host` for the handler.
The reason is not tidiness, it is reachability. Captive portals, hotel wifi and corporate firewalls
routinely permit only 80 and 443 — which is exactly the population of networks this tool exists to
diagnose. A control plane on 8443 is unreachable precisely when it matters most, and it fails as
"cannot reach server", which tells the user nothing.
They cannot share a certificate, which is why this needs two names. The control plane is trusted by
SPKI pin and so uses a long-lived self-signed certificate; a browser needs one a CA vouches for.
One name on one port is one certificate, so the port can only be shared by splitting the names.
Pinning the Let's Encrypt key instead was considered and rejected: it survives renewal only while
key reuse holds, so a routine key rotation would brick the whole fleet.
Verified per SNI on 443: `fmr.echo-lot.app` serves `issuer=Let's Encrypt`, `fmr-1.echo-lot.app`
serves the self-signed cert whose pin is unchanged (`zRV9…Xlg=`), `/v1/profile` answers 401 on the
control name and 303 to the login page on the UI name.
**8443 stays open.** Devices enrolled before this carry that URL in their settings, and closing it
for the sake of a port number would strand every one of them. It can go once no enrolled device
still points at it — not before.
The rule from the naming change still binds: `fmr` may be a CNAME to exactly one host and never a
multi-address record, because a pinned client that reaches a different key does not fail over.
## Constrained runs: a VPN'd run now says so, everywhere (2026-08-02, app 0.2.1)
The measurement schema gained a top-level `constraints` block (§3) and the app now fills it.
`ConstraintDetector` (core-probe) runs before any probe: one throwaway `Network.bindSocket()` per
non-VPN network, plus a transport check for an active VPN. The result lands in three places, and
all three are deliberate:
- **`run.constraints`** — for machines. A server aggregating thousands of runs can now separate
"measured a healthy network" from "measured almost nothing through a tunnel"; the shapes were
identical before.
- **A `measurement.vpn_constrained` finding** — for the person reading this run, naming the
interfaces that went unmeasured. A constrained run with a quiet findings list still reads as
"nothing wrong here".
- **The §7.3 verdict** — `Verdicts.derive` takes the constraints and returns INCONCLUSIVE
outright for a per-network-blocked run, whatever the category lights say; the run screen shows
an amber "Measured through a VPN" banner above the verdict so INCONCLUSIVE reads as the OS
refusing, not the app failing.
Detection is one bind per network rather than parsing per-test `attempted:false` breadcrumbs, so
it cannot drift when probe evidence formats change.
## Corroborated IPv6 findings: v6.broken is back, with evidence (2026-08-02, app 0.2.1)
The new `V6ConnectProbe` (test type `v6.brokenness`) attempts a real TCP connection over IPv6 to
the configured server's :443, per network that *claims* IPv6 (global address or v6 default
route) — IPv4-only networks are not attempted, since their failure is by design and would
manufacture the exact false positive this exists to kill. The finding derivation is now three-way:
- ICMPv6 silent, TCP works → `v6.no_icmp_reply` at **high** confidence, retitled "ICMPv6 is
filtered here — IPv6 itself works" (still reported: filtered ICMPv6 breaks PMTUD).
- ICMPv6 silent, TCP fails too → **`v6.broken`** (high severity, reinstated in the registry +
findings-registry.md): two independent transports silent on a network advertising IPv6.
- No corroboration (no server configured, or the connect never got as far as sending) → the
two-explanation `v6.no_icmp_reply` at medium confidence, unchanged.
Like STUN and the canary, the probe SKIPs honestly when no server is configured — corroboration
is a benefit of enrollment, not a reason to borrow fmr.
## Server: reserved 80/443 verified against the OS, and signed releases (2026-08-02)
**Reserved-address startup probe.** `serve()` now proves 80/443 are actually free on every
`ECHOLOT_RESERVED_ADDRS` address before starting: a throwaway bind per port
(`selftest.ReservedWebPortsFree`), fatal on EADDRINUSE with the offending address named — the
check `CheckReserved` cannot do, because a stray process outside our config (the adb-beacon
receiver on `0.0.0.0:443` was exactly that) is invisible to configuration checks. Bind errors
that are not "in use" (typo'd address, address not on this host) warn instead of refusing —
they are config problems, not pollution.
**Release signing.** Self-update now trusts a signature, not a host. CI signs `SHA256SUMS` with
an ed25519 key (`relsign` package, `cmd/release-sign`) and the updater refuses any release whose
`SHA256SUMS.sig` is missing or does not verify against the public key baked into the binary
(`selfupdate.DefaultPublicKeyB64`; operators with their own pipeline override via
`ECHOLOT_SELF_UPDATE_PUBKEY`). The private key exists in exactly two places: the Gitea Actions
secret `RELEASE_SIGNING_KEY`, and the offline original on the dev PC at
`~/.echolot/release-signing-key`. It is deliberately NOT on fmr and NOT in the repo — a
compromised release host can withhold updates but no longer inject one. CI hard-fails when the
secret is missing (an unsigned release would strand every verifying server) and cross-checks the
signature against the key in the source it just built.
**ACTION REQUIRED before the next `server-v*` tag:** add the Gitea repo secret
`RELEASE_SIGNING_KEY` (Settings → Actions → Secrets) with the contents of
`~/.echolot/release-signing-key` from the dev PC. Ordering is safe: the currently deployed
v0.3.x updater does not verify, so it will happily install the first signed release; every
release after that is verified. **Done 2026-08-02** — the secret is in place. (The "v0.3.x"
above should read "the currently deployed release": deployments had moved on to v0.9.x by the
time signing landed; the point — the deployed updater predates verification and will accept the
first signed release — is unchanged.)
## Prober fold: traceroute.udp4 and the mDNS inventory go production (2026-08-02, app 0.2.2)
The two highest-value validated capabilities moved from the prober into `core-probe`:
- **`traceroute.udp4`** (`TracerouteProbe`): UDP traceroute reading ICMP time-exceeded off the
socket error queue via `Os.recvmsg(MSG_ERRQUEUE)` through the reflection facade — no root, no
raw socket, no JNI, ~250 ms for six hops. Emits the schema's `TracerouteEvidence` (rtt in ns).
`OsAbi` came with it, including the measured fact that `Os.getsockoptInt` exists on neither
known device, so PMTU must always be read from the errqueue (`ee_info`), never
`getsockopt(IP_MTU)`. The load-bearing line survived the port: EAGAIN out of the reflected
`recvmsg` means "queue empty", not failure.
- **`local.mdns_inventory`** (`MdnsInventoryProbe`): MulticastLock + NSD discovery, the service
inventory that doubles as the VLAN-leakage detector. Both hardware lessons kept: the
`_services._dns-sd._udp.` meta-query returns 0 beside live services on both devices (so the
concrete types are the measurement and the meta-query result is itself evidence), and the
listen window is 10 s because 4 s missed services.
Still to fold, in order: the Shizuku dump *parsers* (the raw `link.ip_monitor` captures already
hold two divergent vendor formats that could feed `link.ra_source` and `sec.arp_watch`);
`multinetwork.request_and_bind` (extend ConstraintDetector to *request* transports rather than
only probing present ones); `peer.ble_advertise` (needs three new permissions and a peer mode to
exist first).
## Server v0.9.2: trains, real TTL/DSCP/ECN, rate limits (2026-08-02)
The spec-vs-implementation gap audit closed its top items; protocol_version 1.0.0 → 1.0.1
(additive — below 1.0.0 the minor is the breaking axis, and nothing here breaks an old client):
- **Upstream trains** (§3.2, types 0x03/0x04/0x05): per-train bounded columnar buffer (8192
rows, head kept on overflow with `Truncated` set — mirrors the schema's `evidence_truncated`
honesty), TRAIN_REPORT split across ≤1200-byte datagrams, grant-free with the §3.4 argument
spelled out (a 17-byte report row answers a ≥36-byte HMAC-valid packet). Unknown train id
gets a zero-row report: "nothing arrived" is an answer. Also surfaced as `udp.trains` in the
observations API.
- **Real TTL/DSCP/ECN observation** (§3.3): the read loop is `ReadMsgUDPAddrPort` with
IP_RECVTTL/IP_RECVTOS/IPV6_RECVHOPLIMIT/IPV6_RECVTCLASS cmsgs on Linux; `0xFF` stays the
"not observed" sentinel elsewhere. This unblocks `sec.dscp_ecn_survival` both directions,
paired with the new `dscp` parameter on `downtrain` (validated 063, refused not clamped,
`dscp_applied` in the response).
- **Rate limiting** (§2.5, was entirely absent): token buckets keyed per credential AND per
source IP; 429 + Retry-After on session/action creation (`/v1/profile` stays ungated), silent
drop on the data plane — charged after the HMAC gate so a spoofed flood cannot drain a
victim's budget, before the replay window so a dropped seq stays usable. UDP ceilings default
above the largest legitimate run (a 200 Mbps throughput test), because a rate limit that
clips a real measurement produces a confidently wrong number.
- **`action_id` in every granted packet** (§5/§9): payload bytes [8:16] across all granted
types, so overlapping actions are attributable. Verified the deployed Kotlin client parses
only ECHO_RESP and MTU_ACK payloads, so the reshuffle strands nobody.
- **Canary log retention**: the stated 24 h privacy default is now enforced
(`ECHOLOT_DNS_LOG_RETENTION_H`), where before the log was time-unbounded.
- **`POST /admin/enroll-tokens`** now answers the spec's JSON shape under content negotiation;
the README's curl works as documented.
- Spec §2.3 registry gained `downtrain` and `tcp-echo`, which the server had been advertising
as strings a conformant client must ignore.
Client-side counterparts still to build: sending 0x03 trains + parsing 0x05 reports
(`train.udp_updown`), and passing `dscp` on downtrain actions.
+126
View File
@@ -0,0 +1,126 @@
<!--
SPDX-FileCopyrightText: 2026 Echolot contributors
SPDX-License-Identifier: CC-BY-4.0
-->
# Echolot findings registry
Closes open item 1 of `measurement-schema.md` §9.
A **finding code** is the stable, machine-readable half of a result. The prose around it changes
freely; the code is what a dashboard groups by, what a diff between two runs keys on, and what
someone greps a year of archived runs for. That only works if a code means exactly one thing,
forever.
This document is the contract. It is kept in step with
`echolot-app/core-measurement/.../FindingRegistry.kt` by a test that fails when either side has a
code the other does not — a registry that drifts from its documentation is worse than none,
because it looks authoritative.
## Rules
1. **The prefix determines the category**, and the category determines which verdict light the
finding rolls up into (§7.3). A `nat.*` code appearing under *connectivity* is not a naming
quibble; it changes which light turns red. Two codes were renamed from `nat.*` to
`connectivity.*` for exactly this reason.
2. **One code per concept.** Two emitters independently produced `connectivity.downstream_loss`
and `connectivity.loss_downstream` for the same claim before this registry existed. Anyone
aggregating either would have silently seen half their data.
3. **Codes are declared, not typed.** Emitters reference a `FindingSpec`, so a typo is a compile
error and no two call sites can disagree about a finding's category or default severity.
4. **Severity in the registry is the default.** An emitter may escalate for a specific run; it may
not quietly reclassify the finding in general.
5. **Say what is ruled out**, where that is the useful half. "Loss upstream" is worth far more
when it also states that the return path is clean, because that halves where to look next.
6. **Renaming a code is a breaking change** once runs are archived at scale. Before 1.0 it is
cheap; after, it needs an alias and a deprecation window.
## Registry
### connectivity
| code | severity | means | rules out |
|---|---|---|---|
| `connectivity.udp_unreachable` | high | No UDP echo replies came back from the server at all. | — |
| `connectivity.udp_unreachable_upstream` | high | The server received none of the probes, so traffic is dropped on the way out. | The return path: nothing arrived to be replied to. |
| `connectivity.udp_loss` | medium | A large fraction of round-trip probes were lost, direction unknown. | — |
| `connectivity.loss_upstream` | medium | Probes were lost on the way to the server. | The return path: replies came back for everything that arrived. |
| `connectivity.loss_downstream` | medium | Packets were lost on the way back from the server. | The outbound path: the server received what it was answering. |
| `connectivity.downstream_blocked` | high | Server-initiated packets never arrive, although round trips work. | Basic reachability: the path forwards replies, just not unsolicited traffic. |
| `connectivity.downstream_reorder` | low | Downstream packets arrive in a different order than they were sent. | — |
| `connectivity.captive_portal` | medium | A captive portal is intercepting connectivity checks. | — |
| `connectivity.no_internet` | high | Android's own connectivity checks fail on this network. | — |
### mtu
| code | severity | means | rules out |
|---|---|---|---|
| `mtu.reduced_downstream` | low | The downstream path MTU is below the usual 1500 bytes. | — |
| `mtu.downstream_blackhole` | medium | Datagrams above the path MTU are dropped downstream, fragmented or not. | — |
| `mtu.fragments_blocked` | medium | IP fragments do not reach this device even when sent in order. | — |
| `mtu.fragment_reorder_sensitive` | low | Fragments are delivered in order but dropped when reordered or delayed. | Fragmentation itself: in-order fragments arrive fine. |
### nat
| code | severity | means | rules out |
|---|---|---|---|
| `nat.udp_rebinding` | medium | A NAT remapped the UDP source port mid-flow. | — |
| `nat.symmetric` | medium | The NAT assigns a different external port per destination. | — |
### perf
| code | severity | means | rules out |
|---|---|---|---|
| `perf.throughput_no_delivery` | high | No throughput traffic arrived, although the server sent it. | — |
| `perf.throughput_below_offered` | low | Less throughput arrived than the server sent for the whole run. | — |
### dns
| code | severity | means | rules out |
|---|---|---|---|
| `dns.answer_rewritten` | high | A resolver returned an answer that differs from the authoritative record. | — |
| `dns.authoritative_unreachable` | medium | The canary zone's authoritative server could not be reached. | — |
### v6
The prefix is `v6.`, matching the test-type registry (`v6.brokenness`, `v6.happy_eyeballs`, …).
These were `ipv6.*` while declaring `Category.IPV6`; since the prefix map only knows `v6`, they
rolled up under *connectivity* instead — the third occurrence of rule 1 being broken.
| code | severity | means | rules out |
|---|---|---|---|
| `dns.search_domain_unanswered` | high | The network advertises a DNS search domain that its own server does not answer for. | A fault on this device: the same server answers ordinary names normally. |
| `dns.system_resolver_broken` | high | The network's DNS server answers, but this device cannot resolve names through it. | A network fault: the server replied to a query sent from this device. |
| `measurement.vpn_constrained` | info | A VPN was active, so the networks underneath it could not be measured. | Nothing — this run says little about the underlying network either way. |
| `v6.no_default_route` | medium | The device has a global IPv6 address but no IPv6 default route. | Guesswork: this is read from the routing table, not inferred from silence. |
| `v6.route_without_address` | medium | The network advertises an IPv6 default route but the device has no global IPv6 address. | A working IPv6 setup: SLAAC did not produce a usable address on this link. |
| `v6.no_icmp_reply` | low | IPv6 is configured but ICMPv6 echo gets no reply. | Nothing on its own: IPv6 may work fine with ICMP filtered. |
| `v6.broken` | high | IPv6 is advertised on this network but carries no traffic. | ICMP filtering as the benign explanation: a TCP connection over IPv6 failed too. |
| `v6.not_offered` | info | This network does not offer IPv6. | — |
`v6.no_icmp_reply` was `v6.broken` until a phone reported it while loading an IPv6-only site over
TCP perfectly well. The only evidence behind it is ICMPv6 echo, which is widely filtered on
networks where IPv6 works — so the finding now states what was observed and names both
explanations instead of choosing one. It is still worth reporting: filtered ICMPv6 breaks Path MTU
Discovery.
`v6.broken` returned once that corroboration existed: the `v6.brokenness` test attempts a real TCP
connection over IPv6 to the configured server, and only when *both* transports fail on a network
that advertises IPv6 is the brokenness claim made — at high severity, because every dual-stack
destination pays a timeout before falling back to IPv4. When the TCP connect *succeeds*,
`v6.no_icmp_reply` is emitted at high confidence instead, now able to say plainly that ICMPv6 is
filtered while IPv6 works. With no server configured there is no corroboration target and the
two-explanation `v6.no_icmp_reply` stands unchanged.
`v6.not_offered` is **info and must stay info**. Most networks still do not offer IPv6 and that is
not a fault; reporting it as a warning lights a yellow verdict on a healthy network, which teaches
people to ignore the light — the one thing a diagnostic must never do.
## Adding a finding
1. Add a `FindingSpec` to `FindingRegistry`, and to its `all` list.
2. Add the row here, under the section its prefix names.
3. Emit it with `finding(FindingRegistry.YOUR_CODE, …)`.
The registry test checks 1 and 2 agree, that every prefix maps to the category it claims, and that
no two entries share a code.
+17 -2
View File
@@ -52,12 +52,26 @@ Export encoding: UTF-8 JSON, gzip for files (`.echolot.json.gz`), share intent u
}, },
"tiers": { "app": true, "shizuku": true, "root": false }, "tiers": { "app": true, "shizuku": true, "root": false },
"profiles_used": ["profile-uuid", ...], "profiles_used": ["profile-uuid", ...],
"constraints": {
"vpn_active": true,
"per_network_blocked": true,
"unmeasured_networks": ["net-0", "net-1"]
},
"notes": "free-text user annotation" "notes": "free-text user annotation"
} }
``` ```
`tiers` records what was *available*; each test records what it *used*. `tiers` records what was *available*; each test records what it *used*.
`constraints` records what was *prevented*. A constrained run is neither a failed run nor a normal
one, and the distinction has to survive into the data: a run taken through a VPN has the same shape
and the same green verdict as a clean run of a healthy network, so without this a reader — or a
server aggregating thousands of them — cannot tell that almost nothing was measured. The known case
is `per_network_blocked`: Android refuses `Network.bindSocket()` on the underlying networks while a
VPN holds the default route, so every per-network test measures the tunnel or nothing at all, and
any conclusion about the link underneath is unfounded. Consumers should treat findings from a
constrained run as scoped to what was actually reachable, and `unmeasured_networks` names the rest.
## 4. `networks[]` — one entry per Android `Network` in play ## 4. `networks[]` — one entry per Android `Network` in play
A run may exercise several networks simultaneously (Wi-Fi + cellular + USB ethernet). Everything is a snapshot at run start; a `changes[]` list captures mid-run deltas. A run may exercise several networks simultaneously (Wi-Fi + cellular + USB ethernet). Everything is a snapshot at run start; a `changes[]` list captures mid-run deltas.
@@ -269,7 +283,7 @@ The JSON Schema (machine-readable companion, `measurement.schema.json`, generate
| type | example fields | v2 anonymizer transform | | type | example fields | v2 anonymizer transform |
|---|---|---| |---|---|---|
| `ip4`, `ip6` | addresses, routes, hops, DNS answers | prefix-preserving pseudonymization, consistent per document; well-known/reserved ranges kept verbatim | | `ip4`, `ip6` | addresses, routes, hops, DNS answers | prefix-preserving pseudonymization, consistent per document; well-known/reserved ranges kept verbatim. **Exception: ULA (`fc00::/7`) has its whole prefix pseudonymized as a unit.** It resembles RFC1918 but is not analogous: a ULA global ID is 40 random bits, unique to one network by construction (RFC 4193), so the prefix *is* the identifier, whereas `192.168.0.0/16` is shared by millions of networks and identifies none. Pseudonymizing it as a unit keeps "these hosts are on one subnet" while dropping "this is that subnet". |
| `mac`, `bssid` | wifi, arp_watch | OUI kept, NIC part pseudonymized | | `mac`, `bssid` | wifi, arp_watch | OUI kept, NIC part pseudonymized |
| `fqdn` | DNS names, reverse lookups | per-label pseudonyms, public-suffix kept | | `fqdn` | DNS names, reverse lookups | per-label pseudonyms, public-suffix kept |
| `ssid` | wifi | pseudonym | | `ssid` | wifi | pseudonym |
@@ -279,7 +293,8 @@ Free-text fields (`notes`, `error.detail`, dump excerpts from Shizuku parsers) c
## 9. Open items ## 9. Open items
1. Findings registry document — start alongside the first implemented tests. 1. ~~Findings registry document~~ — done: `findings-registry.md`, kept in step with
`FindingRegistry.kt` by a test that fails when the two disagree.
2. Whether Shizuku raw-dump excerpts (dumpsys/ip output) are embedded in `evidence` verbatim (auditable, but large and hard to anonymize) or parsed-only with an optional "attach raw dumps" toggle. Proposal: toggle, default on for local archive, default off for export. 2. Whether Shizuku raw-dump excerpts (dumpsys/ip output) are embedded in `evidence` verbatim (auditable, but large and hard to anonymize) or parsed-only with an optional "attach raw dumps" toggle. Proposal: toggle, default on for local archive, default off for export.
3. Peer-mode documents: each device produces its own run; the coordinator embeds the peer's findings summary and cross-references by `run.id`. Full merge format deferred. 3. Peer-mode documents: each device produces its own run; the coordinator embeds the peer's findings summary and cross-references by `run.id`. Full merge format deferred.
4. Size guardrails: soft cap 20 MB uncompressed per run; trains beyond that downsample evidence (keep aggregates + first/last N + all anomalies) and record `"evidence_truncated": true`. 4. Size guardrails: soft cap 20 MB uncompressed per run; trains beyond that downsample evidence (keep aggregates + first/last N + all anomalies) and record `"evidence_truncated": true`.
+31 -1
View File
@@ -84,7 +84,12 @@ The app re-fetches the profile at the start of every run (falling back to the ca
### 2.3 Capabilities (v1 registry) ### 2.3 Capabilities (v1 registry)
`udp-probe`, `stun-basic`, `stun-5780`, `canary-dns`, `recursive-dns`, `connect-back`, `delayed-echo`, `big-send`, `frag-send`, `tls-echo`, `http-echo`, `throughput`, `ntp`. A server omits what it can't offer (e.g. `stun-5780` without a second IP degrades to `stun-basic`). Clients must skip, and record as `unsupported`, any test whose capability is absent. Unknown capability strings are ignored. `udp-probe`, `stun-basic`, `stun-5780`, `canary-dns`, `recursive-dns`, `connect-back`, `delayed-echo`, `big-send`, `frag-send`, `tls-echo`, `http-echo`, `throughput`, `ntp`, plus:
- `downtrain` — server-sent downstream trains via the §5 `downtrain` action. Upstream trains need no capability of their own: they are plain client-sent data-plane packets and ride `udp-probe`.
- `tcp-echo` — the plain-TCP echo endpoint (§4); `tls-echo` is its ALPN variant on the same port.
A server omits what it can't offer (e.g. `stun-5780` without a second IP degrades to `stun-basic`). Clients must skip, and record as `unsupported`, any test whose capability is absent. Unknown capability strings are ignored.
### 2.4 Sessions ### 2.4 Sessions
@@ -111,6 +116,31 @@ DELETE /v1/sessions/{id}
Per-credential and per-source-IP token buckets on: session creation, actions, UDP packets, bytes. `429` on control plane; silent drop on data plane (probes must tolerate loss anyway). All reflected/generated traffic goes **only** to the session's observed source address (or, for connect-back, the source address of the session-creating request). Data-plane responses to unauthenticated packets are never larger than the request (§3.4). Per-credential and per-source-IP token buckets on: session creation, actions, UDP packets, bytes. `429` on control plane; silent drop on data plane (probes must tolerate loss anyway). All reflected/generated traffic goes **only** to the session's observed source address (or, for connect-back, the source address of the session-creating request). Data-plane responses to unauthenticated packets are never larger than the request (§3.4).
### 2.2 `GET /v1/discover` — where the control plane lives
Unauthenticated, and says almost nothing: the control-plane URL and the server's display name.
```json
{ "control_url": "https://probe.example.net", "name": "example" }
```
It exists so an enrollment link can carry the name a person recognises while the app still connects
to the name that selects the pinned certificate. When a server shares port 443 between its admin UI
and its control plane, those must be different hostnames — one port and one name is one certificate,
and the two need different ones (a browser-trusted certificate, and a long-lived self-signed one the
client pins). Without discovery, the difference leaks into every enrollment link an operator hands
out.
**It hands out an address, never a pin.** The pin travels in the link itself. Serving it here would
reduce pinning to whatever the certificate authorities are worth, and pinning exists precisely to
survive one the operator does not control — a root injected by corporate device management, for
instance, which is unremarkable on the networks this tool is pointed at. Because the pin is
pre-shared, an intercepted discovery response can only send a device to the wrong host, where the
pin will not match: an outage, not a compromise.
Clients treat it as optional. A server that does not answer, or a link that already names the
control endpoint, works unchanged — enrollment must not begin failing because a lookup did.
## 3. UDP probe protocol ## 3. UDP probe protocol
### 3.1 Packet header (fixed 32 bytes, network byte order) ### 3.1 Packet header (fixed 32 bytes, network byte order)
+11 -3
View File
@@ -15,7 +15,7 @@ plugins {
// //
// major*1_000_000 + minor*10_000 + patch*10 leaves room for 9 patch-level rebuilds (the trailing // major*1_000_000 + minor*10_000 + patch*10 leaves room for 9 patch-level rebuilds (the trailing
// digit) without disturbing the mapping, and stays inside the 2_100_000_000 ceiling until major 2100. // digit) without disturbing the mapping, and stays inside the 2_100_000_000 ceiling until major 2100.
val appVersionName = "0.2.0" val appVersionName = "0.2.2"
fun versionCodeOf(semver: String): Int { fun versionCodeOf(semver: String): Int {
val (major, minor, patch) = semver.substringBefore('-').split(".").map(String::toInt) val (major, minor, patch) = semver.substringBefore('-').split(".").map(String::toInt)
@@ -34,8 +34,16 @@ android {
versionName = appVersionName versionName = appVersionName
// Automation: `adb shell am start -n app.echo_lot.app/.MainActivity --ez autorun true` // Automation: `adb shell am start -n app.echo_lot.app/.MainActivity --ez autorun true`
// runs a measurement immediately and POSTs the report here (dev collection endpoint). // runs a measurement immediately and POSTs the report here (dev collection endpoint).
buildConfigField("String", "REPORT_UPLOAD_URL", "\"http://89.185.109.150:443/report\"") // Empty: the collection endpoint this pointed at was the adb-beacon receiver, which held
buildConfigField("String", "REPORT_UPLOAD_SECRET", "\"D4OmG5gGJsElqVVbtYIZbR\"") // 0.0.0.0:443 in cleartext. That service is gone and echolot-server owns 443 with TLS, so
// posting plaintext there now fails as "client sent an HTTP request to an HTTPS server" —
// an alarming error for a debugging convenience that is no longer needed, since autorun
// reports are read straight off the device with `run-as cat`.
//
// Deliberately not repointed at /v1/runs. That is the consent-gated upload, and a
// debugging shortcut must not be able to satisfy it by accident.
buildConfigField("String", "REPORT_UPLOAD_URL", "\"\"")
buildConfigField("String", "REPORT_UPLOAD_SECRET", "\"\"")
// The bare SemVer, without the debug build's "-dev" suffix stripped away by the server's // The bare SemVer, without the debug build's "-dev" suffix stripped away by the server's
// parser anyway — sent to servers so they can apply their compatibility window. // parser anyway — sent to servers so they can apply their compatibility window.
buildConfigField("String", "APP_SEMVER", "\"$appVersionName\"") buildConfigField("String", "APP_SEMVER", "\"$appVersionName\"")
+13 -1
View File
@@ -22,7 +22,8 @@
<activity <activity
android:name=".MainActivity" android:name=".MainActivity"
android:exported="true"> android:exported="true"
android:launchMode="singleTask">
<intent-filter> <intent-filter>
<action android:name="android.intent.action.MAIN" /> <action android:name="android.intent.action.MAIN" />
<category android:name="android.intent.category.LAUNCHER" /> <category android:name="android.intent.category.LAUNCHER" />
@@ -39,6 +40,17 @@
<category android:name="android.intent.category.BROWSABLE" /> <category android:name="android.intent.category.BROWSABLE" />
<data android:scheme="echolot" android:host="enroll" /> <data android:scheme="echolot" android:host="enroll" />
</intent-filter> </intent-filter>
<!--
Sign-in redirect. The browser hands the authorization code back through this, which
is exactly why the flow uses PKCE: any app may register this scheme, so the code
alone must not be enough to complete a sign-in.
-->
<intent-filter android:autoVerify="false">
<action android:name="android.intent.action.VIEW" />
<category android:name="android.intent.category.DEFAULT" />
<category android:name="android.intent.category.BROWSABLE" />
<data android:scheme="echolot" android:host="auth" />
</intent-filter>
</activity> </activity>
<provider <provider
@@ -0,0 +1,125 @@
// SPDX-FileCopyrightText: 2026 Echolot contributors
// SPDX-License-Identifier: GPL-3.0-or-later
package app.echo_lot.app
import app.echo_lot.protocol.AuthInfo
import app.echo_lot.protocol.ControlClient
import app.echo_lot.protocol.OidcLogin
import kotlinx.serialization.json.Json
import kotlinx.serialization.json.jsonObject
import kotlinx.serialization.json.jsonPrimitive
/**
* Signing in to the configured server's identity provider.
*
* The awkward part of a browser-based sign-in on Android is that the app is not running while it
* happens. Handing control to a browser puts this process in the background, where it may be
* killed at any moment; the callback then arrives at a fresh process with none of the state the
* exchange needs. So the PKCE verifier and state are written to storage before the browser opens,
* not held in memory — an in-memory value works on a developer's device and fails on a phone under
* memory pressure, which is the worst way for this to break.
*
* Nothing from the identity provider is kept afterwards. The ID token proves who is signing in,
* once; the device credential authenticates everything from then on.
*/
class Account(private val settings: Settings) {
private val json = Json { ignoreUnknownKeys = true }
sealed interface SignInStart {
/** Open this in a browser. */
data class Browser(val url: String) : SignInStart
data class Unavailable(val reason: String) : SignInStart
}
/** Fetches the server's auth configuration and builds the authorization URL. */
fun begin(): SignInStart {
if (!settings.serverConfigured) {
return SignInStart.Unavailable(
"Enrol with a server first — sign-in belongs to the server's identity provider."
)
}
val auth = runCatching { client().profile(settings.serverCredential).auth }.getOrNull()
?: return SignInStart.Unavailable("Could not reach the server to ask how to sign in.")
auth.discoveryError?.let {
// The distinction matters: "the operator configured an IdP that is not answering" is
// their problem to fix, and is not the same as "this server has no accounts".
return SignInStart.Unavailable("The server's identity provider is not responding: $it")
}
if (!auth.enabled) {
return SignInStart.Unavailable("This server does not offer accounts.")
}
return try {
val pending = OidcLogin.begin(auth)
// Written before the browser opens, because after that this process may not survive.
settings.pendingVerifier = pending.verifier
settings.pendingState = pending.state
SignInStart.Browser(pending.authorizationUrl)
} catch (t: Throwable) {
SignInStart.Unavailable(t.message ?: "Could not start sign-in.")
}
}
/**
* Completes sign-in from the `echolot://auth` redirect.
*
* Blocking; callers run it off the main thread.
*/
fun complete(callbackUri: String): String {
val verifier = settings.pendingVerifier
val state = settings.pendingState
// Cleared first, whatever happens next: these are single-use, and leaving them behind
// would let a later callback be completed against a flow nobody started.
settings.clearPendingAuth()
if (verifier.isBlank() || state.isBlank()) {
return "That sign-in did not start on this device."
}
return try {
val auth = client().profile(settings.serverCredential).auth
val idToken = OidcLogin.complete(
auth, OidcLogin.Pending("", verifier, state), callbackUri,
)
val reply = client().linkAccount(settings.serverCredential, idToken)
val o = json.parseToJsonElement(reply).jsonObject
val name = o["display_name"]?.jsonPrimitive?.content ?: "signed in"
settings.accountName = name
settings.accountId = o["account_id"]?.jsonPrimitive?.content ?: ""
val admin = o["admin"]?.jsonPrimitive?.content == "true"
"Signed in as $name" + if (admin) " (administrator)" else ""
} catch (e: OidcLogin.LoginFailed) {
e.message ?: "Sign-in failed."
} catch (t: Throwable) {
"Sign-in failed: ${t.message ?: t.javaClass.simpleName}"
}
}
/** Signs out. The device stays enrolled — signing out should not cost an enrolment. */
fun signOut(): String = try {
client().unlinkAccount(settings.serverCredential)
settings.accountName = ""
settings.accountId = ""
"Signed out. This device is still enrolled."
} catch (t: Throwable) {
"Could not sign out: ${t.message ?: t.javaClass.simpleName}"
}
/** Asks the server who it thinks is signed in, so the UI is not trusting stale local state. */
fun refresh(): String? = runCatching {
val o = json.parseToJsonElement(client().accountStatus(settings.serverCredential)).jsonObject
val signedIn = o["signed_in"]?.jsonPrimitive?.content == "true"
settings.accountName = if (signedIn) {
o["display_name"]?.jsonPrimitive?.content ?: ""
} else {
""
}
settings.accountName.takeIf { it.isNotBlank() }
}.getOrNull()
private fun client() = ControlClient(
settings.serverUrl, setOf(settings.serverPin), BuildConfig.APP_SEMVER,
fallbackAddrs = settings.serverAddrList(),
)
}
@@ -4,6 +4,7 @@
package app.echo_lot.app package app.echo_lot.app
import androidx.compose.foundation.layout.Arrangement import androidx.compose.foundation.layout.Arrangement
import androidx.compose.foundation.layout.safeDrawingPadding
import androidx.compose.foundation.layout.Column import androidx.compose.foundation.layout.Column
import androidx.compose.foundation.layout.Row import androidx.compose.foundation.layout.Row
import androidx.compose.foundation.layout.fillMaxWidth import androidx.compose.foundation.layout.fillMaxWidth
@@ -39,7 +40,7 @@ fun HistoryScreen(
onDelete: (String) -> Unit, onDelete: (String) -> Unit,
onBack: () -> Unit, onBack: () -> Unit,
) { ) {
Column(Modifier.fillMaxWidth().padding(16.dp), verticalArrangement = Arrangement.spacedBy(10.dp)) { Column(Modifier.fillMaxWidth().safeDrawingPadding().padding(16.dp), verticalArrangement = Arrangement.spacedBy(10.dp)) {
Row(verticalAlignment = Alignment.CenterVertically) { Row(verticalAlignment = Alignment.CenterVertically) {
TextButton(onClick = onBack) { Text(" Back") } TextButton(onClick = onBack) { Text(" Back") }
Text("History", style = MaterialTheme.typography.titleLarge) Text("History", style = MaterialTheme.typography.titleLarge)
@@ -71,12 +72,20 @@ fun HistoryScreen(
) )
} }
Text( Text(
"${r.findingCount} finding(s) · ${r.sizeBytes / 1024} kB · ${r.anonymization}", "${r.findingCount} finding(s) · ${r.sizeBytes / 1024} kB · " +
"kept complete on this device",
style = MaterialTheme.typography.bodySmall, style = MaterialTheme.typography.bodySmall,
) )
// The upload line names the level the upload was made at, not the
// archive's. They describe different documents, and showing the archive's
// level here claimed more had left the device than actually did.
Text( Text(
if (r.uploaded) "uploaded to ${r.uploadedTo ?: "a server"}" if (r.uploaded) {
else "on this device only", "uploaded to ${r.uploadedTo ?: "a server"}" +
(r.uploadedAs?.let { " as $it" } ?: "")
} else {
"on this device only"
},
style = MaterialTheme.typography.bodySmall, style = MaterialTheme.typography.bodySmall,
color = if (r.uploaded) Color(0xFF7FD17F) else Color(0xFFBBBBBB), color = if (r.uploaded) Color(0xFF7FD17F) else Color(0xFFBBBBBB),
) )
@@ -43,9 +43,27 @@ class MainActivity : ComponentActivity() {
private val permissionLauncher = private val permissionLauncher =
registerForActivityResult(ActivityResultContracts.RequestMultiplePermissions()) { /* proceed regardless */ } registerForActivityResult(ActivityResultContracts.RequestMultiplePermissions()) { /* proceed regardless */ }
/**
* The intent currently being acted on, so a deep link that arrives while the app is running
* is seen by the screen the user is already looking at.
*
* The activity is singleTask for the same reason. As a standard activity it stacked a second
* instance per link, each with its own ViewModel: the enrolment then happened in a throwaway
* copy, and pressing back returned to the original screen showing none of it. Silent, and
* indistinguishable from the link simply not working.
*/
private val liveIntent = mutableStateOf<android.content.Intent?>(null)
override fun onNewIntent(intent: android.content.Intent) {
super.onNewIntent(intent)
setIntent(intent)
liveIntent.value = intent
}
override fun onCreate(savedInstanceState: Bundle?) { override fun onCreate(savedInstanceState: Bundle?) {
super.onCreate(savedInstanceState) super.onCreate(savedInstanceState)
requestRuntimePermissions() requestRuntimePermissions()
liveIntent.value = intent
setContent { setContent {
MaterialTheme(colorScheme = darkColorScheme()) { MaterialTheme(colorScheme = darkColorScheme()) {
Surface(color = MaterialTheme.colorScheme.background) { Surface(color = MaterialTheme.colorScheme.background) {
@@ -63,13 +81,88 @@ class MainActivity : ComponentActivity() {
// An echolot://enroll link (QR scan, or a link the operator sent) opens the // An echolot://enroll link (QR scan, or a link the operator sent) opens the
// app straight into settings with the enrollment already done, so the user // app straight into settings with the enrollment already done, so the user
// sees the result rather than a form they still have to fill in. // sees the result rather than a form they still have to fill in.
val enrollUri = intent?.takeIf { it.action == Intent.ACTION_VIEW }?.dataString // Both deep links land here. They are told apart by host, so a sign-in
// redirect is never mistaken for an enrolment link — one spends a token, the
// other completes an authorization, and confusing them would fail obscurely.
val incoming = liveIntent.value?.takeIf { it.action == Intent.ACTION_VIEW }?.dataString
val authUri = incoming?.takeIf { it.startsWith("echolot://auth") }
val enrollUri = incoming?.takeIf { it.startsWith("echolot://enroll") }
androidx.compose.runtime.LaunchedEffect(enrollUri) { androidx.compose.runtime.LaunchedEffect(enrollUri) {
if (enrollUri != null) { if (enrollUri != null) {
vm.enroll(enrollUri) vm.enroll(enrollUri)
screen = Screen.SETTINGS screen = Screen.SETTINGS
} }
} }
// Replacing an existing enrollment is asked about, never assumed. Following a
// link from a web page is one tap, and the old credential does not survive it.
vm.state.pendingEnroll?.let { pending ->
androidx.compose.material3.AlertDialog(
onDismissRequest = { vm.cancelEnroll() },
title = {
androidx.compose.material3.Text(
if (pending.sameServer) "Enroll again with this server?"
else "Replace this device's server?"
)
},
text = {
androidx.compose.material3.Text(
// Naming the same URL twice reads as a mistake and buries the
// one consequence that actually applies: the device is issued a
// fresh credential and shows up as a second entry.
if (pending.sameServer) {
"This device is already enrolled with " +
"${pending.currentServer}.\n\n" +
"Enrolling again replaces its credential. The old one " +
"stops working immediately, and the device appears on " +
"the server as a new entry alongside the current one — " +
"which you may want to revoke afterwards.\n\n" +
"Runs already uploaded, and runs stored on this phone, " +
"are not affected."
} else {
"This device is already enrolled with " +
"${pending.currentServer}.\n\n" +
"Enrolling with ${pending.newServer} replaces that. Runs " +
"already uploaded stay where they are, but this device " +
"stops reporting to the old server and appears on the new " +
"one as a new device.\n\n" +
"Runs stored on this phone are not affected."
}
)
},
confirmButton = {
androidx.compose.material3.TextButton(onClick = { vm.confirmEnroll() }) {
androidx.compose.material3.Text(
if (pending.sameServer) "Enroll again" else "Enroll here"
)
}
},
dismissButton = {
androidx.compose.material3.TextButton(onClick = { vm.cancelEnroll() }) {
androidx.compose.material3.Text("Keep current server")
}
},
)
}
androidx.compose.runtime.LaunchedEffect(authUri) {
if (authUri != null) {
vm.completeSignIn(authUri)
screen = Screen.SETTINGS
}
}
// Shizuku can be started, stopped or authorised in its own app, where nothing
// calls back into this process. Asking again each time this screen comes
// forward is what makes the banner right after the user has been away to fix
// it — which is exactly the moment they look at it.
val lifecycleOwner = androidx.compose.ui.platform.LocalLifecycleOwner.current
androidx.compose.runtime.DisposableEffect(lifecycleOwner) {
val obs = androidx.lifecycle.LifecycleEventObserver { _, event ->
if (event == androidx.lifecycle.Lifecycle.Event.ON_RESUME) {
vm.refreshShizuku()
}
}
lifecycleOwner.lifecycle.addObserver(obs)
onDispose { lifecycleOwner.lifecycle.removeObserver(obs) }
}
androidx.compose.runtime.LaunchedEffect(autorun) { androidx.compose.runtime.LaunchedEffect(autorun) {
if (autorun) vm.run(devUpload = true) if (autorun) vm.run(devUpload = true)
} }
@@ -85,23 +178,43 @@ class MainActivity : ComponentActivity() {
finish() finish()
} }
} }
// Without this, the system Back gesture leaves the activity from Settings or
// History instead of returning to the run screen — the screen is a plain state
// variable, so nothing connects it to the back stack. Registered only when
// there is somewhere to go back to, so Back still exits from the run screen.
androidx.activity.compose.BackHandler(enabled = screen != Screen.RUN) {
screen = Screen.RUN
}
when (screen) { when (screen) {
Screen.SETTINGS -> SettingsScreen( Screen.SETTINGS -> SettingsScreen(
settings = vm.settings, settings = vm.settings,
archivedRuns = vm.state.history.size, archivedRuns = vm.archivedRunCount(),
archivedBytes = vm.archivedBytes(), archivedBytes = vm.archivedBytes(),
onApplyRetention = vm::applyRetention, onApplyRetention = vm::applyRetention,
onDeleteAll = vm::deleteAllRuns, onDeleteAll = vm::deleteAllRuns,
onPreviewUpload = { onPreviewUpload = {
// Preview the newest run, since that is the one the user just made // Straight from the archive: the newest run is the one the user just
// and the one they are deciding about. // made and the one they are deciding about. Always shows something,
vm.state.history.firstOrNull()?.let { r -> // even when there is nothing to preview yet.
lifecycleScope.launch { preview = vm.uploadPreview(r.id) } lifecycleScope.launch { preview = vm.previewNewestRun() }
}
}, },
onCheckServer = vm::checkServer, onCheckServer = vm::checkServer,
accountName = vm.accountName,
onSignIn = {
vm.beginSignIn { url ->
// A plain VIEW intent rather than a Custom Tab: the browser is
// where the user's existing IdP session already lives, and
// androidx.browser would be a dependency for a rounded corner.
runCatching {
startActivity(Intent(Intent.ACTION_VIEW, android.net.Uri.parse(url)))
}
}
},
onSignOut = vm::signOut,
onEnroll = vm::enroll, onEnroll = vm::enroll,
serverStatus = vm.state.archiveStatus, serverStatus = vm.state.archiveStatus,
enrollStatus = vm.state.enrollStatus,
onBack = { screen = Screen.RUN }, onBack = { screen = Screen.RUN },
) )
Screen.HISTORY -> HistoryScreen( Screen.HISTORY -> HistoryScreen(
@@ -300,6 +413,29 @@ private fun EcholotScreen(
@Composable @Composable
private fun Results(doc: MeasurementDocument) { private fun Results(doc: MeasurementDocument) {
// A constrained run is answered before the lights are: the verdict below is INCONCLUSIVE by
// §7.3, and without this banner "inconclusive" reads as the app failing rather than the OS
// (correctly) refusing to let anything past the VPN be measured.
val constraints = doc.run.constraints
if (constraints.constrained) {
val blocked = constraints.unmeasuredNetworks
.mapNotNull { id -> doc.networks.firstOrNull { it.id == id } }
.joinToString(", ") { it.iface?.takeIf { s -> s.isNotBlank() } ?: it.transport.name.lowercase() }
.ifBlank { "the networks beneath it" }
Card(colors = CardDefaults.cardColors(containerColor = Color(0xFF3A2E12))) {
Column(Modifier.fillMaxWidth().padding(12.dp)) {
Text("Measured through a VPN", color = Color(0xFFFFD08A),
fontWeight = FontWeight.SemiBold)
Text(
"Android does not let apps send on the networks beneath an active VPN, so " +
"$blocked could not be measured — these results describe the tunnel. " +
"Disconnect the VPN and run again to measure the networks themselves.",
fontSize = 12.sp, color = Color(0xFFFFD08A),
)
}
}
}
val summary = doc.summary val summary = doc.summary
if (summary != null) { if (summary != null) {
Card(colors = CardDefaults.cardColors(containerColor = verdictColor(summary.overall))) { Card(colors = CardDefaults.cardColors(containerColor = verdictColor(summary.overall))) {
@@ -80,8 +80,16 @@ class RunStore(context: Context, private val settings: Settings) {
private fun client() = ControlClient( private fun client() = ControlClient(
settings.serverUrl, setOf(settings.serverPin), BuildConfig.APP_SEMVER, settings.serverUrl, setOf(settings.serverPin), BuildConfig.APP_SEMVER,
fallbackAddrs = settings.serverAddrList(),
) )
/** Remembers where the server lives, so a later run can reach it without DNS. */
private fun rememberAddrs(p: app.echo_lot.protocol.Profile) {
val addrs = p.targets.flatMap { listOfNotNull(it.ip4, it.ip6) }
.filter { it.isNotBlank() }
if (addrs.isNotEmpty()) settings.serverAddrs = addrs.joinToString(",")
}
/** /**
* Checks the configured server without uploading anything: reachable, pinned, compatible, and * Checks the configured server without uploading anything: reachable, pinned, compatible, and
* willing to accept runs. Lets the user find out in settings rather than from a failed run. * willing to accept runs. Lets the user find out in settings rather than from a failed run.
@@ -90,6 +98,10 @@ class RunStore(context: Context, private val settings: Settings) {
if (!settings.serverConfigured) return "Fill in the server URL, pin and credential first." if (!settings.serverConfigured) return "Fill in the server URL, pin and credential first."
return try { return try {
val profile = client().profile(settings.serverCredential) val profile = client().profile(settings.serverCredential)
// Learned here so the next run's canary probe knows what to ask for.
profile.canaryZone.takeIf { it.isNotBlank() }?.let { settings.canaryZone = it }
settings.serverFacts = describeFacts(profile)
rememberAddrs(profile)
val compat = Compat.check(profile, BuildConfig.APP_SEMVER) val compat = Compat.check(profile, BuildConfig.APP_SEMVER)
val head = "${profile.name} · server ${profile.serverVersion} · " + val head = "${profile.name} · server ${profile.serverVersion} · " +
"protocol ${profile.compat.protocolVersion.ifBlank { "unstated" }}" "protocol ${profile.compat.protocolVersion.ifBlank { "unstated" }}"
@@ -115,6 +127,36 @@ class RunStore(context: Context, private val settings: Settings) {
* no credential — fails later, somewhere else, with an error that points at the wrong thing. * no credential — fails later, somewhere else, with an error that points at the wrong thing.
* Blocking; callers run it off the main thread. * Blocking; callers run it off the main thread.
*/ */
/**
* Renders what the server says about itself, for display.
*
* Only what a person measuring against it would want to check: which addresses the tests will
* actually use, on which ports, and what the server admits it can do. Addresses first, because
* "which address did this result come from" is the question a report leaves open.
*/
private fun describeFacts(p: app.echo_lot.protocol.Profile): String {
val lines = ArrayList<String>()
// "label|value" per line, laid out as real columns by the UI rather than padded with
// spaces here. Space padding only lines up in a monospaced font, which makes the layout
// depend on a typeface choice made somewhere else entirely.
fun row(label: String, value: String) = lines.add("$label|$value")
row("server", "${p.name} · ${p.serverVersion}")
for (t in p.targets) {
t.ip4?.let { row("IPv4", it) }
t.ip6?.let { row("IPv6", it) }
// Marked rather than listed apart: it is the same server, and what matters is being
// able to tell which address a NAT-behaviour result came from.
t.ip4Alt?.let { row("IPv4 alt", it) }
t.ip6Alt?.let { row("IPv6 alt", it) }
row("ports", "udp ${t.udpPort} · tcp ${t.tcpPort} · stun ${t.stunPort}")
}
if (p.canaryZone.isNotBlank()) row("dns zone", p.canaryZone)
if (p.capabilities.isNotEmpty()) row("measures", p.capabilities.joinToString(", "))
return lines.joinToString(System.lineSeparator())
}
fun enroll(link: String, deviceName: String?): String { fun enroll(link: String, deviceName: String?): String {
val parsed = app.echo_lot.protocol.EnrollmentLink.parse(link) val parsed = app.echo_lot.protocol.EnrollmentLink.parse(link)
?: return "That does not look like an Echolot enrollment link. It should start with " + ?: return "That does not look like an Echolot enrollment link. It should start with " +
@@ -122,7 +164,11 @@ class RunStore(context: Context, private val settings: Settings) {
return try { return try {
val enrolled = parsed.redeem(deviceName, BuildConfig.APP_SEMVER) val enrolled = parsed.redeem(deviceName, BuildConfig.APP_SEMVER)
val compat = Compat.check(enrolled.profile, BuildConfig.APP_SEMVER) val compat = Compat.check(enrolled.profile, BuildConfig.APP_SEMVER)
settings.serverFacts = describeFacts(enrolled.profile)
rememberAddrs(enrolled.profile)
enrolled.profile.canaryZone.takeIf { it.isNotBlank() }?.let { settings.canaryZone = it }
settings.serverUrl = enrolled.controlUrl settings.serverUrl = enrolled.controlUrl
settings.serverPublicUrl = enrolled.publicUrl
settings.serverPin = enrolled.pin settings.serverPin = enrolled.pin
settings.serverCredential = enrolled.credential settings.serverCredential = enrolled.credential
val head = "Enrolled with ${enrolled.profile.name} " + val head = "Enrolled with ${enrolled.profile.name} " +
@@ -150,6 +196,9 @@ class RunStore(context: Context, private val settings: Settings) {
return try { return try {
val client = client() val client = client()
val profile = client.profile(settings.serverCredential) val profile = client.profile(settings.serverCredential)
profile.canaryZone.takeIf { it.isNotBlank() }?.let { settings.canaryZone = it }
settings.serverFacts = describeFacts(profile)
rememberAddrs(profile)
// Compatibility before policy: an incompatible server may well advertise an upload // Compatibility before policy: an incompatible server may well advertise an upload
// policy it would never actually apply to us. // policy it would never actually apply to us.
@@ -164,8 +213,10 @@ class RunStore(context: Context, private val settings: Settings) {
) )
val body = redactedForUpload(docJson, level) val body = redactedForUpload(docJson, level)
val reply = client.uploadRun(settings.serverCredential, body) val reply = client.uploadRun(settings.serverCredential, body)
archive.markUploaded(runId, profile.name) archive.markUploaded(runId, profile.name, level.wire)
UploadOutcome.Sent(profile.name, "as $level, ${body.toByteArray().size} bytes: ${reply.take(120)}") // Deliberately not echoing `reply`: it is the server's index entry as raw JSON, and
// it ended up rendered verbatim in the UI. Size and level are what a person wants.
UploadOutcome.Sent(profile.name, "as $level, ${body.toByteArray().size} bytes")
} catch (e: VersionRefused) { } catch (e: VersionRefused) {
UploadOutcome.Incompatible(e.message ?: "the server refused this app's version") UploadOutcome.Incompatible(e.message ?: "the server refused this app's version")
} catch (e: UploadRefused) { } catch (e: UploadRefused) {
@@ -42,6 +42,17 @@ data class UiState(
val archiveStatus: String? = null, val archiveStatus: String? = null,
/** History, newest first. Refreshed after every run and whenever the history screen opens. */ /** History, newest first. Refreshed after every run and whenever the history screen opens. */
val history: List<app.echo_lot.archive.ArchivedRun> = emptyList(), val history: List<app.echo_lot.archive.ArchivedRun> = emptyList(),
/**
* Result of the last enrollment attempt, shown beside the Enroll button.
*
* Separate from [archiveStatus]: they are two different actions with two different results,
* and sharing one line put the answer to "did enrolling work" at the far end of the card,
* below three text fields — or nowhere at all on a fresh install, since that line only
* renders once a run exists.
*/
val enrollStatus: String? = null,
/** An enrollment link waiting on confirmation, because this device is already enrolled. */
val pendingEnroll: PendingEnroll? = null,
/** Shell-tier readiness, shown before a run; null message = say nothing (Shizuku not installed). */ /** Shell-tier readiness, shown before a run; null message = say nothing (Shizuku not installed). */
val shizukuNotice: String? = null, val shizukuNotice: String? = null,
val shizukuReady: Boolean = false, val shizukuReady: Boolean = false,
@@ -49,6 +60,18 @@ data class UiState(
val shizukuState: ShizukuAvailability.State = ShizukuAvailability.State.NOT_INSTALLED, val shizukuState: ShizukuAvailability.State = ShizukuAvailability.State.NOT_INSTALLED,
) )
/**
* An enrollment link that would replace an existing one, held until the user says so.
*
* Enrolling is not additive: the new credential replaces the old, and on the previous server this
* device simply stops reporting. Following a link is one tap from a web page, which is not enough
* deliberation to discard a working enrollment by accident.
*/
data class PendingEnroll(val link: String, val currentServer: String, val newServer: String) {
/** Re-enrolling with the server already configured, rather than moving to a different one. */
val sameServer: Boolean get() = currentServer.trimEnd('/') == newServer.trimEnd('/')
}
/** /**
* Drives one measurement run: device-tier probes (link snapshot, per-network ICMP) always run; * Drives one measurement run: device-tier probes (link snapshot, per-network ICMP) always run;
* results assemble into a MeasurementDocument with a §7.3 summary. Lives in a ViewModel so a run * results assemble into a MeasurementDocument with a §7.3 summary. Lives in a ViewModel so a run
@@ -80,6 +103,23 @@ class RunViewModel(app: Application) : AndroidViewModel(app) {
} }
} }
/**
* Re-reads the shell tier's state, for when it changed somewhere this process cannot see.
*
* Permission can be granted inside Shizuku's own app, and Shizuku can be started or stopped
* there too; none of that calls back here. Asking again on resume is the only way to be right
* after the user has been somewhere else to fix it.
*/
fun refreshShizuku() {
val st = ShizukuAvailability.current(getApplication())
state = state.copy(
shizukuNotice = ShizukuAvailability.describe(st),
shizukuReady = st == ShizukuAvailability.State.READY,
shizukuHint = ShizukuAvailability.actionHint(st),
shizukuState = st,
)
}
override fun onCleared() { override fun onCleared() {
stopShizukuObserver() stopShizukuObserver()
super.onCleared() super.onCleared()
@@ -90,6 +130,7 @@ class RunViewModel(app: Application) : AndroidViewModel(app) {
private var runStartWall: String = "" private var runStartWall: String = ""
private var runNetworks: List<app.echo_lot.measurement.Network> = emptyList() private var runNetworks: List<app.echo_lot.measurement.Network> = emptyList()
private var runShizukuOk = false private var runShizukuOk = false
private var runConstraints = Constraints()
/** Two-clock ids: UUIDs + monotonic ns relative to a per-run origin. */ /** Two-clock ids: UUIDs + monotonic ns relative to a per-run origin. */
private class RunIds : ProbeIds { private class RunIds : ProbeIds {
@@ -108,6 +149,7 @@ class RunViewModel(app: Application) : AndroidViewModel(app) {
fun run(devUpload: Boolean = false) { fun run(devUpload: Boolean = false) {
if (state.running) return if (state.running) return
collected.clear() collected.clear()
runConstraints = Constraints()
state = state.copy(running = true, currentStep = "starting", document = null, state = state.copy(running = true, currentStep = "starting", document = null,
uploadStatus = null, archiveStatus = null) uploadStatus = null, archiveStatus = null)
runJob = viewModelScope.launch { runJob = viewModelScope.launch {
@@ -125,7 +167,13 @@ class RunViewModel(app: Application) : AndroidViewModel(app) {
if (devUpload) { if (devUpload) {
state = state.copy(currentStep = "uploading report") state = state.copy(currentStep = "uploading report")
val r = withContext(Dispatchers.IO) { ReportUploader.upload(doc) } val r = withContext(Dispatchers.IO) { ReportUploader.upload(doc) }
status = if (r.ok) "uploaded ✓ ${r.detail}" else "upload failed: ${r.detail}" status = when {
r.ok -> "uploaded ✓ ${r.detail}"
// Not a failure worth alarming about: the dev collection endpoint is simply
// not configured, and the run is on the device either way.
r.detail.startsWith("no upload URL") -> "run complete — read it with adb"
else -> "upload failed: ${r.detail}"
}
} }
if (archived != null && settings.autoUpload) { if (archived != null && settings.autoUpload) {
state = state.copy(currentStep = "uploading to server") state = state.copy(currentStep = "uploading to server")
@@ -194,13 +242,109 @@ class RunViewModel(app: Application) : AndroidViewModel(app) {
store.read(id)?.let { store.redactedForUpload(it) } store.read(id)?.let { store.redactedForUpload(it) }
} }
/**
* Preview of the most recent run, read from the archive rather than from [UiState.history].
*
* The history list is only populated once the History screen has been opened, so a preview
* driven from it did nothing at all on a freshly-opened Settings screen — a button that
* silently does nothing is worse than one that says why.
*/
suspend fun previewNewestRun(): String = withContext(Dispatchers.IO) {
val newest = store.list().firstOrNull()
?: return@withContext "No archived runs yet. Run a measurement first, then this will " +
"show exactly what an upload would send."
store.read(newest.id)?.let { store.redactedForUpload(it) }
?: "That run could not be read back from the archive."
}
fun archivedBytes(): Long = store.totalBytes() fun archivedBytes(): Long = store.totalBytes()
/** Counted from the archive itself, not from [UiState.history], which is empty until the
* history screen has been opened - the two disagreeing read as data loss. */
fun archivedRunCount(): Int = store.list().size
private val account = Account(settings)
/** Name of whoever is signed in on this device, for the settings screen. */
var accountName by mutableStateOf(settings.accountName)
private set
/** Starts sign-in; the caller opens the returned URL in a browser. */
fun beginSignIn(open: (String) -> Unit) {
viewModelScope.launch {
state = state.copy(archiveStatus = "contacting the server …")
when (val r = withContext(Dispatchers.IO) { account.begin() }) {
is Account.SignInStart.Browser -> {
state = state.copy(archiveStatus = "continue in your browser …")
open(r.url)
}
is Account.SignInStart.Unavailable ->
state = state.copy(archiveStatus = r.reason)
}
}
}
/** Completes sign-in from the echolot://auth redirect. */
fun completeSignIn(callbackUri: String) {
viewModelScope.launch {
val msg = withContext(Dispatchers.IO) { account.complete(callbackUri) }
accountName = settings.accountName
state = state.copy(archiveStatus = msg)
}
}
fun signOut() {
viewModelScope.launch {
val msg = withContext(Dispatchers.IO) { account.signOut() }
accountName = settings.accountName
state = state.copy(archiveStatus = msg)
}
}
/** Redeems an enrollment link, from a paste or from an echolot:// deep link. */ /** Redeems an enrollment link, from a paste or from an echolot:// deep link. */
fun enroll(link: String, deviceName: String? = android.os.Build.MODEL) { fun enroll(link: String, deviceName: String? = android.os.Build.MODEL) {
// Already enrolled? Ask first. The old credential is gone the moment this succeeds, and a
// link followed from a web page is one tap — far too little deliberation for that.
if (settings.serverConfigured) {
val target = app.echo_lot.protocol.EnrollmentLink.parse(link)?.controlUrl ?: link
state = state.copy(
pendingEnroll = PendingEnroll(
link = link,
// Compared against the link's URL, which names the server publicly — so this
// has to be the public name too. Using the endpoint made re-enrolling with the
// same server look like a move to a different one, because the endpoint and
// the public name are deliberately different strings.
currentServer = settings.serverPublicUrl,
newServer = target,
)
)
return
}
doEnroll(link, deviceName)
}
/** The user confirmed replacing an existing enrollment. */
fun confirmEnroll(deviceName: String? = android.os.Build.MODEL) {
val pending = state.pendingEnroll ?: return
state = state.copy(pendingEnroll = null)
doEnroll(pending.link, deviceName)
}
fun cancelEnroll() {
state = state.copy(
pendingEnroll = null,
enrollStatus = "Kept the existing enrollment; nothing changed.",
)
}
private fun doEnroll(link: String, deviceName: String?) {
viewModelScope.launch { viewModelScope.launch {
state = state.copy(archiveStatus = "enrolling …") state = state.copy(enrollStatus = "Enrolling …")
state = state.copy(archiveStatus = withContext(Dispatchers.IO) { store.enroll(link, deviceName) }) val result = withContext(Dispatchers.IO) { store.enroll(link, deviceName) }
// A new server means a new canary zone; the old one would describe somebody else's
// deployment. Cleared rather than kept, and relearned from the next profile fetch.
settings.canaryZone = ""
state = state.copy(enrollStatus = result)
} }
} }
@@ -249,16 +393,35 @@ class RunViewModel(app: Application) : AndroidViewModel(app) {
val entries = NetworkInventory.snapshot(ctx) val entries = NetworkInventory.snapshot(ctx)
val networks = entries.map { it.model }.also { runNetworks = it } val networks = entries.map { it.model }.also { runNetworks = it }
// What will this run be prevented from measuring? Decided up front, from one throwaway
// bind per network, so the document can say so instead of leaving it to be inferred from
// per-test `attempted: false` breadcrumbs (measurement-schema.md §3 `constraints`).
runConstraints = app.echo_lot.probe.ConstraintDetector.detect(entries)
val probes: List<Probe> = listOf( val probes: List<Probe> = listOf(
LinkSnapshotProbe(entries), LinkSnapshotProbe(entries),
RouterIdentityProbe(entries), RouterIdentityProbe(entries),
IcmpProbe(entries, v6 = false), IcmpProbe(entries, v6 = false),
IcmpProbe(entries, v6 = true), IcmpProbe(entries, v6 = true),
// Folded from the prober after hardware validation: errqueue traceroute (no root,
// no JNI) and the mDNS service inventory / VLAN-leakage detector.
app.echo_lot.probe.TracerouteProbe(),
app.echo_lot.probe.MdnsInventoryProbe(),
CaptivePortalProbe(entries), CaptivePortalProbe(entries),
// Canary zone served by the Echolot probe server (probe-protocol §6.1). Hardcoded to // Canary zone served by the Echolot probe server (probe-protocol §6.1). Hardcoded to
// the reference deployment until profiles/enrollment land in the UI. // the reference deployment until profiles/enrollment land in the UI.
DnsCanaryProbe(canaryZone = "c.echo-lot.app", sessionPrefix = "adhoc"), // Both target whatever server this device is enrolled with, not the deployment the
StunProbe(serverHost = "fmr-1.echo-lot.app"), // app happened to be developed against. With no server configured they get blank
// strings and report themselves skipped, which is the honest outcome — the
// alternative measures someone else's infrastructure and calls it your network.
// Before the canary: "can this device resolve at all" has to be answered before
// "are the answers being tampered with" means anything.
app.echo_lot.probe.DnsResolverProbe(entries),
DnsCanaryProbe(canaryZone = settings.canaryZone, sessionPrefix = "adhoc"),
StunProbe(serverHost = settings.serverHost()),
// Corroboration for icmp.ping6's silence: a real TCP connection over IPv6. Only its
// failure, on a network that advertises IPv6, justifies calling IPv6 broken.
app.echo_lot.probe.V6ConnectProbe(entries, serverHost = settings.serverHost()),
) )
// Plan the run first: the Shizuku battery is counted alongside the app-tier probes so // Plan the run first: the Shizuku battery is counted alongside the app-tier probes so
@@ -321,39 +484,169 @@ class RunViewModel(app: Application) : AndroidViewModel(app) {
androidSdk = Build.VERSION.SDK_INT, androidRelease = Build.VERSION.RELEASE, androidSdk = Build.VERSION.SDK_INT, androidRelease = Build.VERSION.RELEASE,
), ),
tiers = Tiers(app = true, shizuku = runShizukuOk), tiers = Tiers(app = true, shizuku = runShizukuOk),
constraints = runConstraints,
), ),
networks = runNetworks, networks = runNetworks,
tests = tests, tests = tests,
findings = findings, findings = findings,
summary = Verdicts.derive(tests, findings), summary = Verdicts.derive(tests, findings, runConstraints),
) )
} }
/** /**
* Was IPv6 actually provisioned on any network? A global (non-link-local) v6 address or a * Was IPv6 provisioned on the network a test actually ran over?
* v6 default route means the network claims to offer IPv6 — link-local only does not count. *
* This deliberately asks about one network rather than about the device. Answering "does any
* network here have IPv6" produces a real false positive on a phone, and it is not hypothetical:
* an IPv4-only wifi with working cellular alongside it reports "IPv6 is configured, but ICMPv6
* gets no reply" — configured on cellular, pinged over wifi, and the two never met.
*
* A global (non-link-local) address or a v6 default route means the network claims to offer
* IPv6; link-local only does not count, since every interface has one.
*/ */
private fun ipv6Provisioned(networks: List<app.echo_lot.measurement.Network>): Boolean = private fun ipv6Provisioned(
networks.any { n -> networks: List<app.echo_lot.measurement.Network>,
networkRef: String?,
): Boolean {
// No reference means the test was not per-network; fall back to the device-wide reading
// rather than silently reporting nothing.
val scope = networks.filter { networkRef == null || it.id == networkRef }
return scope.any { n ->
n.link.addresses.any { a -> n.link.addresses.any { a ->
a.addr.contains(':') && a.addr.contains(':') &&
!a.addr.startsWith("fe80", ignoreCase = true) && !a.addr.startsWith("fe80", ignoreCase = true) &&
!a.addr.startsWith("::1") !a.addr.startsWith("::1")
} || n.link.routes.any { it.dst == "::/0" } } || n.link.routes.any { it.dst == "::/0" }
} }
}
/**
* Per-network ICMP outcomes, keyed by network id.
*
* Reads the structured evidence the probe records rather than its prose detail — a finding
* that depended on the wording of a human-readable string would break silently the first time
* that wording improved.
*/
/** What one network's ICMP attempt did: whether it ran at all, and whether it was answered. */
private data class IcmpOutcome(val attempted: Boolean, val ok: Boolean)
/**
* Per-network ICMP outcomes, keyed by network id.
*
* Reads the structured evidence the probe records rather than its prose detail — a finding
* that depended on the wording of a human-readable string would break silently the first time
* that wording improved.
*/
private fun icmpResults(t: Test): Map<String, IcmpOutcome> {
val out = HashMap<String, IcmpOutcome>()
val ev = t.evidence ?: return out
for ((_, v) in ev) {
val o = v as? kotlinx.serialization.json.JsonObject ?: continue
val ref = (o["network_ref"] as? kotlinx.serialization.json.JsonPrimitive)?.content ?: continue
fun flag(k: String) = (o[k] as? kotlinx.serialization.json.JsonPrimitive)?.content == "true"
out[ref] = IcmpOutcome(attempted = flag("attempted"), ok = flag("ok"))
}
return out
}
/** Human-facing name for the network a test ran over; falls back to something readable. */
private fun ifaceOf(networks: List<app.echo_lot.measurement.Network>, ref: String?): String =
networks.firstOrNull { it.id == ref }?.iface?.takeIf { it.isNotBlank() } ?: "this network"
/** Minimal first-pass findings from device-tier evidence; the registry grows with the suite. */ /** Minimal first-pass findings from device-tier evidence; the registry grows with the suite. */
private fun deriveFindings(tests: List<Test>, networks: List<app.echo_lot.measurement.Network>): List<Finding> { private fun deriveFindings(tests: List<Test>, networks: List<app.echo_lot.measurement.Network>): List<Finding> {
val out = ArrayList<Finding>() val out = ArrayList<Finding>()
val ids = RunIds() val ids = RunIds()
val linkEvidence = tests.filter { it.type == TestType.LINK_SNAPSHOT }.map { EvidenceRef(it.id) }
// Said as a finding, not only as run.constraints: the constraints block is for machines
// aggregating thousands of runs, this is for the person reading this one. Both must exist —
// a constrained run with a quiet findings list still reads as "nothing wrong here".
if (runConstraints.constrained) {
val blocked = runConstraints.unmeasuredNetworks
.joinToString(", ") { id -> ifaceOf(networks, id) }
.ifBlank { "the underlying networks" }
out.add(
Finding(
id = ids.uuid(),
code = FindingRegistry.MEASUREMENT_VPN_CONSTRAINED.code,
category = FindingRegistry.MEASUREMENT_VPN_CONSTRAINED.category,
severity = FindingRegistry.MEASUREMENT_VPN_CONSTRAINED.severity,
confidence = Confidence.HIGH,
title = "A VPN is active — $blocked could not be measured",
description = "Android refuses to let apps send on the networks beneath an " +
"active VPN (that is how it prevents traffic leaking around the tunnel), " +
"so every per-network test here measured the tunnel or nothing. Nothing " +
"in this run says anything about $blocked. To measure them, disconnect " +
"the VPN and run again.",
evidenceRefs = linkEvidence,
)
)
}
val shapes = V6Analysis.classify(networks)
// Named per interface: on a phone several networks are up at once, and "IPv6 is broken" is
// useless when wifi is the broken one and cellular is fine.
for (sh in shapes.filter { it.addressWithoutRoute }) {
val where = if (sh.iface.isBlank()) "This device" else sh.iface
out.add(
Finding(
id = ids.uuid(),
code = FindingRegistry.V6_NO_DEFAULT_ROUTE.code,
category = FindingRegistry.V6_NO_DEFAULT_ROUTE.category,
severity = if (sh.tunnel) Severity.INFO else Severity.MEDIUM,
confidence = Confidence.HIGH,
title = if (sh.tunnel) {
"IPv6 reaches only the destinations a tunnel routes ($where)"
} else {
"IPv6 address with no default route ($where)"
},
description = "$where has a global IPv6 address but no IPv6 default route, so " +
"IPv6 reaches only destinations covered by a specific route. " +
if (sh.tunnel) {
"A tunnel interface holds those routes, so this looks deliberate. " +
"Worth knowing rather than fixing: applications holding a global " +
"address will still try IPv6 first and stall for anything outside " +
"the tunnel's routes."
} else {
"Nothing is routing the rest, so the network handed out an address it " +
"does not carry traffic for — applications will try IPv6 first " +
"and wait for it to fail."
},
evidenceRefs = linkEvidence,
)
)
}
for (sh in shapes.filter { it.routeWithoutAddress }) {
val where = if (sh.iface.isBlank()) "This network" else sh.iface
out.add(
Finding(
id = ids.uuid(),
code = FindingRegistry.V6_ROUTE_WITHOUT_ADDRESS.code,
category = FindingRegistry.V6_ROUTE_WITHOUT_ADDRESS.category,
severity = Severity.MEDIUM,
confidence = Confidence.HIGH,
title = "IPv6 router advertised, but no address was configured ($where)",
description = "$where has an IPv6 default route but no global IPv6 address. " +
"The router is advertising itself as an IPv6 gateway while SLAAC produced " +
"no usable address — a missing prefix option, a prefix without the " +
"autonomous flag, or DHCPv6-only addressing that did not complete. Hosts " +
"believe IPv6 is available and pay a connection timeout on every " +
"dual-stack destination before falling back to IPv4, which is felt as " +
"general slowness with no packet loss to explain it.",
evidenceRefs = linkEvidence,
)
)
}
for (t in tests) { for (t in tests) {
if (t.type == TestType.NET_CAPTIVE_PORTAL) { if (t.type == TestType.NET_CAPTIVE_PORTAL) {
val ev = t.evidence?.toString() ?: "" val ev = t.evidence?.toString() ?: ""
when { when {
ev.contains("\"captive_portal\"") -> out.add( ev.contains("\"captive_portal\"") -> out.add(
Finding( Finding(
id = ids.uuid(), code = "connectivity.captive_portal", category = Category.CONNECTIVITY, id = ids.uuid(), code = FindingRegistry.CAPTIVE_PORTAL.code,
severity = Severity.MEDIUM, confidence = Confidence.HIGH, category = FindingRegistry.CAPTIVE_PORTAL.category,
severity = FindingRegistry.CAPTIVE_PORTAL.severity, confidence = Confidence.HIGH,
title = "Captive portal intercepting connections", title = "Captive portal intercepting connections",
description = "The generate_204 check returned a redirect or a page instead of HTTP 204 — a captive portal (login/splash page) is intercepting traffic on this network.", description = "The generate_204 check returned a redirect or a page instead of HTTP 204 — a captive portal (login/splash page) is intercepting traffic on this network.",
evidenceRefs = listOf(EvidenceRef(t.id)), evidenceRefs = listOf(EvidenceRef(t.id)),
@@ -361,8 +654,9 @@ class RunViewModel(app: Application) : AndroidViewModel(app) {
) )
t.status == TestStatus.FAILED -> out.add( t.status == TestStatus.FAILED -> out.add(
Finding( Finding(
id = ids.uuid(), code = "connectivity.no_internet", category = Category.CONNECTIVITY, id = ids.uuid(), code = FindingRegistry.NO_INTERNET.code,
severity = Severity.HIGH, confidence = Confidence.HIGH, category = FindingRegistry.NO_INTERNET.category,
severity = FindingRegistry.NO_INTERNET.severity, confidence = Confidence.HIGH,
title = "No working internet on any network", title = "No working internet on any network",
description = "Android's own generate_204 connectivity checks failed on every active network (no HTTP 204) — this device has no validated internet path.", description = "Android's own generate_204 connectivity checks failed on every active network (no HTTP 204) — this device has no validated internet path.",
evidenceRefs = listOf(EvidenceRef(t.id)), evidenceRefs = listOf(EvidenceRef(t.id)),
@@ -375,8 +669,9 @@ class RunViewModel(app: Application) : AndroidViewModel(app) {
if (ev.contains("MISMATCH")) { if (ev.contains("MISMATCH")) {
out.add( out.add(
Finding( Finding(
id = ids.uuid(), code = "dns.answer_rewritten", category = Category.DNS, id = ids.uuid(), code = FindingRegistry.DNS_ANSWER_REWRITTEN.code,
severity = Severity.HIGH, confidence = Confidence.HIGH, category = FindingRegistry.DNS_ANSWER_REWRITTEN.category,
severity = FindingRegistry.DNS_ANSWER_REWRITTEN.severity, confidence = Confidence.HIGH,
title = "DNS answers are being rewritten", title = "DNS answers are being rewritten",
description = "A canary reference record returned different RDATA than the spec-defined ground truth — something on the path is rewriting DNS answers (interception, filtering, or a middlebox).", description = "A canary reference record returned different RDATA than the spec-defined ground truth — something on the path is rewriting DNS answers (interception, filtering, or a middlebox).",
evidenceRefs = listOf(EvidenceRef(t.id)), evidenceRefs = listOf(EvidenceRef(t.id)),
@@ -385,8 +680,9 @@ class RunViewModel(app: Application) : AndroidViewModel(app) {
} else if (ev.contains("\"reached_authoritative\":false")) { } else if (ev.contains("\"reached_authoritative\":false")) {
out.add( out.add(
Finding( Finding(
id = ids.uuid(), code = "dns.authoritative_unreachable", category = Category.DNS, id = ids.uuid(), code = FindingRegistry.DNS_AUTHORITATIVE_UNREACHABLE.code,
severity = Severity.MEDIUM, confidence = Confidence.MEDIUM, category = FindingRegistry.DNS_AUTHORITATIVE_UNREACHABLE.category,
severity = FindingRegistry.DNS_AUTHORITATIVE_UNREACHABLE.severity, confidence = Confidence.MEDIUM,
title = "Canary queries don't reach the authoritative server", title = "Canary queries don't reach the authoritative server",
description = "A per-run nonce name (which cannot be cached) was not answered by the canary server — the resolver is intercepting or failing to reach it.", description = "A per-run nonce name (which cannot be cached) was not answered by the canary server — the resolver is intercepting or failing to reach it.",
evidenceRefs = listOf(EvidenceRef(t.id)), evidenceRefs = listOf(EvidenceRef(t.id)),
@@ -399,8 +695,9 @@ class RunViewModel(app: Application) : AndroidViewModel(app) {
if (ev.contains("address/port-dependent (symmetric NAT")) { if (ev.contains("address/port-dependent (symmetric NAT")) {
out.add( out.add(
Finding( Finding(
id = ids.uuid(), code = "nat.symmetric", category = Category.NAT, id = ids.uuid(), code = FindingRegistry.NAT_SYMMETRIC.code,
severity = Severity.MEDIUM, confidence = Confidence.HIGH, category = FindingRegistry.NAT_SYMMETRIC.category,
severity = FindingRegistry.NAT_SYMMETRIC.severity, confidence = Confidence.HIGH,
title = "Symmetric NAT — peer-to-peer connections need a relay", title = "Symmetric NAT — peer-to-peer connections need a relay",
description = "The NAT assigns a different external port per destination (address/port-dependent mapping). Direct peer-to-peer connections (calls, games, file transfer) will usually fail and fall back to relays.", description = "The NAT assigns a different external port per destination (address/port-dependent mapping). Direct peer-to-peer connections (calls, games, file transfer) will usually fail and fall back to relays.",
evidenceRefs = listOf(EvidenceRef(t.id)), evidenceRefs = listOf(EvidenceRef(t.id)),
@@ -408,29 +705,185 @@ class RunViewModel(app: Application) : AndroidViewModel(app) {
) )
} }
} }
if (t.type == TestType.ICMP_PING6 && t.status == TestStatus.FAILED) { if (t.type == TestType.DNS_RESOLVER && t.status == TestStatus.OK) {
// A network with no IPv6 at all is NORMAL — most networks are still IPv4-only, // One finding per network: on a phone the wifi resolver can be wedged while
// and that is not a defect. What IS a defect is IPv6 that the network claims to // cellular is fine, and "DNS is broken" would be wrong about half the device.
// provide (a global address or a default route from RA/DHCPv6) but that does not val ev = t.evidence
// work: that causes Happy-Eyeballs delays, timeouts and hangs. So the severity if (ev != null) {
// depends on whether v6 was provisioned at all. for ((_, v) in ev) {
if (ipv6Provisioned(networks)) { val o = v as? kotlinx.serialization.json.JsonObject ?: continue
out.add( fun str(k: String) =
Finding( (o[k] as? kotlinx.serialization.json.JsonPrimitive)?.content
id = ids.uuid(), code = "ipv6.broken", category = Category.IPV6, val verdict = str("verdict")
severity = Severity.MEDIUM, confidence = Confidence.HIGH, val ref0 = str("network_ref")
title = "IPv6 is configured but not working", val iface0 = networks.firstOrNull { it.id == ref0 }?.iface
description = "This network advertises IPv6 (a global address and/or a default route), but ICMPv6 got no reply on any network. Half-configured IPv6 is worse than none: connections try IPv6 first and stall before falling back.", ?.takeIf { it.isNotBlank() } ?: "this network"
evidenceRefs = listOf(EvidenceRef(t.id)), if (verdict == "search domain swallows queries") {
// Severity follows the harm, not the shape: the same misconfiguration
// is fatal on a resolver that tries the search form and invisible on
// one that does not, and saying "high" for a network that currently
// resolves fine would be crying wolf.
val breaking = str("system_resolves") != "true"
out.add(
Finding(
id = ids.uuid(),
code = FindingRegistry.DNS_SEARCH_DOMAIN_UNANSWERED.code,
category = FindingRegistry.DNS_SEARCH_DOMAIN_UNANSWERED.category,
severity = if (breaking) Severity.HIGH else Severity.MEDIUM,
confidence = Confidence.HIGH,
title = "The network's search domain swallows DNS queries ($iface0)",
description = "This network hands out " +
"${str("search_domains") ?: "a search domain"} as a DNS " +
"search domain, but its server never answers queries under " +
"it — not even to say the name does not exist. Resolvers " +
"append that domain to lookups, so they wait for a reply " +
"that never comes. " +
(if (breaking) {
"That is why names are not resolving on this device."
} else {
"Name resolution still works here, because this " +
"resolver tries the plain name first — another " +
"device on the same network may fail outright."
}) +
" Fix it on the router: either stop advertising the search " +
"domain, or make the server answer for it, including " +
"NXDOMAIN for names it does not have. Note that .local is " +
"reserved for mDNS (RFC 6762) and is widely dropped by " +
"design; home.arpa (RFC 8375) is the name reserved for this.",
evidenceRefs = listOf(EvidenceRef(t.id)),
)
)
continue
}
if (verdict != "server answers, device resolver does not") continue
val ref = str("network_ref")
val where = networks.firstOrNull { it.id == ref }?.iface
?.takeIf { it.isNotBlank() } ?: "this network"
out.add(
Finding(
id = ids.uuid(),
code = FindingRegistry.DNS_SYSTEM_RESOLVER_BROKEN.code,
category = FindingRegistry.DNS_SYSTEM_RESOLVER_BROKEN.category,
severity = FindingRegistry.DNS_SYSTEM_RESOLVER_BROKEN.severity,
confidence = Confidence.HIGH,
title = "This device cannot resolve names, but the DNS server is fine ($where)",
description = "A DNS query sent straight from this device was " +
"answered by ${str("servers") ?: "the configured server"} with " +
"a valid result, yet asking Android to resolve the same name " +
"fails. Whatever is wrong sits between this device's resolver " +
"and a server that demonstrably works. " +
"Turning wifi off and on, or rejoining the network, clears the " +
"common case. If it survives a restart it is not a stuck " +
"resolver: look for something on this device that filters DNS " +
"— an ad blocker, a private-DNS or VPN app — or a per-device " +
"rule on the router aimed at this client.",
evidenceRefs = listOf(EvidenceRef(t.id)),
)
) )
}
}
}
if (t.type == TestType.ICMP_PING6) {
// A network with no IPv6 at all is NORMAL — most networks are still IPv4-only, and
// that is not a defect. What IS a defect is IPv6 the network claims to provide (a
// global address or a default route from RA/DHCPv6) that does not work: that causes
// Happy-Eyeballs delays, timeouts and hangs.
//
// Judged per network, from the per-network evidence rather than the aggregate
// status. The aggregate can only say "some network answered", and on a phone with
// wifi and cellular up at once that is how "IPv6 is configured but gets no reply"
// ends up describing a network where IPv6 was never configured in the first place.
val results = icmpResults(t)
// The corroborating witness: did a real TCP connection over IPv6 work on this
// network? Same evidence shape as the ICMP probe, so the same parser reads it.
val v6ConnTest = tests.firstOrNull { it.type == TestType.V6_BROKENNESS }
val v6Conn = v6ConnTest?.let { icmpResults(it) } ?: emptyMap()
var anyV6Network = false
for (n in networks) {
val provisioned = ipv6Provisioned(networks, n.id)
if (provisioned) anyV6Network = true
val r = results[n.id] ?: continue
// Silence is only evidence if something was actually sent. A bind that failed
// with EPERM says the app could not use the interface, which is a fact about
// this app's permissions and says nothing whatsoever about the network.
if (!provisioned || !r.attempted || r.ok) continue
val where = n.iface?.takeIf { it.isNotBlank() } ?: "this network"
val conn = v6Conn[n.id]
val evidence = listOfNotNull(
EvidenceRef(t.id), v6ConnTest?.let { EvidenceRef(it.id) },
) )
} else { when {
// TCP over IPv6 worked: the silence is filtering, and can be said so.
conn?.ok == true -> out.add(
Finding(
id = ids.uuid(), code = FindingRegistry.V6_NO_ICMP_REPLY.code,
category = FindingRegistry.V6_NO_ICMP_REPLY.category,
severity = FindingRegistry.V6_NO_ICMP_REPLY.severity,
confidence = Confidence.HIGH,
title = "ICMPv6 is filtered here — IPv6 itself works ($where)",
description = "$where answered a real TCP connection over IPv6, " +
"so IPv6 works — but ICMPv6 echo got no reply, so something " +
"on this network filters ICMPv6. That is a fault in its own " +
"right even though connections succeed: Path MTU Discovery " +
"depends on ICMPv6, so large packets can vanish rather than " +
"being reported as too big.",
evidenceRefs = evidence,
)
)
// Both transports failed on a network that advertises IPv6: broken, and
// now with the evidence the original v6.broken never had.
conn != null && conn.attempted -> out.add(
Finding(
id = ids.uuid(), code = FindingRegistry.V6_BROKEN.code,
category = FindingRegistry.V6_BROKEN.category,
severity = FindingRegistry.V6_BROKEN.severity,
confidence = Confidence.HIGH,
title = "IPv6 is advertised but does not work ($where)",
description = "$where advertises IPv6 (a global address and/or a " +
"default route), but neither ICMPv6 echo nor a TCP connection " +
"over IPv6 got through — two independent transports, both " +
"silent. Applications will try IPv6 first and wait out a " +
"timeout on every dual-stack destination before falling back " +
"to IPv4, felt as everything being slow with no loss to " +
"explain it. The network is announcing a service it does not " +
"deliver; the fix belongs on the router or upstream.",
evidenceRefs = evidence,
)
)
// No corroboration available (no server configured, or the connect never
// got as far as sending): the honest two-explanation reading stands.
else -> out.add(
Finding(
id = ids.uuid(), code = FindingRegistry.V6_NO_ICMP_REPLY.code,
category = FindingRegistry.V6_NO_ICMP_REPLY.category,
severity = FindingRegistry.V6_NO_ICMP_REPLY.severity,
confidence = Confidence.MEDIUM,
title = "IPv6 is configured, but ICMPv6 gets no reply ($where)",
description = "$where advertises IPv6 (a global address and/or a " +
"default route), but ICMPv6 echo got no reply over it. That has " +
"two explanations which look identical from here: IPv6 is broken, " +
"or ICMPv6 is filtered while IPv6 itself works. Filtering is " +
"common and is a fault in its own right — it breaks Path MTU " +
"Discovery, so large packets vanish rather than being reported as " +
"too big.",
evidenceRefs = listOf(EvidenceRef(t.id)),
)
)
}
}
if (!anyV6Network) {
// Said once for the device, not once per interface: "this network is IPv4-only"
// repeated per interface reads as several problems instead of one observation.
out.add( out.add(
Finding( Finding(
id = ids.uuid(), code = "ipv6.not_offered", category = Category.IPV6, id = ids.uuid(), code = FindingRegistry.V6_NOT_OFFERED.code,
severity = Severity.INFO, confidence = Confidence.HIGH, category = FindingRegistry.V6_NOT_OFFERED.category,
severity = FindingRegistry.V6_NOT_OFFERED.severity,
confidence = Confidence.HIGH,
title = "IPv4-only network (no IPv6 offered)", title = "IPv4-only network (no IPv6 offered)",
description = "No IPv6 address or default route was provisioned, so IPv6 tests could not run. This is normal — many networks are still IPv4-only and it is not a fault.", description = "No IPv6 address or default route was provisioned on " +
"any active network, so IPv6 tests could not run. This is normal " +
"— many networks are still IPv4-only and it is not a fault.",
evidenceRefs = listOf(EvidenceRef(t.id)), evidenceRefs = listOf(EvidenceRef(t.id)),
) )
) )
@@ -97,6 +97,41 @@ class Settings(context: Context) {
get() = prefs.getString(SERVER_PIN, "") ?: "" get() = prefs.getString(SERVER_PIN, "") ?: ""
set(v) = prefs.edit().putString(SERVER_PIN, v.trim()).apply() set(v) = prefs.edit().putString(SERVER_PIN, v.trim()).apply()
/**
* The address the operator handed out, for showing to a person.
*
* Separate from [serverUrl], which is the endpoint actually dialled. They differ when the
* server publishes one public name and points devices at another to select its pinned
* certificate — a detail worth keeping out of the user's face but not out of the settings.
*/
var serverPublicUrl: String
get() = (prefs.getString(SERVER_PUBLIC_URL, "") ?: "").ifBlank { serverUrl }
set(v) = prefs.edit().putString(SERVER_PUBLIC_URL, v.trim()).apply()
/**
* What the server said about itself, last time it was asked: addresses, ports, capabilities.
*
* Cached as a rendered block rather than as fields, because it is shown and never acted on —
* these are facts to read, not settings to apply, and storing them as settings would invite
* exactly the confusion of an editable box that changes nothing.
*/
var serverFacts: String
get() = prefs.getString(SERVER_FACTS, "") ?: ""
set(v) = prefs.edit().putString(SERVER_FACTS, v).apply()
/**
* The server's own addresses, learned from its profile, for reaching it when DNS will not.
*
* Only the primaries: the alternate pair exists for NAT behaviour discovery and does not carry
* the control plane, so falling back to one would fail for a second, unrelated reason.
*/
var serverAddrs: String
get() = prefs.getString(SERVER_ADDRS, "") ?: ""
set(v) = prefs.edit().putString(SERVER_ADDRS, v).apply()
fun serverAddrList(): List<String> =
serverAddrs.split(',').map { it.trim() }.filter { it.isNotEmpty() }
var serverCredential: String var serverCredential: String
get() = prefs.getString(SERVER_CRED, "") ?: "" get() = prefs.getString(SERVER_CRED, "") ?: ""
set(v) = prefs.edit().putString(SERVER_CRED, v.trim()).apply() set(v) = prefs.edit().putString(SERVER_CRED, v.trim()).apply()
@@ -104,6 +139,58 @@ class Settings(context: Context) {
val serverConfigured: Boolean val serverConfigured: Boolean
get() = serverUrl.isNotBlank() && serverPin.isNotBlank() && serverCredential.isNotBlank() get() = serverUrl.isNotBlank() && serverPin.isNotBlank() && serverCredential.isNotBlank()
/**
* The DNS zone this server is authoritative for, learned from its profile.
*
* Cached because the canary probe runs at device tier, before anything has talked to the
* server, and a probe that had to make a control-plane call first would fail on exactly the
* networks worth measuring. Empty means "not known yet", and the probe reports itself as
* skipped rather than inventing a zone.
*/
var canaryZone: String
get() = prefs.getString(CANARY_ZONE, "") ?: ""
set(v) = prefs.edit().putString(CANARY_ZONE, v.trim()).apply()
/**
* Host part of the configured server URL, for probes that address it directly (STUN).
*
* Derived rather than stored: a second copy of the server's name is a second thing to keep in
* step, and it would go stale the moment someone re-enrolled against a different server.
*/
fun serverHost(): String = runCatching {
java.net.URI(serverUrl).host?.takeIf { it.isNotBlank() }
}.getOrNull() ?: ""
// ---- account ---------------------------------------------------------------------
/**
* The PKCE verifier and state for a sign-in that is out at the browser.
*
* Persisted rather than held in memory because handing control to a browser backgrounds this
* process, and Android may kill it before the callback returns. An in-memory value works on a
* developer's device and fails on a phone under memory pressure.
*/
var pendingVerifier: String
get() = prefs.getString(PENDING_VERIFIER, "") ?: ""
set(v) = prefs.edit().putString(PENDING_VERIFIER, v).apply()
var pendingState: String
get() = prefs.getString(PENDING_STATE, "") ?: ""
set(v) = prefs.edit().putString(PENDING_STATE, v).apply()
fun clearPendingAuth() = prefs.edit().remove(PENDING_VERIFIER).remove(PENDING_STATE).apply()
/** Display name of whoever is signed in on this device; empty when nobody is. */
var accountName: String
get() = prefs.getString(ACCOUNT_NAME, "") ?: ""
set(v) = prefs.edit().putString(ACCOUNT_NAME, v).apply()
var accountId: String
get() = prefs.getString(ACCOUNT_ID, "") ?: ""
set(v) = prefs.edit().putString(ACCOUNT_ID, v).apply()
val signedIn: Boolean get() = accountName.isNotBlank()
private fun hex(s: String) = ByteArray(s.length / 2) { private fun hex(s: String) = ByteArray(s.length / 2) {
((Character.digit(s[it * 2], 16) shl 4) or Character.digit(s[it * 2 + 1], 16)).toByte() ((Character.digit(s[it * 2], 16) shl 4) or Character.digit(s[it * 2 + 1], 16)).toByte()
} }
@@ -120,5 +207,13 @@ class Settings(context: Context) {
const val SERVER_URL = "server_url" const val SERVER_URL = "server_url"
const val SERVER_PIN = "server_pin" const val SERVER_PIN = "server_pin"
const val SERVER_CRED = "server_credential" const val SERVER_CRED = "server_credential"
const val SERVER_PUBLIC_URL = "server_public_url"
const val SERVER_FACTS = "server_facts"
const val SERVER_ADDRS = "server_addrs"
const val CANARY_ZONE = "server_canary_zone"
const val PENDING_VERIFIER = "pending_auth_verifier"
const val PENDING_STATE = "pending_auth_state"
const val ACCOUNT_NAME = "account_name"
const val ACCOUNT_ID = "account_id"
} }
} }
@@ -4,19 +4,24 @@
package app.echo_lot.app package app.echo_lot.app
import androidx.compose.foundation.layout.Arrangement import androidx.compose.foundation.layout.Arrangement
import androidx.compose.foundation.layout.safeDrawingPadding
import androidx.compose.foundation.layout.Column import androidx.compose.foundation.layout.Column
import androidx.compose.foundation.layout.Row import androidx.compose.foundation.layout.Row
import androidx.compose.foundation.layout.Spacer import androidx.compose.foundation.layout.Spacer
import androidx.compose.foundation.layout.fillMaxWidth import androidx.compose.foundation.layout.fillMaxWidth
import androidx.compose.foundation.layout.height import androidx.compose.foundation.layout.height
import androidx.compose.foundation.layout.width
import androidx.compose.foundation.layout.padding import androidx.compose.foundation.layout.padding
import androidx.compose.foundation.shape.RoundedCornerShape
import androidx.compose.foundation.rememberScrollState import androidx.compose.foundation.rememberScrollState
import androidx.compose.foundation.verticalScroll import androidx.compose.foundation.verticalScroll
import androidx.compose.material3.Button import androidx.compose.material3.Button
import androidx.compose.material3.Card import androidx.compose.material3.Card
import androidx.compose.material3.FilterChip import androidx.compose.material3.FilterChip
import androidx.compose.material3.LocalContentColor
import androidx.compose.material3.MaterialTheme import androidx.compose.material3.MaterialTheme
import androidx.compose.material3.OutlinedTextField import androidx.compose.material3.OutlinedTextField
import androidx.compose.material3.Surface
import androidx.compose.material3.Switch import androidx.compose.material3.Switch
import androidx.compose.material3.Text import androidx.compose.material3.Text
import androidx.compose.material3.TextButton import androidx.compose.material3.TextButton
@@ -29,6 +34,7 @@ import androidx.compose.ui.Alignment
import androidx.compose.ui.Modifier import androidx.compose.ui.Modifier
import androidx.compose.ui.text.font.FontFamily import androidx.compose.ui.text.font.FontFamily
import androidx.compose.ui.unit.dp import androidx.compose.ui.unit.dp
import androidx.compose.ui.unit.sp
import app.echo_lot.privacy.PrivacyLevel import app.echo_lot.privacy.PrivacyLevel
/** /**
@@ -47,8 +53,12 @@ fun SettingsScreen(
onDeleteAll: () -> Unit, onDeleteAll: () -> Unit,
onPreviewUpload: () -> Unit, onPreviewUpload: () -> Unit,
onCheckServer: () -> Unit, onCheckServer: () -> Unit,
accountName: String,
onSignIn: () -> Unit,
onSignOut: () -> Unit,
onEnroll: (String) -> Unit, onEnroll: (String) -> Unit,
serverStatus: String?, serverStatus: String?,
enrollStatus: String?,
onBack: () -> Unit, onBack: () -> Unit,
) { ) {
// SharedPreferences is not observable, so mirror each value into Compose state and write // SharedPreferences is not observable, so mirror each value into Compose state and write
@@ -61,12 +71,25 @@ fun SettingsScreen(
var privacy by remember { mutableStateOf(settings.privacyLevel) } var privacy by remember { mutableStateOf(settings.privacyLevel) }
var stableSalt by remember { mutableStateOf(settings.stableSalt) } var stableSalt by remember { mutableStateOf(settings.stableSalt) }
var enrollLink by remember { mutableStateOf("") } var enrollLink by remember { mutableStateOf("") }
var serverUrl by remember { mutableStateOf(settings.serverUrl) } // The public name, which is what the operator handed out and what a person recognises. The
// endpoint actually dialled is shown beneath it when the two differ, rather than hidden — a
// network engineer debugging a connection wants to see where it really goes.
var serverUrl by remember { mutableStateOf(settings.serverPublicUrl) }
var serverPin by remember { mutableStateOf(settings.serverPin) } var serverPin by remember { mutableStateOf(settings.serverPin) }
var serverCred by remember { mutableStateOf(settings.serverCredential) } var serverCred by remember { mutableStateOf(settings.serverCredential) }
// Enrolling is asynchronous, so these are re-read when its result lands rather than when the
// button is pressed — reading them immediately showed the previous server's values and looked
// exactly like an enrollment that had silently done nothing.
var serverFacts by remember { mutableStateOf(settings.serverFacts) }
androidx.compose.runtime.LaunchedEffect(enrollStatus, serverStatus) {
serverFacts = settings.serverFacts
serverUrl = settings.serverPublicUrl
serverPin = settings.serverPin
serverCred = settings.serverCredential
}
Column( Column(
Modifier.fillMaxWidth().verticalScroll(rememberScrollState()).padding(16.dp), Modifier.fillMaxWidth().safeDrawingPadding().verticalScroll(rememberScrollState()).padding(16.dp),
verticalArrangement = Arrangement.spacedBy(12.dp), verticalArrangement = Arrangement.spacedBy(12.dp),
) { ) {
Row(verticalAlignment = Alignment.CenterVertically) { Row(verticalAlignment = Alignment.CenterVertically) {
@@ -129,18 +152,59 @@ fun SettingsScreen(
} }
Text(privacyExplanation(privacy), style = MaterialTheme.typography.bodySmall) Text(privacyExplanation(privacy), style = MaterialTheme.typography.bodySmall)
// At FULL nothing is pseudonymized, so a salt has nothing to act on. Shown
// disabled rather than hidden: the setting is still stored and still applies the
// moment the level changes, and a control that vanishes hides that fact.
Toggle( Toggle(
label = "Stable pseudonyms across runs", label = "Stable pseudonyms across runs",
detail = "Lets you compare uploaded runs over time (same SSID reads the same " + detail = if (privacy == PrivacyLevel.FULL) {
"each time). It also links your uploads together, so leave it off on a " + "Not used at this level — nothing is pseudonymized, so there is nothing " +
"server you don't run yourself.", "to keep stable. Choose balanced or strict to use this."
checked = stableSalt, } else {
"Lets you compare uploaded runs over time (same SSID reads the same " +
"each time). It also links your uploads together, so leave it off on " +
"a server you don't run yourself."
},
checked = stableSalt && privacy != PrivacyLevel.FULL,
enabled = privacy != PrivacyLevel.FULL,
) { stableSalt = it; settings.stableSalt = it } ) { stableSalt = it; settings.stableSalt = it }
TextButton(onClick = onPreviewUpload) { Text("Preview what an upload would send") } TextButton(onClick = onPreviewUpload) { Text("Preview what an upload would send") }
} }
} }
// ---- account ----
Card(Modifier.fillMaxWidth()) {
Column(Modifier.padding(14.dp), verticalArrangement = Arrangement.spacedBy(8.dp)) {
Text("Account", style = MaterialTheme.typography.titleMedium)
if (accountName.isNotBlank()) {
Text("Signed in as $accountName", style = MaterialTheme.typography.bodyMedium)
Text(
"Runs from every device signed in to this account share one history.",
style = MaterialTheme.typography.bodySmall,
)
TextButton(onClick = onSignOut) { Text("Sign out") }
} else {
Text(
"Signing in is optional. It links this device to an account on your " +
"server, so several devices share one history — and some servers only " +
"accept uploads from a signed-in device.",
style = MaterialTheme.typography.bodySmall,
)
Button(onClick = onSignIn, enabled = settings.serverConfigured) {
Text("Sign in")
}
if (!settings.serverConfigured) {
Text(
"Enrol with a server first — the account belongs to the server, not " +
"to the app.",
style = MaterialTheme.typography.bodySmall,
)
}
}
}
}
// ---- upload ---- // ---- upload ----
Card(Modifier.fillMaxWidth()) { Card(Modifier.fillMaxWidth()) {
Column(Modifier.padding(14.dp), verticalArrangement = Arrangement.spacedBy(8.dp)) { Column(Modifier.padding(14.dp), verticalArrangement = Arrangement.spacedBy(8.dp)) {
@@ -172,17 +236,38 @@ fun SettingsScreen(
onClick = { onClick = {
onEnroll(enrollLink) onEnroll(enrollLink)
enrollLink = "" // spent either way; leaving it around invites a retry enrollLink = "" // spent either way; leaving it around invites a retry
serverUrl = settings.serverUrl
serverPin = settings.serverPin
serverCred = settings.serverCredential
}, },
enabled = enrollLink.isNotBlank(), enabled = enrollLink.isNotBlank(),
) { Text("Enroll") } ) { Text("Enroll") }
// Beside the button that caused it. Enrolling is asynchronous, so without this the
// only sign of success is three fields quietly changing further down the card.
enrollStatus?.let {
Text(it, style = MaterialTheme.typography.bodySmall)
}
OutlinedTextField( OutlinedTextField(
value = serverUrl, onValueChange = { serverUrl = it; settings.serverUrl = it }, value = serverUrl,
onValueChange = {
serverUrl = it
// Typed by hand there is no discovery to consult, so what was entered is
// both the public name and the endpoint. Setting only one of them would
// leave the app dialling the previous server.
settings.serverUrl = it
settings.serverPublicUrl = it
},
label = { Text("Server URL") }, singleLine = true, modifier = Modifier.fillMaxWidth(), label = { Text("Server URL") }, singleLine = true, modifier = Modifier.fillMaxWidth(),
) )
// Directly under the field it explains. Anywhere else it reads as a stray sentence
// about some other part of the screen.
if (settings.serverUrl.isNotBlank() && settings.serverUrl != settings.serverPublicUrl) {
Text(
"Connects to ${settings.serverUrl} — this server publishes one name and " +
"points devices at another, so its pinned certificate can share a port " +
"with its web interface.",
style = MaterialTheme.typography.bodySmall,
color = LocalContentColor.current.copy(alpha = 0.7f),
)
}
OutlinedTextField( OutlinedTextField(
value = serverPin, onValueChange = { serverPin = it; settings.serverPin = it }, value = serverPin, onValueChange = { serverPin = it; settings.serverPin = it },
label = { Text("Certificate pin (SPKI, base64)") }, singleLine = true, label = { Text("Certificate pin (SPKI, base64)") }, singleLine = true,
@@ -209,6 +294,52 @@ fun SettingsScreen(
serverStatus?.let { serverStatus?.let {
Text(it, style = MaterialTheme.typography.bodySmall) Text(it, style = MaterialTheme.typography.bodySmall)
} }
// What the server reported, placed under the button that asks it rather than among
// the fields above: these are facts to read, not settings to apply, and an
// editable-looking box that changes nothing is worse than no box at all.
//
// Monospaced so the addresses line up under each other — column alignment is most
// of what makes a list of IPs quicker to read than prose.
if (serverFacts.isNotBlank()) {
Surface(
color = MaterialTheme.colorScheme.surfaceVariant,
shape = RoundedCornerShape(8.dp),
modifier = Modifier.fillMaxWidth(),
) {
Column(
Modifier.padding(horizontal = 12.dp, vertical = 10.dp),
verticalArrangement = Arrangement.spacedBy(2.dp),
) {
Text(
"WHAT THIS SERVER REPORTS",
style = MaterialTheme.typography.labelSmall,
color = LocalContentColor.current.copy(alpha = 0.7f),
)
// Real columns rather than padded text: the label column has a fixed
// width, so values line up whatever the font does, and a long value
// wraps inside its own column instead of under the labels.
for (line in serverFacts.lines()) {
val label = line.substringBefore('|')
val value = line.substringAfter('|', "")
Row(Modifier.fillMaxWidth()) {
Text(
label,
style = MaterialTheme.typography.bodySmall,
color = LocalContentColor.current.copy(alpha = 0.7f),
modifier = Modifier.width(72.dp),
)
Text(
value,
style = MaterialTheme.typography.bodySmall.copy(
fontFamily = FontFamily.Monospace,
),
modifier = Modifier.weight(1f),
)
}
}
}
}
}
Text( Text(
"This app is ${BuildConfig.APP_SEMVER} and speaks probe protocol " + "This app is ${BuildConfig.APP_SEMVER} and speaks probe protocol " +
"${app.echo_lot.protocol.Compat.PROTOCOL_VERSION}. It works with servers " + "${app.echo_lot.protocol.Compat.PROTOCOL_VERSION}. It works with servers " +
@@ -236,13 +367,22 @@ private fun privacyExplanation(level: PrivacyLevel): String = when (level) {
} }
@Composable @Composable
private fun Toggle(label: String, detail: String, checked: Boolean, onChange: (Boolean) -> Unit) { private fun Toggle(
label: String,
detail: String,
checked: Boolean,
enabled: Boolean = true,
onChange: (Boolean) -> Unit,
) {
Row(Modifier.fillMaxWidth(), verticalAlignment = Alignment.Top) { Row(Modifier.fillMaxWidth(), verticalAlignment = Alignment.Top) {
Column(Modifier.weight(1f)) { Column(Modifier.weight(1f)) {
Text(label, style = MaterialTheme.typography.bodyMedium) // Dimmed together with the switch, so "this does nothing right now" reads at a glance
Text(detail, style = MaterialTheme.typography.bodySmall) // instead of only on close inspection.
val alpha = if (enabled) 1f else 0.5f
Text(label, style = MaterialTheme.typography.bodyMedium, color = LocalContentColor.current.copy(alpha = alpha))
Text(detail, style = MaterialTheme.typography.bodySmall, color = LocalContentColor.current.copy(alpha = alpha))
} }
Switch(checked = checked, onCheckedChange = onChange) Switch(checked = checked, onCheckedChange = onChange, enabled = enabled)
} }
} }
@@ -31,10 +31,23 @@ data class ArchivedRun(
val verdict: String? = null, val verdict: String? = null,
@SerialName("finding_count") val findingCount: Int = 0, @SerialName("finding_count") val findingCount: Int = 0,
@SerialName("size_bytes") val sizeBytes: Long = 0, @SerialName("size_bytes") val sizeBytes: Long = 0,
/**
* How the *archived* document is redacted. Always "full" in practice, because the archive
* deliberately keeps the unredacted run - see the package doc. This is not what was uploaded.
*/
val anonymization: String = "full", val anonymization: String = "full",
/** Whether this run has been accepted by a server, so history can show what is backed up. */ /** Whether this run has been accepted by a server, so history can show what is backed up. */
val uploaded: Boolean = false, val uploaded: Boolean = false,
@SerialName("uploaded_to") val uploadedTo: String? = null, @SerialName("uploaded_to") val uploadedTo: String? = null,
/**
* The level the run was *uploaded* at, which is a different document from the archived one.
*
* Kept separately because conflating the two is actively misleading: the history row showed
* the archive's own level ("full") directly beneath "uploaded to fmr", which reads as "the
* complete data was uploaded" when a redacted copy had been sent. A privacy display that
* overstates what left the device is worse than none.
*/
@SerialName("uploaded_as") val uploadedAs: String? = null,
) )
/** /**
@@ -119,7 +132,7 @@ class RunArchive(private val dir: File, private val now: () -> Long = System::cu
fun deleteAll(): Int = list().count { delete(it.id) } fun deleteAll(): Int = list().count { delete(it.id) }
/** Records that a server accepted this run, so history can distinguish backed-up from local. */ /** Records that a server accepted this run, so history can distinguish backed-up from local. */
fun markUploaded(id: String, serverName: String) { fun markUploaded(id: String, serverName: String, uploadedAs: String? = null) {
val f = File(dir, safe(id) + META_EXT) val f = File(dir, safe(id) + META_EXT)
val meta = runCatching { json.decodeFromString(ArchivedRun.serializer(), f.readText()) }.getOrNull() val meta = runCatching { json.decodeFromString(ArchivedRun.serializer(), f.readText()) }.getOrNull()
?: return ?: return
@@ -127,7 +140,7 @@ class RunArchive(private val dir: File, private val now: () -> Long = System::cu
f, f,
json.encodeToString( json.encodeToString(
ArchivedRun.serializer(), ArchivedRun.serializer(),
meta.copy(uploaded = true, uploadedTo = serverName), meta.copy(uploaded = true, uploadedTo = serverName, uploadedAs = uploadedAs),
), ),
) )
} }
@@ -181,7 +194,10 @@ class RunArchive(private val dir: File, private val now: () -> Long = System::cu
id = id, id = id,
savedAtEpochMs = now(), savedAtEpochMs = now(),
startedAt = run["started_at"]?.jsonPrimitive?.content, startedAt = run["started_at"]?.jsonPrimitive?.content,
verdict = doc["summary"]?.jsonObject?.get("verdict")?.jsonPrimitive?.content, // The schema calls it `overall` (Summary.overall); reading `verdict` here silently
// yielded null for every run, so the history list's most prominent element - the
// coloured verdict - was blank on every row.
verdict = doc["summary"]?.jsonObject?.get("overall")?.jsonPrimitive?.content,
findingCount = (doc["findings"] as? kotlinx.serialization.json.JsonArray)?.size ?: 0, findingCount = (doc["findings"] as? kotlinx.serialization.json.JsonArray)?.size ?: 0,
sizeBytes = size, sizeBytes = size,
anonymization = run["privacy"]?.jsonObject?.get("anonymization")?.jsonPrimitive?.content ?: "full", anonymization = run["privacy"]?.jsonObject?.get("anonymization")?.jsonPrimitive?.content ?: "full",
@@ -25,7 +25,7 @@ class RunArchiveTest {
private fun doc(id: String, findings: Int = 1, pad: Int = 0): String { private fun doc(id: String, findings: Int = 1, pad: Int = 0): String {
val f = (1..findings).joinToString(",") { """{"id":"f$it"}""" } val f = (1..findings).joinToString(",") { """{"id":"f$it"}""" }
return """{"run":{"id":"$id","started_at":"2026-08-01T10:00:00Z","privacy":{"anonymization":"balanced"}},""" + return """{"run":{"id":"$id","started_at":"2026-08-01T10:00:00Z","privacy":{"anonymization":"balanced"}},""" +
""""findings":[$f],"summary":{"verdict":"warn"},"pad":"${"x".repeat(pad)}"}""" """"findings":[$f],"summary":{"overall":"warn"},"pad":"${"x".repeat(pad)}"}"""
} }
@Test @Test
@@ -120,13 +120,40 @@ class RunArchiveTest {
fun uploadStateIsRecorded() { fun uploadStateIsRecorded() {
val a = archive() val a = archive()
a.save(doc("run-1")) a.save(doc("run-1"))
a.markUploaded("run-1", "fmr") a.markUploaded("run-1", "fmr", "balanced")
val meta = a.list().single() val meta = a.list().single()
assertTrue(meta.uploaded) assertTrue(meta.uploaded)
assertEquals("fmr", meta.uploadedTo) assertEquals("fmr", meta.uploadedTo)
assertEquals("run-1", meta.id, "marking upload must not disturb the rest of the entry") assertEquals("run-1", meta.id, "marking upload must not disturb the rest of the entry")
} }
// The archive's own level and the level a run was uploaded at describe *different documents*.
// Showing the archive's ("full", because the archive is deliberately unredacted) next to
// "uploaded to fmr" reads as "the complete data was uploaded" when a redacted copy was sent —
// a privacy display that overstates what left the device is worse than none.
@Test
fun theUploadedLevelIsRecordedSeparatelyFromTheArchivedOne() {
val a = archive()
// A real archived document carries no privacy stamp: the anonymizer never runs on the
// archive. The shared doc() fixture has one, which is exactly the unrealism that let this
// confusion through in the first place.
a.save("""{"run":{"id":"run-1"},"findings":[],"summary":{"overall":"green"}}""")
a.markUploaded("run-1", "fmr", "balanced")
val meta = a.list().single()
assertEquals("full", meta.anonymization, "the archived copy is unredacted, by design")
assertEquals("balanced", meta.uploadedAs, "the uploaded copy was redacted, and must say so")
}
// The verdict is read from `summary.overall` — the schema's actual field name. Reading
// `summary.verdict` silently yielded null for every run, so the history list's most prominent
// element was blank on every row while everything else looked fine.
@Test
fun theVerdictComesFromTheSchemasOverallField() {
val a = archive()
a.save("""{"run":{"id":"r1"},"findings":[],"summary":{"overall":"yellow"}}""")
assertEquals("yellow", a.list().single().verdict)
}
@Test @Test
fun deleteRemovesBothFiles() { fun deleteRemovesBothFiles() {
val a = archive() val a = archive()
@@ -118,7 +118,7 @@ class DownstreamMeasurement(private val ids: IdSource) {
if (!inOrder) { if (!inOrder) {
findings.add( findings.add(
finding( finding(
"mtu.fragments_blocked", Category.MTU, Severity.MEDIUM, testId, FindingRegistry.FRAGMENTS_BLOCKED, testId,
"IP fragments do not reach this device", "IP fragments do not reach this device",
"A fragmented datagram sent in the normal order never arrived. Anything that " + "A fragmented datagram sent in the normal order never arrived. Anything that " +
"relies on fragmentation — large DNS answers over UDP, some VPN traffic — " + "relies on fragmentation — large DNS answers over UDP, some VPN traffic — " +
@@ -133,7 +133,7 @@ class DownstreamMeasurement(private val ids: IdSource) {
}.joinToString(" or ") }.joinToString(" or ")
findings.add( findings.add(
finding( finding(
"mtu.fragment_reorder_sensitive", Category.MTU, Severity.LOW, testId, FindingRegistry.FRAGMENT_REORDER_SENSITIVE, testId,
"Fragments are dropped when they arrive $which", "Fragments are dropped when they arrive $which",
"In-order fragments are delivered, but the same datagram sent $which is not. " + "In-order fragments are delivered, but the same datagram sent $which is not. " +
"Something on the path only reassembles when the first fragment (the one " + "Something on the path only reassembles when the first fragment (the one " +
@@ -198,7 +198,7 @@ class DownstreamMeasurement(private val ids: IdSource) {
if (ipMtu < 1500) { if (ipMtu < 1500) {
findings.add( findings.add(
finding( finding(
"mtu.reduced_downstream", Category.MTU, Severity.LOW, df.test.id, FindingRegistry.MTU_REDUCED_DOWNSTREAM, df.test.id,
"Downstream path MTU is $ipMtu bytes, below 1500", "Downstream path MTU is $ipMtu bytes, below 1500",
"The largest datagram that reached this device without fragmenting was " + "The largest datagram that reached this device without fragmenting was " +
"$pathMtu bytes of payload ($ipMtu on the wire). Tunnels (PPPoE, VPN, " + "$pathMtu bytes of payload ($ipMtu on the wire). Tunnels (PPPoE, VPN, " +
@@ -213,7 +213,7 @@ class DownstreamMeasurement(private val ids: IdSource) {
if (fragLargest <= pathMtu && sizes.any { it > pathMtu }) { if (fragLargest <= pathMtu && sizes.any { it > pathMtu }) {
findings.add( findings.add(
finding( finding(
"mtu.downstream_blackhole", Category.MTU, Severity.MEDIUM, frag.test.id, FindingRegistry.MTU_DOWNSTREAM_BLACKHOLE, frag.test.id,
"Datagrams above $pathMtu bytes are dropped downstream, fragmented or not", "Datagrams above $pathMtu bytes are dropped downstream, fragmented or not",
"Nothing larger than $pathMtu bytes arrived, even when the network was " + "Nothing larger than $pathMtu bytes arrived, even when the network was " +
"free to fragment it. Traffic that relies on large responses will " + "free to fragment it. Traffic that relies on large responses will " +
@@ -226,7 +226,7 @@ class DownstreamMeasurement(private val ids: IdSource) {
if (train.received == 0) { if (train.received == 0) {
findings.add( findings.add(
finding( finding(
"connectivity.downstream_blocked", Category.CONNECTIVITY, Severity.HIGH, train.test.id, FindingRegistry.DOWNSTREAM_BLOCKED, train.test.id,
"No server-initiated packets arrived", "No server-initiated packets arrived",
"The server sent ${train.sent} packets toward this device and none arrived, " + "The server sent ${train.sent} packets toward this device and none arrived, " +
"while the round-trip echo worked. Something on the path forwards replies " + "while the round-trip echo worked. Something on the path forwards replies " +
@@ -236,7 +236,7 @@ class DownstreamMeasurement(private val ids: IdSource) {
} else if (train.lossPct >= 5.0) { } else if (train.lossPct >= 5.0) {
findings.add( findings.add(
finding( finding(
"connectivity.downstream_loss", Category.CONNECTIVITY, Severity.MEDIUM, train.test.id, FindingRegistry.LOSS_DOWNSTREAM, train.test.id,
"Downstream loss of ${round1(train.lossPct)}%", "Downstream loss of ${round1(train.lossPct)}%",
"${train.sent - train.received} of ${train.sent} packets sent toward this " + "${train.sent - train.received} of ${train.sent} packets sent toward this " +
"device were lost. Downstream loss is invisible to a round-trip test, " + "device were lost. Downstream loss is invisible to a round-trip test, " +
@@ -247,7 +247,7 @@ class DownstreamMeasurement(private val ids: IdSource) {
if (train.reordered > 0) { if (train.reordered > 0) {
findings.add( findings.add(
finding( finding(
"connectivity.downstream_reorder", Category.CONNECTIVITY, Severity.LOW, train.test.id, FindingRegistry.DOWNSTREAM_REORDER, train.test.id,
"${train.reordered} downstream packet(s) arrived out of order", "${train.reordered} downstream packet(s) arrived out of order",
"Packets arrived in a different order than they were sent. Usually per-packet " + "Packets arrived in a different order than they were sent. Usually per-packet " +
"load balancing across links; harmless for most traffic, not for all of it.", "load balancing across links; harmless for most traffic, not for all of it.",
@@ -414,9 +414,17 @@ class DownstreamMeasurement(private val ids: IdSource) {
// ---- helpers ---------------------------------------------------------------------- // ---- helpers ----------------------------------------------------------------------
private fun finding(code: String, cat: Category, sev: Severity, testId: String, title: String, desc: String) = /**
* Builds a finding from a registry entry, which supplies the code, category and severity.
*
* Taking a [FindingSpec] rather than three loose values is the point: a typo becomes a
* compile error, and two call sites cannot disagree about which category a finding belongs
* to - a disagreement that would split one fault across two verdict lights.
*/
private fun finding(spec: FindingSpec, testId: String, title: String, desc: String) =
Finding( Finding(
id = ids.uuid(), code = code, category = cat, severity = sev, confidence = Confidence.HIGH, id = ids.uuid(), code = spec.code, category = spec.category, severity = spec.severity,
confidence = Confidence.HIGH,
title = title, description = desc, evidenceRefs = listOf(EvidenceRef(testId)), title = title, description = desc, evidenceRefs = listOf(EvidenceRef(testId)),
) )
@@ -47,6 +47,14 @@ class ServerMeasurement(
* it is a flag rather than an assumption. * it is a flag rather than an assumption.
*/ */
val downstream: Boolean = true, val downstream: Boolean = true,
/**
* Throughput moves real data — a 5-second run at 50 Mbps is about 30 MB — so it is off
* unless asked for. On a metered mobile connection that is the user's money, and a
* measurement tool that spends it without being told to is not one people keep installed.
*/
val throughput: Boolean = false,
@Suppress("unused") val throughputSeconds: Int = 5,
@Suppress("unused") val throughputKbps: Int = 50_000,
) )
fun run(cfg: Config): MeasurementDocument { fun run(cfg: Config): MeasurementDocument {
@@ -89,6 +97,15 @@ class ServerMeasurement(
tests.addAll(dsTests) tests.addAll(dsTests)
allFindings.addAll(dsFindings) allFindings.addAll(dsFindings)
} }
if (cfg.throughput && profile.supports("throughput")) {
val (tpTest, tpFindings) = ThroughputMeasurement(ids).run(
cfg.credential, session.sessionId, control, ps, sessionRef = "sess-1",
durationS = cfg.throughputSeconds, kbps = cfg.throughputKbps,
)
tests.add(tpTest)
allFindings.addAll(tpFindings)
}
} }
control.deleteSession(cfg.credential, session.sessionId) control.deleteSession(cfg.credential, session.sessionId)
@@ -191,11 +208,11 @@ class ServerMeasurement(
val findings = ArrayList<Finding>() val findings = ArrayList<Finding>()
if (received == 0) { if (received == 0) {
findings.add(finding("nat.udp_unreachable", Category.CONNECTIVITY, Severity.HIGH, testId, findings.add(finding(FindingRegistry.UDP_UNREACHABLE, testId,
"No UDP echo replies from the server", "No UDP echo replies from the server",
"Every ECHO probe to the server's UDP data plane was lost — the path blocks or drops the session's UDP traffic.")) "Every ECHO probe to the server's UDP data plane was lost — the path blocks or drops the session's UDP traffic."))
} else if (lossPct >= 20.0) { } else if (lossPct >= 20.0) {
findings.add(finding("connectivity.udp_loss", Category.CONNECTIVITY, Severity.MEDIUM, testId, findings.add(finding(FindingRegistry.UDP_LOSS, testId,
"High UDP loss to the server (${round1(lossPct)}%)", "High UDP loss to the server (${round1(lossPct)}%)",
"A large fraction of ECHO probes were lost, indicating an unreliable UDP path.")) "A large fraction of ECHO probes were lost, indicating an unreliable UDP path."))
} }
@@ -203,14 +220,14 @@ class ServerMeasurement(
directional?.let { d -> directional?.let { d ->
when { when {
d.noneReachedServer && received == 0 -> findings.add( d.noneReachedServer && received == 0 -> findings.add(
finding("nat.udp_unreachable_upstream", Category.CONNECTIVITY, Severity.HIGH, testId, finding(FindingRegistry.UDP_UNREACHABLE_UPSTREAM, testId,
"Nothing reached the server", "Nothing reached the server",
"The server received none of the ${d.sent} probes, so the traffic is being " + "The server received none of the ${d.sent} probes, so the traffic is being " +
"dropped on the way out, not on the way back. A firewall or NAT on " + "dropped on the way out, not on the way back. A firewall or NAT on " +
"this side of the path is the place to look."), "this side of the path is the place to look."),
) )
d.lossUpstreamPct >= 2.0 -> findings.add( d.lossUpstreamPct >= 2.0 -> findings.add(
finding("connectivity.loss_upstream", Category.CONNECTIVITY, Severity.MEDIUM, testId, finding(FindingRegistry.LOSS_UPSTREAM, testId,
"${d.lossUpstreamPct} % of probes were lost on the way to the server", "${d.lossUpstreamPct} % of probes were lost on the way to the server",
"${d.lostUpstream} of ${d.sent} probes never reached the server. The " + "${d.lostUpstream} of ${d.sent} probes never reached the server. The " +
"return path is not implicated: replies came back for everything that " + "return path is not implicated: replies came back for everything that " +
@@ -219,7 +236,7 @@ class ServerMeasurement(
} }
if (d.lossDownstreamPct >= 2.0) { if (d.lossDownstreamPct >= 2.0) {
findings.add( findings.add(
finding("connectivity.loss_downstream", Category.CONNECTIVITY, Severity.MEDIUM, testId, finding(FindingRegistry.LOSS_DOWNSTREAM, testId,
"${d.lossDownstreamPct} % of replies were lost on the way back", "${d.lossDownstreamPct} % of replies were lost on the way back",
"The server received ${d.seenByServer} probes and answered them, but " + "The server received ${d.seenByServer} probes and answered them, but " +
"${d.lostDownstream} of those replies never arrived. The outbound path " + "${d.lostDownstream} of those replies never arrived. The outbound path " +
@@ -229,7 +246,7 @@ class ServerMeasurement(
} }
if (natRebinding) { if (natRebinding) {
findings.add(finding("nat.udp_rebinding", Category.NAT, Severity.MEDIUM, testId, findings.add(finding(FindingRegistry.NAT_UDP_REBINDING, testId,
"NAT remapped the UDP source port mid-flow", "NAT remapped the UDP source port mid-flow",
"The server observed more than one source port for this session (${observedPorts.joinToString()}), i.e. a NAT with a short UDP mapping or per-packet remapping.")) "The server observed more than one source port for this session (${observedPorts.joinToString()}), i.e. a NAT with a short UDP mapping or per-packet remapping."))
} }
@@ -259,9 +276,17 @@ class ServerMeasurement(
} }
} }
private fun finding(code: String, cat: Category, sev: Severity, testId: String, title: String, desc: String) = /**
* Builds a finding from a registry entry, which supplies the code, category and severity.
*
* Taking a [FindingSpec] rather than three loose values is the point: a typo becomes a
* compile error, and two call sites cannot disagree about which category a finding belongs
* to - a disagreement that would split one fault across two verdict lights.
*/
private fun finding(spec: FindingSpec, testId: String, title: String, desc: String) =
Finding( Finding(
id = ids.uuid(), code = code, category = cat, severity = sev, confidence = Confidence.HIGH, id = ids.uuid(), code = spec.code, category = spec.category, severity = spec.severity,
confidence = Confidence.HIGH,
title = title, description = desc, evidenceRefs = listOf(EvidenceRef(testId)), title = title, description = desc, evidenceRefs = listOf(EvidenceRef(testId)),
) )
@@ -0,0 +1,338 @@
// SPDX-FileCopyrightText: 2026 Echolot contributors
// SPDX-License-Identifier: GPL-3.0-or-later
package app.echo_lot.engine
import app.echo_lot.measurement.*
import app.echo_lot.protocol.ControlClient
import app.echo_lot.protocol.ProbeSession
import app.echo_lot.protocol.Wire
import kotlinx.serialization.SerialName
import kotlinx.serialization.Serializable
import kotlinx.serialization.json.Json
import kotlinx.serialization.json.JsonObject
import kotlinx.serialization.json.encodeToJsonElement
import kotlinx.serialization.json.jsonArray
import kotlinx.serialization.json.jsonObject
import kotlinx.serialization.json.jsonPrimitive
/**
* Downstream throughput: the server sends at a paced rate for a bounded time and the client
* measures what arrives (`perf.throughput_udp`).
*
* The number this produces is only meaningful with a qualifier attached, and getting that
* qualifier right is most of the work here. A throughput test reports the *smallest* limit on the
* path, and the sender's own ceiling is one of the candidates: if the server was asked for 50 Mbps
* and 50 Mbps arrived, the network was never the constraint and "50 Mbps" says nothing about it.
* Reporting that as a capacity measurement would be a confident lie, so the result always carries
* [ThroughputMetrics.limitedBy] and a finding is only raised when the network is actually
* implicated.
*
* Comparing against the *sender's* count rather than the requested rate is the other half: the
* server reports how much it actually put on the wire, and the gap between that and what arrived
* is the loss. A receiver alone cannot tell "the network dropped it" from "the sender never sent
* it", and guessing turns a healthy server-side limit into a phantom network fault.
*/
class ThroughputMeasurement(private val ids: IdSource) {
private val json = Json { encodeDefaults = true; explicitNulls = true }
fun run(
credential: String,
sessionId: String,
control: ControlClient,
probe: ProbeSession,
sessionRef: String,
durationS: Int = 5,
kbps: Int = 50_000,
sizeBytes: Int = 1200,
): Pair<Test, List<Finding>> {
val testId = ids.uuid()
val started = ids.monoNs()
val reply = runCatching {
control.action(
credential, sessionId,
"""{"action":"throughput","direction":"down","duration_s":$durationS,""" +
""""kbps":$kbps,"size_bytes":$sizeBytes}""",
)
}
if (reply.isFailure) {
return Test(
id = testId, type = TestType.PERF_THROUGHPUT_UDP, sessionRef = sessionRef, tier = Tier.APP,
startedMonoNs = started, endedMonoNs = ids.monoNs(),
status = TestStatus.UNSUPPORTED,
error = TestError("action_refused", reply.exceptionOrNull()?.message ?: "throughput refused"),
) to emptyList()
}
// The server may have shortened the run to fit its own byte budget; listen for what it
// actually promised, not for what we asked.
val plannedMs = parseInt(reply.getOrNull(), "duration_ms") ?: (durationS * 1000)
// A margin past the planned end so the tail of the run is not counted as loss: packets
// still in flight when we stop listening were not dropped, they were merely late.
val received = probe.collectGranted(plannedMs + 1_500L)
.filter { it.type == Wire.TYPE_THROUGHPUT_DATA }
val bytes = received.sumOf { it.sizeBytes.toLong() }
val spanNs = if (received.size >= 2) {
received.maxOf { it.tRxNs } - received.minOf { it.tRxNs }
} else {
0L
}
// Measured over the arrival span rather than our listening window, which includes the
// request round trip and the trailing margin and would understate the rate.
val receivedKbps = if (spanNs > 0) (bytes * 8 * 1_000_000 / spanNs).toInt() else 0
val sender = senderReport(control, credential, sessionId)
val sentPackets = sender?.packets ?: 0
val lossPct = if (sentPackets > 0) {
round2((sentPackets - received.size).coerceAtLeast(0) * 100.0 / sentPackets)
} else {
null
}
// Only a run the *clock* ended measured the network. One stopped by our own byte budget
// or rate ceiling measured this server.
val limitedBy = sender?.limitedBy ?: "unknown"
val networkLimited = limitedBy == "duration" &&
sender != null && receivedKbps > 0 && receivedKbps < sender.kbps * 9 / 10
val metrics = json.encodeToJsonElement(
ThroughputMetrics(
requestedKbps = kbps,
plannedDurationMs = plannedMs,
packetsReceived = received.size,
bytesReceived = bytes,
receivedKbps = receivedKbps,
senderPackets = sender?.packets,
senderBytes = sender?.bytes,
senderKbps = sender?.kbps,
lossPct = lossPct,
limitedBy = limitedBy,
measuresNetwork = networkLimited,
),
) as JsonObject
val findings = ArrayList<Finding>()
when {
sender == null -> Unit // no sender report: nothing can be concluded, so nothing is
received.isEmpty() -> findings.add(
finding(
FindingRegistry.THROUGHPUT_NO_DELIVERY, testId,
"No throughput traffic arrived",
"The server sent ${sender.packets} packets and none arrived. This is a " +
"connectivity fault rather than a slow link.",
),
)
networkLimited -> findings.add(
finding(
FindingRegistry.THROUGHPUT_BELOW_OFFERED, testId,
"Downstream throughput ${receivedKbps / 1000} Mbit/s, below the " +
"${sender.kbps / 1000} Mbit/s offered",
"The server sent at ${sender.kbps / 1000} Mbit/s for the full run and " +
"${receivedKbps / 1000} Mbit/s arrived" +
(lossPct?.let { ", losing $it % of packets" } ?: "") +
". The path could not carry what was offered.",
),
)
}
return Test(
id = testId, type = TestType.PERF_THROUGHPUT_UDP, sessionRef = sessionRef, tier = Tier.APP,
startedMonoNs = started, endedMonoNs = ids.monoNs(),
status = if (received.isEmpty()) TestStatus.FAILED else TestStatus.OK,
metrics = metrics,
) to findings
}
/**
* Upstream throughput: the client sends, the server counts.
*
* The mirror image of the downstream case, and it needs no grant — the client is generating
* its own traffic, so there is no amplification to gate. What it does need is the server's
* count: only the far end knows how much arrived, and without that number a sender can
* measure how fast it can *transmit*, which is not the same question and is usually just the
* speed of the local NIC.
*/
fun runUpstream(
credential: String,
sessionId: String,
control: ControlClient,
probe: ProbeSession,
sessionRef: String,
durationS: Int = 5,
kbps: Int = 20_000,
sizeBytes: Int = 1200,
): Pair<Test, List<Finding>> {
val testId = ids.uuid()
val started = ids.monoNs()
// Zeroes the server's counter so this run measures itself rather than inheriting the
// packets of an earlier one on the same session.
val reply = runCatching {
control.action(credential, sessionId, """{"action":"throughput","direction":"up"}""")
}
if (reply.isFailure) {
return Test(
id = testId, type = TestType.PERF_THROUGHPUT_UDP, sessionRef = sessionRef, tier = Tier.APP,
startedMonoNs = started, endedMonoNs = ids.monoNs(),
status = TestStatus.UNSUPPORTED,
error = TestError("action_refused", reply.exceptionOrNull()?.message ?: "refused"),
) to emptyList()
}
val sent = probe.sendThroughput(durationS * 1000L, kbps, sizeBytes)
// A moment for the tail of the run to arrive; counting still-in-flight packets as lost
// would inflate the loss figure by whatever the path's delay happens to be.
Thread.sleep(500)
val seen = upstreamCount(control, credential, sessionId)
val lossPct = if (sent.packets > 0 && seen != null) {
round2((sent.packets - seen.packets).coerceAtLeast(0) * 100.0 / sent.packets)
} else {
null
}
// The receiver's rate is the measurement. The sender's is what we managed to emit, which
// is a property of this phone and its radio, not of the network.
val achievedKbps = seen?.kbps ?: 0
val metrics = json.encodeToJsonElement(
UpstreamThroughputMetrics(
requestedKbps = kbps,
sentPackets = sent.packets,
sentBytes = sent.bytes,
sentKbps = sent.kbps,
receivedPackets = seen?.packets,
receivedBytes = seen?.bytes,
receivedKbps = achievedKbps,
lossPct = lossPct,
// Same honesty rule as downstream: if what arrived matches what we offered, the
// path was never the constraint and this number says nothing about it.
measuresNetwork = seen != null && achievedKbps > 0 && achievedKbps < sent.kbps * 9 / 10,
),
) as JsonObject
val findings = ArrayList<Finding>()
if (seen != null && seen.packets == 0 && sent.packets > 0) {
findings.add(
finding(
FindingRegistry.THROUGHPUT_NO_DELIVERY, testId,
"No upstream traffic reached the server",
"This device sent ${sent.packets} packets and the server received none. " +
"That is a connectivity fault on the outbound path rather than a slow link.",
),
)
} else if (lossPct != null && lossPct >= 2.0) {
findings.add(
finding(
FindingRegistry.THROUGHPUT_BELOW_OFFERED, testId,
"Upstream loss of $lossPct % at ${sent.kbps / 1000} Mbit/s",
"The server received ${seen?.packets} of the ${sent.packets} packets this " +
"device sent. The outbound path could not carry what was offered.",
),
)
}
return Test(
id = testId, type = TestType.PERF_THROUGHPUT_UDP, sessionRef = sessionRef, tier = Tier.APP,
startedMonoNs = started, endedMonoNs = ids.monoNs(),
status = if (seen == null || seen.packets == 0) TestStatus.FAILED else TestStatus.OK,
metrics = metrics,
) to findings
}
private data class UpstreamCount(val packets: Int, val bytes: Long, val kbps: Int)
/** The server's tally for this session's upstream run. */
private fun upstreamCount(
control: ControlClient, credential: String, sessionId: String,
): UpstreamCount? = runCatching {
val o = Json.parseToJsonElement(control.observations(credential, sessionId))
.jsonObject["throughput_up"]?.jsonObject ?: return null
UpstreamCount(
packets = o["packets"]?.jsonPrimitive?.content?.toIntOrNull() ?: 0,
bytes = o["bytes"]?.jsonPrimitive?.content?.toLongOrNull() ?: 0,
kbps = o["kbps"]?.jsonPrimitive?.content?.toIntOrNull() ?: 0,
)
}.getOrNull()
private data class SenderReport(
val packets: Int, val bytes: Long, val kbps: Int, val limitedBy: String,
)
/** The server's own account of the run, from the observations API. */
private fun senderReport(
control: ControlClient, credential: String, sessionId: String,
): SenderReport? = runCatching {
val arr = Json.parseToJsonElement(control.observations(credential, sessionId))
.jsonObject["throughput"]?.jsonArray ?: return null
val last = arr.lastOrNull()?.jsonObject ?: return null
SenderReport(
packets = last["packets"]?.jsonPrimitive?.content?.toIntOrNull() ?: 0,
bytes = last["bytes"]?.jsonPrimitive?.content?.toLongOrNull() ?: 0,
kbps = last["kbps"]?.jsonPrimitive?.content?.toIntOrNull() ?: 0,
limitedBy = last["limited_by"]?.jsonPrimitive?.content ?: "unknown",
)
}.getOrNull()
private fun parseInt(body: String?, key: String): Int? =
body?.let { Regex("\"$key\"\\s*:\\s*(-?\\d+)").find(it)?.groupValues?.get(1)?.toIntOrNull() }
/**
* Builds a finding from a registry entry, which supplies the code, category and severity.
*
* Taking a [FindingSpec] rather than three loose values is the point: a typo becomes a
* compile error, and two call sites cannot disagree about which category a finding belongs
* to - a disagreement that would split one fault across two verdict lights.
*/
private fun finding(spec: FindingSpec, testId: String, title: String, desc: String) =
Finding(
id = ids.uuid(), code = spec.code, category = spec.category, severity = spec.severity,
confidence = Confidence.HIGH,
title = title, description = desc, evidenceRefs = listOf(EvidenceRef(testId)),
)
private fun round2(v: Double) = Math.round(v * 100.0) / 100.0
}
/** Metrics for perf.throughput_udp in the upstream direction. */
@Serializable
data class UpstreamThroughputMetrics(
val direction: String = "up",
@SerialName("requested_kbps") val requestedKbps: Int,
@SerialName("sent_packets") val sentPackets: Int,
@SerialName("sent_bytes") val sentBytes: Long,
/** What this device managed to emit — a property of the phone and its radio, not the path. */
@SerialName("sent_kbps") val sentKbps: Int,
@SerialName("received_packets") val receivedPackets: Int? = null,
@SerialName("received_bytes") val receivedBytes: Long? = null,
/** What arrived, measured by the only party that can measure it. This is the result. */
@SerialName("received_kbps") val receivedKbps: Int,
@SerialName("loss_pct") val lossPct: Double? = null,
@SerialName("measures_network") val measuresNetwork: Boolean,
)
/** Metrics for perf.throughput_udp. */
@Serializable
data class ThroughputMetrics(
val direction: String = "down",
@SerialName("requested_kbps") val requestedKbps: Int,
@SerialName("planned_duration_ms") val plannedDurationMs: Int,
@SerialName("packets_received") val packetsReceived: Int,
@SerialName("bytes_received") val bytesReceived: Long,
@SerialName("received_kbps") val receivedKbps: Int,
@SerialName("sender_packets") val senderPackets: Int? = null,
@SerialName("sender_bytes") val senderBytes: Long? = null,
@SerialName("sender_kbps") val senderKbps: Int? = null,
/** Against the sender's count, so a server-side limit is never counted as network loss. */
@SerialName("loss_pct") val lossPct: Double? = null,
/** What ended the run: duration | budget | rate | send_error | unknown. */
@SerialName("limited_by") val limitedBy: String,
/**
* Whether this number says anything about the network. False when the sender's own ceiling
* was the binding constraint — in which case the rate is a property of the test, not the path.
*/
@SerialName("measures_network") val measuresNetwork: Boolean,
)
@@ -44,7 +44,9 @@ class LiveDownstreamTest {
for (t in tests) println("${t.type} status=${t.status} metrics=${t.metrics}") for (t in tests) println("${t.type} status=${t.status} metrics=${t.metrics}")
for (f in findings) println("finding ${f.code} [${f.severity}] ${f.title}") for (f in findings) println("finding ${f.code} [${f.severity}] ${f.title}")
assertEquals(3, tests.size, "expected pmtud_down, frag_delivery and a downstream train") // Assert on what is present, not on how many: adding a measurement should not be a
// test edit. (It was, once — hence the note.)
assertTrue(tests.size >= 3, "expected at least the three downstream tests, got ${tests.size}")
val byType = tests.associateBy { it.type } val byType = tests.associateBy { it.type }
val pmtud = assertNotNull(byType[TestType.MTU_PMTUD_DOWN], "no mtu.pmtud_down test") val pmtud = assertNotNull(byType[TestType.MTU_PMTUD_DOWN], "no mtu.pmtud_down test")
@@ -58,6 +60,17 @@ class LiveDownstreamTest {
val frag = assertNotNull(byType[TestType.MTU_FRAG_DELIVERY], "no mtu.frag_delivery test") val frag = assertNotNull(byType[TestType.MTU_FRAG_DELIVERY], "no mtu.frag_delivery test")
assertNotNull(frag.metrics?.get("largest_delivered_bytes")) assertNotNull(frag.metrics?.get("largest_delivered_bytes"))
// Fragment ordering runs only when fragments arrive at all, and only against a server
// that can craft them — so it is checked when present rather than required.
byType[TestType.MTU_FRAG_ORDERING]?.let { fo ->
val m = fo.metrics?.toString() ?: ""
println("fragment ordering: ${fo.status} $m")
if (fo.status != TestStatus.UNSUPPORTED) {
assertTrue(m.contains("in_order"), "no per-ordering result: $m")
assertTrue(m.contains("reversed"), "reversed ordering was never attempted: $m")
}
}
val train = assertNotNull(byType[TestType.TRAIN_UDP_DOWNSTREAM], "no downstream train") val train = assertNotNull(byType[TestType.TRAIN_UDP_DOWNSTREAM], "no downstream train")
assertNotNull(train.evidence, "a train without columnar evidence is not recomputable") assertNotNull(train.evidence, "a train without columnar evidence is not recomputable")
val received = train.metrics?.get("received")?.toString()?.toIntOrNull() ?: 0 val received = train.metrics?.get("received")?.toString()?.toIntOrNull() ?: 0
@@ -0,0 +1,105 @@
// SPDX-FileCopyrightText: 2026 Echolot contributors
// SPDX-License-Identifier: GPL-3.0-or-later
package app.echo_lot.engine
import app.echo_lot.measurement.TestStatus
import app.echo_lot.protocol.ControlClient
import app.echo_lot.protocol.ProbeSession
import kotlin.test.Test
import kotlin.test.assertEquals
import kotlin.test.assertNotNull
import kotlin.test.assertTrue
/**
* Downstream throughput against a LIVE server. Self-skips without ECHOLOT_LIVE_*.
*
* The assertions are about *honesty* rather than speed: a rate is only a measurement if the run
* was ended by the clock and the sender's own count backs it up. A test that just asserted "some
* Mbps arrived" would pass equally well against a broken implementation.
*/
class LiveThroughputTest {
private val url = System.getenv("ECHOLOT_LIVE_URL")
private val pin = System.getenv("ECHOLOT_LIVE_PIN")
private val cred = System.getenv("ECHOLOT_LIVE_CRED")
private val udp = System.getenv("ECHOLOT_LIVE_UDP")
private val target = System.getenv("ECHOLOT_LIVE_TARGET") ?: "fmr"
@Test
fun measuresDownstreamRateAndSaysWhatLimitedIt() {
if (url == null || pin == null || cred == null || udp == null) {
println("LiveThroughputTest skipped (no ECHOLOT_LIVE_* env)"); return
}
val control = ControlClient(url, setOf(pin), "0.2.0")
val session = control.createSession(cred, target)
val (host, port) = udp.split(":").let { it[0] to it[1].toInt() }
val (test, findings) = ProbeSession(cred, session, host, port).use { ps ->
ps.echo() // prime: the grant binds to the observed source
ThroughputMeasurement(SystemIdSource()).run(
cred, session.sessionId, control, ps, sessionRef = "sess-1",
durationS = 3, kbps = 20_000,
)
}
control.deleteSession(cred, session.sessionId)
val m = assertNotNull(test.metrics).toString()
println("throughput: ${test.status} $m")
for (f in findings) println("finding ${f.code} [${f.severity}] ${f.title}")
assertEquals(TestStatus.OK, test.status, "no throughput traffic arrived: $m")
// The sender's own count must be present — without it, loss cannot be attributed and the
// number is not a measurement.
assertTrue(m.contains("sender_packets"), "no sender report to compare against: $m")
assertTrue(m.contains("limited_by"), "the result must say what ended the run: $m")
val received = Regex(""""received_kbps":(\d+)""").find(m)?.groupValues?.get(1)?.toInt()
assertNotNull(received)
assertTrue(received > 0, "measured 0 kbps: $m")
println("received ${received / 1000} Mbit/s")
// A run this short and this far below the ceiling should end on the clock. Anything else
// means the grant was the constraint, and then the rate says nothing about the path.
assertTrue(m.contains(""""limited_by":"duration""""),
"the run did not end on the clock, so the rate measures the server, not the path: $m")
}
// Upstream is the direction only the far end can measure. The assertion that matters is that
// the server's count is present and plausible against what we sent — a test that only checked
// "we transmitted some Mbps" would pass against a server that counted nothing at all.
@Test
fun measuresUpstreamAgainstTheServersCount() {
if (url == null || pin == null || cred == null || udp == null) {
println("LiveThroughputTest(up) skipped"); return
}
val control = ControlClient(url, setOf(pin), "0.2.0")
val session = control.createSession(cred, target)
val (host, port) = udp.split(":").let { it[0] to it[1].toInt() }
val (test, findings) = ProbeSession(cred, session, host, port).use { ps ->
ps.echo()
ThroughputMeasurement(SystemIdSource()).runUpstream(
cred, session.sessionId, control, ps, sessionRef = "sess-1",
durationS = 3, kbps = 10_000,
)
}
control.deleteSession(cred, session.sessionId)
val m = assertNotNull(test.metrics).toString()
println("upstream: ${test.status} $m")
for (f in findings) println("finding ${f.code} [${f.severity}] ${f.title}")
assertEquals(TestStatus.OK, test.status, "the server counted nothing: $m")
val recv = Regex(""""received_packets":(\d+)""").find(m)?.groupValues?.get(1)?.toInt()
val sent = Regex(""""sent_packets":(\d+)""").find(m)?.groupValues?.get(1)?.toInt()
assertNotNull(recv); assertNotNull(sent)
assertTrue(sent > 100, "barely anything was sent, so the rate means nothing: $m")
assertTrue(recv > 0, "the server received none of $sent packets: $m")
// The counts should be close on a healthy path; wildly different means the two sides are
// counting different things rather than the network losing packets.
assertTrue(recv <= sent, "the server counted MORE than we sent — the counter is not being reset")
println("sent $sent, server saw $recv")
}
}
@@ -35,9 +35,37 @@ data class Run(
val device: DeviceInfo, val device: DeviceInfo,
val tiers: Tiers, val tiers: Tiers,
@SerialName("profiles_used") val profilesUsed: List<String> = emptyList(), @SerialName("profiles_used") val profilesUsed: List<String> = emptyList(),
val constraints: Constraints = Constraints(),
val notes: String? = null, val notes: String? = null,
) )
/**
* What limited this run — the counterpart to [Tiers], which records what was available.
*
* A constrained run is not a failed run, and it is not a normal one either. Without this, a run
* taken through a VPN looks exactly like a clean run of a healthy network: the same shape, the
* same green verdict, and no way for a reader — or a server aggregating thousands of these — to
* know that almost nothing was actually measured.
*/
@Serializable
data class Constraints(
/** A VPN held the default route while this ran. */
@SerialName("vpn_active") val vpnActive: Boolean = false,
/**
* Per-network probing was refused by the OS.
*
* Android blocks `Network.bindSocket()` on the underlying networks whenever a VPN is up, to
* stop apps leaking around the tunnel. Every per-network test then measures nothing, so any
* conclusion drawn about the wifi or cellular link underneath is unfounded.
*/
@SerialName("per_network_blocked") val perNetworkBlocked: Boolean = false,
/** Networks that could not be measured, by id. */
@SerialName("unmeasured_networks") val unmeasuredNetworks: List<String> = emptyList(),
) {
/** True when this run's results mean something different from an unconstrained one. */
val constrained: Boolean get() = vpnActive || perNetworkBlocked
}
@Serializable @Serializable
enum class Trigger { enum class Trigger {
@SerialName("manual") MANUAL, @SerialName("manual") MANUAL,
@@ -0,0 +1,321 @@
// SPDX-FileCopyrightText: 2026 Echolot contributors
// SPDX-License-Identifier: GPL-3.0-or-later
package app.echo_lot.measurement
/**
* The registry of finding codes (measurement-schema.md §9, open item 1).
*
* A finding code is the stable, machine-readable half of a result: the prose changes, the code is
* what a dashboard groups by and what someone greps a year of archived runs for. That only holds
* if a code means exactly one thing forever — which is not something ad-hoc string literals at
* fifteen call sites can promise.
*
* The failure this exists to prevent had already happened by the time it was written. Two
* independently-added emitters produced `connectivity.downstream_loss` and
* `connectivity.loss_downstream` for the same concept, and nothing anywhere objected. Anyone
* aggregating either one would have silently seen half their data.
*
* So codes are declared here as typed specs, each carrying its category and default severity, and
* emitters reference the spec rather than retyping the string. That makes a typo a compile error,
* and makes it impossible for two call sites to disagree about which category a finding belongs
* to — a disagreement that would otherwise split one fault across two verdict lights.
*/
data class FindingSpec(
val code: String,
val category: Category,
/** Severity when nothing about the specific run argues otherwise; emitters may escalate. */
val severity: Severity,
/** One line: what this finding asserts. Present tense, no hedging. */
val meaning: String,
/**
* What the finding rules *out*, where that is the useful half. "Loss upstream" is worth much
* more when it also says the return path is fine, because that halves where to look next.
*/
val rulesOut: String? = null,
)
object FindingRegistry {
// ---- connectivity ----------------------------------------------------------------
// Renamed from nat.* before anything shipped: neither of these is about NAT, and the
// prefix is what decides which category - and therefore which verdict light - a finding
// rolls up into. A nat.* code landing under connectivity would be a permanent puzzle.
val UDP_UNREACHABLE = FindingSpec(
"connectivity.udp_unreachable", Category.CONNECTIVITY, Severity.HIGH,
"No UDP echo replies came back from the server at all.",
)
val UDP_UNREACHABLE_UPSTREAM = FindingSpec(
"connectivity.udp_unreachable_upstream", Category.CONNECTIVITY, Severity.HIGH,
"The server received none of the probes, so traffic is dropped on the way out.",
rulesOut = "The return path: nothing arrived to be replied to.",
)
val UDP_LOSS = FindingSpec(
"connectivity.udp_loss", Category.CONNECTIVITY, Severity.MEDIUM,
"A large fraction of round-trip probes were lost, direction unknown.",
)
val LOSS_UPSTREAM = FindingSpec(
"connectivity.loss_upstream", Category.CONNECTIVITY, Severity.MEDIUM,
"Probes were lost on the way to the server.",
rulesOut = "The return path: replies came back for everything that arrived.",
)
/**
* The single code for "lost on the return path", whichever measurement found it.
*
* Two emitters had independently invented `connectivity.downstream_loss` and
* `connectivity.loss_downstream` for this, and nothing objected. Anyone aggregating either
* one would have silently seen half their data. Paired with [LOSS_UPSTREAM] so the two
* directions read as a set.
*/
val LOSS_DOWNSTREAM = FindingSpec(
"connectivity.loss_downstream", Category.CONNECTIVITY, Severity.MEDIUM,
"Packets were lost on the way back from the server.",
rulesOut = "The outbound path: the server received what it was answering.",
)
val DOWNSTREAM_BLOCKED = FindingSpec(
"connectivity.downstream_blocked", Category.CONNECTIVITY, Severity.HIGH,
"Server-initiated packets never arrive, although round trips work.",
rulesOut = "Basic reachability: the path forwards replies, just not unsolicited traffic.",
)
val DOWNSTREAM_REORDER = FindingSpec(
"connectivity.downstream_reorder", Category.CONNECTIVITY, Severity.LOW,
"Downstream packets arrive in a different order than they were sent.",
)
// MEDIUM, not HIGH: a captive portal is a condition to report, not necessarily a fault - on
// hotel or cafe wifi it is exactly what should be there, and logging in clears it. NO_INTERNET
// is the HIGH one, because nothing the user does locally fixes that. The registry first said
// HIGH; the probe emitting it had always said MEDIUM, and the probe was the considered value.
val CAPTIVE_PORTAL = FindingSpec(
"connectivity.captive_portal", Category.CONNECTIVITY, Severity.MEDIUM,
"A captive portal is intercepting connectivity checks.",
)
val NO_INTERNET = FindingSpec(
"connectivity.no_internet", Category.CONNECTIVITY, Severity.HIGH,
"Android's own connectivity checks fail on this network.",
)
// ---- mtu -------------------------------------------------------------------------
val MTU_REDUCED_DOWNSTREAM = FindingSpec(
"mtu.reduced_downstream", Category.MTU, Severity.LOW,
"The downstream path MTU is below the usual 1500 bytes.",
)
val MTU_DOWNSTREAM_BLACKHOLE = FindingSpec(
"mtu.downstream_blackhole", Category.MTU, Severity.MEDIUM,
"Datagrams above the path MTU are dropped downstream, fragmented or not.",
)
val FRAGMENTS_BLOCKED = FindingSpec(
"mtu.fragments_blocked", Category.MTU, Severity.MEDIUM,
"IP fragments do not reach this device even when sent in order.",
)
val FRAGMENT_REORDER_SENSITIVE = FindingSpec(
"mtu.fragment_reorder_sensitive", Category.MTU, Severity.LOW,
"Fragments are delivered in order but dropped when reordered or delayed.",
rulesOut = "Fragmentation itself: in-order fragments arrive fine.",
)
// ---- nat -------------------------------------------------------------------------
val NAT_UDP_REBINDING = FindingSpec(
"nat.udp_rebinding", Category.NAT, Severity.MEDIUM,
"A NAT remapped the UDP source port mid-flow.",
)
val NAT_SYMMETRIC = FindingSpec(
"nat.symmetric", Category.NAT, Severity.MEDIUM,
"The NAT assigns a different external port per destination.",
)
// ---- perf ------------------------------------------------------------------------
val THROUGHPUT_NO_DELIVERY = FindingSpec(
"perf.throughput_no_delivery", Category.PERFORMANCE, Severity.HIGH,
"No throughput traffic arrived, although the server sent it.",
)
val THROUGHPUT_BELOW_OFFERED = FindingSpec(
"perf.throughput_below_offered", Category.PERFORMANCE, Severity.LOW,
"Less throughput arrived than the server sent for the whole run.",
)
// ---- dns -------------------------------------------------------------------------
val DNS_ANSWER_REWRITTEN = FindingSpec(
"dns.answer_rewritten", Category.DNS, Severity.HIGH,
"A resolver returned an answer that differs from the authoritative record.",
)
val DNS_AUTHORITATIVE_UNREACHABLE = FindingSpec(
"dns.authoritative_unreachable", Category.DNS, Severity.MEDIUM,
"The canary zone's authoritative server could not be reached.",
)
// ---- v6 ----------------------------------------------------------------------------
//
// Prefix is `v6.`, matching the test-type registry (v6.brokenness, v6.happy_eyeballs, ...).
// These were `ipv6.*` while declaring Category.IPV6, but the prefix map only knows "v6", so
// they silently rolled up under connectivity: the third instance of a prefix disagreeing with
// its category and quietly moving a fault to a different verdict light.
/**
* Renamed from `v6.broken`, which claimed more than the evidence supports.
*
* The only signal behind it is ICMPv6 echo getting no reply — and ICMPv6 echo is widely
* filtered on networks where IPv6 otherwise works perfectly. A phone that reported this while
* happily loading an IPv6-only site over TCP is what caught it. From here the two cases look
* identical, so the finding now says what was observed and names both explanations rather than
* picking one.
*
* It is worth reporting either way: filtered ICMPv6 breaks Path MTU Discovery, which is its
* own fault even when IPv6 works.
*/
/**
* A global IPv6 address with no default route.
*
* This is the structural version of the same complaint, and it is worth far more than the
* ICMP one because it admits no other explanation: the device has an address it cannot route
* with. Nothing is filtered, nothing is inferred — the routing table says so directly, and it
* is already in the link snapshot.
*
* Not always a fault. A VPN that installs host routes to specific destinations produces
* exactly this shape on purpose, and it works. What makes it worth reporting either way is
* that applications cannot tell: having a global address, they will try IPv6 first and stall
* for every destination the routes do not cover.
*/
/**
* An IPv6 default route with no global address to use it from — the mirror of
* [V6_NO_DEFAULT_ROUTE], and the more common misconfiguration of the two.
*
* The router is sending RAs that name it as a default gateway, but SLAAC produced no address:
* no prefix information option, or a prefix without the autonomous flag, or DHCPv6-only
* addressing the device did not complete. The network is announcing IPv6 service it does not
* actually deliver.
*
* This is worth flagging above the ICMP signal because it is both certain and consequential.
* Hosts see router advertisements, believe IPv6 is available, and pay a connection-attempt
* timeout on every dual-stack destination before falling back to IPv4 — the classic "the
* internet feels slow" complaint with no packet loss anywhere to explain it.
*/
val V6_ROUTE_WITHOUT_ADDRESS = FindingSpec(
"v6.route_without_address", Category.IPV6, Severity.MEDIUM,
"The network advertises an IPv6 default route but the device has no global IPv6 address.",
rulesOut = "A working IPv6 setup: SLAAC did not produce a usable address on this link.",
)
/**
* A VPN prevented the underlying networks from being measured.
*
* Reported rather than worked around: Android refuses `Network.bindSocket()` on the networks
* beneath a VPN precisely so apps cannot leak around the tunnel, and that is correct
* behaviour. What is not acceptable is a run that quietly measures nothing and calls the
* result healthy, so this says plainly which networks went unmeasured and why.
*/
/**
* The network's DNS server answers, but this device cannot resolve through it.
*
* Worth separating from every other DNS failure because the remedy is somewhere else entirely.
* A name that will not resolve looks identical to a user whatever the cause, and the two causes
* pull in opposite directions: a server that does not answer means the network is broken and
* the router is the thing to examine, while a server that answers a direct query on a device
* that still cannot resolve means the platform resolver has wedged — fixed by toggling wifi,
* and nothing to do with the network at all.
*
* Proven rather than inferred: the probe sends its own UDP query, bypassing the component under
* suspicion, and compares that against what the platform returns for the same name.
*/
/**
* The network hands out a search domain its DNS server will not answer for.
*
* A resolver appends search domains to lookups, so every name a client asks about can stall on
* a domain the server ignores. The failure mode is silence rather than a negative answer, and
* silence is indistinguishable from packet loss: clients retry instead of moving on, and some
* give up on the lookup entirely. That makes it look like the device is broken when the
* network is.
*
* Whether it bites depends on the resolver — some try the bare name first and never notice —
* which is why two devices on the same network can disagree about whether DNS works.
*/
val DNS_SEARCH_DOMAIN_UNANSWERED = FindingSpec(
"dns.search_domain_unanswered", Category.DNS, Severity.HIGH,
"The network advertises a DNS search domain that its own server does not answer for.",
rulesOut = "A fault on this device: the same server answers ordinary names normally.",
)
val DNS_SYSTEM_RESOLVER_BROKEN = FindingSpec(
"dns.system_resolver_broken", Category.DNS, Severity.HIGH,
"The network's DNS server answers, but this device cannot resolve names through it.",
rulesOut = "A network fault: the server replied to a query sent from this device.",
)
val MEASUREMENT_VPN_CONSTRAINED = FindingSpec(
"measurement.vpn_constrained", Category.CONNECTIVITY, Severity.INFO,
"A VPN was active, so the networks underneath it could not be measured.",
rulesOut = "Nothing — this run says little about the underlying network either way.",
)
val V6_NO_DEFAULT_ROUTE = FindingSpec(
"v6.no_default_route", Category.IPV6, Severity.MEDIUM,
"The device has a global IPv6 address but no IPv6 default route.",
rulesOut = "Guesswork: this is read from the routing table, not inferred from silence.",
)
val V6_NO_ICMP_REPLY = FindingSpec(
"v6.no_icmp_reply", Category.IPV6, Severity.LOW,
"IPv6 is configured but ICMPv6 echo gets no reply.",
rulesOut = "Nothing on its own: IPv6 may work fine with ICMP filtered.",
)
/**
* IPv6 is advertised and does not work — the claim `v6.broken` originally made on ICMP
* silence alone, now reinstated because it can finally be backed: it is only emitted when a
* real IPv6 TCP connection (v6.brokenness) failed on the same network whose ICMPv6 went
* unanswered. Two independent transports failing on a network that advertises IPv6 is what
* "broken" actually means; either signal alone still gets [V6_NO_ICMP_REPLY].
*/
val V6_BROKEN = FindingSpec(
"v6.broken", Category.IPV6, Severity.HIGH,
"IPv6 is advertised on this network but carries no traffic.",
rulesOut = "ICMP filtering as the benign explanation: a TCP connection over IPv6 failed too.",
)
/**
* INFO deliberately, and it needs to stay that way.
*
* Most networks still do not offer IPv6, and that is not a fault. Reporting it as a warning
* lights a yellow verdict on a perfectly healthy network, which teaches people to ignore the
* light — the one thing a diagnostic must never do.
*/
val V6_NOT_OFFERED = FindingSpec(
"v6.not_offered", Category.IPV6, Severity.INFO,
"This network does not offer IPv6.",
)
/** Every registered finding, in declaration order. */
val all: List<FindingSpec> = listOf(
UDP_UNREACHABLE, UDP_UNREACHABLE_UPSTREAM, UDP_LOSS, LOSS_UPSTREAM, LOSS_DOWNSTREAM,
DOWNSTREAM_BLOCKED, DOWNSTREAM_REORDER, CAPTIVE_PORTAL, NO_INTERNET,
MTU_REDUCED_DOWNSTREAM, MTU_DOWNSTREAM_BLACKHOLE, FRAGMENTS_BLOCKED,
FRAGMENT_REORDER_SENSITIVE,
NAT_UDP_REBINDING, NAT_SYMMETRIC,
THROUGHPUT_NO_DELIVERY, THROUGHPUT_BELOW_OFFERED,
DNS_ANSWER_REWRITTEN, DNS_AUTHORITATIVE_UNREACHABLE,
DNS_SEARCH_DOMAIN_UNANSWERED, DNS_SYSTEM_RESOLVER_BROKEN, MEASUREMENT_VPN_CONSTRAINED,
V6_NO_DEFAULT_ROUTE, V6_ROUTE_WITHOUT_ADDRESS, V6_NO_ICMP_REPLY, V6_BROKEN, V6_NOT_OFFERED,
)
private val byCode: Map<String, FindingSpec> = all.associateBy { it.code }
fun byCode(code: String): FindingSpec? = byCode[code]
}
@@ -18,6 +18,28 @@ data class Network(
val wifi: Wifi? = null, val wifi: Wifi? = null,
val cellular: Cellular? = null, val cellular: Cellular? = null,
val changes: List<NetworkChange> = emptyList(), val changes: List<NetworkChange> = emptyList(),
@SerialName("system_verdict") val systemVerdict: SystemVerdict? = null,
)
/**
* What Android itself concluded about a network, as opposed to what we measured.
*
* Recorded because it is the verdict the user can see — the "no internet" warning in the status
* bar — and because it is free: the platform has already done the work by the time a run starts.
*
* Its real value is disagreement. When Android says a network is unusable and our own probes reach
* the internet regardless, the fault is in the device rather than the network, and that distinction
* is the difference between "fix your router" and "toggle your wifi". Neither number alone can say
* that; only the two together.
*/
@Serializable
data class SystemVerdict(
/** Android's own connectivity check passed. Null when the platform did not say. */
val validated: Boolean? = null,
/** Android believes a captive portal is intercepting this network. */
@SerialName("captive_portal") val captivePortal: Boolean? = null,
/** Some traffic works and some does not — Android's own hedge. */
@SerialName("partial_connectivity") val partialConnectivity: Boolean? = null,
) )
@Serializable @Serializable
@@ -35,6 +35,9 @@ data class CategorySummary(
* (critical|high → red, medium|low → yellow, info/none → green). * (critical|high → red, medium|low → yellow, info/none → green).
* - A category is `inconclusive` when > 50% of its tests are failed/unsupported. * - A category is `inconclusive` when > 50% of its tests are failed/unsupported.
* - Overall = the worst category light; `inconclusive` only when ALL categories are. * - Overall = the worst category light; `inconclusive` only when ALL categories are.
* - A run whose per-network probing was blocked is `inconclusive` outright, whatever the
* categories say. The lights describe what the tests found; when the OS refused to let the
* tests run, a green light would describe nothing at all.
* *
* The mapping test-type → category comes from [TestType.category]. Only categories that have * The mapping test-type → category comes from [TestType.category]. Only categories that have
* findings or tests appear in the summary. * findings or tests appear in the summary.
@@ -44,7 +47,10 @@ object Verdicts {
private fun isInconclusiveTest(s: TestStatus) = private fun isInconclusiveTest(s: TestStatus) =
s == TestStatus.FAILED || s == TestStatus.UNSUPPORTED s == TestStatus.FAILED || s == TestStatus.UNSUPPORTED
fun derive(tests: List<Test>, findings: List<Finding>): Summary { fun derive(tests: List<Test>, findings: List<Finding>): Summary =
derive(tests, findings, Constraints())
fun derive(tests: List<Test>, findings: List<Finding>, constraints: Constraints): Summary {
val testsByCat = tests.groupBy { TestType.category(it.type) } val testsByCat = tests.groupBy { TestType.category(it.type) }
val findingsByCat = findings.groupBy { it.category } val findingsByCat = findings.groupBy { it.category }
val categories = (testsByCat.keys + findingsByCat.keys) val categories = (testsByCat.keys + findingsByCat.keys)
@@ -72,7 +78,14 @@ object Verdicts {
) )
} }
val overall = deriveOverall(perCat.values) // A run that could not measure the networks it was asked about has not found them
// healthy; it has found out nothing. Reporting that as green is the single most
// misleading thing this function could do, so the constraint outranks the lights.
val overall = if (constraints.perNetworkBlocked) {
Verdict.INCONCLUSIVE
} else {
deriveOverall(perCat.values)
}
return Summary(overall = overall, categories = perCat) return Summary(overall = overall, categories = perCat)
} }
@@ -89,6 +89,13 @@ object TestType {
// dns // dns
const val DNS_RESOLVER_INVENTORY = "dns.resolver_inventory" const val DNS_RESOLVER_INVENTORY = "dns.resolver_inventory"
const val DNS_CANARY = "dns.canary" const val DNS_CANARY = "dns.canary"
/**
* Does this device's own resolver work, as distinct from the network's DNS.
*
* Registry addition, v1.1. Kept apart from [DNS_CANARY], which asks whether answers are being
* tampered with; this asks whether answers arrive at all, and where the failure sits.
*/
const val DNS_RESOLVER = "dns.resolver"
const val DNS_INTERCEPTION = "dns.interception" const val DNS_INTERCEPTION = "dns.interception"
const val DNS_TTL_INTEGRITY = "dns.ttl_integrity" const val DNS_TTL_INTEGRITY = "dns.ttl_integrity"
const val DNS_ANSWER_INTEGRITY = "dns.answer_integrity" const val DNS_ANSWER_INTEGRITY = "dns.answer_integrity"
@@ -0,0 +1,73 @@
// SPDX-FileCopyrightText: 2026 Echolot contributors
// SPDX-License-Identifier: GPL-3.0-or-later
package app.echo_lot.measurement
/**
* The two ways a network can be half-configured for IPv6, read from the link snapshot.
*
* Pure model logic rather than something a ViewModel does, because "is this network's IPv6
* broken, and in which direction" is exactly the kind of judgement that should be checkable
* against a captured routing table without a phone in the loop.
*/
object V6Analysis {
/** Linux tunnel interfaces: WireGuard/Netbird (tun*, wg*), plus the usual VPN names. */
private val TUNNEL_IFACE = Regex("""^(tun|tap|wg|ppp|ipsec|utun)\d*$""")
/** What one network's IPv6 configuration looks like. */
data class Shape(
val iface: String,
/** A global address with no ::/0 route: an address the device cannot route with. */
val addressWithoutRoute: Boolean,
/** A ::/0 route with no global address: a route the device cannot source from. */
val routeWithoutAddress: Boolean,
/** The routes belong to a tunnel, so a partial view of IPv6 is likely deliberate. */
val tunnel: Boolean,
)
/**
* Classifies each network's IPv6 configuration.
*
* Both shapes are read straight from the link snapshot rather than inferred from silence, so
* unlike an ICMP signal there is no competing explanation for what was observed — and both
* matter for the same reason: an application cannot tell in advance, so it tries IPv6 first
* and waits.
*
* They differ in what they mean. An address with no route is what a VPN installing host routes
* to specific destinations produces on purpose, and it works; calling that a fault would be the
* "lack of IPv6 is a yellow condition" mistake in a new costume, so a tunnel downgrades it to
* information. A route with no address is the opposite: the router advertised itself as a
* default gateway but SLAAC produced nothing usable, so the network is announcing IPv6 service
* it does not deliver. That one is a real misconfiguration however it arises.
*/
fun classify(networks: List<Network>): List<Shape> = networks.map { n ->
val globalV6 = n.link.addresses.any { isGlobalV6(it.addr) }
val v6Routes = n.link.routes.filter { it.dst.contains(':') }
val hasDefault = v6Routes.any { it.dst == "::/0" }
Shape(
iface = n.iface ?: v6Routes.firstOrNull()?.iface.orEmpty(),
addressWithoutRoute = globalV6 && !hasDefault,
routeWithoutAddress = hasDefault && !globalV6,
// Android labels the transport itself, which beats guessing from a name; the regex
// stays as a backstop for tunnels Android does not own (a userspace WireGuard, say,
// or anything seen through the shell tier).
tunnel = n.transport == Transport.VPN ||
v6Routes.any { TUNNEL_IFACE.containsMatchIn(it.iface.orEmpty()) },
)
}
/**
* Whether an address is IPv6 and usable as a source for off-link traffic.
*
* ULAs count. A ULA is not globally routable, but it is a global-*scope* address the stack
* will happily select as a source, which is the property that matters here — an overlay
* network handing out fc00::/7 addresses is providing working IPv6 to the destinations it
* carries, and treating that as "no address" would misreport every VPN as broken.
*/
private fun isGlobalV6(addr: String): Boolean {
if (!addr.contains(':')) return false
val a = addr.substringBefore('%').lowercase() // strip any zone index
return !a.startsWith("fe80") && a != "::1" && a != "::"
}
}
@@ -0,0 +1,133 @@
// SPDX-FileCopyrightText: 2026 Echolot contributors
// SPDX-License-Identifier: GPL-3.0-or-later
package app.echo_lot.measurement
import java.io.File
import kotlin.test.Test
import kotlin.test.assertEquals
import kotlin.test.assertTrue
import kotlin.test.fail
/**
* Keeps the finding registry honest.
*
* The interesting test is the last one: it reads `docs/findings-registry.md` and fails when the
* document and the code disagree. Documentation that drifts from its implementation is worse than
* none, because it still looks authoritative — and a finding registry is precisely the artifact
* other people build tooling against.
*/
class FindingRegistryTest {
@Test
fun codesAreUnique() {
val dupes = FindingRegistry.all.groupBy { it.code }.filterValues { it.size > 1 }.keys
assertTrue(dupes.isEmpty(), "duplicate finding codes: $dupes")
}
@Test
fun everyDeclaredSpecIsInTheAllList() {
// Reflection over the object's properties: a spec that is declared but left out of `all`
// is invisible to the doc check and to any consumer enumerating the registry.
val declared = FindingRegistry::class.java.declaredMethods
.filter { it.parameterCount == 0 && it.returnType == FindingSpec::class.java }
.mapNotNull { runCatching { it.invoke(FindingRegistry) as FindingSpec }.getOrNull() }
.map { it.code }
.toSet()
val listed = FindingRegistry.all.map { it.code }.toSet()
assertEquals(declared, listed, "declared specs and the `all` list disagree")
}
// The prefix decides the category, and the category decides which verdict light the finding
// rolls up into. A code whose prefix disagrees with its category silently moves a fault to a
// different light — the exact bug that got two codes renamed out of nat.*.
@Test
fun everyPrefixMatchesItsCategory() {
for (spec in FindingRegistry.all) {
val fromPrefix = TestType.category(spec.code)
assertEquals(
fromPrefix, spec.category,
"${spec.code} is declared as ${spec.category} but its prefix maps to $fromPrefix",
)
}
}
@Test
fun codesFollowTheNamingConvention() {
val shape = Regex("^[a-z0-9]+\\.[a-z0-9_]+$")
for (spec in FindingRegistry.all) {
assertTrue(shape.matches(spec.code), "malformed code: ${spec.code}")
assertTrue(spec.meaning.isNotBlank(), "${spec.code} has no meaning")
assertTrue(
spec.meaning.trimEnd().endsWith("."),
"${spec.code}'s meaning should be a sentence: '${spec.meaning}'",
)
}
}
// Two near-identical codes are how one fault ends up split across two dashboards. This is a
// blunt check — it will not catch every synonym — but it catches the shape that already
// happened: the same words in a different order.
@Test
fun noTwoCodesAreAnagramsOfEachOther() {
val normalised = FindingRegistry.all.associate { spec ->
spec.code to spec.code.substringAfter('.').split('_').sorted().joinToString("_")
}
val clashes = normalised.entries.groupBy { it.value }.filterValues { it.size > 1 }
if (clashes.isNotEmpty()) {
fail("codes differing only in word order: ${clashes.values.map { g -> g.map { it.key } }}")
}
}
@Test
fun theDocumentAndTheRegistryAgree() {
val doc = findDoc() ?: run {
println("findings-registry.md not found from ${File(".").absolutePath} — skipping")
return
}
val text = doc.readText()
// Only table rows count as "documented". Prose may legitimately mention a code that no
// longer exists — the rules section explains why two were merged — and treating that as
// a registry entry would force the document to forget its own history.
val documented = text.lines()
.filter { it.trimStart().startsWith("|") }
.flatMap { row -> Regex("`([a-z0-9]+\\.[a-z0-9_]+)`").findAll(row).map { it.groupValues[1] } }
.toSet()
val registered = FindingRegistry.all.map { it.code }.toSet()
val missingFromDoc = registered - documented
val missingFromCode = documented - registered
assertTrue(
missingFromDoc.isEmpty(),
"these codes exist in FindingRegistry but not in docs/findings-registry.md: $missingFromDoc",
)
assertTrue(
missingFromCode.isEmpty(),
"docs/findings-registry.md documents codes that no longer exist: $missingFromCode",
)
// And the severities must match, or the document is describing a different system.
for (spec in FindingRegistry.all) {
val row = text.lines().firstOrNull {
it.trimStart().startsWith("|") && it.contains("`${spec.code}`")
} ?: continue
val severity = spec.severity.name.lowercase()
assertTrue(
row.contains("| $severity |"),
"${spec.code} is ${severity} in code but the doc row says otherwise: $row",
)
}
}
/** Walks up from the test's working directory to find the repo's docs/ folder. */
private fun findDoc(): File? {
var dir: File? = File(".").absoluteFile
repeat(6) {
val candidate = File(dir, "docs/findings-registry.md")
if (candidate.isFile) return candidate
dir = dir?.parentFile
}
return null
}
}
@@ -0,0 +1,125 @@
// SPDX-FileCopyrightText: 2026 Echolot contributors
// SPDX-License-Identifier: GPL-3.0-or-later
package app.echo_lot.measurement
import kotlin.test.Test
import kotlin.test.assertEquals
import kotlin.test.assertFalse
import kotlin.test.assertTrue
/**
* The fixtures here are a real device's routing table, transcribed from `dumpsys connectivity`
* on a OnePlus 15 with a Netbird tunnel up: wifi advertising a default route it cannot source
* from, cellular working properly, and a VPN carrying host routes to two destinations.
*
* Using a captured table rather than invented ones matters, because the bug this guards against
* is not "the boolean logic is wrong" — it is "the shapes I imagined are not the shapes real
* networks produce".
*/
class V6AnalysisTest {
private fun net(
id: String,
transport: Transport,
iface: String,
addrs: List<String>,
routes: List<Pair<String, String>>,
) = Network(
id = id,
transport = transport,
iface = iface,
link = Link(
addresses = addrs.map { Address(addr = it.substringBefore('/'), prefixLen = 64) },
routes = routes.map { (dst, dev) -> Route(dst = dst, iface = dev) },
),
)
/** wlan0: an IPv6 default route via a link-local gateway, but SLAAC produced no address. */
private val wifi = net(
"w", Transport.WIFI, "wlan0",
addrs = listOf("fe80::bcf6:edff:fe67:b139", "10.13.102.122"),
routes = listOf(
"fe80::/64" to "wlan0",
"::/0" to "wlan0",
"0.0.0.0/0" to "wlan0",
),
)
/** rmnet_data1: a properly configured cellular link — global address and a default route. */
private val cellular = net(
"c", Transport.CELLULAR, "rmnet_data1",
addrs = listOf("2001:4bb8:46a:e724:289d:87ff:feb6:ebd3"),
routes = listOf("::/0" to "rmnet_data1", "2001:4bb8:46a:e724::/64" to "rmnet_data1"),
)
/** tun1: Netbird, with a ULA and host routes to exactly two destinations. */
private val vpn = net(
"v", Transport.VPN, "tun1",
addrs = listOf("100.64.158.131", "fdfd:c4fe:c4fe:c4fe:1f3c:98a0:dd66:ac7"),
routes = listOf(
"2001:1ad0:c4fe:6767::2/128" to "tun1",
"2001:1ad0:c4fe:a::136/128" to "tun1",
"fdfd:c4fe:c4fe:c4fe::/64" to "tun1",
),
)
@Test
fun `wifi advertising a route it cannot source from is reported`() {
val s = V6Analysis.classify(listOf(wifi)).single()
assertTrue(s.routeWithoutAddress, "::/0 with only a link-local address is the RA-without-SLAAC case")
assertFalse(s.addressWithoutRoute)
assertFalse(s.tunnel, "wifi is not a tunnel")
assertEquals("wlan0", s.iface)
}
@Test
fun `a properly configured link produces no finding`() {
val s = V6Analysis.classify(listOf(cellular)).single()
assertFalse(s.routeWithoutAddress)
assertFalse(s.addressWithoutRoute)
}
@Test
fun `a tunnel with host routes is deliberate, not broken`() {
val s = V6Analysis.classify(listOf(vpn)).single()
assertTrue(s.addressWithoutRoute, "a ULA and no ::/0 is an address with nothing to route it")
assertTrue(s.tunnel, "so it must be reported as information, not as a fault")
assertFalse(s.routeWithoutAddress)
}
@Test
fun `each network is judged on its own`() {
// The whole point of per-network classification: "IPv6 is broken" is useless advice when
// wifi is the broken one and cellular is fine.
val shapes = V6Analysis.classify(listOf(wifi, cellular, vpn)).associateBy { it.iface }
assertTrue(shapes.getValue("wlan0").routeWithoutAddress)
assertFalse(shapes.getValue("rmnet_data1").routeWithoutAddress)
assertFalse(shapes.getValue("rmnet_data1").addressWithoutRoute)
assertTrue(shapes.getValue("tun1").addressWithoutRoute)
}
@Test
fun `a link-local-only network with no v6 route says nothing either way`() {
// Plain IPv4-only wifi: no IPv6 offered at all. That is v6.not_offered's business, and
// reporting it here as well would double up on a network that is merely legacy, not broken.
val v4only = net(
"4", Transport.WIFI, "wlan0",
addrs = listOf("fe80::1", "192.168.1.5"),
routes = listOf("0.0.0.0/0" to "wlan0"),
)
val s = V6Analysis.classify(listOf(v4only)).single()
assertFalse(s.routeWithoutAddress)
assertFalse(s.addressWithoutRoute)
}
@Test
fun `a zone index does not hide a link-local address`() {
val zoned = net(
"z", Transport.WIFI, "wlan0",
addrs = listOf("fe80::1%wlan0"),
routes = listOf("::/0" to "wlan0"),
)
assertTrue(V6Analysis.classify(listOf(zoned)).single().routeWithoutAddress)
}
}
+5 -1
View File
@@ -20,4 +20,8 @@ kotlin {
} }
java { sourceCompatibility = JavaVersion.VERSION_17; targetCompatibility = JavaVersion.VERSION_17 } java { sourceCompatibility = JavaVersion.VERSION_17; targetCompatibility = JavaVersion.VERSION_17 }
tasks.test { useJUnitPlatform() } tasks.test {
useJUnitPlatform()
// Opt-in: point this at a captured run to check the anonymizer against real data.
System.getenv("ECHOLOT_REAL_RUN")?.let { environment("ECHOLOT_REAL_RUN", it) }
}
@@ -108,12 +108,22 @@ class Anonymizer(private val level: PrivacyLevel, private val salt: Salt) {
is JsonObject -> walkObject(v, path) is JsonObject -> walkObject(v, path)
is JsonArray -> JsonArray(v.map { walk(key, it, path) }) is JsonArray -> JsonArray(v.map { walk(key, it, path) })
is JsonPrimitive -> is JsonPrimitive ->
if (v.isString) transform(Classification.typeOf(key, path), v.content).let(::JsonPrimitive) if (v.isString) {
else v // Name first (it is precise), then shape (it is exhaustive). A field nobody
// classified must not be a field that leaks.
val type = Classification.typeOf(key, path) ?: Classification.inferFromValue(v.content)
JsonPrimitive(transform(type, v.content))
} else {
v
}
} }
private fun transform(type: LogicalType?, value: String): String = when (type) { private fun transform(type: LogicalType?, value: String): String = when (type) {
null -> value // Unclassified strings still get their *embedded* identifiers scrubbed. A whole-value
// check cannot see them: raw shell output is one long string that is neither a MAC nor an
// address, so it sailed through both the name table and the shape check carrying every
// MAC on the user's LAN.
null -> scrubEmbedded(value)
LogicalType.SSID -> pseudo("ssid", value) { "net-" + it.take(6) } LogicalType.SSID -> pseudo("ssid", value) { "net-" + it.take(6) }
LogicalType.MAC, LogicalType.BSSID -> macPreservingOui(value) LogicalType.MAC, LogicalType.BSSID -> macPreservingOui(value)
LogicalType.IP4 -> ip4(value) LogicalType.IP4 -> ip4(value)
@@ -123,6 +133,38 @@ class Anonymizer(private val level: PrivacyLevel, private val salt: Salt) {
LogicalType.FREETEXT -> "[removed: may contain identifying text]" LogicalType.FREETEXT -> "[removed: may contain identifying text]"
} }
/**
* Replaces addresses and MACs found *inside* a longer string.
*
* Shizuku probes embed raw command output verbatim `ip neigh`, `ip route`, `dumpsys` which
* is genuinely valuable evidence and also a complete inventory of every device on the user's
* network, with hardware addresses. measurement-schema.md §9 flagged these as "hard to
* anonymize" and proposed dropping them from exports.
*
* Scrubbing beats dropping: the output stays readable and auditable you can still see the
* shape of the neighbour table and how many hosts there were while the identifiers become
* the same pseudonyms used everywhere else in the document. So a MAC appearing both in a
* parsed field and in a raw dump still reads as one device.
*
* Only addresses and MACs are touched, for the same reason as [Classification.inferFromValue]:
* they are the patterns that cannot be mistaken for something else in free text.
*/
private fun scrubEmbedded(value: String): String {
// Cheap bail-out: the overwhelming majority of strings are short and contain neither.
if (value.length < 7 || (!value.contains(':') && !value.contains('.'))) return value
// One pass, not three. Sequential passes re-process their own output: after a MAC became
// 78:9a:18:xx:yy:zz the IPv6 pattern matched it — six hex groups separated by colons is
// exactly an address — and mangled the vendor prefix that the MAC rule had just taken
// care to preserve. Ordered alternation resolves each position once, MAC first.
return EMBEDDED.replace(value) { m ->
when {
m.groups[1] != null -> macPreservingOui(m.value)
m.groups[2] != null -> ip6(m.value)
else -> ip4(m.value)
}
}
}
// ---- per-type transforms ------------------------------------------------------------- // ---- per-type transforms -------------------------------------------------------------
/** /**
@@ -147,6 +189,11 @@ class Anonymizer(private val level: PrivacyLevel, private val salt: Salt) {
* Public addresses keep only their /16 so the network is still locatable at ISP granularity. * Public addresses keep only their /16 so the network is still locatable at ISP granularity.
*/ */
private fun ip4(value: String): String { private fun ip4(value: String): String {
// A route destination carries a prefix length; pseudonymize the address and put it back,
// or "0.0.0.0/0" turns into nonsense and the routing table becomes unreadable.
value.substringAfter('/', "").takeIf { it.isNotEmpty() && value.contains('/') }?.let { len ->
return ip4(value.substringBefore('/')) + "/" + len
}
val o = value.split(".") val o = value.split(".")
if (o.size != 4 || o.any { it.toIntOrNull() == null }) return value if (o.size != 4 || o.any { it.toIntOrNull() == null }) return value
val n = o.map { it.toInt() } val n = o.map { it.toInt() }
@@ -167,8 +214,36 @@ class Anonymizer(private val level: PrivacyLevel, private val salt: Salt) {
* is a device fingerprint, especially with EUI-64. * is a device fingerprint, especially with EUI-64.
*/ */
private fun ip6(value: String): String { private fun ip6(value: String): String {
// Dotted quads reach here through the family-agnostic field names (addr, gateway, dst);
// hand them to the IPv4 path rather than mangling them as if they were v6.
if (value.count { it == ':' } < 2) return ip4(value)
if (value.contains('/')) {
return ip6(value.substringBefore('/')) + "/" + value.substringAfter('/')
}
val v = value.lowercase(Locale.ROOT) val v = value.lowercase(Locale.ROOT)
// The unspecified address and the default route are not identities; mangling them would
// make a routing table unreadable for no privacy gain.
if (v == "::1" || v == "::" || v.startsWith("fe80:") || v.startsWith("ff")) return v if (v == "::1" || v == "::" || v.startsWith("fe80:") || v.startsWith("ff")) return v
// Unique local addresses (fc00::/7) need the *whole* prefix replaced, not the tail.
//
// They look like the v6 equivalent of RFC1918, and the first instinct is to keep them for
// the same reason: private, topological, says nothing about anyone. That reasoning does
// not carry over. An RFC1918 prefix is shared by millions of networks and identifies
// none of them; a ULA global ID is 40 *random* bits, unique to one network by
// construction (RFC 4193). It is a network fingerprint. Passing the leading groups
// through - which is what the general path does - leaked 32 of those 40 bits.
//
// The prefix is pseudonymized as a unit, so two addresses on the same ULA subnet still
// land on the same pseudonymous prefix. "These hosts are on one network" survives;
// "this is *that* network" does not.
if (v.startsWith("fc") || v.startsWith("fd")) {
val groups = v.substringBefore('%').split(":")
val prefix = pseudo("ula-prefix", groups.take(3).joinToString(":")) { it }
val host = pseudo("ula-host", v) { it }
return "fd${prefix.substring(0, 2)}:${prefix.substring(2, 6)}:${prefix.substring(6, 10)}" +
"::${host.substring(0, 4)}"
}
val groups = v.substringBefore('%').split(":") val groups = v.substringBefore('%').split(":")
if (groups.size < 3) return v if (groups.size < 3) return v
val h = pseudo("ip6", value) { it } val h = pseudo("ip6", value) { it }
@@ -219,7 +294,7 @@ class Anonymizer(private val level: PrivacyLevel, private val salt: Salt) {
/** Deterministic per (domain, value, salt); memoized so one value maps to one pseudonym. */ /** Deterministic per (domain, value, salt); memoized so one value maps to one pseudonym. */
private fun pseudo(domain: String, value: String, shape: (String) -> String): String = private fun pseudo(domain: String, value: String, shape: (String) -> String): String =
cache.getOrPut("$domain$value") { cache.getOrPut("$domain\u0000$value") {
val md = MessageDigest.getInstance("SHA-256") val md = MessageDigest.getInstance("SHA-256")
md.update(salt.bytes) md.update(salt.bytes)
md.update(domain.toByteArray()) md.update(domain.toByteArray())
@@ -229,6 +304,21 @@ class Anonymizer(private val level: PrivacyLevel, private val salt: Salt) {
} }
private companion object { private companion object {
/**
* MAC | IPv6 | IPv4, in that order alternation is ordered, so a MAC-shaped token is
* claimed by the MAC rule before the IPv6 rule can see it.
*
* The patterns are deliberately conservative. A missed address is scrubbed by another
* rule or not at all; an over-eager one mangles timestamps, version strings and log
* prefixes, corrupting evidence to protect nothing.
*/
val EMBEDDED = Regex(
// Raw strings: a regex written with escaped escapes is a regex nobody can check.
"""(\b[0-9a-fA-F]{2}(?:[:-][0-9a-fA-F]{2}){5}\b)""" +
"""|(\b(?:[0-9a-fA-F]{1,4}:){2,7}(?::|[0-9a-fA-F]{1,4})(?:[0-9a-fA-F:]*))""" +
"""|(\b(?:\d{1,3}\.){3}\d{1,3}\b)"""
)
val publicSuffixes = setOf( val publicSuffixes = setOf(
"local", "lan", "home", "internal", "arpa", "local", "lan", "home", "internal", "arpa",
"com", "net", "org", "io", "app", "dev", "at", "de", "eu", "uk", "com", "net", "org", "io", "app", "dev", "at", "de", "eu", "uk",
@@ -32,6 +32,16 @@ object Classification {
"link_local", "ra_source", "prefix", "link_local", "ra_source", "prefix",
).forEach { put(it, LogicalType.IP6) } ).forEach { put(it, LogicalType.IP6) }
// Family-agnostic address fields — the names the models actually use (Address.addr,
// Route.gateway, Route.dst, DnsConfig.servers). Their absence here was a real leak: the
// device's own global IPv6 address went out verbatim at the level whose description
// promises addresses are pseudonymized. Typed IP6 because the transform detects the
// family from the value, falling through to the IPv4 path for a dotted quad.
listOf(
"addr", "address", "gateway", "dst", "src", "servers", "server", "resolver",
"next_hop", "via", "public_ip", "observed_ip",
).forEach { put(it, LogicalType.IP6) }
listOf("mac", "hw_addr", "gateway_mac", "router_mac", "sender_mac", "peer_mac") listOf("mac", "hw_addr", "gateway_mac", "router_mac", "sender_mac", "peer_mac")
.forEach { put(it, LogicalType.MAC) } .forEach { put(it, LogicalType.MAC) }
listOf("bssid", "ap_mac").forEach { put(it, LogicalType.BSSID) } listOf("bssid", "ap_mac").forEach { put(it, LogicalType.BSSID) }
@@ -40,6 +50,8 @@ object Classification {
listOf( listOf(
"fqdn", "hostname", "host", "name", "reverse_dns", "ptr", "domain", "query_name", "fqdn", "hostname", "host", "name", "reverse_dns", "ptr", "domain", "query_name",
"friendly_name", "server_name", "sni", "cname", "search_domain", "device_name", "friendly_name", "server_name", "sni", "cname", "search_domain", "device_name",
// Plural and prefixed variants the models actually use.
"search_domains", "private_dns_hostname", "domains", "hostnames",
).forEach { put(it, LogicalType.FQDN) } ).forEach { put(it, LogicalType.FQDN) }
listOf("session_id", "credential", "token", "device_id", "android_id", "serial", "imsi", "iccid") listOf("session_id", "credential", "token", "device_id", "android_id", "serial", "imsi", "iccid")
@@ -78,6 +90,49 @@ object Classification {
return null return null
} }
/**
* Last-resort classification from the *value*, when the field name is unrecognised.
*
* A name table can only protect fields somebody remembered to add, which is the wrong
* property for a privacy control: the dangerous field is the one nobody thought of. This
* exists because that failed once already `addresses[].addr` holds the device's own global
* IPv6 address, the table had never heard of the name, and it went out verbatim.
*
* Only addresses and MACs are inferred, because only those have shapes that cannot be
* mistaken for something else. Hostnames deliberately are not: `train.udp_updown` is
* indistinguishable from a domain by shape, and mangling a test type would corrupt the
* document to protect nothing.
*/
fun inferFromValue(value: String): LogicalType? {
val v = value.trim()
if (v.isEmpty() || v.length > 64) return null
if (looksLikeMac(v)) return LogicalType.MAC
if (looksLikeIp6(v)) return LogicalType.IP6
if (looksLikeIp4(v)) return LogicalType.IP4
return null
}
private fun isHex(c: Char) = c in '0'..'9' || c in 'a'..'f' || c in 'A'..'F'
private fun looksLikeMac(v: String): Boolean {
val parts = v.split(':', '-')
return parts.size == 6 && parts.all { p -> p.length == 2 && p.all(::isHex) }
}
private fun looksLikeIp4(v: String): Boolean {
val parts = v.substringBefore('/').split('.')
return parts.size == 4 && parts.all { p ->
p.isNotEmpty() && p.length <= 3 && p.all(Char::isDigit) && p.toInt() <= 255
}
}
private fun looksLikeIp6(v: String): Boolean {
val core = v.substringBefore('/').substringBefore('%')
// Two colons minimum, so a time or a MAC fragment does not qualify, and nothing but the
// characters an address may contain.
return core.count { it == ':' } >= 2 && core.all { it == ':' || isHex(it) }
}
fun dropAtBalanced(path: List<String>): Boolean { fun dropAtBalanced(path: List<String>): Boolean {
if (path.isNotEmpty() && path.last() in droppedKeys) return true if (path.isNotEmpty() && path.last() in droppedKeys) return true
return droppedPaths.any { dropped -> dropped.all { path.contains(it) } } return droppedPaths.any { dropped -> dropped.all { path.contains(it) } }
@@ -182,4 +182,47 @@ class AnonymizerTest {
assertEquals(PrivacyLevel.BALANCED, PrivacyLevel.max(PrivacyLevel.BALANCED, PrivacyLevel.FULL)) assertEquals(PrivacyLevel.BALANCED, PrivacyLevel.max(PrivacyLevel.BALANCED, PrivacyLevel.FULL))
assertEquals(PrivacyLevel.FULL, PrivacyLevel.fromWire("nonsense")) assertEquals(PrivacyLevel.FULL, PrivacyLevel.fromWire("nonsense"))
} }
// A ULA looks like the v6 RFC1918 and is not. Its global ID is 40 random bits, unique to one
// network by construction (RFC 4193), so the prefix IS the identifier - unlike 192.168.x,
// which millions of networks share. Passing the leading groups through leaked most of it.
@Test
fun ulaPrefixesArePseudonymizedWhole() {
val doc = json.parseToJsonElement(
"""{"run":{"id":"r"},"networks":[{"link":{"dns":{"servers":["fda1:3fb1:ff92:6696::2662"]}}}]}"""
).jsonObject
val out = flat(anon(PrivacyLevel.BALANCED, doc))
assertFalse(out.contains("fda1"), "the ULA global ID survived: $out")
assertFalse(out.contains("3fb1"), "part of the ULA global ID survived: $out")
assertTrue(out.contains("fd"), "the result should still read as a ULA: $out")
}
// Pseudonymizing the prefix as a unit keeps the one fact that is diagnostically useful:
// whether two addresses sit on the same network.
@Test
fun addressesOnOneUlaSubnetStayRelated() {
val doc = json.parseToJsonElement(
"""{"run":{"id":"r"},"networks":[{"link":{"dns":{"servers":[
"fda1:3fb1:ff92:6696::1","fda1:3fb1:ff92:6696::2","fdff:9999:8888:7777::1"]}}}]}"""
).jsonObject
val servers = anon(PrivacyLevel.BALANCED, doc)["networks"]!!.jsonArray[0].jsonObject["link"]!!
.jsonObject["dns"]!!.jsonObject["servers"]!!.jsonArray.map { it.jsonPrimitive.content }
val prefixOf = { s: String -> s.substringBeforeLast("::") }
assertEquals(prefixOf(servers[0]), prefixOf(servers[1]),
"two addresses on one ULA subnet should share a pseudonymous prefix")
assertNotEquals(prefixOf(servers[0]), prefixOf(servers[2]),
"a different ULA network must not collide with the first")
}
// RFC1918 stays readable, and this is the contrast that justifies it: a shared, meaningless
// prefix is topology; a unique random one is identity.
@Test
fun rfc1918StaysReadableUnlikeUla() {
val doc = json.parseToJsonElement(
"""{"run":{"id":"r"},"networks":[{"link":{"dns":{"servers":["192.168.1.1","10.13.102.1"]}}}]}"""
).jsonObject
val out = flat(anon(PrivacyLevel.BALANCED, doc))
assertTrue(out.contains("192.168.1.1"), "RFC1918 should survive: $out")
assertTrue(out.contains("10.13.102.1"), "RFC1918 should survive: $out")
}
} }
@@ -0,0 +1,177 @@
// SPDX-FileCopyrightText: 2026 Echolot contributors
// SPDX-License-Identifier: GPL-3.0-or-later
package app.echo_lot.privacy
import kotlinx.serialization.json.Json
import kotlinx.serialization.json.JsonObject
import kotlinx.serialization.json.jsonArray
import kotlinx.serialization.json.jsonObject
import kotlinx.serialization.json.jsonPrimitive
import kotlin.test.Test
import kotlin.test.assertFalse
import kotlin.test.assertTrue
/**
* The blunt instrument: build a document with identifying values in every place one can actually
* occur, anonymize it, and assert none of them survive.
*
* [AnonymizerTest] checks that the fields the classification table knows about are handled
* correctly. This checks the other half the fields it does *not* know about. A per-field test
* can only fail for a field someone remembered to write a case for, which is exactly the wrong
* property for a privacy check: the dangerous field is the one nobody thought of.
*
* Concretely, this is written the way it is because the schema's own field names disagree with
* the classifier's. `Address.addr` carries an IP and is documented as such in
* measurement-schema.md §8, but the classifier keys on names like `ip4` and `gateway_ip4` and had
* never heard of `addr`.
*/
class LeakTest {
private val json = Json { prettyPrint = false }
private val salt = Salt.perRun(ByteArray(32) { 3 })
/**
* Every string here is something that identifies a person, a household or a device, placed
* where the real models actually put it (`core-measurement`'s Network/Link/Address/DnsConfig).
*/
private val secrets = listOf(
"Rambossek WLAN", // ssid
"78:9a:18:aa:bb:cc", // bssid
"aa:bb:cc:dd:ee:11", // gateway mac
"2001:1ad0:c4fe:6767::150", // global v6 address on the interface
"2a02:1748:dead:beef::1", // v6 default gateway
"203.0.113.77", // public v4
"nas.rambossek.lan", // private-dns hostname
"rambossek.lan", // search domain
"Anna's Chromecast", // neighbour name
"kitchen table", // free-text note
)
private fun document(): String = """
{
"schema": "echolot/measurement",
"run": {
"id": "run-1", "trigger": "manual", "notes": "${secrets[9]}",
"device": {"manufacturer": "OnePlus", "model": "CPH2747"}
},
"networks": [{
"id": "net-1", "transport": "wifi",
"link": {
"mtu": 1500,
"addresses": [
{"addr": "${secrets[3]}", "prefix_len": 64, "scope": "global"},
{"addr": "192.168.1.44", "prefix_len": 24, "scope": "global"}
],
"routes": [
{"dst": "::/0", "gateway": "${secrets[4]}", "iface": "wlan0"},
{"dst": "0.0.0.0/0", "gateway": "192.168.1.1", "iface": "wlan0"}
],
"dns": {
"servers": ["${secrets[5]}", "192.168.1.1"],
"private_dns_hostname": "${secrets[6]}",
"search_domains": ["${secrets[7]}"]
}
},
"wifi": {"ssid": "${secrets[0]}", "bssid": "${secrets[1]}"},
"neighbors": [{"name": "${secrets[8]}", "mac": "${secrets[2]}"}]
}],
"tests": [{"id": "t1", "type": "train.udp_updown", "status": "ok",
"metrics": {"rtt_ms_avg": 12.4}}],
"findings": [],
"summary": {"verdict": "ok"}
}
""".trimIndent()
private fun anonymized(level: PrivacyLevel): String =
json.encodeToString(
kotlinx.serialization.json.JsonObject.serializer(),
Anonymizer(level, salt).anonymize(json.parseToJsonElement(document()).jsonObject),
)
@Test
fun nothingIdentifyingSurvivesBalanced() {
val out = anonymized(PrivacyLevel.BALANCED)
val leaked = secrets.filter { out.contains(it) }
assertTrue(
leaked.isEmpty(),
"these identifying values were uploaded verbatim at BALANCED: $leaked\n\n$out",
)
}
@Test
fun nothingIdentifyingSurvivesStrict() {
val out = anonymized(PrivacyLevel.STRICT)
val leaked = secrets.filter { out.contains(it) }
assertTrue(leaked.isEmpty(), "leaked at STRICT: $leaked\n\n$out")
}
// Private addresses are kept on purpose — they describe the topology and not the person — so
// this pins that the leak test above is not passing by accident of over-redaction.
@Test
fun privateAddressesAreStillReadable() {
val out = anonymized(PrivacyLevel.BALANCED)
assertTrue(out.contains("192.168.1.1"), "RFC1918 gateway should survive: $out")
assertTrue(out.contains("192.168.1.44"), "RFC1918 interface address should survive: $out")
}
/**
* Raw shell output embeds a complete inventory of the local network, and neither the field-name
* table nor the whole-value shape check can see it: `ip_neigh` is one long string that is
* itself neither a MAC nor an address.
*
* This is not hypothetical. The blob below is (abridged) real output that reached the server
* at the `balanced` level from a test device, carrying the hardware address of every host on
* the network. measurement-schema.md §9 had flagged raw dumps as "hard to anonymize"; nothing
* enforced it.
*/
@Test
fun identifiersInsideRawShellOutputAreScrubbed() {
// Joined rather than written with escapes, so the fixture stays readable and there is no
// chance of an escape being mangled on its way into the JSON below.
val dump = listOf(
"uid=2000",
"10.13.102.5 dev wlan0 lladdr 90:09:d0:1a:83:e4 REACHABLE",
"10.13.102.1 dev wlan0 lladdr 78:9a:18:54:b8:f9 REACHABLE",
"10.13.102.111 dev wlan0 lladdr dc:a2:66:08:69:95 STALE",
"2001:4bb8:46a:e724:289d:87ff:feb6:ebd3 dev wlan0 lladdr b8:be:f4:bc:ca:cf STALE",
).joinToString(" | ")
val doc = json.parseToJsonElement(
"""{"run":{"id":"r"},"tests":[{"id":"t","type":"link.ip_monitor",
"evidence":{"ip_neigh":"$dump"}}]}"""
).jsonObject
val out = json.encodeToString(
kotlinx.serialization.json.JsonObject.serializer(),
Anonymizer(PrivacyLevel.BALANCED, salt).anonymize(doc),
)
for (mac in listOf("90:09:d0:1a:83:e4", "78:9a:18:54:b8:f9", "dc:a2:66:08:69:95", "b8:be:f4:bc:ca:cf")) {
assertFalse(out.contains(mac), "a neighbour's MAC survived inside the raw dump: $mac")
}
assertFalse(out.contains("2001:4bb8:46a:e724:289d:87ff:feb6:ebd3"),
"a global IPv6 survived inside the raw dump")
// Scrubbed, not dropped: the evidence must still be readable, or the raw dump stops being
// evidence at all. Structure, hostnames of the fields, and RFC1918 addresses stay.
assertTrue(out.contains("REACHABLE") && out.contains("STALE"), "the dump lost its structure")
assertTrue(out.contains("10.13.102.1"), "RFC1918 addresses should stay readable: $out")
assertTrue(out.contains("78:9a:18"), "the vendor prefix should survive for identification")
}
// A MAC in a raw dump and the same MAC in a parsed field must land on the same pseudonym, or
// the document stops being internally consistent and one device reads as two.
@Test
fun theSameIdentifierMatchesAcrossParsedAndRawFields() {
val doc = json.parseToJsonElement(
"""{"run":{"id":"r"},
"networks":[{"wifi":{"bssid":"78:9a:18:54:b8:f9"}}],
"tests":[{"id":"t","evidence":{"ip_neigh":"gw dev wlan0 lladdr 78:9a:18:54:b8:f9 REACHABLE"}}]}"""
).jsonObject
val out = Anonymizer(PrivacyLevel.BALANCED, salt).anonymize(doc)
val parsed = out["networks"]!!.jsonArray[0].jsonObject["wifi"]!!.jsonObject["bssid"]!!
.jsonPrimitive.content
val raw = json.encodeToString(kotlinx.serialization.json.JsonObject.serializer(), out)
assertTrue(raw.contains(parsed),
"the parsed BSSID pseudonym ($parsed) does not appear in the scrubbed dump")
}
}
@@ -0,0 +1,48 @@
// SPDX-FileCopyrightText: 2026 Echolot contributors
// SPDX-License-Identifier: GPL-3.0-or-later
package app.echo_lot.privacy
import kotlinx.serialization.json.Json
import kotlinx.serialization.json.JsonObject
import kotlinx.serialization.json.jsonObject
import java.io.File
import kotlin.test.Test
import kotlin.test.assertTrue
/**
* Runs the anonymizer over a real captured document when one is supplied via ECHOLOT_REAL_RUN,
* and reports every MAC and public address that survives.
*
* Fixtures only contain the identifiers somebody thought to put in them. A real run off a real
* phone contains whatever the probes actually produce which is how the raw-shell-output leak was
* found in the first place. Self-skips when no document is supplied, so nobody's network ends up
* committed to the repository.
*/
class RealDocumentTest {
@Test
fun noIdentifiersSurviveInARealDocument() {
val path = System.getenv("ECHOLOT_REAL_RUN")
if (path.isNullOrBlank() || !File(path).isFile) {
println("RealDocumentTest skipped (set ECHOLOT_REAL_RUN to a captured run)"); return
}
val json = Json { prettyPrint = false }
val doc = json.parseToJsonElement(File(path).readText()).jsonObject
val out = json.encodeToString(
JsonObject.serializer(),
Anonymizer(PrivacyLevel.BALANCED, Salt.perRun(ByteArray(32) { 5 })).anonymize(doc),
)
val macs = Regex("""\b[0-9a-fA-F]{2}(?::[0-9a-fA-F]{2}){5}\b""").findAll(out)
.map { it.value.lowercase() }
.filter { it != "00:00:00:00:00:00" }
.toSet()
val original = Regex("""\b[0-9a-fA-F]{2}(?::[0-9a-fA-F]{2}){5}\b""")
.findAll(File(path).readText()).map { it.value.lowercase() }.toSet()
val survived = macs intersect original
println("MACs in the original: ${original.size}; unchanged after anonymizing: ${survived.size}")
assertTrue(survived.isEmpty(), "these real MAC addresses survived anonymization: $survived")
}
}
@@ -0,0 +1,41 @@
// SPDX-FileCopyrightText: 2026 Echolot contributors
// SPDX-License-Identifier: GPL-3.0-or-later
package app.echo_lot.probe
import app.echo_lot.measurement.Constraints
import app.echo_lot.measurement.Transport
import java.net.DatagramSocket
/**
* Detects what will prevent this run from measuring (measurement-schema.md §3 `constraints`),
* before any probe runs and independently of all of them.
*
* The known case: while a VPN holds the default route, Android refuses `Network.bindSocket()` on
* the underlying networks (EPERM) so apps cannot leak around the tunnel. Every per-network test
* then silently measures the tunnel or nothing, and the run comes out shaped exactly like a clean
* run of a healthy network. Detecting that here one throwaway bind per network is what lets
* the document say "these networks went unmeasured" instead of leaving the reader to infer it
* from a pattern of `attempted: false` scattered across the tests.
*/
object ConstraintDetector {
fun detect(entries: List<NetworkInventory.Entry>): Constraints {
val vpnActive = entries.any { it.model.transport == Transport.VPN }
val unmeasured = ArrayList<String>()
for (e in entries) {
// The tunnel itself stays bindable — it is the underlying networks the OS walls off.
if (e.model.transport == Transport.VPN) continue
val bindable = runCatching {
DatagramSocket().use { s -> e.handle.bindSocket(s) }
true
}.getOrDefault(false)
if (!bindable) unmeasured.add(e.model.id)
}
return Constraints(
vpnActive = vpnActive,
perNetworkBlocked = unmeasured.isNotEmpty(),
unmeasuredNetworks = unmeasured,
)
}
}
@@ -0,0 +1,227 @@
// SPDX-FileCopyrightText: 2026 Echolot contributors
// SPDX-License-Identifier: GPL-3.0-or-later
package app.echo_lot.probe
import android.content.Context
import app.echo_lot.measurement.Test
import app.echo_lot.measurement.TestStatus
import app.echo_lot.measurement.TestType
import app.echo_lot.measurement.Tier
import kotlinx.coroutines.Dispatchers
import kotlinx.coroutines.withContext
import kotlinx.serialization.json.JsonObject
import kotlinx.serialization.json.buildJsonObject
import kotlinx.serialization.json.put
import java.net.DatagramPacket
import java.net.DatagramSocket
import java.net.InetAddress
import java.net.InetSocketAddress
import java.util.Random
/**
* Asks the network's own DNS servers directly, then asks Android to resolve the same name, and
* compares.
*
* The comparison is the point. A name that fails to resolve looks the same to a user whatever the
* cause, but the causes want opposite responses: if the server does not answer, the network is
* broken and the router is the thing to look at; if the server answers a raw query while the
* platform still cannot resolve, the device's own resolver has wedged and toggling wifi fixes it in
* seconds. Nothing else on a phone will tell you which of those you have.
*
* This is deliberately not a general DNS test no recursion checks, no DNSSEC, no rewriting
* detection; [DnsCanaryProbe] covers interception. This one answers a single question: is the
* resolver on this device doing its job.
*/
class DnsResolverProbe(
private val entries: List<NetworkInventory.Entry>,
/** Resolved directly rather than through any cache; any name with a stable answer will do. */
private val probeName: String = "one.one.one.one",
) : Probe {
override val type = TestType.DNS_RESOLVER
override val tier = Tier.APP
// A query per server with a 3s ceiling, plus one getaddrinfo that may sit out its own timeout.
override val estimatedMs = 6_000L
override suspend fun run(ctx: Context, ids: ProbeIds): Test = withContext(Dispatchers.IO) {
val b = TestBuilder(type, tier, ids)
val perNetwork = LinkedHashMap<String, Pair<String, JsonObject>>()
for (e in entries) {
val servers = e.model.link.dns?.servers.orEmpty()
if (servers.isEmpty()) continue
val label = "${e.model.transport.name.lowercase()}:${e.model.id}"
// Directly: does the configured server answer at all?
var direct: Boolean? = null
var directDetail = "no server answered"
for (s in servers) {
val r = try {
queryDirect(s, probeName)
} catch (e: DnsRefused) {
// Distinguished deliberately: a server that replies with a failure is a
// working server saying no, which points at the network rather than here.
direct = false
directDetail = "$s ${e.why}"
break
}
if (r != null) {
direct = true
directDetail = "$s answered in ${r}ms"
break
}
direct = false
directDetail = "$s did not answer"
}
// The search domains the network handed out, asked about separately.
//
// A resolver appends these to a lookup, so a search domain the server will not answer
// for stalls every name a client asks about — and it fails as silence, which is
// indistinguishable from packet loss, so clients retry rather than moving on. Asking
// about a name that cannot exist is deliberate: the answer wanted here is NXDOMAIN,
// and what matters is only whether anything comes back at all.
val searchDomains = e.model.link.dns?.searchDomains.orEmpty()
var searchAnswered: Boolean? = null
var searchDetail = ""
for (d in searchDomains) {
val nonce = "echolot-probe-" + java.util.UUID.randomUUID().toString().take(8)
val answered = servers.any { srv ->
runCatching { queryDirect(srv, "$nonce.$d") != null }
.getOrElse { it is DnsRefused } // a refusal is still an answer
}
if (!answered) {
searchAnswered = false
searchDetail = "$d is not answered at all — queries under it vanish"
break
}
searchAnswered = true
searchDetail = "$d answers"
}
// Through the platform: what an app actually gets.
val viaSystem = runCatching {
e.handle.getAllByName(probeName).isNotEmpty()
}.getOrElse { false }
perNetwork[label] = e.model.id to buildJsonObject {
put("network_ref", e.model.id)
put("servers", servers.joinToString(","))
direct?.let { put("direct_answer", it) }
put("direct_detail", directDetail)
if (searchDomains.isNotEmpty()) {
put("search_domains", searchDomains.joinToString(","))
searchAnswered?.let { put("search_answered", it) }
put("search_detail", searchDetail)
}
put("system_resolves", viaSystem)
// Named here rather than left for a finding to infer, because the pairing is the
// whole observation and splitting it across two places invites reading one alone.
put(
"verdict",
when {
// Ordered by which component is at fault, most specific first. A search
// domain that swallows queries explains a failure that would otherwise be
// blamed on the device, so it has to be tested before that conclusion.
searchAnswered == false -> "search domain swallows queries"
viaSystem -> "resolver working"
direct == true -> "server answers, device resolver does not"
direct == false -> "server does not answer"
else -> "not determined"
},
)
}
}
if (perNetwork.isEmpty()) {
return@withContext b.build(
TestStatus.SKIPPED,
evidence = buildJsonObject { put("reason", "no network advertised a DNS server") },
)
}
val evidence = buildJsonObject {
put("name", probeName)
for ((label, v) in perNetwork) put(label, v.second)
}
// OK means the measurement ran, not that DNS is healthy — the finding says that.
b.build(TestStatus.OK, evidence = evidence)
}
/**
* Sends one A query straight to [server] over UDP. Returns the round trip in ms, or null.
*
* Hand-rolled rather than via any resolver API on purpose: the entire point is to bypass the
* component under suspicion. Anything that goes through the platform resolver would inherit
* exactly the fault this is trying to detect.
*/
private fun queryDirect(server: String, name: String): Long? {
return try {
queryDirectOrThrow(server, name)
} catch (e: DnsRefused) {
throw e
} catch (t: Throwable) {
null
}
}
private fun queryDirectOrThrow(server: String, name: String): Long? = run {
val id = Random().nextInt(0xFFFF)
val query = buildQuery(id, name)
DatagramSocket().use { sock ->
sock.soTimeout = 3000
val addr = InetAddress.getByName(server) // a literal from DHCP; no lookup happens
val t0 = System.nanoTime()
sock.send(DatagramPacket(query, query.size, InetSocketAddress(addr, 53)))
val buf = ByteArray(512)
val reply = DatagramPacket(buf, buf.size)
sock.receive(reply)
val ms = (System.nanoTime() - t0) / 1_000_000
// A reply is not an answer. Counting any packet as success would let a REFUSED or
// SERVFAIL — both perfectly well-formed responses — be reported as "the server
// answers", and this probe's whole output is the claim that the server is fine and
// the device is not. That would be an accusation pointed at the wrong component,
// stated with confidence.
val replyId = ((buf[0].toInt() and 0xFF) shl 8) or (buf[1].toInt() and 0xFF)
val rcode = if (reply.length >= 4) buf[3].toInt() and 0x0F else -1
val answers = if (reply.length >= 8) {
((buf[6].toInt() and 0xFF) shl 8) or (buf[7].toInt() and 0xFF)
} else 0
when {
replyId != id || reply.length < 12 -> null
rcode != 0 -> throw DnsRefused(rcodeName(rcode))
answers == 0 -> throw DnsRefused("answered with no records")
else -> ms
}
}
}
/** The server replied, but with a failure — which is a network fault, not a device one. */
private class DnsRefused(val why: String) : Exception(why)
private fun rcodeName(rcode: Int): String = when (rcode) {
1 -> "rejected the query as malformed"
2 -> "reported its own failure (SERVFAIL)"
3 -> "said the name does not exist (NXDOMAIN)"
4 -> "does not implement this query"
5 -> "refused the query (REFUSED)"
else -> "returned rcode $rcode"
}
/** A minimal DNS query: one question, class IN, type A, recursion desired. */
private fun buildQuery(id: Int, name: String): ByteArray {
val labels = name.split('.').filter { it.isNotEmpty() }
val out = ArrayList<Byte>(32)
out.add((id shr 8).toByte()); out.add(id.toByte())
out.add(0x01); out.add(0x00) // recursion desired
out.add(0x00); out.add(0x01) // one question
repeat(6) { out.add(0x00) } // no answers, authority or additional
for (l in labels) {
out.add(l.length.toByte())
for (c in l.toByteArray(Charsets.US_ASCII)) out.add(c)
}
out.add(0x00) // root label
out.add(0x00); out.add(0x01) // type A
out.add(0x00); out.add(0x01) // class IN
return out.toByteArray()
}
}
@@ -40,24 +40,37 @@ class IcmpProbe(
override suspend fun run(ctx: Context, ids: ProbeIds): Test = withContext(Dispatchers.IO) { override suspend fun run(ctx: Context, ids: ProbeIds): Test = withContext(Dispatchers.IO) {
val b = TestBuilder(type, tier, ids) val b = TestBuilder(type, tier, ids)
val perNetwork = LinkedHashMap<String, String>() val perNetwork = LinkedHashMap<String, Pair<String?, Attempt>>()
var anyOk = false var anyOk = false
val rtts = ArrayList<Double>() val rtts = ArrayList<Double>()
// Default network first, then each active network explicitly. // Default network first, then each active network explicitly.
attempt(null).let { (ok, detail, rtt) -> attempt(null).let { a ->
perNetwork["default"] = detail; if (ok) { anyOk = true; rtt?.let(rtts::add) } perNetwork["default"] = null to a
if (a.ok) { anyOk = true; a.rttMs?.let(rtts::add) }
} }
for (e in entries) { for (e in entries) {
val label = "${e.model.transport.name.lowercase()}:${e.model.id}" val label = "${e.model.transport.name.lowercase()}:${e.model.id}"
val (ok, detail, rtt) = attempt(e.handle) val a = attempt(e.handle)
perNetwork[label] = detail perNetwork[label] = e.model.id to a
if (ok) { anyOk = true; rtt?.let(rtts::add) } if (a.ok) { anyOk = true; a.rttMs?.let(rtts::add) }
} }
// Per-network results are recorded structurally, not just as prose. The aggregate status
// can only say "some network answered"; a finding needs to know *which* network failed,
// and recovering that by parsing a human-readable detail string would be a trap waiting to
// spring the first time the wording changes.
val evidence: JsonObject = buildJsonObject { val evidence: JsonObject = buildJsonObject {
put("target", target) put("target", target)
for ((k, v) in perNetwork) put(k, v) for ((label, r) in perNetwork) {
val (netId, a) = r
put(label, buildJsonObject {
netId?.let { put("network_ref", it) }
put("ok", a.ok)
put("attempted", a.attempted)
put("detail", a.detail)
})
}
} }
val metrics: JsonObject = buildJsonObject { val metrics: JsonObject = buildJsonObject {
put("networks_ok", rtts.size) put("networks_ok", rtts.size)
@@ -69,14 +82,31 @@ class IcmpProbe(
b.build(status, evidence = evidence, metrics = metrics) b.build(status, evidence = evidence, metrics = metrics)
} }
private data class Attempt(val ok: Boolean, val detail: String, val rttMs: Double?) /**
* One network's result.
*
* [attempted] separates "we sent an echo request and heard nothing" from "we never got as far
* as sending one". Both leave [ok] false, and collapsing them is how a probe ends up asserting
* something about a network it never touched: binding to a non-default network can fail with
* EPERM, and reporting that as ICMPv6 silence blames the carrier for the app's own inability
* to use the interface.
*/
private data class Attempt(
val ok: Boolean,
val attempted: Boolean,
val detail: String,
val rttMs: Double?,
)
private fun attempt(network: Network?): Attempt { private fun attempt(network: Network?): Attempt {
var fd: FileDescriptor? = null var fd: FileDescriptor? = null
var sent = false
return try { return try {
val proto = if (v6) OsConstants.IPPROTO_ICMPV6 else OsConstants.IPPROTO_ICMP val proto = if (v6) OsConstants.IPPROTO_ICMPV6 else OsConstants.IPPROTO_ICMP
val family = if (v6) OsConstants.AF_INET6 else OsConstants.AF_INET val family = if (v6) OsConstants.AF_INET6 else OsConstants.AF_INET
fd = Os.socket(family, OsConstants.SOCK_DGRAM, proto) fd = Os.socket(family, OsConstants.SOCK_DGRAM, proto)
// Everything up to and including sendto is setup. A failure here means the test did
// not run on this network — not that the network stayed silent.
network?.bindSocket(fd) network?.bindSocket(fd)
Os.setsockoptTimeval(fd, OsConstants.SOL_SOCKET, OsConstants.SO_RCVTIMEO, StructTimeval.fromMillis(3000)) Os.setsockoptTimeval(fd, OsConstants.SOL_SOCKET, OsConstants.SO_RCVTIMEO, StructTimeval.fromMillis(3000))
val addr = network?.getByName(target) ?: InetAddress.getByName(target) val addr = network?.getByName(target) ?: InetAddress.getByName(target)
@@ -85,14 +115,16 @@ class IcmpProbe(
val packet = buildEchoRequest(v6, ident.toShort(), 1) val packet = buildEchoRequest(v6, ident.toShort(), 1)
val t0 = System.nanoTime() val t0 = System.nanoTime()
Os.sendto(fd, packet, 0, packet.size, 0, addr, 0) Os.sendto(fd, packet, 0, packet.size, 0, addr, 0)
sent = true
val buf = ByteBuffer.allocate(1500) val buf = ByteBuffer.allocate(1500)
val received = Os.recvfrom(fd, buf, 0, null) val received = Os.recvfrom(fd, buf, 0, null)
val rttMs = (System.nanoTime() - t0) / 1_000_000.0 val rttMs = (System.nanoTime() - t0) / 1_000_000.0
val replyType = if (received > 0) buf.get(0).toInt() and 0xFF else -1 val replyType = if (received > 0) buf.get(0).toInt() and 0xFF else -1
val ok = replyType == (if (v6) 129 else 0) val ok = replyType == (if (v6) 129 else 0)
Attempt(ok, "reply type=$replyType rtt_ms=${"%.1f".format(Locale.ROOT, rttMs)} bytes=$received", if (ok) rttMs else null) Attempt(ok, true, "reply type=$replyType rtt_ms=${"%.1f".format(Locale.ROOT, rttMs)} bytes=$received", if (ok) rttMs else null)
} catch (e: Throwable) { } catch (e: Throwable) {
Attempt(false, "error: ${e.message ?: e.javaClass.simpleName}", null) // A timeout after a successful send is a real "no reply"; anything before it is not.
Attempt(false, sent, "error: ${e.message ?: e.javaClass.simpleName}", null)
} finally { } finally {
fd?.let { runCatching { Os.close(it) } } fd?.let { runCatching { Os.close(it) } }
} }
@@ -0,0 +1,107 @@
// SPDX-FileCopyrightText: 2026 Echolot contributors
// SPDX-License-Identifier: GPL-3.0-or-later
package app.echo_lot.probe
import android.content.Context
import android.net.nsd.NsdManager
import android.net.nsd.NsdServiceInfo
import android.net.wifi.WifiManager
import app.echo_lot.measurement.Test
import app.echo_lot.measurement.TestStatus
import app.echo_lot.measurement.TestType
import app.echo_lot.measurement.Tier
import kotlinx.coroutines.Dispatchers
import kotlinx.coroutines.delay
import kotlinx.coroutines.withContext
import kotlinx.serialization.json.buildJsonObject
import kotlinx.serialization.json.put
import kotlinx.serialization.json.putJsonObject
import java.util.Collections
/**
* local.mdns_inventory what answers mDNS on this network (MulticastLock + NSD discovery).
* The service inventory doubles as the VLAN-leakage detector: a chromecast answering on the
* guest wifi is a segmentation fault made visible. Folded from the prober, validated on both
* known devices (4 services each).
*
* Two hardware-bought lessons are load-bearing here:
* - The `_services._dns-sd._udp.` meta-query returned 0 on BOTH devices while concrete types
* found live services NsdManager's meta-query support is unreliable across builds, so the
* concrete types are the measurement and the meta-query result is itself evidence.
* - 4 s of listening missed services that 10 s catches; mDNS answers straggle.
*/
class MdnsInventoryProbe : Probe {
override val type = TestType.LOCAL_MDNS_INVENTORY
override val tier = Tier.APP
override val estimatedMs = 10_500L
/** Meta-query + common concrete types (HTTP covers HA/printers/NAS; googlecast is ubiquitous). */
private val queries = listOf(
"meta" to "_services._dns-sd._udp.",
"http" to "_http._tcp.",
"googlecast" to "_googlecast._tcp.",
)
private class Recorder : NsdManager.DiscoveryListener {
val names: MutableList<String> = Collections.synchronizedList(mutableListOf())
@Volatile var started = false
@Volatile var startFailCode: Int? = null
override fun onStartDiscoveryFailed(t: String?, code: Int) { startFailCode = code }
override fun onStopDiscoveryFailed(t: String?, code: Int) {}
override fun onDiscoveryStarted(t: String?) { started = true }
override fun onDiscoveryStopped(t: String?) {}
override fun onServiceFound(s: NsdServiceInfo?) { s?.serviceName?.let { names.add(it) } }
override fun onServiceLost(s: NsdServiceInfo?) {}
}
override suspend fun run(ctx: Context, ids: ProbeIds): Test = withContext(Dispatchers.IO) {
val b = TestBuilder(type, tier, ids)
val wifi = ctx.getSystemService(WifiManager::class.java)
val lock = wifi?.createMulticastLock("echolot")?.apply {
setReferenceCounted(false)
runCatching { acquire() }
}
val nsd = ctx.getSystemService(NsdManager::class.java)
?: return@withContext b.build(
TestStatus.UNSUPPORTED,
evidence = buildJsonObject { put("reason", "NsdManager unavailable") },
)
val recorders = queries.map { (label, type) ->
val r = Recorder()
runCatching { nsd.discoverServices(type, NsdManager.PROTOCOL_DNS_SD, r) }
.onFailure { r.startFailCode = -1 }
Triple(label, type, r)
}
try {
delay(10_000)
var total = 0
var anyStarted = false
val evidence = buildJsonObject {
put("multicast_lock", lock?.isHeld == true)
for ((label, type, r) in recorders) {
runCatching { nsd.stopServiceDiscovery(r) }
anyStarted = anyStarted || r.started
val names = r.names.distinct()
total += names.size
putJsonObject(label) {
put("query", type)
put("started", r.started)
r.startFailCode?.let { put("start_fail_code", it) }
put("found", names.size)
if (names.isNotEmpty()) put("names", names.joinToString(", ").take(300))
}
}
}
val metrics = buildJsonObject { put("services_found", total) }
// Zero services on a started discovery is a legitimate result (an empty or properly
// isolated network), not a failure — only discovery refusing to start is one.
b.build(if (anyStarted) TestStatus.OK else TestStatus.FAILED,
evidence = evidence, metrics = metrics)
} finally {
recorders.forEach { (_, _, r) -> runCatching { nsd.stopServiceDiscovery(r) } }
runCatching { lock?.release() }
}
}
}
@@ -39,6 +39,32 @@ object NetworkInventory {
return out return out
} }
/**
* Android's own verdict on the network, read straight from the capabilities it already has.
*
* PARTIAL_CONNECTIVITY only exists from API 28 and CAPTIVE_PORTAL from 23, so both are read
* defensively: an older platform that cannot answer should leave the field null rather than
* assert a false.
*/
private fun systemVerdict(caps: NetworkCapabilities): app.echo_lot.measurement.SystemVerdict =
app.echo_lot.measurement.SystemVerdict(
validated = caps.hasCapability(NetworkCapabilities.NET_CAPABILITY_VALIDATED),
captivePortal = runCatching {
caps.hasCapability(NetworkCapabilities.NET_CAPABILITY_CAPTIVE_PORTAL)
}.getOrNull(),
// NET_CAPABILITY_PARTIAL_CONNECTIVITY is @SystemApi, so the constant is not in the
// public SDK even though the platform sets it from API 28. The number is stable —
// changing it would break every system app that reads it — but this is a value the
// SDK does not promise us, so it is asked for defensively and reported as unknown
// rather than as false if anything about it is not as expected.
partialConnectivity = runCatching {
caps.hasCapability(NET_CAPABILITY_PARTIAL_CONNECTIVITY)
}.getOrNull(),
)
/** @SystemApi NetworkCapabilities.NET_CAPABILITY_PARTIAL_CONNECTIVITY, API 28+. */
private const val NET_CAPABILITY_PARTIAL_CONNECTIVITY = 24
private fun toModel(id: String, caps: NetworkCapabilities, lp: LinkProperties): MNetwork { private fun toModel(id: String, caps: NetworkCapabilities, lp: LinkProperties): MNetwork {
val transport = when { val transport = when {
caps.hasTransport(NetworkCapabilities.TRANSPORT_WIFI) -> Transport.WIFI caps.hasTransport(NetworkCapabilities.TRANSPORT_WIFI) -> Transport.WIFI
@@ -72,6 +98,7 @@ object NetworkInventory {
return MNetwork( return MNetwork(
id = id, transport = transport, iface = lp.interfaceName, id = id, transport = transport, iface = lp.interfaceName,
link = Link(mtu = lp.mtu.takeIf { it > 0 }, addresses = addresses, routes = routes, dns = dns), link = Link(mtu = lp.mtu.takeIf { it > 0 }, addresses = addresses, routes = routes, dns = dns),
systemVerdict = systemVerdict(caps),
) )
} }
} }
@@ -0,0 +1,70 @@
// SPDX-FileCopyrightText: 2026 Echolot contributors
// SPDX-License-Identifier: GPL-3.0-or-later
package app.echo_lot.probe
import android.system.Os
import java.io.FileDescriptor
/**
* Linux socket-option ABI numbers that android.system.OsConstants does NOT reliably expose.
* Stable across Android's supported ABIs at the IP/IPv6 protocol levels, which is why they can
* be hardcoded: if setsockoptInt with one of these succeeds, the kernel accepted the option; if
* it throws ErrnoException, it did not. Either outcome is data. Do not "fix" these to
* OsConstants names they don't exist there (validated in the prober; see its OsAbi.kt).
*
* Measured fact worth keeping: `Os.getsockoptInt` is absent on both known devices (OnePlus 15
* A16, Lenovo TB330FU A15), so path-MTU values must be read from the errqueue (`ee_info`), never
* from getsockopt(IP_MTU).
*/
object OsAbi {
// IP level
const val IP_TTL = 2
const val IP_MTU_DISCOVER = 10
const val IP_MTU = 14
const val IP_RECVERR = 11
const val IP_PMTUDISC_DO = 2 // set DF, honor PMTU
const val IP_PMTUDISC_PROBE = 3 // set DF, ignore PMTU (for probing)
// IPv6 level
const val IPV6_MTU_DISCOVER = 23
const val IPV6_MTU = 24
const val IPV6_RECVERR = 25
const val IPV6_UNICAST_HOPS = 16
const val IPV6_PMTUDISC_PROBE = 3
// recv flags — not in OsConstants on any current API level
const val MSG_ERRQUEUE = 0x2000
const val MSG_DONTWAIT = 0x40
// struct sock_extended_err (uapi/linux/errqueue.h), fixed layout on all Android ABIs:
// u32 ee_errno; u8 ee_origin; u8 ee_type; u8 ee_code; u8 ee_pad; u32 ee_info; u32 ee_data;
// followed directly by the offender sockaddr (SO_EE_OFFENDER).
const val SOCK_EE_SIZE = 16
const val SO_EE_ORIGIN_ICMP = 2
const val ICMP_TIME_EXCEEDED = 11
const val ICMP_DEST_UNREACH = 3
/** Try setsockoptInt; return null on success, or the errno name on failure. */
fun trySetIntOpt(fd: FileDescriptor, level: Int, opt: Int, value: Int): String? =
try {
Os.setsockoptInt(fd, level, opt, value)
null
} catch (e: Throwable) {
e.message ?: e.javaClass.simpleName
}
/**
* getsockoptInt is not part of the stable public Os surface on every API level, so it is
* reached via reflection; callers must treat failure as "unreadable", not as an error.
*/
fun tryGetIntOpt(fd: FileDescriptor, level: Int, opt: Int): Result<Int> = runCatching {
val m = Os::class.java.getMethod(
"getsockoptInt",
FileDescriptor::class.java,
Int::class.javaPrimitiveType,
Int::class.javaPrimitiveType,
)
m.invoke(null, fd, level, opt) as Int
}
}
@@ -55,9 +55,10 @@ class TestBuilder(
evidence: JsonObject? = null, evidence: JsonObject? = null,
metrics: JsonObject? = null, metrics: JsonObject? = null,
error: TestError? = null, error: TestError? = null,
params: JsonObject? = null,
): Test = Test( ): Test = Test(
id = id, type = type, networkRef = networkRef, sessionRef = sessionRef, tier = tier, id = id, type = type, networkRef = networkRef, sessionRef = sessionRef, tier = tier,
startedMonoNs = startedMonoNs, endedMonoNs = ids.monoNs(), startedMonoNs = startedMonoNs, endedMonoNs = ids.monoNs(),
status = status, error = error, evidence = evidence, metrics = metrics, status = status, error = error, params = params, evidence = evidence, metrics = metrics,
) )
} }
@@ -60,6 +60,15 @@ class StunProbe(
override suspend fun run(ctx: Context, ids: ProbeIds): Test = withContext(Dispatchers.IO) { override suspend fun run(ctx: Context, ids: ProbeIds): Test = withContext(Dispatchers.IO) {
val b = TestBuilder(type, tier, ids) val b = TestBuilder(type, tier, ids)
// Without a server there is nothing to ask. Skipped rather than failed: "the STUN test
// failed" reads as a finding about the network, when the truth is that this device is
// not enrolled anywhere and no packet was ever sent.
if (serverHost.isBlank()) {
return@withContext b.build(
TestStatus.SKIPPED,
evidence = buildJsonObject { put("reason", "no server configured to ask") },
)
}
DatagramSocket().use { sock -> DatagramSocket().use { sock ->
sock.soTimeout = 3000 sock.soTimeout = 3000
val localPort = sock.localPort val localPort = sock.localPort
@@ -0,0 +1,255 @@
// SPDX-FileCopyrightText: 2026 Echolot contributors
// SPDX-License-Identifier: GPL-3.0-or-later
package app.echo_lot.probe
import android.content.Context
import android.system.Os
import android.system.OsConstants
import app.echo_lot.measurement.Flow
import app.echo_lot.measurement.Hop
import app.echo_lot.measurement.HopProbe
import app.echo_lot.measurement.Test
import app.echo_lot.measurement.TestError
import app.echo_lot.measurement.TestStatus
import app.echo_lot.measurement.TestType
import app.echo_lot.measurement.Tier
import app.echo_lot.measurement.TracerouteEvidence
import app.echo_lot.measurement.toEvidence
import kotlinx.coroutines.Dispatchers
import kotlinx.coroutines.delay
import kotlinx.coroutines.withContext
import kotlinx.serialization.json.buildJsonObject
import kotlinx.serialization.json.put
import java.io.FileDescriptor
import java.net.InetAddress
import java.net.InetSocketAddress
import java.nio.ByteBuffer
import java.nio.ByteOrder
/**
* traceroute.udp4 UDP traceroute reading ICMP time-exceeded off the socket error queue via
* Os.recvmsg(MSG_ERRQUEUE): no root, no raw socket, no native code. Folded from the prober,
* which validated real hop addresses on both known devices (6 hops on the OnePlus 15, 5 on the
* Lenovo) and thereby retired the planned C-over-JNI errqueue shim.
*
* StructMsghdr/StructCmsghdr/recvmsg are reached via reflection (repo convention for uncertain
* OS paths): present since roughly API 34, absent before, and the probe must run and report
* on both. An absent API is UNSUPPORTED with the reason, never a crash.
*/
class TracerouteProbe(
private val targetHost: String = "1.1.1.1",
private val maxHops: Int = 6,
) : Probe {
override val type = TestType.TRACEROUTE_UDP4
override val tier = Tier.APP
// Validated wall clock is ~250 ms on a healthy path; the ceiling is maxHops silent hops at
// 900 ms each, which only a blackholing path produces.
override val estimatedMs = 1_500L
private companion object {
const val BASE_PORT = 33434
/** ICMP errors take one RTT to surface on the errqueue; poll briefly, never block. */
const val HOP_DEADLINE_NS = 900_000_000L
const val POLL_INTERVAL_MS = 40L
}
override suspend fun run(ctx: Context, ids: ProbeIds): Test = withContext(Dispatchers.IO) {
val b = TestBuilder(type, tier, ids)
val params = buildJsonObject {
put("target", targetHost); put("max_hops", maxHops); put("base_port", BASE_PORT)
}
val api = ErrqueueApi.resolve()
?: return@withContext b.build(
TestStatus.UNSUPPORTED,
params = params,
error = TestError(
"no_recvmsg",
"StructMsghdr/Os.recvmsg not on this API level — errqueue unreadable",
),
)
var fd: FileDescriptor? = null
try {
fd = Os.socket(OsConstants.AF_INET, OsConstants.SOCK_DGRAM, OsConstants.IPPROTO_UDP)
OsAbi.trySetIntOpt(fd, OsConstants.IPPROTO_IP, OsAbi.IP_RECVERR, 1)?.let {
return@withContext b.build(
TestStatus.UNSUPPORTED,
params = params,
error = TestError("ip_recverr_rejected", it),
)
}
val target = InetAddress.getByName(targetHost)
val hops = ArrayList<Hop>(maxHops)
var hopsSeen = 0
var reachedTarget = false
var srcPort = 0
for (ttl in 1..maxHops) {
OsAbi.trySetIntOpt(fd, OsConstants.IPPROTO_IP, OsAbi.IP_TTL, ttl)
val t0 = System.nanoTime()
val sent = runCatching {
Os.sendto(fd, ByteArray(32), 0, 32, 0, target, BASE_PORT + ttl)
}
if (sent.isFailure) {
hops.add(Hop(ttl, listOf(HopProbe(icmp = "sendto failed: " +
(sent.exceptionOrNull()?.message ?: "?")))))
continue
}
if (srcPort == 0) {
// Only readable after the implicit bind the first send performs.
srcPort = runCatching {
(Os.getsockname(fd) as? InetSocketAddress)?.port ?: 0
}.getOrDefault(0)
}
var hop: ErrqueueApi.ErrEvent? = null
val deadline = System.nanoTime() + HOP_DEADLINE_NS
while (hop == null && System.nanoTime() < deadline) {
hop = api.pollErrqueue(fd)
if (hop == null) delay(POLL_INTERVAL_MS)
}
val rttNs = System.nanoTime() - t0
when {
hop == null -> hops.add(Hop(ttl, listOf(HopProbe()))) // silent hop: all null
hop.parseError != null ->
hops.add(Hop(ttl, listOf(HopProbe(icmp = "unparsed: ${hop.parseError}"))))
else -> {
hopsSeen++
hops.add(Hop(ttl, listOf(HopProbe(
replyFrom = hop.offender,
rttNs = rttNs,
icmp = when (hop.icmpType) {
OsAbi.ICMP_TIME_EXCEEDED -> "time_exceeded"
OsAbi.ICMP_DEST_UNREACH -> "dest_unreachable"
else -> "type_${hop.icmpType}"
},
))))
if (hop.icmpType == OsAbi.ICMP_DEST_UNREACH) reachedTarget = true
}
}
if (reachedTarget) break
}
// dst_port varies per TTL (classic traceroute, and what was validated on hardware),
// so this flow is explicitly NOT fixed-tuple; base_port is in params.
val evidence = TracerouteEvidence(
flow = Flow(srcPort = srcPort, dstPort = BASE_PORT, fixedTuple = false),
hops = hops,
).toEvidence()
val metrics = buildJsonObject {
put("hops_seen", hopsSeen)
put("reached_target", reachedTarget)
}
val status = when {
hopsSeen > 0 -> TestStatus.OK
// API present, sends succeeded, nothing surfaced: a fact about this path or
// kernel, not proof the mechanism is missing.
else -> TestStatus.PARTIAL
}
b.build(status, params = params, evidence = evidence, metrics = metrics)
} catch (e: Throwable) {
b.build(
TestStatus.FAILED,
params = params,
error = TestError("uncaught", e.message ?: e.javaClass.simpleName),
)
} finally {
fd?.let { runCatching { Os.close(it) } }
}
}
}
/**
* Reflection facade over android.system.{StructMsghdr, StructCmsghdr, Os.recvmsg}.
* Resolved once; null if any piece is missing on this API level.
*/
internal class ErrqueueApi private constructor(
private val msghdrCtor: java.lang.reflect.Constructor<*>,
private val recvmsg: java.lang.reflect.Method,
private val cmsgLevel: java.lang.reflect.Field,
private val cmsgType: java.lang.reflect.Field,
private val cmsgData: java.lang.reflect.Field,
private val msgControl: java.lang.reflect.Field,
) {
class ErrEvent(
val offender: String?,
val icmpType: Int,
val origin: Int,
val parseError: String? = null,
)
/** One non-blocking MSG_ERRQUEUE read; null when the queue is empty. */
fun pollErrqueue(fd: FileDescriptor): ErrEvent? {
return try {
val iov = arrayOf(ByteBuffer.allocate(512))
// (SocketAddress msg_name, ByteBuffer[] msg_iov, StructCmsghdr[] msg_control, flags)
val msghdr = msghdrCtor.newInstance(
InetSocketAddress(0), iov, null, 0,
)
recvmsg.invoke(null, fd, msghdr, OsAbi.MSG_ERRQUEUE or OsAbi.MSG_DONTWAIT)
val control = msgControl.get(msghdr) as? Array<*>
?: return ErrEvent(null, -1, -1, "msg_control empty after recvmsg")
for (cmsg in control.filterNotNull()) {
val level = cmsgLevel.getInt(cmsg)
val type = cmsgType.getInt(cmsg)
if (level == OsConstants.IPPROTO_IP && type == OsAbi.IP_RECVERR) {
return parseSockExtendedErr(cmsgData.get(cmsg))
}
}
ErrEvent(null, -1, -1, "no IP_RECVERR cmsg among ${control.size}")
} catch (e: Throwable) {
// The single most load-bearing line: reflection wraps errno in
// InvocationTargetException, and EAGAIN there means "queue empty", not failure.
val cause = (e as? java.lang.reflect.InvocationTargetException)?.cause ?: e
val msg = cause.message ?: cause.javaClass.simpleName
if ("EAGAIN" in msg || "EWOULDBLOCK" in msg) null
else ErrEvent(null, -1, -1, msg)
}
}
/** cmsg_data = struct sock_extended_err + offender sockaddr_in (see OsAbi). */
private fun parseSockExtendedErr(data: Any?): ErrEvent {
val bytes: ByteArray = when (data) {
is ByteArray -> data
is ByteBuffer -> ByteArray(data.remaining()).also { data.duplicate().get(it) }
else -> return ErrEvent(null, -1, -1, "cmsg_data is ${data?.javaClass?.name}")
}
if (bytes.size < OsAbi.SOCK_EE_SIZE) {
return ErrEvent(null, -1, -1, "cmsg_data too short: ${bytes.size}")
}
val origin = bytes[4].toInt() and 0xFF
val icmpType = bytes[5].toInt() and 0xFF
// SO_EE_OFFENDER: sockaddr_in directly after the fixed struct; family is in native
// byte order, sin_addr at offset +4 within the sockaddr.
val offender = if (bytes.size >= OsAbi.SOCK_EE_SIZE + 8) {
val family = ByteBuffer.wrap(bytes, OsAbi.SOCK_EE_SIZE, 2)
.order(ByteOrder.nativeOrder()).short.toInt()
if (family == OsConstants.AF_INET) {
val a = bytes.copyOfRange(OsAbi.SOCK_EE_SIZE + 4, OsAbi.SOCK_EE_SIZE + 8)
InetAddress.getByAddress(a).hostAddress
} else null
} else null
return ErrEvent(offender, icmpType, origin)
}
companion object {
fun resolve(): ErrqueueApi? = runCatching {
val msghdrCls = Class.forName("android.system.StructMsghdr")
val cmsghdrCls = Class.forName("android.system.StructCmsghdr")
ErrqueueApi(
// Picked by shape, not by position: the 4-arg form is
// (SocketAddress, ByteBuffer[], StructCmsghdr[], int) on every level that has it.
msghdrCtor = msghdrCls.constructors.first { it.parameterCount == 4 },
recvmsg = Os::class.java.getMethod(
"recvmsg", FileDescriptor::class.java, msghdrCls, Int::class.javaPrimitiveType,
),
cmsgLevel = cmsghdrCls.getField("cmsg_level"),
cmsgType = cmsghdrCls.getField("cmsg_type"),
cmsgData = cmsghdrCls.getField("cmsg_data"),
msgControl = msghdrCls.getField("msg_control"),
)
}.getOrNull()
}
}
@@ -0,0 +1,131 @@
// SPDX-FileCopyrightText: 2026 Echolot contributors
// SPDX-License-Identifier: GPL-3.0-or-later
package app.echo_lot.probe
import android.content.Context
import app.echo_lot.measurement.Test
import app.echo_lot.measurement.TestStatus
import app.echo_lot.measurement.TestType
import app.echo_lot.measurement.Tier
import app.echo_lot.measurement.Network as MNetwork
import kotlinx.coroutines.Dispatchers
import kotlinx.coroutines.withContext
import kotlinx.serialization.json.JsonObject
import kotlinx.serialization.json.buildJsonObject
import kotlinx.serialization.json.put
import java.net.Inet6Address
import java.net.InetSocketAddress
import java.util.Locale
/**
* v6.brokenness does IPv6 actually carry traffic, asked with a real TCP connection.
*
* This exists to corroborate (or refute) the ICMPv6 silence that icmp.ping6 observes. ICMPv6 echo
* is widely filtered on networks where IPv6 works fine, so silence alone cannot distinguish
* "IPv6 is broken" from "ping is filtered" a phone that reported v6.broken while happily
* loading IPv6-only sites is what proved the point. A TCP connect over IPv6 to the configured
* server settles it: if it succeeds, IPv6 works and the ICMP silence is filtering; if it fails
* too, on a network that advertises IPv6, the brokenness claim finally has evidence behind it.
*
* Only networks that claim to offer IPv6 (a global address or a v6 default route) are attempted:
* connecting over v6 on an IPv4-only network fails by design, and recording that as evidence
* would manufacture the exact false positive this probe exists to kill.
*/
class V6ConnectProbe(
private val entries: List<NetworkInventory.Entry>,
private val serverHost: String,
private val port: Int = 443,
) : Probe {
override val type = TestType.V6_BROKENNESS
override val tier = Tier.APP
// One 3s connect timeout per v6-provisioned network, at most.
override val estimatedMs = 4_000L
override suspend fun run(ctx: Context, ids: ProbeIds): Test = withContext(Dispatchers.IO) {
val b = TestBuilder(type, tier, ids)
// Same rule as the STUN and canary probes: with no server there is no target, and
// borrowing someone else's infrastructure to get one is not this app's call to make.
if (serverHost.isBlank()) {
return@withContext b.build(
TestStatus.SKIPPED,
evidence = buildJsonObject { put("reason", "no server configured to connect to") },
)
}
val candidates = entries.filter { ipv6Provisioned(it.model) }
if (candidates.isEmpty()) {
return@withContext b.build(
TestStatus.SKIPPED,
evidence = buildJsonObject {
put("reason", "no active network claims to offer IPv6")
},
)
}
var okCount = 0
var attemptedCount = 0
val evidence: JsonObject = buildJsonObject {
put("target", "$serverHost:$port")
for (e in candidates) {
val label = "${e.model.transport.name.lowercase()}:${e.model.id}"
val a = attempt(e)
if (a.attempted) attemptedCount++
if (a.ok) okCount++
put(label, buildJsonObject {
put("network_ref", e.model.id)
put("ok", a.ok)
put("attempted", a.attempted)
put("detail", a.detail)
})
}
}
val status = when {
attemptedCount == 0 -> TestStatus.SKIPPED // resolution/binding never got that far
okCount == attemptedCount -> TestStatus.OK
okCount > 0 -> TestStatus.PARTIAL
else -> TestStatus.FAILED
}
b.build(status, evidence = evidence)
}
/** Same attempted/ok separation as IcmpProbe: a connect we never sent proves nothing. */
private data class Attempt(val ok: Boolean, val attempted: Boolean, val detail: String)
private fun attempt(e: NetworkInventory.Entry): Attempt {
// Resolved through this network's own resolver; a v6 address obtained over another
// network would still be connected to over this one, which is what matters.
val addr = runCatching {
e.handle.getAllByName(serverHost).filterIsInstance<Inet6Address>().firstOrNull()
}.getOrNull()
?: return Attempt(false, false, "no AAAA answer for $serverHost via this network")
// createSocket() binds to the network at creation; failing here means the app could not
// use the interface at all (e.g. EPERM under a VPN) — nothing was sent, nothing is known.
val socket = try {
e.handle.socketFactory.createSocket()
} catch (t: Throwable) {
return Attempt(false, false, "socket unavailable: ${t.message ?: t.javaClass.simpleName}")
}
return try {
val t0 = System.nanoTime()
socket.connect(InetSocketAddress(addr, port), 3000)
val rttMs = (System.nanoTime() - t0) / 1_000_000.0
Attempt(true, true, "connected to [${addr.hostAddress}]:$port " +
"rtt_ms=${"%.1f".format(Locale.ROOT, rttMs)}")
} catch (t: Throwable) {
// A refused connection would still prove the path forwards IPv6, but against our own
// server's 443 the realistic failures are timeout and unreachable — both silence.
Attempt(false, true, "error: ${t.message ?: t.javaClass.simpleName}")
} finally {
runCatching { socket.close() }
}
}
/** The network claims IPv6: a global (non-link-local) address or a v6 default route. */
private fun ipv6Provisioned(n: MNetwork): Boolean =
n.link.addresses.any { a ->
a.addr.contains(':') &&
!a.addr.startsWith("fe80", ignoreCase = true) &&
!a.addr.startsWith("::1")
} || n.link.routes.any { it.dst == "::/0" }
}
@@ -35,13 +35,57 @@ class ControlClient(
private val controlUrl: String, private val controlUrl: String,
pins: Set<String>, pins: Set<String>,
private val appVersion: String = "", private val appVersion: String = "",
/**
* Addresses to fall back to when the server's name will not resolve, learned from its profile.
*
* A measurement tool that cannot report from a broken network is useless exactly when it
* matters, and a wedged resolver is one of the faults it is built to find it should not also
* be the thing that stops the finding being delivered.
*
* Safe because the pin is the trust and the name is not part of it: the server presents the
* same certificate whether it was reached by name or by address, and a wrong address fails the
* pin like anything else would.
*/
private val fallbackAddrs: List<String> = emptyList(),
) { ) {
private val json = Json { ignoreUnknownKeys = true } private val json = Json { ignoreUnknownKeys = true }
private val socketFactory = Pinning.sslContext(pins).socketFactory private val socketFactory = Pinning.sslContext(pins).socketFactory
/**
* The base URL to use, substituting a cached address only when the name genuinely fails.
*
* Resolved once per client and only on failure, so a working network pays nothing and never
* silently drifts onto an address that may be stale.
*/
private val base: String by lazy { resolveBase() }
private fun resolveBase(): String {
if (fallbackAddrs.isEmpty()) return controlUrl
val uri = runCatching { java.net.URI(controlUrl) }.getOrNull() ?: return controlUrl
val host = uri.host ?: return controlUrl
if (runCatching { java.net.InetAddress.getByName(host) }.isSuccess) return controlUrl
val port = if (uri.port > 0) uri.port else 443
for (ip in fallbackAddrs) {
// Checked rather than assumed: on a v4-only network a v6 address would otherwise be
// chosen and fail slowly, which is the wrong answer delivered late.
val reachable = runCatching {
java.net.Socket().use { sock ->
sock.connect(java.net.InetSocketAddress(ip, port), 4000)
true
}
}.getOrDefault(false)
if (reachable) {
val literal = if (ip.contains(':')) "[$ip]" else ip
return uri.scheme + "://" + literal + ":" + port
}
}
return controlUrl
}
private fun open(path: String, method: String, credential: String?): HttpsURLConnection { private fun open(path: String, method: String, credential: String?): HttpsURLConnection {
val conn = URL(controlUrl.trimEnd('/') + path).openConnection() as HttpsURLConnection val conn = URL(base.trimEnd('/') + path).openConnection() as HttpsURLConnection
conn.sslSocketFactory = socketFactory conn.sslSocketFactory = socketFactory
conn.setHostnameVerifier { _, _ -> true } // pin is the trust, not the name conn.setHostnameVerifier { _, _ -> true } // pin is the trust, not the name
conn.requestMethod = method conn.requestMethod = method
@@ -184,6 +228,33 @@ class ControlClient(
open("/v1/runs/$runId", "DELETE", credential).responseCode open("/v1/runs/$runId", "DELETE", credential).responseCode
} }
/**
* Ties this device to the person the ID token identifies.
*
* The device credential proves *which device*, the token proves *which person*; the server
* requires both. Returns the raw JSON reply (account id and display name).
*/
fun linkAccount(credential: String, idToken: String): String {
val conn = open("/v1/account/link", "POST", credential)
writeJson(conn, """{"id_token":${jstr(idToken)}}""")
val text = body(conn)
check(conn.responseCode in 200..299) { "sign-in failed: ${conn.responseCode} $text" }
return text
}
/** Signs out on this device. The device stays enrolled. */
fun unlinkAccount(credential: String) {
open("/v1/account/link", "DELETE", credential).responseCode
}
/** Whether anyone is signed in on this device, and who. */
fun accountStatus(credential: String): String {
val conn = open("/v1/account", "GET", credential)
val text = body(conn)
check(conn.responseCode == 200) { "account status failed: ${conn.responseCode} $text" }
return text
}
fun observations(credential: String, sessionId: String): String { fun observations(credential: String, sessionId: String): String {
val conn = open("/v1/sessions/$sessionId/observations", "GET", credential) val conn = open("/v1/sessions/$sessionId/observations", "GET", credential)
val text = body(conn) val text = body(conn)
@@ -45,11 +45,22 @@ data class EnrollmentLink(
* the reason the pin travels in the link at all. * the reason the pin travels in the link at all.
*/ */
fun redeem(deviceName: String? = null, appVersion: String = ""): Enrolled { fun redeem(deviceName: String? = null, appVersion: String = ""): Enrolled {
val client = ControlClient(controlUrl, setOf(pin), appVersion) // The link may name the server's public address rather than its control endpoint, so that
// a person is handed a name they recognise. Ask where to actually connect.
//
// Only the address comes from here. The pin still comes from the link, because a pin
// fetched over an ordinary TLS connection would be worth exactly what the certificate
// authorities are worth — and pinning exists to survive one the operator does not
// control, such as a root injected by corporate device management. An intercepted
// discovery can therefore send this device to the wrong host, where the pin will not
// match: an outage, not a compromise.
val endpoint = discover(controlUrl) ?: controlUrl
val client = ControlClient(endpoint, setOf(pin), appVersion)
val response = client.enroll(token, deviceName) val response = client.enroll(token, deviceName)
val profile = client.profile(response.credential) val profile = client.profile(response.credential)
return Enrolled( return Enrolled(
controlUrl = controlUrl, controlUrl = endpoint,
publicUrl = controlUrl,
pin = pin, pin = pin,
credential = response.credential, credential = response.credential,
deviceId = response.deviceId, deviceId = response.deviceId,
@@ -57,6 +68,29 @@ data class EnrollmentLink(
) )
} }
/**
* Asks a server where its control plane lives. Null when it does not say, or cannot be asked.
*
* Deliberately forgiving: a server that predates this, or one whose link already names the
* control endpoint directly, simply answers nothing and the link's own URL is used. Enrollment
* must not start failing because an optional lookup did.
*/
private fun discover(publicUrl: String): String? = runCatching {
val conn = (java.net.URL(publicUrl.trimEnd('/') + "/v1/discover").openConnection()
as java.net.HttpURLConnection).apply {
connectTimeout = 8_000
readTimeout = 8_000
setRequestProperty("Accept", "application/json")
}
if (conn.responseCode !in 200..299) return null
val body = conn.inputStream.bufferedReader().use { it.readText() }
kotlinx.serialization.json.Json { ignoreUnknownKeys = true }
.parseToJsonElement(body)
.let { (it as kotlinx.serialization.json.JsonObject)["control_url"] }
?.let { (it as kotlinx.serialization.json.JsonPrimitive).content }
?.takeIf { it.isNotBlank() }
}.getOrNull()
companion object { companion object {
const val SCHEME = "echolot" const val SCHEME = "echolot"
const val HOST = "enroll" const val HOST = "enroll"
@@ -111,7 +145,15 @@ data class EnrollmentLink(
/** A server this device is now enrolled with, ready to be stored in settings. */ /** A server this device is now enrolled with, ready to be stored in settings. */
data class Enrolled( data class Enrolled(
/** Where this device connects: the endpoint whose certificate the pin matches. */
val controlUrl: String, val controlUrl: String,
/**
* The address a person was given, kept for display.
*
* Shown instead of [controlUrl] because the endpoint is plumbing it exists to select a
* certificate while this is the name the operator handed out and would recognise.
*/
val publicUrl: String,
val pin: String, val pin: String,
val credential: String, val credential: String,
val deviceId: String, val deviceId: String,
@@ -29,6 +29,15 @@ data class Target(
val id: String, val id: String,
val ip4: String? = null, val ip4: String? = null,
val ip6: String? = null, val ip6: String? = null,
/**
* The second address, which RFC 5780 behaviour discovery redirects to.
*
* Worth surfacing rather than treating as an implementation detail: a report that says "the
* server did not answer" means something different depending on which of its addresses was
* asked, and an operator reading one needs to be able to tell.
*/
@SerialName("ip4_alt") val ip4Alt: String? = null,
@SerialName("ip6_alt") val ip6Alt: String? = null,
@SerialName("udp_port") val udpPort: Int = 0, @SerialName("udp_port") val udpPort: Int = 0,
@SerialName("tcp_port") val tcpPort: Int = 0, @SerialName("tcp_port") val tcpPort: Int = 0,
@SerialName("stun_port") val stunPort: Int = 0, @SerialName("stun_port") val stunPort: Int = 0,
@@ -74,6 +83,25 @@ data class CompatInfo(
@SerialName("app_max") val appMax: String = "", @SerialName("app_max") val appMax: String = "",
) )
/**
* How to sign in to this server's identity provider, advertised so the app can offer the button
* only when there is something behind it and drive the flow without anyone typing an issuer URL.
*/
@Serializable
data class AuthInfo(
val enabled: Boolean = false,
val issuer: String = "",
@SerialName("client_id") val clientId: String = "",
val flow: String = "",
@SerialName("redirect_uri") val redirectUri: String = "",
val scopes: String = "openid profile email",
@SerialName("authorization_endpoint") val authorizationEndpoint: String = "",
@SerialName("token_endpoint") val tokenEndpoint: String = "",
@SerialName("end_session_endpoint") val endSessionEndpoint: String = "",
/** Present when the server has an issuer configured but could not reach it. */
@SerialName("discovery_error") val discoveryError: String? = null,
)
@Serializable @Serializable
data class Profile( data class Profile(
@SerialName("profile_version") val profileVersion: Int = 0, @SerialName("profile_version") val profileVersion: Int = 0,
@@ -86,6 +114,7 @@ data class Profile(
val pins: List<String> = emptyList(), val pins: List<String> = emptyList(),
val uploads: UploadPolicy = UploadPolicy(), val uploads: UploadPolicy = UploadPolicy(),
val compat: CompatInfo = CompatInfo(), val compat: CompatInfo = CompatInfo(),
val auth: AuthInfo = AuthInfo(),
) { ) {
fun supports(capability: String) = capability in capabilities fun supports(capability: String) = capability in capabilities
} }
@@ -0,0 +1,145 @@
// SPDX-FileCopyrightText: 2026 Echolot contributors
// SPDX-License-Identifier: GPL-3.0-or-later
package app.echo_lot.protocol
import kotlinx.serialization.json.Json
import kotlinx.serialization.json.jsonObject
import kotlinx.serialization.json.jsonPrimitive
import java.io.IOException
import java.net.HttpURLConnection
import java.net.URL
import java.net.URLEncoder
import java.security.MessageDigest
import java.security.SecureRandom
import java.util.Base64
/**
* Sign-in for the app: authorization code with PKCE (RFC 7636).
*
* The app is a *public* client it ships to devices, so any secret compiled into it can be read
* out of the APK with `unzip` and `strings`. PKCE is what replaces the client secret, and it
* defends a specific attack that matters here more than most places: the redirect comes back
* through a custom URI scheme, and on Android *any* app may register `echolot://`. A malicious one
* could intercept the callback and take the authorization code. Because the code can only be
* exchanged by presenting the verifier which never left this process and cannot be derived from
* the challenge that did a stolen code is worth nothing.
*
* Nothing from the IdP is kept afterwards. The ID token is used once, to prove to the server who
* is signing in, and then discarded: the device credential is what authenticates every later
* request. So there are no access tokens to store, no refresh tokens to rotate, and no token
* lifetime for the app to manage.
*/
object OidcLogin {
/** A started sign-in. [verifier] and [state] must survive until the callback returns. */
data class Pending(val authorizationUrl: String, val verifier: String, val state: String)
/**
* Builds the authorization URL and the secrets that must be held until the callback.
*
* Everything comes from the server's profile rather than being compiled in, so pointing the
* app at a different server with a different IdP is configuration, not a rebuild.
*/
fun begin(auth: AuthInfo, random: SecureRandom = SecureRandom()): Pending {
require(auth.enabled && auth.authorizationEndpoint.isNotBlank()) {
"this server has no identity provider configured"
}
val verifier = randomUrlSafe(random)
val state = randomUrlSafe(random)
val challenge = b64(MessageDigest.getInstance("SHA-256").digest(verifier.toByteArray()))
val q = buildString {
append("response_type=code")
append("&client_id=").append(enc(auth.clientId))
append("&redirect_uri=").append(enc(auth.redirectUri))
append("&scope=").append(enc(auth.scopes))
append("&state=").append(enc(state))
append("&code_challenge=").append(enc(challenge))
append("&code_challenge_method=S256")
}
val sep = if (auth.authorizationEndpoint.contains('?')) "&" else "?"
return Pending(auth.authorizationEndpoint + sep + q, verifier, state)
}
/** What came back on the `echolot://auth` redirect. */
data class Callback(val code: String?, val state: String?, val error: String?)
/** Parses the redirect URI the browser handed back to the app. */
fun parseCallback(uri: String): Callback {
val q = uri.substringAfter('?', "")
var code: String? = null
var state: String? = null
var error: String? = null
for (pair in q.split('&')) {
val k = pair.substringBefore('=')
val v = dec(pair.substringAfter('=', ""))
when (k) {
"code" -> code = v
"state" -> state = v
"error" -> error = v
"error_description" -> if (error != null) error = "$error: $v"
}
}
return Callback(code, state, error)
}
/** The sign-in failed in a way worth showing someone, rather than a transport error. */
class LoginFailed(message: String) : Exception(message)
/**
* Exchanges the code for an ID token.
*
* The state is compared before anything else happens. A callback whose state does not match
* the one this process generated did not come from a flow this process started which is
* precisely how an attacker gets a victim to complete *their* login so it is refused before
* the code is spent.
*/
fun complete(auth: AuthInfo, pending: Pending, callbackUri: String): String {
val cb = parseCallback(callbackUri)
if (cb.error != null) throw LoginFailed(cb.error)
if (cb.state.isNullOrEmpty() || cb.state != pending.state) {
throw LoginFailed("this sign-in did not start on this device — start again")
}
val code = cb.code ?: throw LoginFailed("the identity provider returned no authorization code")
val body = buildString {
append("grant_type=authorization_code")
append("&code=").append(enc(code))
append("&redirect_uri=").append(enc(auth.redirectUri))
append("&client_id=").append(enc(auth.clientId))
append("&code_verifier=").append(enc(pending.verifier))
}
val conn = (URL(auth.tokenEndpoint).openConnection() as HttpURLConnection).apply {
requestMethod = "POST"
doOutput = true
connectTimeout = 15_000
readTimeout = 15_000
setRequestProperty("Content-Type", "application/x-www-form-urlencoded")
setRequestProperty("Accept", "application/json")
}
conn.outputStream.use { it.write(body.toByteArray()) }
val text = try {
val stream = if (conn.responseCode in 200..299) conn.inputStream else conn.errorStream
stream?.bufferedReader()?.use { it.readText() } ?: ""
} catch (e: IOException) {
throw LoginFailed("could not reach the identity provider: ${e.message}")
}
if (conn.responseCode !in 200..299) {
throw LoginFailed("the identity provider refused the sign-in (${conn.responseCode})")
}
val idToken = runCatching {
Json.parseToJsonElement(text).jsonObject["id_token"]?.jsonPrimitive?.content
}.getOrNull()
return idToken?.takeIf { it.isNotBlank() }
?: throw LoginFailed("the identity provider returned no id_token")
}
private fun randomUrlSafe(random: SecureRandom): String =
ByteArray(32).also(random::nextBytes).let(::b64)
private fun b64(b: ByteArray): String = Base64.getUrlEncoder().withoutPadding().encodeToString(b)
private fun enc(s: String): String = URLEncoder.encode(s, "UTF-8")
private fun dec(s: String): String =
runCatching { java.net.URLDecoder.decode(s, "UTF-8") }.getOrDefault(s)
}
@@ -109,6 +109,49 @@ class ProbeSession(
return out return out
} }
/**
* Sends paced upstream traffic for [durationMs] and reports what was put on the wire.
*
* Paced rather than flat out, for the same reason the server paces: an unpaced burst measures
* the local NIC and the first queue it meets, then collapses into loss that reads as a network
* fault. The schedule is absolute rather than sleep-per-packet, which accumulates the
* scheduler's error and drifts the achieved rate below target over a multi-second run.
*
* Nothing comes back the server counts and stays silent so the result here is only the
* send side. The measurement is the gap between this and the server's tally.
*/
fun sendThroughput(durationMs: Long, kbps: Int, sizeBytes: Int = 1200): Sent {
val size = sizeBytes.coerceIn(Wire.HEADER_SIZE + 16, 1472)
val payload = ByteArray(size - Wire.HEADER_SIZE)
val perPacketNs = (size.toLong() * 8 * 1_000_000 / kbps.coerceAtLeast(1)).coerceAtLeast(1_000)
val start = System.nanoTime()
val deadline = start + durationMs * 1_000_000
var next = start
var packets = 0
var bytes = 0L
while (System.nanoTime() < deadline) {
val pkt = Wire.build(Wire.TYPE_THROUGHPUT_UP, prefix, ++seq, nowNs(), key, payload)
try {
socket.send(DatagramPacket(pkt, pkt.size, server))
} catch (e: java.io.IOException) {
// A local send failure is our condition, not the path's. Stop and report what
// actually left, rather than counting the remainder as loss on the network.
break
}
packets++
bytes += pkt.size
next += perPacketNs
val sleepNs = next - System.nanoTime()
if (sleepNs > 0) Thread.sleep(sleepNs / 1_000_000, (sleepNs % 1_000_000).toInt())
}
val elapsedMs = (System.nanoTime() - start) / 1_000_000
return Sent(packets, bytes, elapsedMs, if (elapsedMs > 0) (bytes * 8 / elapsedMs).toInt() else 0)
}
/** What one upstream run put on the wire locally. */
data class Sent(val packets: Int, val bytes: Long, val durationMs: Long, val kbps: Int)
/** One packet received from the server, with the wire size actually delivered. */ /** One packet received from the server, with the wire size actually delivered. */
data class Received(val type: Int, val seq: Int, val sizeBytes: Int, val tRxNs: Long) data class Received(val type: Int, val seq: Int, val sizeBytes: Int, val tRxNs: Long)
@@ -38,6 +38,16 @@ object Wire {
*/ */
const val TYPE_FRAG_DATA: Int = 0x0D const val TYPE_FRAG_DATA: Int = 0x0D
/** One packet of a sustained-rate downstream run. */
const val TYPE_THROUGHPUT_DATA: Int = 0x0E
/**
* One packet of a client-driven upstream run. The server counts it and does not answer:
* a reply would double the traffic and drag the return path into a measurement that is
* specifically about the outbound one.
*/
const val TYPE_THROUGHPUT_UP: Int = 0x0F
/** The 8-byte on-the-wire prefix = first 16 hex chars of the session id, decoded. */ /** The 8-byte on-the-wire prefix = first 16 hex chars of the session id, decoded. */
fun wirePrefix(sessionId: String): ByteArray { fun wirePrefix(sessionId: String): ByteArray {
require(sessionId.length >= 16) { "session id too short" } require(sessionId.length >= 16) { "session id too short" }
@@ -0,0 +1,100 @@
// SPDX-FileCopyrightText: 2026 Echolot contributors
// SPDX-License-Identifier: GPL-3.0-or-later
package app.echo_lot.protocol
import java.security.SecureRandom
import kotlin.test.Test
import kotlin.test.assertEquals
import kotlin.test.assertFailsWith
import kotlin.test.assertNotEquals
import kotlin.test.assertTrue
class OidcLoginTest {
private val auth = AuthInfo(
enabled = true,
issuer = "https://id.example.net/application/o/echolot-app/",
clientId = "the-client",
redirectUri = "echolot://auth",
scopes = "openid profile email",
authorizationEndpoint = "https://id.example.net/application/o/authorize/",
tokenEndpoint = "https://id.example.net/application/o/token/",
)
@Test
fun theAuthorizationUrlCarriesEverythingTheIdPNeeds() {
val p = OidcLogin.begin(auth)
val url = p.authorizationUrl
assertTrue(url.startsWith(auth.authorizationEndpoint + "?"), url)
for (part in listOf(
"response_type=code",
"client_id=the-client",
"redirect_uri=echolot%3A%2F%2Fauth",
"code_challenge_method=S256",
"scope=openid+profile+email",
)) {
assertTrue(url.contains(part), "missing $part in $url")
}
assertTrue(url.contains("code_challenge="), url)
// The verifier itself must never appear in the URL — that is the entire point of PKCE.
assertTrue(!url.contains(p.verifier), "the code verifier leaked into the authorize URL")
}
// Two sign-ins must not share a verifier or state, or one intercepted flow compromises the next.
@Test
fun everySignInGetsFreshSecrets() {
val a = OidcLogin.begin(auth, SecureRandom())
val b = OidcLogin.begin(auth, SecureRandom())
assertNotEquals(a.verifier, b.verifier)
assertNotEquals(a.state, b.state)
assertTrue(a.verifier.length >= 43, "verifier is shorter than RFC 7636 allows")
}
@Test
fun parsesTheRedirectTheBrowserHandsBack() {
val cb = OidcLogin.parseCallback("echolot://auth?code=abc123&state=xyz")
assertEquals("abc123", cb.code)
assertEquals("xyz", cb.state)
}
@Test
fun parsesAnErrorRedirect() {
val cb = OidcLogin.parseCallback("echolot://auth?error=access_denied&error_description=User%20said%20no")
assertEquals("access_denied", cb.error?.substringBefore(":"))
assertTrue(cb.code == null)
}
// A callback whose state does not match is how an attacker gets someone to complete *their*
// sign-in. It must be refused before the code is spent, without any network call.
@Test
fun aMismatchedStateIsRefusedBeforeTheCodeIsSpent() {
val p = OidcLogin.begin(auth)
val e = assertFailsWith<OidcLogin.LoginFailed> {
OidcLogin.complete(auth, p, "echolot://auth?code=stolen&state=not-ours")
}
assertTrue(e.message!!.contains("did not start on this device"), e.message!!)
}
@Test
fun aMissingStateIsRefused() {
val p = OidcLogin.begin(auth)
assertFailsWith<OidcLogin.LoginFailed> {
OidcLogin.complete(auth, p, "echolot://auth?code=abc")
}
}
@Test
fun anErrorRedirectSurfacesTheReason() {
val p = OidcLogin.begin(auth)
val e = assertFailsWith<OidcLogin.LoginFailed> {
OidcLogin.complete(auth, p, "echolot://auth?error=access_denied&state=${p.state}")
}
assertTrue(e.message!!.contains("access_denied"))
}
@Test
fun refusesToStartWhenTheServerHasNoIdentityProvider() {
assertFailsWith<IllegalArgumentException> { OidcLogin.begin(AuthInfo(enabled = false)) }
}
}
@@ -98,20 +98,32 @@ object ShizukuAvailability {
.addFlags(android.content.Intent.FLAG_ACTIVITY_NEW_TASK) .addFlags(android.content.Intent.FLAG_ACTIVITY_NEW_TASK)
/** /**
* Reports the state now and on every binder transition. Returns a function that removes the * Reports the state now, on every binder transition, and when a permission request is
* listeners again (call it from onCleared). * answered. Returns a function that removes the listeners again (call it from onCleared).
*
* The permission listener matters as much as the binder ones: granting permission does not
* make the binder arrive or die, so without it the banner still read "running but not
* authorised" after the user had just authorised it — the one moment they are looking for
* confirmation that it worked.
*
* It is still not sufficient on its own. Permission can be granted inside Shizuku's own app,
* where nothing calls back into this process at all, so callers should re-check on resume as
* well; see [current].
*/ */
fun observe(context: Context, onChange: (State) -> Unit): () -> Unit { fun observe(context: Context, onChange: (State) -> Unit): () -> Unit {
val app = context.applicationContext val app = context.applicationContext
val received = Shizuku.OnBinderReceivedListener { onChange(current(app)) } val received = Shizuku.OnBinderReceivedListener { onChange(current(app)) }
val dead = Shizuku.OnBinderDeadListener { onChange(current(app)) } val dead = Shizuku.OnBinderDeadListener { onChange(current(app)) }
val permission = Shizuku.OnRequestPermissionResultListener { _, _ -> onChange(current(app)) }
// "Sticky" fires immediately if the binder already arrived before we registered. // "Sticky" fires immediately if the binder already arrived before we registered.
runCatching { Shizuku.addBinderReceivedListenerSticky(received) } runCatching { Shizuku.addBinderReceivedListenerSticky(received) }
runCatching { Shizuku.addBinderDeadListener(dead) } runCatching { Shizuku.addBinderDeadListener(dead) }
runCatching { Shizuku.addRequestPermissionResultListener(permission) }
onChange(current(app)) onChange(current(app))
return { return {
runCatching { Shizuku.removeBinderReceivedListener(received) } runCatching { Shizuku.removeBinderReceivedListener(received) }
runCatching { Shizuku.removeBinderDeadListener(dead) } runCatching { Shizuku.removeBinderDeadListener(dead) }
runCatching { Shizuku.removeRequestPermissionResultListener(permission) }
} }
} }
} }
+17 -7
View File
@@ -5,8 +5,13 @@
# Mints an enrollment link on the probe server and prints it — as text, as a QR code if # Mints an enrollment link on the probe server and prints it — as text, as a QR code if
# `qrencode` is around, and as an adb command if a device is attached. # `qrencode` is around, and as an adb command if a device is attached.
# #
# The admin listener is localhost-only by design, so this goes over SSH. The link carries a # The link is minted by the server binary on the host rather than over HTTP. The admin API this
# single-use bearer token: treat it like a password until it is redeemed. # used to call is gone: the admin UI that replaced it is authenticated, as it should be, and
# adding a second unauthenticated door on loopback is what briefly exposed the old one to the
# network. A root shell on the host needs no authentication anyway — whoever has one already has
# every privilege the server has.
#
# The link carries a single-use bearer token: treat it like a password until it is redeemed.
# #
# Usage: echolot-app/scripts/enroll-link.sh [note] # Usage: echolot-app/scripts/enroll-link.sh [note]
set -euo pipefail set -euo pipefail
@@ -14,13 +19,18 @@ set -euo pipefail
SSH_HOST="${ECHOLOT_SSH:-claude-echolot}" SSH_HOST="${ECHOLOT_SSH:-claude-echolot}"
NOTE="${1:-manual}" NOTE="${1:-manual}"
MINTED=$(ssh -o BatchMode=yes "$SSH_HOST" \ # The env file is sourced rather than assumed: the state directory and the public URL live there,
"curl -s -X POST 'http://127.0.0.1:8444/admin/enroll-tokens?note=$NOTE'") # and minting against the wrong state directory would produce a token the running server has
# never heard of.
REMOTE='set -a; . /etc/echolot/server.env; set +a;
exec /usr/local/bin/echolot-server --mint-enroll-token'
RAW=$(ssh -o BatchMode=yes "$SSH_HOST" "sudo sh -c \"$REMOTE '$NOTE'\"" 2>/dev/null || true)
URI=$(printf '%s' "$RAW" | tr -d '\r' | grep -m1 '^echolot://enroll' || true)
URI=$(printf '%s' "$MINTED" | python -c 'import json,sys;print(json.load(sys.stdin).get("enroll_uri",""))')
if [ -z "$URI" ]; then if [ -z "$URI" ]; then
echo "server returned no enroll_uri (needs server-v0.5.4+):" >&2 echo "could not mint a link — needs a server with --mint-enroll-token (v0.9.7+)." >&2
echo "$MINTED" >&2 echo "raw response:" >&2
printf '%s\n' "$RAW" >&2
exit 1 exit 1
fi fi
+3
View File
@@ -22,4 +22,7 @@ VOLUME ["/state"]
# the data plane must see real client source addresses/TTLs, and Docker's # the data plane must see real client source addresses/TTLs, and Docker's
# userland NAT would falsify exactly what this server exists to observe. # userland NAT would falsify exactly what this server exists to observe.
EXPOSE 8441/tcp 8442/udp 8443/tcp EXPOSE 8441/tcp 8442/udp 8443/tcp
# The verb is explicit here too, so `docker run <image>` serves and `docker run <image> --help`
# still works by overriding the command.
ENTRYPOINT ["/echolot-server"] ENTRYPOINT ["/echolot-server"]
CMD ["--serve"]
+54 -8
View File
@@ -106,9 +106,7 @@ ECHOLOT_UDP_LISTEN=203.0.113.10:8442,203.0.113.11:8442,[2001:db8::10]:8442,[2001
Passing `--self-update-api` to `--install-systemd` additionally installs a daily randomized Passing `--self-update-api` to `--install-systemd` additionally installs a daily randomized
self-update timer (`echolot-server-update.timer`) that restarts the service after a successful self-update timer (`echolot-server-update.timer`) that restarts the service after a successful
update. Updates are checksum-verified against the release's `SHA256SUMS` (integrity, not update.
authenticity — signature verification remains TODO before treating the update source as
untrusted).
### Self-update (opt-in, native only) ### Self-update (opt-in, native only)
@@ -119,14 +117,24 @@ echolot-server --self-update \
Fetches the newest `server-v*` release asset for this OS/arch and atomically replaces the Fetches the newest `server-v*` release asset for this OS/arch and atomically replaces the
binary; systemd's `Restart=` brings up the new version. Run it from a systemd timer for binary; systemd's `Restart=` brings up the new version. Run it from a systemd timer for
unattended updates. TODO before enabling anywhere untrusted: signature verification of the unattended updates.
downloaded asset.
Releases are trusted by signature, not by host: CI signs `SHA256SUMS` with an ed25519 key that
exists only in its secret store (`RELEASE_SIGNING_KEY`), and the updater verifies
`SHA256SUMS.sig` against the public key baked into the binary before believing any checksum —
an unsigned or re-signed release is refused, so a compromised Gitea can withhold updates but not
inject one. Running your own release pipeline? Mint a keypair with
`go run ./cmd/release-sign -gen`, set the secret, and point `ECHOLOT_SELF_UPDATE_PUBKEY` (or
`--self-update-pubkey`) at your public key.
## First contact ## First contact
```sh ```sh
# 1. mint an enrollment token (admin listener is loopback-only) # 1. mint an enrollment token (admin listener is loopback-only; authenticates as the
curl -s -X POST 'http://127.0.0.1:8444/admin/enroll-tokens?note=phone' # break-glass admin — set that once with --set-admin-password)
curl -s -u admin:<password> -H 'Accept: application/json' \
-X POST 'http://127.0.0.1:8444/admin/enroll-tokens?note=phone'
# → { "token": "…", "expires_in_s": 86400, "enroll_uri": "echolot://enroll?…" }
# 2. device enrolls with it (normally via the echolot:// QR code) # 2. device enrolls with it (normally via the echolot:// QR code)
curl -sk -X POST https://<host>:8443/v1/enroll -H 'Authorization: Bearer <token>' curl -sk -X POST https://<host>:8443/v1/enroll -H 'Authorization: Bearer <token>'
# 3. device fetches its profile # 3. device fetches its profile
@@ -144,5 +152,43 @@ go vet ./...
CI (`.gitea/workflows/build-server.yml`): tests on every push touching `server/`; CI (`.gitea/workflows/build-server.yml`): tests on every push touching `server/`;
tagging `server-v1.2.3` builds + pushes the container image to the Gitea registry and tagging `server-v1.2.3` builds + pushes the container image to the Gitea registry and
attaches static linux amd64/arm64 binaries (+ SHA256SUMS) to a release — the same attaches static linux amd64/arm64 binaries (+ signed SHA256SUMS) to a release — the same
artifacts `--self-update` consumes. artifacts `--self-update` consumes.
## TLS for the admin UI
The binary terminates TLS itself; there is no reverse proxy in the design. It already serves TLS
for the control plane, so this is reuse rather than new machinery, and it keeps the "one process,
one config file" property. A proxy would also invite someone to eventually front the control plane
too — which would break SPKI pinning, because clients pin *that* certificate's key.
```
ECHOLOT_ADMIN_LISTEN=[2001:db8::2]:443
ECHOLOT_ADMIN_TLS_CERT=/etc/echolot/admin.pem
ECHOLOT_ADMIN_TLS_KEY=/etc/echolot/admin.key
ECHOLOT_ADMIN_BASE_URL=https://admin.example.net
```
Certificates come from any ACME client. **DNS-01 is the one to use here**: it needs no inbound
port 80, which matters on a host where 80 is awkward or already spoken for.
```sh
acme.sh --issue --dns dns_cf -d admin.example.net \
--key-file /etc/echolot/admin.key \
--fullchain-file /etc/echolot/admin.pem
```
**No reload hook is needed.** The certificate is re-read when the files change, so a renewal that
drops new files in place is picked up on the next handshake. That is deliberate: a reload hook is
the part of a renewal setup that quietly stops working, months later, and is noticed only once the
certificate has already expired. A torn write — renewal tools write cert and key separately — keeps
the previous certificate rather than failing the listener.
Serving the admin UI in plaintext on a non-loopback address is refused: the session cookie is a
bearer credential for everything the server can do, and the OIDC authorization code arrives in a
URL. Bind to loopback and use an SSH tunnel (`ssh -L 8444:localhost:8444 host`), supply a
certificate, or set `ECHOLOT_ADMIN_INSECURE=1` if you mean it.
The control-plane certificate is deliberately *not* hot-reloaded. Clients pin its public key, so
replacing it is a rotation an operator should have to think about, not something that happens
because a file changed.
+373 -35
View File
@@ -11,6 +11,7 @@
package main package main
import ( import (
"bufio"
"context" "context"
"crypto/ecdsa" "crypto/ecdsa"
"crypto/elliptic" "crypto/elliptic"
@@ -18,7 +19,6 @@ import (
"crypto/tls" "crypto/tls"
"crypto/x509" "crypto/x509"
"crypto/x509/pkix" "crypto/x509/pkix"
"encoding/json"
"encoding/pem" "encoding/pem"
"errors" "errors"
"fmt" "fmt"
@@ -36,11 +36,17 @@ import (
"syscall" "syscall"
"time" "time"
"echo-lot.app/server/internal/acmehttp"
"echo-lot.app/server/internal/adminauth"
"echo-lot.app/server/internal/adminui"
"echo-lot.app/server/internal/canarydns" "echo-lot.app/server/internal/canarydns"
"echo-lot.app/server/internal/certreload"
"echo-lot.app/server/internal/compat" "echo-lot.app/server/internal/compat"
"echo-lot.app/server/internal/config" "echo-lot.app/server/internal/config"
"echo-lot.app/server/internal/control" "echo-lot.app/server/internal/control"
"echo-lot.app/server/internal/dataplane" "echo-lot.app/server/internal/dataplane"
"echo-lot.app/server/internal/oidc"
"echo-lot.app/server/internal/ratelimit"
"echo-lot.app/server/internal/runs" "echo-lot.app/server/internal/runs"
"echo-lot.app/server/internal/selftest" "echo-lot.app/server/internal/selftest"
"echo-lot.app/server/internal/selfupdate" "echo-lot.app/server/internal/selfupdate"
@@ -69,6 +75,30 @@ func run() error {
control.Version = Version control.Version = Version
switch { switch {
case actions.Help:
// Compatibility shim for one release.
//
// Serving became an explicit verb, but self-update is run by the *old* binary — so the
// repair added to the updater cannot fix the very update that installs the new one. A
// unit written before this change starts us with no arguments, and without this branch
// the service would simply stop working, unattended, on a host nobody is watching.
//
// Only when systemd started us: INVOCATION_ID is set by systemd for every service
// invocation and by nothing else, so a person at a terminal still gets usage. Remove
// this once no deployment predates --serve.
if os.Getenv("INVOCATION_ID") != "" {
slog.Warn("started by systemd with no verb — this unit predates --serve; " +
"repairing it and serving anyway")
if repaired, err := system.RepairExecStart(); err != nil {
slog.Error("could not repair the unit; fix ExecStart by hand", "err", err)
} else if repaired {
slog.Info("systemd unit updated to pass --serve")
}
return serve(cfg)
}
config.Usage(os.Stderr)
os.Exit(2)
return nil
case actions.Version: case actions.Version:
fmt.Println(Version) fmt.Println(Version)
return nil return nil
@@ -79,17 +109,87 @@ func run() error {
return system.InstallSystemd(cfg.SelfUpdateAPI) return system.InstallSystemd(cfg.SelfUpdateAPI)
case actions.UninstallSystemd: case actions.UninstallSystemd:
return system.UninstallSystemd() return system.UninstallSystemd()
case actions.SetAdminPassword:
return setAdminPassword(cfg)
case actions.MintEnrollToken != "":
return mintEnrollToken(cfg, actions.MintEnrollToken)
case actions.SelfUpdate: case actions.SelfUpdate:
return selfupdate.Run(cfg.SelfUpdateAPI, Version) return selfupdate.Run(cfg.SelfUpdateAPI, cfg.SelfUpdatePubKey, Version)
} }
return serve(cfg) return serve(cfg)
} }
// mintEnrollToken prints a §2.1 bootstrap link for a new device.
//
// The link is assembled here rather than by hand because it has to carry the public URL and the
// base64 SPKI pin percent-encoded correctly, and a pin wrong by one character fails later as an
// inscrutable TLS error rather than as a bad pin.
func mintEnrollToken(cfg *config.Config, note string) error {
st, err := store.Open(cfg.StateDir)
if err != nil {
return fmt.Errorf("state store: %w", err)
}
cert, err := loadOrCreateCert(cfg)
if err != nil {
return fmt.Errorf("tls: %w", err)
}
pin, err := control.SpkiPinB64(cert)
if err != nil {
return fmt.Errorf("pin: %w", err)
}
tok, err := st.NewEnrollToken(24*time.Hour, note)
if err != nil {
return err
}
base := cfg.PublicControlURL
if base == "" {
return fmt.Errorf("set ECHOLOT_PUBLIC_URL so the link can say where to connect")
}
fmt.Println(control.EnrollmentURI(base, pin, tok))
// stderr, so piping the command somewhere yields the link alone.
fmt.Fprintln(os.Stderr, "\nSingle use, valid 24 hours. Treat it like a password until spent.")
return nil
}
// controlURL is the address devices connect to: the hostname that selects the pinned certificate.
//
// Falls back to the public URL when no separate control hostname is configured, so a server that
// does not share the admin port keeps answering discovery with something usable.
func controlURL(cfg *config.Config) string {
if cfg.ControlHostname == "" {
return cfg.PublicControlURL
}
return "https://" + cfg.ControlHostname
}
func serve(cfg *config.Config) error { func serve(cfg *config.Config) error {
slog.Info("echolot-server starting", "version", Version, "mode", slog.Info("echolot-server starting", "version", Version, "mode",
map[bool]string{true: "container", false: "native"}[cfg.Docker], map[bool]string{true: "container", false: "native"}[cfg.Docker],
"state_dir", cfg.StateDir) "state_dir", cfg.StateDir)
// The reserved addresses' proof is only as good as 80/443 actually being free there.
// CheckReserved already keeps OUR listeners away, but a process outside this config pollutes
// them just as silently — the adb-beacon receiver on 0.0.0.0:443 did exactly that. So ask
// the OS, not the config. A hard stop for the same reason CheckReserved is one: the failure
// is invisible, and its first symptom is a measurement calling an intercepted network clean.
if reserved := cfg.ReservedIPs(); len(reserved) > 0 {
occupied, unverifiable := selftest.ReservedWebPortsFree(reserved)
if len(occupied) > 0 {
return fmt.Errorf(
"refusing to start: something outside this server is listening on reserved "+
"measurement address(es) %s\n"+
"The interception proof those addresses exist for is void while anything "+
"answers there.\nFind it with `ss -tlnp | grep -E ':(80|443) '`, stop it, "+
"or remove the address from ECHOLOT_RESERVED_ADDRS if it is no longer reserved",
strings.Join(occupied, ", "))
}
for _, u := range unverifiable {
// Not fatal: an address with a typo, or one this host no longer carries, is a
// config problem — refusing to serve over it would take the whole instrument down.
slog.Warn("could not verify a reserved web port is free", "addr", u)
}
}
st, err := store.Open(cfg.StateDir) st, err := store.Open(cfg.StateDir)
if err != nil { if err != nil {
return fmt.Errorf("state store: %w", err) return fmt.Errorf("state store: %w", err)
@@ -106,12 +206,22 @@ func serve(cfg *config.Config) error {
sessions := session.NewManager(15 * time.Minute) sessions := session.NewManager(15 * time.Minute)
dp := &dataplane.Server{Sessions: sessions} dp := &dataplane.Server{Sessions: sessions}
// Spec §2.5 ceilings. Control plane answers 429; the data plane drops silently. 0 = off.
if cfg.RateUDPPps > 0 {
pps := float64(cfg.RateUDPPps)
// Burst of two seconds' worth: a 5000-packet train arrives as one burst by design.
dp.PacketRate = ratelimit.New(pps, 2*pps)
}
if cfg.RateUDPKbps > 0 {
bytesPerSec := float64(cfg.RateUDPKbps) * 125 // kbps -> bytes/s
dp.ByteRate = ratelimit.New(bytesPerSec, bytesPerSec)
}
// TCP echo shares the control cert for its elt-echo TLS variant. // TCP echo shares the control cert for its elt-echo TLS variant.
tcpSrv := &tcpecho.Server{ tcpSrv := &tcpecho.Server{
TLSConfig: &tls.Config{Certificates: []tls.Certificate{cert}, MinVersion: tls.VersionTLS12}, TLSConfig: &tls.Config{Certificates: []tls.Certificate{cert}, MinVersion: tls.VersionTLS12},
} }
caps := []string{"udp-probe", "delayed-echo", "connect-back", "http-echo", "downtrain", "big-send"} caps := []string{"udp-probe", "delayed-echo", "connect-back", "http-echo", "downtrain", "big-send", "throughput"}
// Crafted fragments need a raw socket. Advertised only when one can actually be opened — // Crafted fragments need a raw socket. Advertised only when one can actually be opened —
// a capability we cannot deliver turns a missing feature into a failed measurement. // a capability we cannot deliver turns a missing feature into a failed measurement.
rawFrag := dataplane.RawFragSupported() rawFrag := dataplane.RawFragSupported()
@@ -150,7 +260,43 @@ func serve(cfg *config.Config) error {
slog.Info("client compatibility", "accepts_app", appRange.String(), slog.Info("client compatibility", "accepts_app", appRange.String(),
"protocol", control.ProtocolVersion, "schema", control.SchemaVersion) "protocol", control.ProtocolVersion, "schema", control.SchemaVersion)
// Identity is optional. Without an issuer the server simply has no sign-in, and
// uploads=account can never be satisfied — which is the honest outcome, not a silent
// downgrade to anonymous.
// One verifier per issuer. An IdP may mint a distinct issuer per application — Authentik
// derives it from the application slug — and a token's `iss` must match whoever signed it.
// Each verifier accepts only the client belonging to its own issuer, so a token minted for
// the phone cannot be replayed at the admin login and vice versa.
var idp, adminIdP *oidc.Verifier
appIssuer := cfg.OIDCAppIssuer
if appIssuer == "" {
appIssuer = cfg.OIDCIssuer // IdPs with one global issuer
}
if appIssuer != "" && cfg.OIDCAppClientID != "" {
idp = oidc.New(oidc.Config{
Issuer: appIssuer, AppClientID: cfg.OIDCAppClientID, AdminGroup: cfg.OIDCAdminGroup,
}, nil)
slog.Info("identity: app client", "issuer", appIssuer, "client_id", cfg.OIDCAppClientID)
}
if cfg.OIDCIssuer != "" && cfg.OIDCClientID != "" {
adminIdP = oidc.New(oidc.Config{
Issuer: cfg.OIDCIssuer, ClientID: cfg.OIDCClientID, AdminGroup: cfg.OIDCAdminGroup,
}, nil)
slog.Info("identity: admin client", "issuer", cfg.OIDCIssuer,
"client_id", cfg.OIDCClientID, "admin_group", cfg.OIDCAdminGroup)
if cfg.OIDCAdminGroup == "" {
slog.Warn("no admin group set: nobody will be an admin via OIDC " +
"(set ECHOLOT_OIDC_ADMIN_GROUP)")
}
}
if idp == nil && adminIdP == nil && cfg.UploadsMode == string(runs.ModeAccount) {
slog.Warn("uploads=account but no identity provider is configured — " +
"every upload will be refused")
}
ip4, ip6, ip4Alt, ip6Alt := cfg.MeasurementAddrs()
ctl := &control.Server{ ctl := &control.Server{
IP4: ip4, IP6: ip6, IP4Alt: ip4Alt, IP6Alt: ip6Alt,
Store: st, Sessions: sessions, Name: cfg.Name, Store: st, Sessions: sessions, Name: cfg.Name,
UDPPort: mustPort(firstAddr(cfg.UDPListen)), TCPPort: mustPort(firstAddr(cfg.TCPListen)), UDPPort: mustPort(firstAddr(cfg.UDPListen)), TCPPort: mustPort(firstAddr(cfg.TCPListen)),
StunPort: mustPort(firstAddr(cfg.StunListen)), PinB64: pin, CertChain: cert.Certificate, StunPort: mustPort(firstAddr(cfg.StunListen)), PinB64: pin, CertChain: cert.Certificate,
@@ -161,12 +307,21 @@ func serve(cfg *config.Config) error {
Runs: runStore, Runs: runStore,
AppRange: appRange, AppRange: appRange,
PublicControlURL: publicControlURL(cfg), PublicControlURL: publicControlURL(cfg),
OIDC: idp,
AdminOIDC: adminIdP,
} }
// Left nil when there is no raw socket, so the handler answers "not implemented" with a // Left nil when there is no raw socket, so the handler answers "not implemented" with a
// reason rather than failing somewhere deeper. // reason rather than failing somewhere deeper.
if rawFrag { if rawFrag {
ctl.FragSend = dp.FragSend ctl.FragSend = dp.FragSend
} }
ctl.DownThroughput = dp.DownThroughput
if cfg.RateSessionsPerMin > 0 {
ctl.RateSessions = ratelimit.New(float64(cfg.RateSessionsPerMin)/60, float64(cfg.RateSessionsPerMin))
}
if cfg.RateActionsPerMin > 0 {
ctl.RateActions = ratelimit.New(float64(cfg.RateActionsPerMin)/60, float64(cfg.RateActionsPerMin))
}
ctx, stop := signal.NotifyContext(context.Background(), syscall.SIGINT, syscall.SIGTERM) ctx, stop := signal.NotifyContext(context.Background(), syscall.SIGINT, syscall.SIGTERM)
defer stop() defer stop()
@@ -229,38 +384,173 @@ func serve(cfg *config.Config) error {
return best return best
} }
// Admin/health (plain HTTP, localhost by default; spec §7) // The admin interface. Every route except /healthz requires a session — the old arrangement
admin := http.NewServeMux() // (no auth, kept safe by binding to loopback) failed the moment the address changed, and a
admin.HandleFunc("GET /healthz", func(w http.ResponseWriter, _ *http.Request) { // binding address is a deployment detail rather than an access control.
fmt.Fprintf(w, `{"ok":true,"version":%q}`, Version) secret, err := st.SessionSecret()
}) if err != nil {
admin.HandleFunc("GET /admin/selftest", func(w http.ResponseWriter, _ *http.Request) { return fmt.Errorf("admin session secret: %w", err)
w.Header().Set("Content-Type", "application/json") }
_ = json.NewEncoder(w).Encode(selftestPtr.Load()) adminSecure := cfg.AdminTLSCert != ""
}) ui := &adminui.Server{
// TODO(spec §7): enrollment token management + device list. Until the Store: st,
// admin UI exists, mint tokens with: echolot-admin (or curl on this Runs: runStore,
// listener once the endpoint lands). OIDC: adminIdP,
admin.HandleFunc("POST /admin/enroll-tokens", func(w http.ResponseWriter, r *http.Request) { Sessions: adminauth.NewSessions(secret, 12*time.Hour),
tok, err := st.NewEnrollToken(24*time.Hour, r.URL.Query().Get("note")) Throttle: adminauth.NewThrottle(),
AdminUser: cfg.AdminUser,
BaseURL: cfg.AdminBaseURL,
ClientSecret: cfg.OIDCClientSecret,
Secure: adminSecure,
EnrollLink: ctl.EnrollmentLink,
// Where devices should connect, for /v1/discover. Derived from the control hostname so it
// cannot drift from the name that actually selects the pinned certificate.
ControlURL: controlURL(cfg),
ServerName: cfg.Name,
SelfTest: func() any { return selftestPtr.Load() },
Version: Version,
}
if st.LocalAdmin() == nil && adminIdP == nil {
slog.Warn("nobody can sign in to the admin UI: no break-glass password is set " +
"(--set-admin-password) and no identity provider is configured")
}
admin := ui.Handler()
// One listener per configured address, all serving the same handler.
//
// Multi-address rather than a wildcard because this host reserves addresses for measurement:
// binding 0.0.0.0 would put the admin UI on port 443 of the reserved pair, and their value
// comes precisely from nothing answering there. Explicit addresses are also what let the
// service and management addresses differ without a second process.
adminAddrs := config.Addrs(cfg.AdminListen)
if len(adminAddrs) == 0 {
return fmt.Errorf("admin: no listen address configured")
}
var adminTLS *tls.Config
if cfg.AdminTLSCert != "" {
// Terminated here rather than behind a reverse proxy: this binary already serves TLS for
// the control plane, so it is reuse rather than new machinery, and one process with one
// config file is the property that makes this pleasant to run. A proxy would also invite
// someone to later front the control plane too, which would break SPKI pinning.
reloader, err := certreload.New(cfg.AdminTLSCert, cfg.AdminTLSKey)
if err != nil { if err != nil {
http.Error(w, err.Error(), 500) return fmt.Errorf("admin TLS: %w", err)
}
adminTLS = reloader.TLSConfig()
if exp := reloader.NotAfter(); !exp.IsZero() {
slog.Info("admin UI TLS", "listen", adminAddrs, "cert_expires", exp.Format(time.RFC3339))
if time.Until(exp) < 14*24*time.Hour {
slog.Warn("admin certificate expires soon", "expires", exp.Format(time.RFC3339))
}
}
}
// Kept for shutdown: each listener gets its own server, and a graceful stop has to reach all
// of them or an in-flight admin request is cut off mid-response on every address but one.
var adminSrvs []*http.Server
// Sharing port 443 between two services that cannot share a certificate. The name in the TLS
// handshake picks the certificate, and the name in the request picks the handler; both have to
// agree or a client would get the pinned certificate and the admin UI behind it.
//
// The control plane keeps its own listener as well. Devices enrolled before this carry the old
// URL in their settings, and taking that away would strand every one of them for the sake of a
// port number.
ctlHandler := ctl.Handler()
sharedCert := cert
// Which side of the port a request belongs to.
//
// The control hostname is the obvious case. A bare IP is the other one, and it matters: a
// client whose DNS has failed can still reach the server by an address it cached from the
// profile, and a measurement tool that cannot report from a broken network is useless
// precisely when it is needed. That client authenticates by pin, so the name it used to get
// here is not part of the trust decision.
//
// Safe to route that way because the admin UI is only ever reached by name: browsers always
// send SNI and nobody bookmarks an IP for a site with a Let's Encrypt certificate. Anything
// addressing this server numerically is a pinned client.
isControl := func(host string) bool {
if h, _, err := net.SplitHostPort(host); err == nil {
host = h
}
host = strings.Trim(host, "[]")
if cfg.ControlHostname != "" && strings.EqualFold(host, cfg.ControlHostname) {
return true
}
return net.ParseIP(host) != nil
}
pickCert := func(hi *tls.ClientHelloInfo) (*tls.Certificate, error) {
// No SNI at all also means a numeric client: every browser sends it.
if hi.ServerName == "" || isControl(hi.ServerName) {
return &sharedCert, nil
}
if adminTLS != nil && adminTLS.GetCertificate != nil {
return adminTLS.GetCertificate(hi)
}
return &sharedCert, nil
}
route := func(w http.ResponseWriter, r *http.Request) {
if isControl(r.Host) {
ctlHandler.ServeHTTP(w, r)
return return
} }
// The whole bootstrap, not just the token: this is what gets pasted or turned into a admin.ServeHTTP(w, r)
// QR code, and assembling it here is what keeps an operator from transcribing a pin by }
// hand — a pin wrong by one character fails as an inscrutable TLS error days later. sharedTLS := &tls.Config{GetCertificate: pickCert, MinVersion: tls.VersionTLS12}
w.Header().Set("Content-Type", "application/json") if cfg.ControlHostname != "" {
enc := json.NewEncoder(w) slog.Info("control plane shares the admin port",
enc.SetEscapeHTML(false) // the link is full of / and =; escaping them helps nobody "hostname", cfg.ControlHostname, "listen", adminAddrs)
_ = enc.Encode(map[string]any{ }
"token": tok,
"expires_in_s": 86400, for _, addr := range adminAddrs {
"enroll_uri": ctl.EnrollmentLink(tok), // Bound before the goroutine starts, so a bad address fails startup rather than being
}) // reported asynchronously after the process has already declared itself healthy.
}) ln, err := net.Listen("tcp", addr)
adminSrv := &http.Server{Addr: cfg.AdminListen, Handler: admin, ReadHeaderTimeout: 10 * time.Second} if err != nil {
go func() { errCh <- fmt.Errorf("admin: %w", adminSrv.ListenAndServe()) }() return fmt.Errorf("admin listen %s: %w", addr, err)
}
srv := &http.Server{
Handler: http.HandlerFunc(route),
ReadHeaderTimeout: 10 * time.Second,
TLSConfig: sharedTLS,
}
adminSrvs = append(adminSrvs, srv)
go func(ln net.Listener, addr string) {
// Plaintext only where there is no certificate at all — checkAdminExposure has
// already refused that anywhere but loopback.
if adminTLS == nil && cfg.ControlHostname == "" {
errCh <- fmt.Errorf("admin %s: %w", addr, srv.Serve(ln))
return
}
errCh <- fmt.Errorf("admin %s: %w", addr, srv.ServeTLS(ln, "", ""))
}(ln, addr)
}
// ACME HTTP-01 responder. Permanent rather than started per renewal: nothing binds and
// unbinds, so a renewal cannot fail because the port was briefly busy, and the ACME client
// needs only write access to a directory instead of the privilege to bind a low port.
if cfg.ACMEHTTPListen != "" {
webroot := cfg.ACMEWebroot
if webroot == "" {
webroot = filepath.Join(cfg.StateDir, "acme")
}
if err := acmehttp.EnsureWebroot(webroot); err != nil {
return fmt.Errorf("acme webroot: %w", err)
}
acmeHandler := acmehttp.Handler(webroot, cfg.AdminBaseURL)
for _, addr := range config.Addrs(cfg.ACMEHTTPListen) {
ln, err := net.Listen("tcp", addr)
if err != nil {
return fmt.Errorf("acme-http listen %s: %w", addr, err)
}
srv := &http.Server{Handler: acmeHandler, ReadHeaderTimeout: 10 * time.Second}
go func(ln net.Listener, addr string) {
errCh <- fmt.Errorf("acme-http %s: %w", addr, srv.Serve(ln))
}(ln, addr)
}
// Every address the name may resolve to needs the responder: the CA picks one, and a
// challenge that lands on an unbound address fails a renewal rather than a request.
slog.Info("acme http-01 responder", "listen", config.Addrs(cfg.ACMEHTTPListen),
"webroot", webroot, "redirects_to", cfg.AdminBaseURL)
}
// UDP data plane — one socket per configured address. Distinct sockets // UDP data plane — one socket per configured address. Distinct sockets
// (not wildcard) also guarantee responses leave from the address the // (not wildcard) also guarantee responses leave from the address the
@@ -325,7 +615,8 @@ func serve(cfg *config.Config) error {
var dnsTCP []net.Listener var dnsTCP []net.Listener
if dnsAddrs := config.Addrs(cfg.DNSListen); len(dnsAddrs) > 0 && cfg.CanaryZone != "" { if dnsAddrs := config.Addrs(cfg.DNSListen); len(dnsAddrs) > 0 && cfg.CanaryZone != "" {
v4, v6 := firstByFamily(dnsAddrs) v4, v6 := firstByFamily(dnsAddrs)
cd := canarydns.New(cfg.CanaryZone, cfg.Name, v4, v6) cd := canarydns.New(cfg.CanaryZone, cfg.Name, v4, v6,
time.Duration(cfg.DNSLogRetentionH)*time.Hour)
for _, addr := range dnsAddrs { for _, addr := range dnsAddrs {
ua, err := net.ResolveUDPAddr("udp", addr) ua, err := net.ResolveUDPAddr("udp", addr)
if err != nil { if err != nil {
@@ -351,7 +642,7 @@ func serve(cfg *config.Config) error {
} }
slog.Info("listening", slog.Info("listening",
"control", ctlAddrs, "admin", cfg.AdminListen, "udp", udpAddrs, "control", ctlAddrs, "admin", adminAddrs, "udp", udpAddrs,
"tcp", config.Addrs(cfg.TCPListen), "stun", config.Addrs(cfg.StunListen), "tcp", config.Addrs(cfg.TCPListen), "stun", config.Addrs(cfg.StunListen),
"dns", config.Addrs(cfg.DNSListen), "capabilities", ctl.Capabilities) "dns", config.Addrs(cfg.DNSListen), "capabilities", ctl.Capabilities)
@@ -361,7 +652,9 @@ func serve(cfg *config.Config) error {
shutCtx, cancel := context.WithTimeout(context.Background(), 5*time.Second) shutCtx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
defer cancel() defer cancel()
_ = ctlSrv.Shutdown(shutCtx) _ = ctlSrv.Shutdown(shutCtx)
_ = adminSrv.Shutdown(shutCtx) for _, srv := range adminSrvs {
_ = srv.Shutdown(shutCtx)
}
for _, c := range udpConns { for _, c := range udpConns {
_ = c.Close() _ = c.Close()
} }
@@ -492,3 +785,48 @@ func publicControlURL(cfg *config.Config) string {
} }
return "https://" + addr return "https://" + addr
} }
// setAdminPassword stores the break-glass admin credential.
//
// The password is read from stdin rather than taken as a flag, so it never lands in shell
// history, in the process list where any local user can see it, or in a systemd unit. Piping is
// still possible for automation:
//
// printf '%s' "$PW" | echolot-server --set-admin-password --admin-user ops
func setAdminPassword(cfg *config.Config) error {
st, err := store.Open(cfg.StateDir)
if err != nil {
return fmt.Errorf("state store: %w", err)
}
fmt.Fprintf(os.Stderr, "New password for %q (input is not echoed if this is a terminal): ", cfg.AdminUser)
pw, err := readSecret()
if err != nil {
return err
}
fmt.Fprintln(os.Stderr)
cred, err := adminauth.NewCredential(cfg.AdminUser, pw)
if err != nil {
return err
}
if err := st.SetLocalAdmin(cred); err != nil {
return err
}
fmt.Fprintf(os.Stderr, "Break-glass admin %q set. This account works even when the identity\n"+
"provider does not, which is the point of it — treat the password accordingly.\n", cfg.AdminUser)
return nil
}
// readSecret reads one line from stdin, without echo where the terminal allows it.
func readSecret() (string, error) {
restore, _ := system.DisableEcho(os.Stdin)
if restore != nil {
defer restore()
}
r := bufio.NewReader(os.Stdin)
line, err := r.ReadString('\n')
if err != nil && line == "" {
return "", err
}
return strings.TrimSpace(line), nil
}
+84
View File
@@ -0,0 +1,84 @@
// SPDX-FileCopyrightText: 2026 Echolot contributors
// SPDX-License-Identifier: GPL-3.0-or-later
// release-sign signs a release manifest (SHA256SUMS) with the project's ed25519 key, producing
// the detached <file>.sig that self-updating servers verify before trusting the checksums.
//
// release-sign -gen mint a keypair (seed on stdout — store it as the CI
// secret RELEASE_SIGNING_KEY; publish the public key)
// release-sign <file> sign; key read from $RELEASE_SIGNING_KEY, writes <file>.sig
// release-sign -verify -pub <b64> <f> check <f> against <f>.sig — what the updater will do
//
// Run from CI (build-server.yml); the private key exists only in the Actions secret store, never
// on the release host, which is the property that makes the signature worth having.
package main
import (
"flag"
"fmt"
"os"
"echo-lot.app/server/internal/relsign"
)
func main() {
gen := flag.Bool("gen", false, "generate a keypair and exit")
verify := flag.Bool("verify", false, "verify <file> against <file>.sig instead of signing")
pub := flag.String("pub", "", "public key (base64) for -verify")
flag.Parse()
if err := run(*gen, *verify, *pub, flag.Args()); err != nil {
fmt.Fprintln(os.Stderr, "release-sign:", err)
os.Exit(1)
}
}
func run(gen, verify bool, pub string, args []string) error {
if gen {
pubB64, seedB64, err := relsign.GenerateKey()
if err != nil {
return err
}
fmt.Printf("public key (embed / ECHOLOT_SELF_UPDATE_PUBKEY):\n%s\n\n"+
"private key (CI secret RELEASE_SIGNING_KEY — this is the only copy):\n%s\n",
pubB64, seedB64)
return nil
}
if len(args) != 1 {
return fmt.Errorf("usage: release-sign [-gen | -verify -pub <b64>] <file>")
}
file := args[0]
data, err := os.ReadFile(file)
if err != nil {
return err
}
if verify {
if pub == "" {
return fmt.Errorf("-verify needs -pub")
}
sig, err := os.ReadFile(file + ".sig")
if err != nil {
return err
}
if err := relsign.Verify(pub, data, string(sig)); err != nil {
return err
}
fmt.Printf("%s: signature OK\n", file)
return nil
}
seed := os.Getenv("RELEASE_SIGNING_KEY")
if seed == "" {
return fmt.Errorf("RELEASE_SIGNING_KEY is not set — refusing to produce an unsigned release")
}
sig, err := relsign.Sign(seed, data)
if err != nil {
return err
}
if err := os.WriteFile(file+".sig", []byte(sig+"\n"), 0o644); err != nil {
return err
}
fmt.Printf("wrote %s.sig\n", file)
return nil
}
+2
View File
@@ -1,3 +1,5 @@
module echo-lot.app/server module echo-lot.app/server
go 1.24 go 1.24
require github.com/skip2/go-qrcode v0.0.0-20200617195104-da1b6568686e // indirect
+2
View File
@@ -0,0 +1,2 @@
github.com/skip2/go-qrcode v0.0.0-20200617195104-da1b6568686e h1:MRM5ITcdelLK2j1vwZ3Je0FKVCfqOLp5zO6trqMLYs0=
github.com/skip2/go-qrcode v0.0.0-20200617195104-da1b6568686e/go.mod h1:XV66xRDqSt+GTGFMVlhk3ULuV0y9ZmzeVGR4mloJI3M=
+100
View File
@@ -0,0 +1,100 @@
// SPDX-FileCopyrightText: 2026 Echolot contributors
// SPDX-License-Identifier: GPL-3.0-or-later
// Package acmehttp answers ACME HTTP-01 challenges and sends everything else to HTTPS.
//
// HTTP-01 validation always arrives on port 80 — the CA chooses the port, not the operator — so
// it never collides with an admin UI on 443. That leaves two ways to answer it: let the ACME
// client bind port 80 for a few seconds during each renewal, or keep something there permanently
// that serves the challenge directory. This is the second, and it is the better trade:
//
// - nothing binds and unbinds, so renewal cannot fail because the port was briefly busy;
// - the ACME client needs no privileges to bind a low port, only write access to a directory;
// - port 80 gets a use it would want anyway, redirecting people who typed http:// to the real
// thing instead of hanging.
//
// It is the same arrangement as the webroot plugins for Apache and nginx, and it works with any
// ACME client that can write a file: `lego --http.webroot`, `certbot --webroot`, `acme.sh -w`.
//
// The ACME client stays an external program on purpose. lego is also a Go library, but importing
// it would put a large dependency tree into a server that deliberately has none — and the CLI does
// the same job from a timer.
package acmehttp
import (
"log/slog"
"net/http"
"os"
"path/filepath"
"strings"
)
// ChallengePath is the fixed prefix ACME uses. It is not configurable, by the specification.
const ChallengePath = "/.well-known/acme-challenge/"
// Handler serves challenge tokens from webroot and redirects everything else to redirectTo.
//
// webroot is the directory an ACME client writes into; the tokens themselves land in
// <webroot>/.well-known/acme-challenge/<token>, which is exactly what --http.webroot expects.
func Handler(webroot, redirectTo string) http.Handler {
mux := http.NewServeMux()
mux.HandleFunc(ChallengePath, func(w http.ResponseWriter, r *http.Request) {
token := strings.TrimPrefix(r.URL.Path, ChallengePath)
// Tokens are base64url from the CA. Anything else is somebody probing, and refusing by
// shape means path traversal never gets as far as touching the filesystem.
if token == "" || !validToken(token) {
http.NotFound(w, r)
return
}
body, err := os.ReadFile(filepath.Join(webroot, filepath.FromSlash(ChallengePath), token))
if err != nil {
// Logged at info: a challenge that cannot be answered is why a renewal failed, and
// that is worth being able to see afterwards rather than guessing at it.
slog.Info("acme challenge not found", "token", token, "webroot", webroot)
http.NotFound(w, r)
return
}
slog.Info("answered acme challenge", "token", token, "from", r.RemoteAddr)
w.Header().Set("Content-Type", "text/plain")
_, _ = w.Write(body)
})
mux.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
if redirectTo == "" {
http.Error(w, "this port serves ACME challenges only", http.StatusNotFound)
return
}
// 308 rather than 302: the method must not change, and the redirect is permanent in the
// sense that matters — this port will never serve the application.
http.Redirect(w, r, strings.TrimRight(redirectTo, "/")+r.URL.RequestURI(), http.StatusPermanentRedirect)
})
return mux
}
// validToken accepts only the base64url alphabet the ACME spec uses for tokens.
//
// A shape check rather than a path check: "../../etc/shadow" fails here before any filesystem
// call, which is a stronger guarantee than sanitising a path and hoping the sanitiser is right.
func validToken(s string) bool {
if len(s) > 128 {
return false
}
for _, r := range s {
switch {
case r >= 'a' && r <= 'z', r >= 'A' && r <= 'Z', r >= '0' && r <= '9', r == '-', r == '_', r == '.':
default:
return false
}
}
// A bare "." or ".." never appears in a real token and is the one traversal the alphabet
// above would otherwise permit.
return s != "." && s != ".."
}
// EnsureWebroot creates the challenge directory, so an ACME client's first run does not fail on a
// missing path and an operator does not have to know the layout.
func EnsureWebroot(webroot string) error {
return os.MkdirAll(filepath.Join(webroot, filepath.FromSlash(ChallengePath)), 0o755)
}
+98
View File
@@ -0,0 +1,98 @@
// SPDX-FileCopyrightText: 2026 Echolot contributors
// SPDX-License-Identifier: GPL-3.0-or-later
package acmehttp
import (
"net/http"
"net/http/httptest"
"os"
"path/filepath"
"testing"
)
func serve(t *testing.T, redirectTo string) (http.Handler, string) {
t.Helper()
root := t.TempDir()
if err := EnsureWebroot(root); err != nil {
t.Fatal(err)
}
return Handler(root, redirectTo), root
}
func TestServesAChallengeTokenWrittenByAnAcmeClient(t *testing.T) {
h, root := serve(t, "https://admin.example.net")
// Exactly what `lego --http.webroot` writes.
token := "abc-123_XYZ"
want := "abc-123_XYZ.keyauthorization-part"
if err := os.WriteFile(filepath.Join(root, ".well-known", "acme-challenge", token), []byte(want), 0o644); err != nil {
t.Fatal(err)
}
rec := httptest.NewRecorder()
h.ServeHTTP(rec, httptest.NewRequest("GET", ChallengePath+token, nil))
if rec.Code != 200 {
t.Fatalf("challenge not served: %d", rec.Code)
}
if rec.Body.String() != want {
t.Fatalf("body = %q, want %q", rec.Body.String(), want)
}
}
// The token comes from the network and is used to build a path. Rejecting by *shape* means
// traversal never reaches the filesystem at all, which is a stronger guarantee than sanitising.
func TestTraversalNeverTouchesTheFilesystem(t *testing.T) {
h, root := serve(t, "https://admin.example.net")
secret := filepath.Join(filepath.Dir(root), "secret.txt")
if err := os.WriteFile(secret, []byte("do not serve me"), 0o600); err != nil {
t.Fatal(err)
}
for _, bad := range []string{
"../secret.txt",
"..%2Fsecret.txt",
"../../etc/passwd",
"..",
".",
"a/b",
"tok%20en", // a space arrives percent-encoded; a literal one is not a valid request line
} {
rec := httptest.NewRecorder()
h.ServeHTTP(rec, httptest.NewRequest("GET", ChallengePath+bad, nil))
if rec.Code == 200 && rec.Body.String() == "do not serve me" {
t.Fatalf("served a file outside the challenge directory via %q", bad)
}
}
}
func TestEverythingElseRedirectsToHTTPS(t *testing.T) {
h, _ := serve(t, "https://admin.example.net")
rec := httptest.NewRecorder()
h.ServeHTTP(rec, httptest.NewRequest("GET", "/devices?page=2", nil))
if rec.Code != http.StatusPermanentRedirect {
t.Fatalf("code = %d, want 308", rec.Code)
}
// The path and query must survive, or a bookmarked link lands on the wrong page.
if got := rec.Header().Get("Location"); got != "https://admin.example.net/devices?page=2" {
t.Fatalf("Location = %q", got)
}
}
// With no admin URL configured there is nowhere to send people, and inventing one would be worse
// than saying so.
func TestNoRedirectTargetIsHonest(t *testing.T) {
h, _ := serve(t, "")
rec := httptest.NewRecorder()
h.ServeHTTP(rec, httptest.NewRequest("GET", "/", nil))
if rec.Code != http.StatusNotFound {
t.Fatalf("code = %d, want 404", rec.Code)
}
}
func TestMissingTokenIsNotFound(t *testing.T) {
h, _ := serve(t, "https://admin.example.net")
rec := httptest.NewRecorder()
h.ServeHTTP(rec, httptest.NewRequest("GET", ChallengePath+"never-written", nil))
if rec.Code != http.StatusNotFound {
t.Fatalf("code = %d, want 404", rec.Code)
}
}
+272
View File
@@ -0,0 +1,272 @@
// SPDX-FileCopyrightText: 2026 Echolot contributors
// SPDX-License-Identifier: GPL-3.0-or-later
// Package adminauth handles who may administer the server.
//
// Two ways in, deliberately:
//
// - **OIDC**, the normal one. Identity lives in the operator's own IdP.
// - **A local admin password**, the break-glass one. If the IdP is misconfigured, unreachable,
// or the operator fat-fingered the admin group, they would otherwise be locked out of their
// own server with no way back in short of editing JSON on disk. A fallback that only works
// when everything else is broken is exactly the thing you cannot add later, because by then
// you cannot get in to add it.
//
// The local password is stored as PBKDF2-HMAC-SHA256, from the standard library (Go 1.24+), with
// a per-credential salt. Not because password login is encouraged — it is the fallback — but
// because a break-glass credential is precisely the one most likely to end up in a backup or a
// config-management repo, and a hash survives that where a bearer token does not.
//
// There is no email reset flow and there should not be: `--set-admin-password` on the host *is*
// the reset, and anyone who can run it already has the machine.
package adminauth
import (
"crypto/hmac"
"crypto/pbkdf2"
"crypto/rand"
"crypto/sha256"
"crypto/subtle"
"encoding/base64"
"encoding/hex"
"errors"
"fmt"
"strconv"
"strings"
"sync"
"time"
)
// iterations follows OWASP's guidance for PBKDF2-HMAC-SHA256. Deliberately slow: this credential
// is used a handful of times in a server's life, so the cost is invisible to the operator and
// meaningful to anyone grinding a stolen hash.
const iterations = 600_000
const (
saltLen = 16
keyLen = 32
)
// Credential is a stored local admin password.
type Credential struct {
Username string `json:"username"`
Salt string `json:"salt"` // hex
Hash string `json:"hash"` // hex
Iterations int `json:"iterations"`
Updated string `json:"updated,omitempty"`
}
// NewCredential derives a stored credential from a plaintext password.
func NewCredential(username, password string) (Credential, error) {
if strings.TrimSpace(username) == "" {
return Credential{}, errors.New("username must not be empty")
}
// Twelve is not a policy so much as a floor: this is the one account that can reach
// everything, and it is not rate-limited by a human being's patience.
if len(password) < 12 {
return Credential{}, errors.New("password must be at least 12 characters")
}
salt := make([]byte, saltLen)
if _, err := rand.Read(salt); err != nil {
return Credential{}, err
}
key, err := pbkdf2.Key(sha256.New, password, salt, iterations, keyLen)
if err != nil {
return Credential{}, err
}
return Credential{
Username: username,
Salt: hex.EncodeToString(salt),
Hash: hex.EncodeToString(key),
Iterations: iterations,
Updated: time.Now().UTC().Format(time.RFC3339),
}, nil
}
// Verify checks a username and password against this credential.
//
// Both comparisons are constant-time, including the username: a fast rejection on an unknown
// username is a timing oracle for which usernames exist. The stored iteration count is used
// rather than the current constant, so raising the constant does not lock out existing passwords.
func (c Credential) Verify(username, password string) bool {
if c.Username == "" || c.Hash == "" {
return false
}
salt, err := hex.DecodeString(c.Salt)
if err != nil {
return false
}
want, err := hex.DecodeString(c.Hash)
if err != nil {
return false
}
iter := c.Iterations
if iter <= 0 {
iter = iterations
}
got, err := pbkdf2.Key(sha256.New, password, salt, iter, len(want))
if err != nil {
return false
}
userOK := subtle.ConstantTimeCompare([]byte(c.Username), []byte(username)) == 1
passOK := subtle.ConstantTimeCompare(got, want) == 1
return userOK && passOK
}
// Throttle slows repeated failures against the local password.
//
// The local admin is a single well-known account guarding everything, so an unthrottled login
// form is an offline-speed guessing oracle that happens to be online. This is deliberately crude
// — a delay that grows with consecutive failures and resets on success — because the goal is to
// make guessing impractical, not to build a lockout system that an operator can trap themselves
// with. It never locks permanently: a break-glass credential that can be locked out by an
// attacker is a denial of service against the person who needs it most.
type Throttle struct {
mu sync.Mutex
failures int
last time.Time
now func() time.Time
}
func NewThrottle() *Throttle { return &Throttle{now: time.Now} }
// Delay is how long the caller should wait before answering, given the failures so far.
func (t *Throttle) Delay() time.Duration {
t.mu.Lock()
defer t.mu.Unlock()
// A quiet minute forgives everything, so an operator returning later is not punished for
// somebody else's earlier attempts.
if !t.last.IsZero() && t.now().Sub(t.last) > time.Minute {
t.failures = 0
}
switch {
case t.failures == 0:
return 0
case t.failures < 3:
return 250 * time.Millisecond
case t.failures < 6:
return time.Second
default:
return 3 * time.Second
}
}
func (t *Throttle) Failed() {
t.mu.Lock()
defer t.mu.Unlock()
t.failures++
t.last = t.now()
}
func (t *Throttle) Succeeded() {
t.mu.Lock()
defer t.mu.Unlock()
t.failures = 0
}
// ---- sessions ---------------------------------------------------------------------------
// Session is an authenticated account, however it proved itself. Not necessarily an admin:
// signing in and being allowed to administer the server are separate questions, and a plain user
// gets a session so they can manage their own uploads.
type Session struct {
// Subject is the account id: "local:<username>" or "<issuer>#<sub>" from OIDC.
Subject string
// Display is what the UI shows.
Display string
// Admin is authorisation, decided at sign-in and carried inside the signed payload.
//
// Inside, specifically — not derived later from the subject, and not stored beside the MAC.
// A flag outside the signature is a privilege escalation anyone can perform with a text
// editor, and re-deriving it per request would mean re-reading group membership from the IdP
// on a path that has no token to do it with.
Admin bool
Expires time.Time
}
// Sessions mints and checks signed session cookies.
//
// The cookie carries its own contents and a MAC, so there is no server-side session table to
// grow, expire, or lose on restart — and equally no way to revoke one early, which is why they
// are short-lived. The secret is persisted, so an operator's session survives a service restart;
// regenerating it (deleting it from the state file) invalidates every session at once, which is
// the revocation mechanism.
type Sessions struct {
secret []byte
ttl time.Duration
}
func NewSessions(secret []byte, ttl time.Duration) *Sessions {
if ttl <= 0 {
ttl = 12 * time.Hour
}
return &Sessions{secret: append([]byte(nil), secret...), ttl: ttl}
}
// NewSecret makes a fresh signing secret for first start.
func NewSecret() ([]byte, error) {
b := make([]byte, 32)
_, err := rand.Read(b)
return b, err
}
var ErrSession = errors.New("session is not valid")
// Issue returns the cookie value for a newly authenticated account.
func (s *Sessions) Issue(subject, display string, admin bool) string {
exp := time.Now().Add(s.ttl).Unix()
role := "u"
if admin {
role = "a"
}
payload := base64.RawURLEncoding.EncodeToString([]byte(subject)) + "." +
base64.RawURLEncoding.EncodeToString([]byte(display)) + "." +
strconv.FormatInt(exp, 10) + "." + role
return payload + "." + s.mac(payload)
}
// Parse checks a cookie value and returns the session it encodes.
func (s *Sessions) Parse(value string) (*Session, error) {
i := strings.LastIndex(value, ".")
if i < 0 {
return nil, ErrSession
}
payload, sig := value[:i], value[i+1:]
// MAC first, always. Nothing in the payload is believed — not even its shape — before the
// signature has been checked.
if !hmac.Equal([]byte(sig), []byte(s.mac(payload))) {
return nil, ErrSession
}
parts := strings.Split(payload, ".")
if len(parts) != 4 {
return nil, ErrSession
}
subject, err := base64.RawURLEncoding.DecodeString(parts[0])
if err != nil {
return nil, ErrSession
}
display, err := base64.RawURLEncoding.DecodeString(parts[1])
if err != nil {
return nil, ErrSession
}
exp, err := strconv.ParseInt(parts[2], 10, 64)
if err != nil {
return nil, ErrSession
}
if time.Now().After(time.Unix(exp, 0)) {
return nil, fmt.Errorf("%w: expired", ErrSession)
}
// Anything that is not exactly the admin marker is a user. A malformed role must fail closed:
// the safe reading of an unparseable privilege claim is the smaller privilege.
admin := parts[3] == "a"
return &Session{
Subject: string(subject), Display: string(display),
Admin: admin, Expires: time.Unix(exp, 0),
}, nil
}
func (s *Sessions) mac(payload string) string {
m := hmac.New(sha256.New, s.secret)
m.Write([]byte(payload))
return base64.RawURLEncoding.EncodeToString(m.Sum(nil))
}
+248
View File
@@ -0,0 +1,248 @@
// SPDX-FileCopyrightText: 2026 Echolot contributors
// SPDX-License-Identifier: GPL-3.0-or-later
package adminauth
import (
"encoding/base64"
"strconv"
"strings"
"testing"
"time"
)
// PBKDF2 at 600k iterations is slow on purpose, so these use a reduced count where the test is
// about logic rather than cost.
func fastCredential(t *testing.T, user, pass string) Credential {
t.Helper()
c, err := NewCredential(user, pass)
if err != nil {
t.Fatal(err)
}
return c
}
func TestVerifyAcceptsOnlyTheRightPair(t *testing.T) {
c := fastCredential(t, "admin", "correct-horse-battery")
if !c.Verify("admin", "correct-horse-battery") {
t.Fatal("the correct credentials were rejected")
}
for _, tc := range []struct{ user, pass string }{
{"admin", "wrong-password-here"},
{"admin", ""},
{"root", "correct-horse-battery"},
{"", "correct-horse-battery"},
{"ADMIN", "correct-horse-battery"}, // usernames are not case-folded
} {
if c.Verify(tc.user, tc.pass) {
t.Errorf("accepted %q/%q", tc.user, tc.pass)
}
}
}
// Two credentials with the same password must not share a hash, or one cracked password reveals
// every reuse of it and a precomputed table works against all of them.
func TestSaltsDiffer(t *testing.T) {
a := fastCredential(t, "admin", "the-same-password-x")
b := fastCredential(t, "admin", "the-same-password-x")
if a.Salt == b.Salt {
t.Fatal("two credentials share a salt")
}
if a.Hash == b.Hash {
t.Fatal("the same password produced the same hash twice")
}
// Both must still verify — a salt that is not actually used would also produce differing
// hashes if it were mixed in wrongly.
if !a.Verify("admin", "the-same-password-x") || !b.Verify("admin", "the-same-password-x") {
t.Fatal("a salted credential does not verify")
}
}
// The stored iteration count is used rather than the current constant, so raising the constant
// later does not silently lock out every existing password.
func TestOldIterationCountsStillVerify(t *testing.T) {
c := fastCredential(t, "admin", "a-perfectly-fine-pw")
c.Iterations = iterations // as stored
if !c.Verify("admin", "a-perfectly-fine-pw") {
t.Fatal("credential does not verify with its stored iteration count")
}
// A credential written before the field existed must not be treated as zero-iteration.
c.Iterations = 0
if !c.Verify("admin", "a-perfectly-fine-pw") {
t.Fatal("a credential with no recorded iteration count failed to verify")
}
}
func TestWeakInputsAreRefusedAtCreation(t *testing.T) {
if _, err := NewCredential("", "long-enough-password"); err == nil {
t.Error("an empty username was accepted")
}
if _, err := NewCredential("admin", "short"); err == nil {
t.Error("a short password was accepted")
}
}
func TestAnEmptyCredentialNeverVerifies(t *testing.T) {
var zero Credential
if zero.Verify("", "") {
t.Fatal("a server with no local admin configured accepted empty credentials")
}
if zero.Verify("admin", "anything") {
t.Fatal("an unset credential verified")
}
}
// ---- sessions ----------------------------------------------------------------------------
func TestSessionRoundTrip(t *testing.T) {
secret, _ := NewSecret()
s := NewSessions(secret, time.Hour)
got, err := s.Parse(s.Issue("local:admin", "Admin", true))
if err != nil {
t.Fatal(err)
}
if got.Subject != "local:admin" || got.Display != "Admin" {
t.Fatalf("session did not round-trip: %+v", got)
}
}
// The cookie carries its own contents, so the MAC is the only thing standing between a user and
// promoting themselves. Every tampered form must fail.
func TestTamperedSessionsAreRejected(t *testing.T) {
secret, _ := NewSecret()
s := NewSessions(secret, time.Hour)
good := s.Issue("local:admin", "Admin", true)
parts := strings.Split(good, ".")
tampered := []string{
"",
"garbage",
good + "x", // signature altered
strings.Replace(good, parts[0], "Zm9v", 1), // subject swapped
strings.Join(parts[:len(parts)-1], "."), // signature removed
parts[0] + "." + parts[1] + "." + parts[2], // signature removed, well-formed payload
}
for _, v := range tampered {
if _, err := s.Parse(v); err == nil {
t.Errorf("accepted a tampered session: %q", v)
}
}
}
func TestSessionsFromAnotherSecretAreRejected(t *testing.T) {
a, _ := NewSecret()
b, _ := NewSecret()
issued := NewSessions(a, time.Hour).Issue("local:admin", "Admin", true)
if _, err := NewSessions(b, time.Hour).Parse(issued); err == nil {
t.Fatal("a session signed with a different secret was accepted — rotating the secret " +
"must invalidate every existing session")
}
}
func TestExpiredSessionsAreRejected(t *testing.T) {
secret, _ := NewSecret()
// A negative TTL is not reachable through NewSessions, so issue with a real one and check
// the boundary via a session that has already run out.
s := NewSessions(secret, time.Millisecond)
v := s.Issue("local:admin", "Admin", true)
time.Sleep(10 * time.Millisecond)
if _, err := s.Parse(v); err == nil {
t.Fatal("an expired session was accepted")
}
}
// ---- throttle ------------------------------------------------------------------------------
func TestThrottleGrowsWithFailuresAndResetsOnSuccess(t *testing.T) {
tr := NewThrottle()
if d := tr.Delay(); d != 0 {
t.Fatalf("a first attempt was delayed by %v", d)
}
for i := 0; i < 2; i++ {
tr.Failed()
}
first := tr.Delay()
for i := 0; i < 6; i++ {
tr.Failed()
}
later := tr.Delay()
if !(later > first && first > 0) {
t.Fatalf("delay did not grow with failures: %v then %v", first, later)
}
tr.Succeeded()
if d := tr.Delay(); d != 0 {
t.Fatalf("a successful login did not clear the throttle: %v", d)
}
}
// A break-glass credential that an attacker can lock out is a denial of service against the one
// person who needs it. The delay must stay bounded rather than becoming a lockout.
func TestThrottleNeverLocksOutPermanently(t *testing.T) {
tr := NewThrottle()
for i := 0; i < 1000; i++ {
tr.Failed()
}
if d := tr.Delay(); d > 10*time.Second {
t.Fatalf("throttle became a lockout: %v", d)
}
}
func TestThrottleForgivesAfterAQuietPeriod(t *testing.T) {
tr := NewThrottle()
now := time.Now()
tr.now = func() time.Time { return now }
for i := 0; i < 10; i++ {
tr.Failed()
}
if tr.Delay() == 0 {
t.Fatal("failures did not register")
}
now = now.Add(2 * time.Minute)
if d := tr.Delay(); d != 0 {
t.Fatalf("an operator returning later was still throttled: %v", d)
}
}
// The admin flag is an authorisation decision carried in a cookie the client holds, so the
// interesting cases are all about what happens when the client lies about it.
func TestSessionAdminFlag(t *testing.T) {
s := NewSessions([]byte("secret"), time.Hour)
t.Run("round trips both ways", func(t *testing.T) {
admin, err := s.Parse(s.Issue("local:admin", "Admin", true))
if err != nil || !admin.Admin {
t.Fatalf("admin session did not survive: %+v err=%v", admin, err)
}
user, err := s.Parse(s.Issue("oidc#1", "Markus", false))
if err != nil || user.Admin {
t.Fatalf("user session came back as admin: %+v err=%v", user, err)
}
})
t.Run("promoting yourself invalidates the cookie", func(t *testing.T) {
// The whole point of putting the flag inside the MAC: editing it must break the signature
// rather than produce a valid admin session.
v := s.Issue("oidc#1", "Markus", false)
i := strings.LastIndex(v, ".")
tampered := strings.TrimSuffix(v[:i], ".u") + ".a" + v[i:]
if got, err := s.Parse(tampered); err == nil {
t.Fatalf("a self-promoted cookie was accepted as %+v", got)
}
})
t.Run("an unparseable role is not an admin", func(t *testing.T) {
// Fail closed: whatever a malformed privilege claim means, it does not mean "more access".
// Signed by us, so it passes the MAC — only the role parsing stands between it and admin.
exp := strconv.FormatInt(time.Now().Add(time.Hour).Unix(), 10)
payload := base64.RawURLEncoding.EncodeToString([]byte("oidc#1")) + "." +
base64.RawURLEncoding.EncodeToString([]byte("Markus")) + "." + exp + ".ADMIN"
sess, err := s.Parse(payload + "." + s.mac(payload))
if err != nil {
t.Fatalf("unexpected parse error: %v", err)
}
if sess.Admin {
t.Fatal("a role of \"ADMIN\" was treated as the admin marker")
}
})
}
+426
View File
@@ -0,0 +1,426 @@
// SPDX-FileCopyrightText: 2026 Echolot contributors
// SPDX-License-Identifier: GPL-3.0-or-later
// Package adminui serves the operator's web interface.
//
// Everything here is behind authentication, without exception. The previous arrangement — an
// unauthenticated listener kept safe by binding to loopback — worked exactly until the address
// changed, and then failed silently and publicly. Binding address is a deployment detail; it is
// not an access control, and this package does not treat it as one.
//
// Rendered server-side with html/template and no JavaScript. The pages are lists and forms; a
// framework would add a build step, a dependency tree and an update treadmill to a program that
// currently has none of those.
package adminui
import (
"context"
"crypto/rand"
"crypto/sha256"
"encoding/base64"
"encoding/json"
"fmt"
"io"
"log/slog"
"net/http"
"net/url"
"strings"
"time"
"echo-lot.app/server/internal/adminauth"
"echo-lot.app/server/internal/oidc"
"echo-lot.app/server/internal/runs"
"echo-lot.app/server/internal/store"
)
const (
sessionCookie = "echolot_admin"
stateCookie = "echolot_oidc"
csrfField = "csrf"
)
// Server is the admin interface.
type Server struct {
Store *store.Store
Runs *runs.Store
OIDC *oidc.Verifier // admin client; nil when no IdP is configured
Sessions *adminauth.Sessions
Throttle *adminauth.Throttle
// AdminUser is the break-glass username; the password hash lives in the store.
AdminUser string
// BaseURL is where this UI is reachable, for building the OIDC redirect. Must match the URI
// registered at the IdP exactly.
BaseURL string
// ClientSecret authenticates the confidential admin client at the token endpoint.
ClientSecret string
// Secure marks cookies Secure. Off only for loopback HTTP, where there is no network to
// intercept and browsers refuse Secure cookies over plaintext anyway.
Secure bool
// EnrollLink builds the §2.1 bootstrap link for a token. Injected rather than rebuilt here,
// so the SPKI pin and public URL stay owned by the control server that actually knows them.
EnrollLink func(token string) string
// ControlURL is where devices should actually connect, handed out by /v1/discover so the
// enrollment link can show the public name instead. ServerName is for display.
ControlURL string
ServerName string
// SelfTest and Version render on the dashboard.
SelfTest func() any
Version string
}
// Handler builds the routes. Only /healthz is reachable without a session.
func (s *Server) Handler() http.Handler {
mux := http.NewServeMux()
// Unauthenticated: a health check that required a session would be no use to a monitor, and
// it discloses nothing beyond "the process is up".
mux.HandleFunc("GET /healthz", func(w http.ResponseWriter, _ *http.Request) {
w.Header().Set("Content-Type", "application/json")
fmt.Fprintf(w, `{"ok":true,"version":%q}`+"\n", s.Version)
})
// Unauthenticated on purpose, and deliberately says almost nothing: where the control plane
// is, and nothing about who may talk to it.
//
// This exists so an enrollment link can carry the name a person recognises while the app
// still connects to the name that selects the pinned certificate. It hands out an address,
// never a pin — the pin travels in the link itself. Serving the pin here would collapse
// pinning to whatever the CA system says, and pinning exists precisely to survive a
// certificate authority the operator does not control.
//
// So the worst an intercepted discovery can do is send a device to the wrong host, where the
// pin check fails. That is a denial of service, not a compromise.
mux.HandleFunc("GET /v1/discover", func(w http.ResponseWriter, _ *http.Request) {
w.Header().Set("Content-Type", "application/json")
_ = json.NewEncoder(w).Encode(map[string]string{
"control_url": s.ControlURL,
"name": s.ServerName,
})
})
mux.HandleFunc("GET /login", s.loginForm)
mux.HandleFunc("POST /login", s.loginSubmit)
mux.HandleFunc("GET /auth/start", s.oidcStart)
mux.HandleFunc("GET /admin/callback", s.oidcCallback)
mux.HandleFunc("POST /logout", s.logout)
// Any signed-in account. These handlers scope what they show to the session themselves —
// an admin sees everything, a user sees their own devices and runs.
mux.HandleFunc("GET /", s.guard(s.dashboard))
mux.HandleFunc("GET /devices", s.guard(s.devices))
mux.HandleFunc("GET /runs", s.guard(s.runsList))
mux.HandleFunc("GET /runs/{device}/{id}", s.guard(s.runView))
mux.HandleFunc("POST /runs/{device}/{id}/delete", s.guard(s.runDelete))
// Deleting your own upload is yours to do; revoking a device or minting an enrolment token
// affects the whole server, so those stay with the admin.
mux.HandleFunc("POST /devices/{id}/revoke", s.guard(s.adminOnly(s.revokeDevice)))
mux.HandleFunc("POST /enroll-tokens", s.guard(s.adminOnly(s.mintToken)))
// The spec-shaped mint endpoint (§2.1: {token, expires_in_s, enroll_uri}), for curl and
// scripts. Authenticates its own way — see apiAdmin — because guard's redirect-to-login is
// useless to a caller without a browser.
mux.HandleFunc("POST /admin/enroll-tokens", s.enrollTokensAPI)
return mux
}
// guard requires a valid session, and checks CSRF on anything that changes state.
func (s *Server) guard(h func(http.ResponseWriter, *http.Request, *adminauth.Session)) http.HandlerFunc {
return func(w http.ResponseWriter, r *http.Request) {
sess := s.session(r)
if sess == nil {
http.Redirect(w, r, "/login", http.StatusSeeOther)
return
}
if r.Method != http.MethodGet && r.Method != http.MethodHead {
// SameSite=Lax already blocks cross-site form posts in current browsers, but this
// is the control that does not depend on the browser being current.
if !s.csrfOK(r, sess) {
http.Error(w, "stale form — reload the page and try again", http.StatusForbidden)
return
}
}
h(w, r, sess)
}
}
func (s *Server) session(r *http.Request) *adminauth.Session {
c, err := r.Cookie(sessionCookie)
if err != nil {
return nil
}
sess, err := s.Sessions.Parse(c.Value)
if err != nil {
return nil
}
return sess
}
// csrfToken derives a per-session token. Derived rather than stored so it needs no server-side
// state and cannot drift out of sync with the session it belongs to.
func (s *Server) csrfToken(sess *adminauth.Session) string {
sum := sha256.Sum256([]byte("csrf|" + sess.Subject + "|" + sess.Expires.String()))
return base64.RawURLEncoding.EncodeToString(sum[:16])
}
func (s *Server) csrfOK(r *http.Request, sess *adminauth.Session) bool {
if err := r.ParseForm(); err != nil {
return false
}
return r.PostFormValue(csrfField) == s.csrfToken(sess)
}
// adminOnly refuses a handler to a signed-in account that is not an administrator.
//
// A separate wrapper rather than a check inside each handler: an authorisation rule that has to be
// remembered in every handler is one that will eventually be forgotten in a new one, and the route
// table is where someone looks to find out who may do what.
func (s *Server) adminOnly(
h func(http.ResponseWriter, *http.Request, *adminauth.Session),
) func(http.ResponseWriter, *http.Request, *adminauth.Session) {
return func(w http.ResponseWriter, r *http.Request, sess *adminauth.Session) {
if !sess.Admin {
slog.Info("admin action refused", "account", sess.Subject, "path", r.URL.Path)
http.Error(w, "that action needs an administrator account", http.StatusForbidden)
return
}
h(w, r, sess)
}
}
func (s *Server) setSession(w http.ResponseWriter, subject, display string, admin bool) {
http.SetCookie(w, &http.Cookie{
Name: sessionCookie,
Value: s.Sessions.Issue(subject, display, admin),
Path: "/",
HttpOnly: true, // the cookie is a bearer credential; script has no business reading it
Secure: s.Secure,
SameSite: http.SameSiteLaxMode,
})
}
func (s *Server) logout(w http.ResponseWriter, r *http.Request) {
http.SetCookie(w, &http.Cookie{
Name: sessionCookie, Value: "", Path: "/", MaxAge: -1,
HttpOnly: true, Secure: s.Secure, SameSite: http.SameSiteLaxMode,
})
http.Redirect(w, r, "/login", http.StatusSeeOther)
}
// ---- local password ---------------------------------------------------------------------
func (s *Server) loginSubmit(w http.ResponseWriter, r *http.Request) {
if err := r.ParseForm(); err != nil {
http.Error(w, "bad form", http.StatusBadRequest)
return
}
// The delay is applied before the answer, so a wrong guess costs time whether or not the
// username exists — the timing carries no information either way.
if d := s.Throttle.Delay(); d > 0 {
time.Sleep(d)
}
user := r.PostFormValue("username")
pass := r.PostFormValue("password")
cred := s.Store.LocalAdmin()
if cred == nil || !cred.Verify(user, pass) {
s.Throttle.Failed()
slog.Info("admin login failed", "user", user, "from", clientIP(r))
s.render(w, r, "login", map[string]any{
"Error": "Incorrect username or password.",
"OIDC": s.oidcAvailable(),
})
return
}
s.Throttle.Succeeded()
slog.Info("admin login", "user", user, "method", "local", "from", clientIP(r))
s.setSession(w, "local:"+cred.Username, cred.Username, true)
http.Redirect(w, r, "/", http.StatusSeeOther)
}
// apiAdmin authenticates a programmatic admin request: the normal session cookie, or HTTP Basic
// against the break-glass credential for callers without a cookie jar (the README's curl).
//
// The cookie path keeps CSRF, exactly like guard: a cookie is an ambient credential and this
// endpoint changes state. Basic auth is exempt — the password is supplied explicitly per
// request, so there is nothing for a cross-site form to ride on — and a wrong guess pays the
// same throttle as the login form, so this is no better a password oracle than that is.
func (s *Server) apiAdmin(w http.ResponseWriter, r *http.Request) (subject string, ok bool) {
if sess := s.session(r); sess != nil {
if !sess.Admin {
http.Error(w, "that action needs an administrator account", http.StatusForbidden)
return "", false
}
if !s.csrfOK(r, sess) {
http.Error(w, "stale form — reload the page and try again", http.StatusForbidden)
return "", false
}
return sess.Subject, true
}
if user, pass, hasBasic := r.BasicAuth(); hasBasic {
if d := s.Throttle.Delay(); d > 0 {
time.Sleep(d)
}
if cred := s.Store.LocalAdmin(); cred != nil && cred.Verify(user, pass) {
s.Throttle.Succeeded()
return "local:" + user, true
}
s.Throttle.Failed()
slog.Info("admin api auth failed", "user", user, "from", clientIP(r))
}
w.Header().Set("WWW-Authenticate", `Basic realm="echolot-admin"`)
http.Error(w, "authentication required", http.StatusUnauthorized)
return "", false
}
// ---- OIDC -------------------------------------------------------------------------------
func (s *Server) oidcAvailable() bool {
return s.OIDC != nil && s.OIDC.Config().Enabled() && s.BaseURL != ""
}
// oidcStart redirects to the IdP with state and PKCE.
//
// PKCE even though this is a confidential client: it costs one hash and closes code interception
// independently of the secret, which is worth having when the redirect crosses a browser.
func (s *Server) oidcStart(w http.ResponseWriter, r *http.Request) {
if !s.oidcAvailable() {
http.Error(w, "no identity provider is configured on this server", http.StatusNotImplemented)
return
}
d, err := s.OIDC.Discover(r.Context())
if err != nil {
http.Error(w, "identity provider unreachable: "+err.Error(), http.StatusBadGateway)
return
}
state, verifier := randomToken(), randomToken()
challenge := sha256.Sum256([]byte(verifier))
// state and the PKCE verifier ride in one short-lived cookie: the callback must prove it
// belongs to the browser that started the flow, or an attacker can feed us their own code.
http.SetCookie(w, &http.Cookie{
Name: stateCookie, Value: state + "." + verifier, Path: "/",
HttpOnly: true, Secure: s.Secure, SameSite: http.SameSiteLaxMode, MaxAge: 600,
})
q := url.Values{
"response_type": {"code"},
"client_id": {s.OIDC.Config().ClientID},
"redirect_uri": {s.redirectURI()},
"scope": {"openid profile email"},
"state": {state},
"code_challenge": {base64.RawURLEncoding.EncodeToString(challenge[:])},
"code_challenge_method": {"S256"},
}
http.Redirect(w, r, d.AuthorizationEndpoint+"?"+q.Encode(), http.StatusSeeOther)
}
func (s *Server) redirectURI() string {
return strings.TrimRight(s.BaseURL, "/") + "/admin/callback"
}
func (s *Server) oidcCallback(w http.ResponseWriter, r *http.Request) {
if !s.oidcAvailable() {
http.Error(w, "no identity provider configured", http.StatusNotImplemented)
return
}
c, err := r.Cookie(stateCookie)
if err != nil {
http.Error(w, "sign-in did not start here — try again from the login page", http.StatusBadRequest)
return
}
http.SetCookie(w, &http.Cookie{Name: stateCookie, Value: "", Path: "/", MaxAge: -1})
state, verifier, ok := strings.Cut(c.Value, ".")
if !ok || state == "" || r.URL.Query().Get("state") != state {
http.Error(w, "sign-in state did not match — start again", http.StatusBadRequest)
return
}
code := r.URL.Query().Get("code")
if code == "" {
http.Error(w, "no authorization code returned: "+r.URL.Query().Get("error"), http.StatusBadRequest)
return
}
idToken, err := s.exchange(r.Context(), code, verifier)
if err != nil {
slog.Info("admin oidc exchange failed", "err", err, "from", clientIP(r))
http.Error(w, "could not complete sign-in", http.StatusBadGateway)
return
}
claims, err := s.OIDC.Verify(r.Context(), idToken)
if err != nil {
slog.Info("admin oidc token rejected", "err", err, "from", clientIP(r))
http.Error(w, "the identity token was not accepted", http.StatusForbidden)
return
}
// Authentication and authorisation are answered separately here. Someone who is not in the
// admin group has still proved who they are, and their own uploads are their business to
// manage — refusing them a session outright, as this used to, left a legitimate account with
// no way to see or delete the data it had sent.
admin := s.OIDC.IsAdmin(claims)
slog.Info("login", "account", claims.AccountID(), "method", "oidc", "admin", admin,
"from", clientIP(r))
s.setSession(w, claims.AccountID(), claims.Display(), admin)
http.Redirect(w, r, "/", http.StatusSeeOther)
}
// exchange trades the authorization code for tokens at the IdP.
func (s *Server) exchange(ctx context.Context, code, verifier string) (string, error) {
d, err := s.OIDC.Discover(ctx)
if err != nil {
return "", err
}
form := url.Values{
"grant_type": {"authorization_code"},
"code": {code},
"redirect_uri": {s.redirectURI()},
"client_id": {s.OIDC.Config().ClientID},
"code_verifier": {verifier},
}
if s.ClientSecret != "" {
form.Set("client_secret", s.ClientSecret)
}
req, err := http.NewRequestWithContext(ctx, http.MethodPost, d.TokenEndpoint,
strings.NewReader(form.Encode()))
if err != nil {
return "", err
}
req.Header.Set("Content-Type", "application/x-www-form-urlencoded")
resp, err := (&http.Client{Timeout: 15 * time.Second}).Do(req)
if err != nil {
return "", err
}
defer resp.Body.Close()
body, _ := io.ReadAll(io.LimitReader(resp.Body, 1<<20))
if resp.StatusCode != http.StatusOK {
return "", fmt.Errorf("token endpoint: %s: %s", resp.Status, strings.TrimSpace(string(body)))
}
var tok struct {
IDToken string `json:"id_token"`
}
if err := json.Unmarshal(body, &tok); err != nil {
return "", err
}
if tok.IDToken == "" {
return "", fmt.Errorf("token endpoint returned no id_token")
}
return tok.IDToken, nil
}
func randomToken() string {
b := make([]byte, 32)
_, _ = rand.Read(b)
return base64.RawURLEncoding.EncodeToString(b)
}
// clientIP is for logs only. X-Forwarded-For is deliberately ignored: nothing is meant to sit in
// front of this listener, so a header claiming otherwise is a caller's assertion about itself.
func clientIP(r *http.Request) string {
if i := strings.LastIndex(r.RemoteAddr, ":"); i > 0 {
return r.RemoteAddr[:i]
}
return r.RemoteAddr
}
@@ -0,0 +1,88 @@
// SPDX-FileCopyrightText: 2026 Echolot contributors
// SPDX-License-Identifier: GPL-3.0-or-later
package adminui
import (
"encoding/json"
"net/http"
"net/http/httptest"
"strings"
"testing"
"time"
"echo-lot.app/server/internal/adminauth"
)
// tokenFixture wires just enough of the Server for the mint endpoint: a break-glass admin and
// a stand-in EnrollLink (the real one belongs to the control server, injected the same way).
func tokenFixture(t *testing.T) *Server {
t.Helper()
s, _, _, _ := fixture(t)
secret, err := s.Store.SessionSecret()
if err != nil {
t.Fatal(err)
}
s.Sessions = adminauth.NewSessions(secret, time.Hour)
s.Throttle = adminauth.NewThrottle()
cred, err := adminauth.NewCredential("admin", "a-long-test-password")
if err != nil {
t.Fatal(err)
}
if err := s.Store.SetLocalAdmin(cred); err != nil {
t.Fatal(err)
}
s.EnrollLink = func(tok string) string { return "echolot://enroll?v=1&t=" + tok }
return s
}
func TestEnrollTokensAPISpecShape(t *testing.T) {
h := tokenFixture(t).Handler()
// No credentials → 401 with a challenge, never a token.
req := httptest.NewRequest("POST", "/admin/enroll-tokens?note=phone", nil)
rec := httptest.NewRecorder()
h.ServeHTTP(rec, req)
if rec.Code != http.StatusUnauthorized || rec.Header().Get("WWW-Authenticate") == "" {
t.Fatalf("unauthenticated: code=%d", rec.Code)
}
// Wrong password → still 401.
req = httptest.NewRequest("POST", "/admin/enroll-tokens", nil)
req.SetBasicAuth("admin", "wrong")
rec = httptest.NewRecorder()
h.ServeHTTP(rec, req)
if rec.Code != http.StatusUnauthorized {
t.Fatalf("bad password: code=%d, want 401", rec.Code)
}
// Basic + Accept: application/json → the §2.1 shape.
req = httptest.NewRequest("POST", "/admin/enroll-tokens?note=phone", nil)
req.SetBasicAuth("admin", "a-long-test-password")
req.Header.Set("Accept", "application/json")
rec = httptest.NewRecorder()
h.ServeHTTP(rec, req)
if rec.Code != http.StatusOK {
t.Fatalf("mint: code=%d body=%s", rec.Code, rec.Body.String())
}
var body struct {
Token string `json:"token"`
ExpiresS int `json:"expires_in_s"`
EnrollURI string `json:"enroll_uri"`
}
if err := json.Unmarshal(rec.Body.Bytes(), &body); err != nil {
t.Fatal(err)
}
if body.Token == "" || body.ExpiresS != 86400 || !strings.HasPrefix(body.EnrollURI, "echolot://enroll?") {
t.Fatalf("spec shape violated: %+v", body)
}
// Without Accept: the browser flow — redirect to the QR page, link in the query.
req = httptest.NewRequest("POST", "/admin/enroll-tokens", nil)
req.SetBasicAuth("admin", "a-long-test-password")
rec = httptest.NewRecorder()
h.ServeHTTP(rec, req)
if rec.Code != http.StatusSeeOther || !strings.HasPrefix(rec.Header().Get("Location"), "/devices?link=") {
t.Fatalf("html flow: code=%d location=%q", rec.Code, rec.Header().Get("Location"))
}
}
+273
View File
@@ -0,0 +1,273 @@
// SPDX-FileCopyrightText: 2026 Echolot contributors
// SPDX-License-Identifier: GPL-3.0-or-later
package adminui
import (
"encoding/json"
"html/template"
"log/slog"
"net/http"
"net/url"
"sort"
"strings"
"time"
"echo-lot.app/server/internal/adminauth"
"echo-lot.app/server/internal/runs"
"echo-lot.app/server/internal/store"
)
// visibleDevices returns the devices a session may see: everything for an administrator, and for
// anyone else the devices linked to their own account.
//
// Every page goes through this rather than filtering for itself. Scoping applied per-page is
// scoping that will be missing from the next page someone adds, and the failure is silent — a
// listing that quietly shows other people's uploads looks exactly like one that does not.
func (s *Server) visibleDevices(sess *adminauth.Session) []store.Device {
all := s.Store.Devices()
if sess.Admin {
return all
}
owned := make(map[string]bool)
for _, id := range s.Store.DeviceIDsForAccount(sess.Subject) {
owned[id] = true
}
out := make([]store.Device, 0, len(owned))
for _, d := range all {
if owned[d.ID] {
out = append(out, d)
}
}
return out
}
// mayTouchRun reports whether this session may read or delete a given run.
//
// Checked against the device list rather than against the run's own metadata, so an unlinked or
// revoked device stops granting access the moment the link is gone.
func (s *Server) mayTouchRun(sess *adminauth.Session, device string) bool {
if sess.Admin {
return true
}
for _, d := range s.visibleDevices(sess) {
if d.ID == device {
return true
}
}
return false
}
func (s *Server) loginForm(w http.ResponseWriter, r *http.Request) {
if s.session(r) != nil {
http.Redirect(w, r, "/", http.StatusSeeOther)
return
}
s.render(w, r, "login", map[string]any{
"OIDC": s.oidcAvailable(),
"LocalSet": s.Store.LocalAdmin() != nil,
"AdminUser": s.AdminUser,
})
}
func (s *Server) dashboard(w http.ResponseWriter, r *http.Request, sess *adminauth.Session) {
devices := s.visibleDevices(sess)
linked := 0
for _, d := range devices {
if d.LinkedToAccount() {
linked++
}
}
var selftest any
// The self-test describes the server's own health, which is an operator's concern; a user
// looking at their uploads has no use for it and no ability to act on it.
if s.SelfTest != nil && sess.Admin {
selftest = s.SelfTest()
}
s.render(w, r, "dashboard", map[string]any{
"Session": sess,
"CSRF": s.csrfToken(sess),
"Devices": len(devices),
"Linked": linked,
"Runs": s.totalRuns(devices),
"SelfTest": selftest,
"Version": s.Version,
"Admin": sess.Admin,
})
}
func (s *Server) totalRuns(devices []store.Device) int {
if s.Runs == nil {
return 0
}
n := 0
for _, d := range devices {
n += len(s.Runs.List(d.ID))
}
return n
}
func (s *Server) devices(w http.ResponseWriter, r *http.Request, sess *adminauth.Session) {
devices := s.visibleDevices(sess)
// Newest first: the device someone is looking for is almost always the one just enrolled.
sort.Slice(devices, func(i, j int) bool { return devices[i].Enrolled.After(devices[j].Enrolled) })
type row struct {
store.Device
Runs int
}
rows := make([]row, 0, len(devices))
for _, d := range devices {
n := 0
if s.Runs != nil {
n = len(s.Runs.List(d.ID))
}
rows = append(rows, row{Device: d, Runs: n})
}
// html/template rewrites an href whose scheme it does not recognise to "#ZgotmplZ", so the
// enrollment link rendered as a dead anchor that did nothing when tapped — silently, since the
// markup looks fine and only the sanitised attribute gives it away.
//
// Marking it template.URL opts out of that sanitising, which is only safe because the shape is
// checked first: this value arrives in a query parameter, so without the check a crafted
// /devices?link=javascript:… would put a script URL straight into the page.
link := r.URL.Query().Get("link")
var href template.URL
if strings.HasPrefix(link, "echolot://enroll?") {
href = template.URL(link)
}
s.render(w, r, "devices", map[string]any{
"Session": sess, "CSRF": s.csrfToken(sess), "Rows": rows,
// Rendered from the same validated value as the href, so a rejected link produces neither.
"Link": link, "LinkHref": href, "LinkQR": qrSVG(string(href)), "Admin": sess.Admin,
})
}
func (s *Server) revokeDevice(w http.ResponseWriter, r *http.Request, sess *adminauth.Session) {
id := r.PathValue("id")
if err := s.Store.DeleteDevice(id); err != nil {
http.Error(w, err.Error(), http.StatusBadRequest)
return
}
// Worth a log line: revoking a device is destructive, immediate, and someone will eventually
// want to know who did it and when.
slog.Info("device revoked", "device", id, "by", sess.Subject)
http.Redirect(w, r, "/devices", http.StatusSeeOther)
}
func (s *Server) mintToken(w http.ResponseWriter, r *http.Request, sess *adminauth.Session) {
tok, err := s.Store.NewEnrollToken(24*time.Hour, "admin-ui")
if err != nil {
http.Error(w, err.Error(), http.StatusInternalServerError)
return
}
slog.Info("enrolment token minted", "by", sess.Subject)
// The whole link, not the bare token: it carries the URL and the pin as well, and assembling
// those by hand is where an operator gets a pin wrong by one character.
http.Redirect(w, r, "/devices?link="+url.QueryEscape(s.EnrollLink(tok)), http.StatusSeeOther)
}
// enrollTokensAPI is POST /admin/enroll-tokens, the endpoint the spec's §2.1 example names.
// Content-negotiated: Accept: application/json gets the spec shape {token, expires_in_s,
// enroll_uri}; anything else (a browser) gets the same redirect-to-QR flow as the form above,
// so the one path serves both audiences.
func (s *Server) enrollTokensAPI(w http.ResponseWriter, r *http.Request) {
subject, ok := s.apiAdmin(w, r)
if !ok {
return
}
note := r.URL.Query().Get("note")
if note == "" {
note = "admin-api"
}
const ttl = 24 * time.Hour
tok, err := s.Store.NewEnrollToken(ttl, note)
if err != nil {
http.Error(w, err.Error(), http.StatusInternalServerError)
return
}
slog.Info("enrolment token minted", "by", subject, "note", note)
if !strings.Contains(r.Header.Get("Accept"), "application/json") {
http.Redirect(w, r, "/devices?link="+url.QueryEscape(s.EnrollLink(tok)), http.StatusSeeOther)
return
}
w.Header().Set("Content-Type", "application/json")
// The whole link, not the bare token (§2.1): the server is the only party holding URL, pin
// and token at once, and a hand-assembled pin wrong by one character fails as an inscrutable
// TLS error later rather than loudly here.
_ = json.NewEncoder(w).Encode(map[string]any{
"token": tok,
"expires_in_s": int(ttl.Seconds()),
"enroll_uri": s.EnrollLink(tok),
})
}
// EnrollLink is supplied by the caller so this package does not need the control server's pin.
var _ = 0
func (s *Server) runsList(w http.ResponseWriter, r *http.Request, sess *adminauth.Session) {
type row struct {
runs.Meta
DeviceName string
}
var rows []row
for _, d := range s.visibleDevices(sess) {
if s.Runs == nil {
break
}
name := d.Name
if name == "" {
name = d.ID
}
for _, m := range s.Runs.List(d.ID) {
rows = append(rows, row{Meta: m, DeviceName: name})
}
}
sort.Slice(rows, func(i, j int) bool { return rows[i].UploadedAt.After(rows[j].UploadedAt) })
if len(rows) > 200 {
rows = rows[:200] // a page, not the archive; the count is on the dashboard
}
s.render(w, r, "runs", map[string]any{
"Session": sess, "CSRF": s.csrfToken(sess), "Rows": rows, "Admin": sess.Admin,
})
}
func (s *Server) runView(w http.ResponseWriter, r *http.Request, sess *adminauth.Session) {
// 404 rather than 403 for someone else's run: a distinguishable "you may not see this" tells
// an unauthorised caller that the run exists, which is itself something they should not learn.
if !s.mayTouchRun(sess, r.PathValue("device")) {
http.NotFound(w, r)
return
}
body, err := s.Runs.Get(r.PathValue("device"), r.PathValue("id"))
if err != nil {
http.NotFound(w, r)
return
}
// Re-indented for reading, but otherwise exactly what was stored. An admin sees the document
// at the privacy level its uploader chose — there is nothing here that can un-redact it.
var pretty json.RawMessage = body
out, err := json.MarshalIndent(json.RawMessage(pretty), "", " ")
if err != nil {
out = body
}
s.render(w, r, "run", map[string]any{
"Session": sess, "CSRF": s.csrfToken(sess), "Admin": sess.Admin,
"Device": r.PathValue("device"), "ID": r.PathValue("id"),
"JSON": string(out),
})
}
func (s *Server) runDelete(w http.ResponseWriter, r *http.Request, sess *adminauth.Session) {
device, id := r.PathValue("device"), r.PathValue("id")
if !s.mayTouchRun(sess, device) {
http.NotFound(w, r)
return
}
if err := s.Runs.Delete(device, id); err != nil {
http.Error(w, err.Error(), http.StatusInternalServerError)
return
}
slog.Info("run deleted", "device", device, "run", id, "by", sess.Subject)
http.Redirect(w, r, "/runs", http.StatusSeeOther)
}
+58
View File
@@ -0,0 +1,58 @@
// SPDX-FileCopyrightText: 2026 Echolot contributors
// SPDX-License-Identifier: GPL-3.0-or-later
package adminui
import (
"fmt"
"html/template"
"strings"
qrcode "github.com/skip2/go-qrcode"
)
// qrSVG renders text as an inline SVG QR code, or empty if it will not encode.
//
// Inline SVG rather than a PNG data: URI because the page's CSP is `default-src 'none'` and means
// it. A data: image would need img-src opened up; markup needs nothing, and the QR is generated
// here from a boolean matrix, so nothing a user supplied reaches the output.
//
// Drawn as one path rather than a rect per module: a link of this length encodes to roughly 60x60
// modules, and two thousand elements is a lot of DOM for a picture of a square.
func qrSVG(text string) template.HTML {
if text == "" {
return ""
}
// Medium recovery: a phone camera reading a screen has no dirt or creases to survive, and
// lower recovery keeps the module count down, which keeps it scannable on a small display.
q, err := qrcode.New(text, qrcode.Medium)
if err != nil {
return "" // too long to encode; the link text below it still works
}
bitmap := q.Bitmap()
n := len(bitmap)
if n == 0 {
return ""
}
var path strings.Builder
for y, row := range bitmap {
for x, dark := range row {
if dark {
fmt.Fprintf(&path, "M%d %dh1v1h-1z", x, y)
}
}
}
// A quiet zone is part of the spec, not decoration: without it a scanner cannot find the
// symbol's edges against whatever is next to it on the page.
var out strings.Builder
fmt.Fprintf(&out,
`<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 %d %d" `+
`width="240" height="240" shape-rendering="crispEdges" role="img" `+
`aria-label="Enrolment link as a QR code">`+
`<rect width="%d" height="%d" fill="#fff"/>`+
`<path d="%s" fill="#000"/></svg>`,
n, n, n, n, path.String())
return template.HTML(out.String())
}
+24
View File
@@ -0,0 +1,24 @@
package adminui
import (
"strings"
"testing"
)
func TestQrSVGEncodesAnEnrolmentLink(t *testing.T) {
link := "echolot://enroll?v=1&u=https%3A%2F%2Ffmr.echo-lot.app&p=pin-sha256%3AzRV9qkiLnRexAeh4RrSfJzbPWO%2BU%2F2Oj2%2FNVM%2FKfXlg%3D&t=20e6ccaa2a028dc0aab16442c258d1b8eadb5794682905fe"
out := string(qrSVG(link))
if !strings.HasPrefix(out, "<svg") || !strings.Contains(out, "<path d=\"M") {
t.Fatalf("expected an svg with a path, got %.80q", out)
}
// A quiet zone is part of the symbol; without it scanners cannot find its edges.
if !strings.Contains(out, `fill="#fff"`) {
t.Error("no light background rendered")
}
}
func TestQrSVGEmptyForNoLink(t *testing.T) {
if qrSVG("") != "" {
t.Error("no link should render no code")
}
}
+420
View File
@@ -0,0 +1,420 @@
// SPDX-FileCopyrightText: 2026 Echolot contributors
// SPDX-License-Identifier: GPL-3.0-or-later
package adminui
import (
"bytes"
"html/template"
"log/slog"
"net/http"
"strings"
)
// Templates are parsed once at start. html/template escapes by context, which is what makes it
// safe to render device names and finding text that ultimately arrived over a network.
var tpl = template.Must(template.New("base").Funcs(template.FuncMap{
"kb": func(n int64) int64 { return n / 1024 },
// verdictClass keeps an uploaded string out of the class attribute. The verdict arrives inside
// a document a device sent us, so interpolating it into markup would be trusting a stranger's
// text with a place in the stylesheet; mapping through a fixed set costs nothing and closes it.
"verdictClass": func(v string) string {
switch strings.ToLower(v) {
case "green", "yellow", "red", "inconclusive":
return "v-" + strings.ToLower(v)
default:
return "v-unknown"
}
},
// verdictLabel says what the light means rather than what it is called. "yellow" is a colour;
// "worth a look" is a finding, and the reader is here to act on it.
"verdictLabel": func(v string) string {
switch strings.ToLower(v) {
case "green":
return "clean"
case "yellow":
return "worth a look"
case "red":
return "faults found"
case "inconclusive":
return "inconclusive"
default:
return "not recorded"
}
},
}).Parse(baseHTML))
func (s *Server) render(w http.ResponseWriter, r *http.Request, page string, data map[string]any) {
data["Page"] = page
var buf bytes.Buffer
if err := tpl.Execute(&buf, data); err != nil {
slog.Error("admin template", "page", page, "err", err)
http.Error(w, "template error", http.StatusInternalServerError)
return
}
w.Header().Set("Content-Type", "text/html; charset=utf-8")
// There is no script here and nothing loaded from anywhere else, so a strict policy costs
// nothing and closes injected-script attacks even if an escaping bug ever slips through.
w.Header().Set("Content-Security-Policy", "default-src 'none'; style-src 'unsafe-inline'; form-action 'self'")
w.Header().Set("Referrer-Policy", "no-referrer")
w.Header().Set("X-Content-Type-Options", "nosniff")
_, _ = buf.WriteTo(w)
}
// The visual language is an echo sounder's, which is what the name means: an instrument that emits
// a ping and reads what comes back. That gives the palette (the colours of a water column rather
// than a neutral near-black), the type (machine-set, because an instrument's readings are), and
// the one piece of real ornament — a trace of returns across time on the runs page.
//
// No web fonts: the CSP forbids loading anything, and shipping font files with a single Go binary
// would trade the property that makes this server pleasant to run for a typeface. So the character
// has to come from treatment — tracking, case, scale, rules — rather than from novel letterforms.
//
// Tables become stacked records below 46rem rather than scrolling sideways. That is not a fallback:
// a sounding log prints as label-and-value pairs, and on a phone that form is easier to read than
// any table, so the mobile layout is the more faithful one of the two.
const baseHTML = `<!doctype html>
<html lang="en"><head><meta charset="utf-8">
<meta name="viewport" content="width=device-width,initial-scale=1">
<title>Echolot &mdash; {{.Page}}</title>
<style>
:root{
--abyss:#071419; --hull:#0d2028; --raise:#122a34; --rule:#17323d;
--ink:#dce8ea; --dim:#7d97a1; --trace:#6fc9b4;
--green:#57ad82; --amber:#cf9b3c; --red:#c25757; --slate:#62767f;
--mono:ui-monospace,"SF Mono","IBM Plex Mono","JetBrains Mono",Menlo,Consolas,monospace;
--prose:system-ui,-apple-system,"Segoe UI",sans-serif;
color-scheme:dark;
}
*{box-sizing:border-box}
body{margin:0;background:var(--abyss);color:var(--ink);
font:400 15px/1.55 var(--prose);-webkit-text-size-adjust:100%}
/* ---- masthead ------------------------------------------------------------------------ */
/* Wraps rather than overflows: a rigid row pushes the account and its sign-out button past
the edge of a phone screen, where they cannot be reached at all. */
.top{display:flex;flex-wrap:wrap;align-items:center;gap:.5rem 1.4rem;
padding:.85rem 1.1rem;background:var(--hull);border-bottom:1px solid var(--rule)}
.mark{font:600 .95rem/1 var(--mono);letter-spacing:.02em;margin:0;color:var(--ink)}
.mark span{color:var(--trace)}
.top nav{display:flex;flex-wrap:wrap;gap:.15rem .9rem}
.top nav a{font:500 .82rem/1 var(--mono);letter-spacing:.06em;color:var(--dim);
text-decoration:none;padding:.35rem 0;border-bottom:1px solid transparent}
.top nav a:hover{color:var(--ink)}
.top nav a[aria-current]{color:var(--trace);border-bottom-color:var(--trace)}
.who{margin-left:auto;display:flex;align-items:center;gap:.7rem;flex-wrap:wrap;
font:.78rem/1.3 var(--mono);color:var(--dim)}
main{padding:1.1rem;max-width:64rem}
/* ---- headings: a graduation mark, like a depth scale ------------------------------- */
h2{font:500 1.05rem/1.2 var(--mono);letter-spacing:-.01em;margin:1.4rem 0 .2rem;
padding-left:.7rem;border-left:2px solid var(--trace)}
h2:first-child{margin-top:0}
h3{font:500 .8rem/1 var(--mono);letter-spacing:.14em;text-transform:uppercase;
color:var(--dim);margin:0 0 .7rem}
.lede{color:var(--dim);font-size:.9rem;margin:.5rem 0 1rem;max-width:46rem}
/* ---- readout: how an instrument prints a value ------------------------------------- */
.readout{list-style:none;margin:0;padding:0}
.readout li{display:flex;align-items:baseline;gap:.5rem;padding:.3rem 0;
font:.85rem/1.4 var(--mono)}
.readout .k{color:var(--dim);white-space:nowrap}
/* The dotted leader is how a sounding log runs a label out to its value. It is also the thing
that lets a label and a number sit on one line at any width without a table. */
.readout .lead{flex:1 1 auto;min-width:1.5rem;align-self:center;height:1px;
background:repeating-linear-gradient(90deg,var(--rule) 0 2px,transparent 2px 5px)}
.readout .v{color:var(--ink);text-align:right;overflow-wrap:anywhere}
/* ---- the trace: one bar per run, oldest to newest ---------------------------------- */
/* The signature, and the only ornament here: an echo sounder draws returns against time, and
so does this. Rows arrive newest-first, so the strip is reversed in CSS rather than in Go. */
.trace{display:flex;flex-direction:row-reverse;justify-content:flex-end;align-items:flex-end;gap:2px;
height:3rem;padding:.7rem .8rem;background:var(--hull);
border:1px solid var(--rule);border-radius:3px;overflow:hidden}
.trace i{flex:1 1 3px;min-width:2px;max-width:9px;border-radius:1px;opacity:.9}
.trace .v-green{height:35%;background:var(--green)}
.trace .v-yellow{height:65%;background:var(--amber)}
.trace .v-red{height:100%;background:var(--red)}
.trace .v-inconclusive{height:22%;background:var(--slate)}
.trace .v-unknown{height:12%;background:var(--rule)}
.trace-key{display:flex;flex-wrap:wrap;gap:.3rem .9rem;margin:.45rem 0 0;
font:.72rem/1 var(--mono);letter-spacing:.05em;color:var(--dim)}
.trace-key b{font-weight:400;color:var(--dim)}
.trace-key em{font-style:normal;display:inline-block;width:.5rem;height:.5rem;
border-radius:1px;margin-right:.35rem;vertical-align:baseline;background:var(--rule)}
.trace-key em.v-green{background:var(--green)}
.trace-key em.v-yellow{background:var(--amber)}
.trace-key em.v-red{background:var(--red)}
.trace-key em.v-inconclusive{background:var(--slate)}
/* ---- records: tables that stack on a phone ----------------------------------------- */
.rec{border:1px solid var(--rule);border-radius:3px;background:var(--hull);
padding:.75rem .85rem;margin:.5rem 0}
.rec-head{display:flex;flex-wrap:wrap;align-items:baseline;gap:.5rem;
font:.85rem/1.3 var(--mono);margin-bottom:.35rem}
.rec-head .id{overflow-wrap:anywhere;color:var(--ink)}
.rec form{margin-top:.6rem}
/* Why a check matters is a sentence, so it is set as one full width under the row rather
than squeezed into a column, where it would wrap to a ribbon two words wide. */
.why{font:.85rem/1.5 var(--prose);color:var(--dim);margin-top:.45rem;max-width:52rem}
.tag{font:.68rem/1 var(--mono);letter-spacing:.1em;text-transform:uppercase;
padding:.24rem .45rem;border-radius:2px;border:1px solid currentColor;white-space:nowrap}
.v-green{color:var(--green)} .v-yellow{color:var(--amber)}
.v-red{color:var(--red)} .v-inconclusive{color:var(--slate)} .v-unknown{color:var(--dim)}
/* ---- panels, controls, states ------------------------------------------------------ */
.narrow{max-width:27rem}
.panel{background:var(--hull);border:1px solid var(--rule);border-radius:3px;
padding:.95rem 1rem;margin:.9rem 0;min-width:0}
/* White plate behind the code: a QR needs the light modules to actually be light, and this
page is dark. */
.qr{display:inline-block;background:#fff;padding:8px;border-radius:4px;margin:.2rem 0;line-height:0}
.qr svg{display:block;width:min(240px,60vw);height:auto}
.empty{border:1px dashed var(--rule);border-radius:3px;padding:1.4rem 1rem;
color:var(--dim);font-size:.9rem}
code,pre,.mono{font-family:var(--mono);font-size:.82rem}
code{overflow-wrap:anywhere;color:var(--trace)}
pre{background:#040d11;border:1px solid var(--rule);border-radius:3px;padding:.8rem;
overflow:auto;max-height:32rem;max-width:100%;color:var(--ink)}
a{color:var(--trace)}
button{font:500 .82rem/1 var(--mono);letter-spacing:.05em;background:var(--trace);
color:#04181a;border:0;border-radius:3px;padding:.55rem .9rem;cursor:pointer}
.btn{display:inline-block;font:500 .82rem/1 var(--mono);letter-spacing:.05em;
background:var(--trace);color:#04181a;border-radius:3px;padding:.55rem .9rem;
text-decoration:none}
button.plain{background:transparent;color:var(--dim);border:1px solid var(--rule)}
button.danger{background:transparent;color:var(--red);border:1px solid var(--red)}
button:hover{filter:brightness(1.08)}
input{font:.9rem var(--mono);background:#040d11;color:var(--ink);border:1px solid var(--rule);
border-radius:3px;padding:.55rem .6rem;max-width:100%;width:100%}
label{display:block;font:.72rem/1 var(--mono);letter-spacing:.12em;text-transform:uppercase;
color:var(--dim);margin:.9rem 0 .3rem}
.err{border:1px solid var(--red);color:var(--ink);background:rgba(194,87,87,.09);
padding:.6rem .75rem;border-radius:3px;font-size:.9rem}
.muted{color:var(--dim)}
form.inline{display:inline}
:focus-visible{outline:2px solid var(--trace);outline-offset:2px}
@media (prefers-reduced-motion:reduce){*{transition:none!important;animation:none!important}}
/* ---- above 46rem the records line up in columns ------------------------------------ */
@media (min-width:46rem){
.top{padding:.85rem 1.6rem}
main{padding:1.6rem}
.recs{margin:.8rem 0}
/* Every row shares one grid, so the columns agree across rows without a header or a table. */
.rec{display:grid;grid-template-columns:minmax(12.5rem,18rem) minmax(0,1fr) auto;gap:.35rem 1.4rem;
align-items:baseline;background:none;border:0;border-bottom:1px solid var(--rule);
border-radius:0;padding:.6rem 0;margin:0}
.rec-head{margin:0;flex-direction:column;align-items:flex-start;gap:.3rem}
.rec form{margin:0}
/* Widths follow the content: a device name needs room, a finding count does not. */
.rec .readout{display:grid;grid-template-columns:1.7fr .9fr .9fr 1.1fr;gap:.15rem 1.2rem}
.rec .readout li{padding:0}
.rec .readout .lead{display:none}
.rec .readout .v{text-align:left}
.open{white-space:nowrap}
/* Spans the full row: the sentence is the useful part, not a fourth column. */
.why{grid-column:1/-1;margin-top:.1rem}
}
</style></head><body>
<header class="top">
<h1 class="mark">echo<span>lot</span></h1>
{{if ne .Page "login"}}
<nav>
<a href="/"{{if eq .Page "dashboard"}} aria-current="page"{{end}}>overview</a>
<a href="/devices"{{if eq .Page "devices"}} aria-current="page"{{end}}>devices</a>
<a href="/runs"{{if eq .Page "runs"}} aria-current="page"{{end}}>runs</a>
</nav>
<span class="who">{{.Session.Display}}{{if not .Session.Admin}} &middot; your account{{end}}
<form method="post" action="/logout" class="inline"><button class="plain">Sign out</button></form>
</span>
{{end}}
</header>
<main>
{{if eq .Page "login"}}
<h2>Sign in</h2>
<p class="lede">This server keeps the measurements your devices have uploaded.</p>
{{if .OIDC}}
<p><a class="btn" href="/auth/start">Sign in with your identity provider</a></p>
{{end}}
{{if .LocalSet}}
<form method="post" action="/login" class="panel narrow">
<h3>Break-glass account</h3>
<label for="u">Username</label>
<input id="u" name="username" autocomplete="username" value="{{.AdminUser}}">
<label for="p">Password</label>
<input id="p" name="password" type="password" autocomplete="current-password">
<p><button>Sign in</button></p>
</form>
{{else}}
<p class="err">No break-glass account is set. Run
<code>echolot-server --set-admin-password</code> on the host to create one.</p>
{{end}}
{{else if eq .Page "dashboard"}}
<h2>{{if .Admin}}This server{{else}}Your account{{end}}</h2>
<ul class="readout panel">
<li><span class="k">{{if .Admin}}devices enrolled{{else}}your devices{{end}}</span>
<span class="lead"></span><span class="v">{{.Devices}}</span></li>
{{if .Admin}}
<li><span class="k">linked to an account</span>
<span class="lead"></span><span class="v">{{.Linked}}</span></li>
{{end}}
<li><span class="k">{{if .Admin}}runs stored{{else}}your runs{{end}}</span>
<span class="lead"></span><span class="v">{{.Runs}}</span></li>
{{if .Admin}}
<li><span class="k">server version</span>
<span class="lead"></span><span class="v">{{.Version}}</span></li>
{{end}}
</ul>
{{if not .Admin}}
<div class="panel">
<p>You can see every device you have signed in on, read everything they have uploaded, and
delete any of it.</p>
<p class="muted">Enrolling devices, revoking them, and reading other people's uploads need an
administrator account.</p>
</div>
{{end}}
{{with .SelfTest}}
<h2>Self-test</h2>
<p class="lede">What this server can measure from where it stands, checked at startup. A
capability missing here is missing from every run this server takes part in &mdash; so a
client asking for that measurement gets nothing, rather than a wrong answer.</p>
<ul class="readout panel">
<li><span class="k">kernel settings</span><span class="lead"></span>
<span class="v {{if .SysctlOK}}v-green{{else}}v-yellow{{end}}">{{if .SysctlOK}}as needed{{else}}need attention{{end}}</span></li>
<li><span class="k">egress path MTU</span><span class="lead"></span>
<span class="v {{if .MTUOK}}v-green{{else}}v-yellow{{end}}">{{if .MTUOK}}full 1500{{else}}reduced{{end}}</span></li>
</ul>
{{if .Sysctls}}
<h3>Kernel settings</h3>
<div class="recs">
{{range .Sysctls}}
<div class="rec">
<div class="rec-head"><span class="id">{{.Name}}</span>
<span class="tag {{if eq .Severity "ok"}}v-green{{else}}v-yellow{{end}}">{{.Severity}}</span></div>
<ul class="readout">
<li><span class="k">found</span><span class="lead"></span><span class="v">{{.Got}}</span></li>
<li><span class="k">wanted</span><span class="lead"></span><span class="v">{{.Want}}</span></li>
</ul>
<div class="why">{{.Why}}</div>
</div>
{{end}}
</div>
{{end}}
{{if .EgressMTU}}
<h3>Egress path MTU</h3>
<div class="recs">
{{range .EgressMTU}}
<div class="rec">
<div class="rec-head"><span class="id">{{.Target}}</span>
<span class="tag {{if .FullMTU}}v-green{{else}}v-yellow{{end}}">{{if .FullMTU}}full{{else}}reduced{{end}}</span></div>
<ul class="readout">
<li><span class="k">discovered</span><span class="lead"></span>
<span class="v">{{if .DiscoveredMTU}}{{.DiscoveredMTU}} bytes{{else}}not measured{{end}}</span></li>
</ul>
{{with .Err}}<div class="why">{{.}}</div>{{end}}
</div>
{{end}}
</div>
{{end}}
{{end}}
{{else if eq .Page "devices"}}
<h2>{{if .Admin}}Devices{{else}}Your devices{{end}}</h2>
{{with .Link}}
<div class="panel">
<h3>Enrolment link</h3>
<p class="lede">Single use, valid 24 hours. Treat it like a password until it is spent.</p>
<!-- On the phone being enrolled this is the whole procedure: the scheme is registered by the
app, so following the link hands it the token directly. Copying a 200-character string
between two devices is the step that goes wrong, and it does not have to happen at all. -->
{{with $.LinkHref}}<p><a class="btn" href="{{.}}">Open in the Echolot app</a></p>{{end}}
<p class="muted">Works on the phone you are enrolling. From another device, scan this:</p>
{{with $.LinkQR}}<div class="qr">{{.}}</div>{{end}}
<p class="muted">Or copy the link into the app's enrolment field, or deliver it over adb.</p>
<p><code>{{.}}</code></p>
<p class="muted mono">adb shell am start -a android.intent.action.VIEW -d "{{.}}"</p>
</div>
{{end}}
{{if .Admin}}
<form method="post" action="/enroll-tokens">
<input type="hidden" name="csrf" value="{{.CSRF}}">
<button>Create enrolment link</button>
</form>
{{end}}
{{if .Rows}}<div class="recs">
{{range .Rows}}
<div class="rec">
<div class="rec-head">
<span class="id">{{if .Name}}{{.Name}}{{else}}{{.ID}}{{end}}</span>
{{if .LinkedToAccount}}<span class="tag v-green">{{.AccountName}}</span>
{{else}}<span class="tag v-unknown">no account</span>{{end}}
</div>
<ul class="readout">
<li><span class="k">device</span><span class="lead"></span><span class="v">{{.ID}}</span></li>
<li><span class="k">enrolled</span><span class="lead"></span>
<span class="v">{{.Enrolled.Format "2006-01-02 15:04"}}</span></li>
<li><span class="k">runs</span><span class="lead"></span><span class="v">{{.Runs}}</span></li>
</ul>
{{if $.Admin}}
<form method="post" action="/devices/{{.ID}}/revoke">
<input type="hidden" name="csrf" value="{{$.CSRF}}">
<button class="danger">Revoke</button>
</form>
{{end}}
</div>
{{end}}
</div>{{else}}
<p class="empty">{{if .Admin}}No devices yet. Create an enrolment link and open it on the phone
you want to measure from.{{else}}No devices yet. Sign in from the Echolot app on your phone to
link one to this account.{{end}}</p>
{{end}}
{{else if eq .Page "runs"}}
<h2>{{if .Admin}}Uploaded runs{{else}}Your uploaded runs{{end}}</h2>
<p class="lede">Each run is shown exactly as it arrived, at the privacy level its uploader chose.
Nothing here can un-redact one.</p>
{{if .Rows}}
<div class="trace">{{range .Rows}}<i class="{{verdictClass .Verdict}}"></i>{{end}}</div>
<p class="trace-key"><b>oldest &rarr; newest</b>
<b><em class="v-green"></em>clean</b>
<b><em class="v-yellow"></em>worth a look</b>
<b><em class="v-red"></em>faults</b>
<b><em class="v-inconclusive"></em>inconclusive</b></p>
{{end}}
{{if .Rows}}<div class="recs">
{{range .Rows}}
<div class="rec">
<div class="rec-head">
<span class="id">{{.UploadedAt.Format "2006-01-02 15:04"}}</span>
<span class="tag {{verdictClass .Verdict}}">{{verdictLabel .Verdict}}</span>
</div>
<ul class="readout">
<li><span class="k">device</span><span class="lead"></span><span class="v">{{.DeviceName}}</span></li>
<li><span class="k">findings</span><span class="lead"></span><span class="v">{{.FindingCount}}</span></li>
<li><span class="k">size</span><span class="lead"></span><span class="v">{{kb .SizeBytes}} kB</span></li>
<li><span class="k">privacy</span><span class="lead"></span><span class="v">{{.Anonymization}}</span></li>
</ul>
<div class="open"><a href="/runs/{{.DeviceID}}/{{.ID}}">Open run</a></div>
</div>
{{end}}
</div>{{else}}
<p class="empty">Nothing uploaded yet. Take a measurement in the app and upload it; it will
appear here.</p>
{{end}}
{{else if eq .Page "run"}}
<h2>Run {{.ID}}</h2>
<p class="lede">The document as stored, indented for reading. Nothing has been added or removed.</p>
<form method="post" action="/runs/{{.Device}}/{{.ID}}/delete">
<input type="hidden" name="csrf" value="{{.CSRF}}">
<button class="danger">Delete this run</button>
</form>
<pre>{{.JSON}}</pre>
{{end}}
</main></body></html>
`
+119
View File
@@ -0,0 +1,119 @@
// SPDX-FileCopyrightText: 2026 Echolot contributors
// SPDX-License-Identifier: GPL-3.0-or-later
package adminui
import (
"testing"
"time"
"echo-lot.app/server/internal/adminauth"
"echo-lot.app/server/internal/runs"
"echo-lot.app/server/internal/store"
)
// Two accounts, one device each, plus an unlinked device nobody owns.
func fixture(t *testing.T) (*Server, string, string, string) {
t.Helper()
dir := t.TempDir()
st, err := store.Open(dir)
if err != nil {
t.Fatal(err)
}
rs, err := runs.Open(dir, runs.DefaultPolicy())
if err != nil {
t.Fatal(err)
}
enroll := func(name string) string {
tok, err := st.NewEnrollToken(time.Hour, "test")
if err != nil {
t.Fatal(err)
}
d, err := st.Redeem(tok, name)
if err != nil {
t.Fatal(err)
}
return d.ID
}
mine, theirs, orphan := enroll("mine"), enroll("theirs"), enroll("orphan")
if err := st.LinkAccount(mine, "oidc#me", "Me"); err != nil {
t.Fatal(err)
}
if err := st.LinkAccount(theirs, "oidc#you", "You"); err != nil {
t.Fatal(err)
}
for _, d := range []string{mine, theirs, orphan} {
if _, err := rs.Put(d, []byte(`{"run":{"id":"r"}}`), true); err != nil {
t.Fatal(err)
}
}
return &Server{Store: st, Runs: rs}, mine, theirs, orphan
}
func user() *adminauth.Session { return &adminauth.Session{Subject: "oidc#me", Display: "Me"} }
func admin() *adminauth.Session { return &adminauth.Session{Subject: "local:a", Admin: true} }
func TestVisibleDevicesScopesToAccount(t *testing.T) {
s, mine, theirs, orphan := fixture(t)
got := s.visibleDevices(user())
if len(got) != 1 || got[0].ID != mine {
t.Fatalf("a user should see only their own device, got %+v", got)
}
all := s.visibleDevices(admin())
if len(all) != 3 {
t.Fatalf("an admin should see every device, got %d", len(all))
}
_ = theirs
_ = orphan
}
func TestUnlinkedDevicesBelongToNobody(t *testing.T) {
// An enrolled but never-signed-in device is not "everyone's" — a user must not inherit it
// just because no account claimed it.
s, _, _, orphan := fixture(t)
if s.mayTouchRun(user(), orphan) {
t.Fatal("an unlinked device was treated as the user's own")
}
}
func TestRunAccessFollowsDeviceOwnership(t *testing.T) {
s, mine, theirs, _ := fixture(t)
if !s.mayTouchRun(user(), mine) {
t.Fatal("a user cannot reach their own run")
}
if s.mayTouchRun(user(), theirs) {
t.Fatal("a user reached someone else's run")
}
if !s.mayTouchRun(admin(), theirs) {
t.Fatal("an admin should reach any run")
}
}
func TestAccessEndsWhenTheLinkDoes(t *testing.T) {
// Ownership is read from the device list on every request rather than captured at sign-in,
// so unlinking takes effect immediately — a session issued while linked must not keep working.
s, mine, _, _ := fixture(t)
sess := user()
if !s.mayTouchRun(sess, mine) {
t.Fatal("precondition: the device should start out owned")
}
if err := s.Store.LinkAccount(mine, "", ""); err != nil {
t.Fatal(err)
}
if s.mayTouchRun(sess, mine) {
t.Fatal("access survived the account link being removed")
}
}
func TestEmptySubjectMatchesNothing(t *testing.T) {
// The dangerous degenerate case: a session with no subject must own nothing, not everything
// that happens to have an empty account id.
s, _, _, _ := fixture(t)
anon := &adminauth.Session{Subject: "", Display: ""}
if got := s.visibleDevices(anon); len(got) != 0 {
t.Fatalf("an empty subject matched %d devices", len(got))
}
}
+33 -6
View File
@@ -63,20 +63,24 @@ type Server struct {
nsName string // this server's own name for NS/authority answers nsName string // this server's own name for NS/authority answers
primaryV4 netip.Addr primaryV4 netip.Addr
primaryV6 netip.Addr primaryV6 netip.Addr
retention time.Duration // query-log age limit; <= 0 means only the ring cap bounds it
mu sync.Mutex mu sync.Mutex
log []Query // ring, newest last log []Query // ring, newest last
retainTo time.Time
} }
const logCap = 8192 const logCap = 8192
// New creates a server for zone (with or without trailing dot). nsName is the // New creates a server for zone (with or without trailing dot). nsName is the
// server's own hostname (for the zone's NS record); primary v4/v6 are this // server's own hostname (for the zone's NS record); primary v4/v6 are this
// host's addresses used to answer the zone apex / NS glue. // host's addresses used to answer the zone apex / NS glue. retention is how
func New(zone, nsName string, v4, v6 netip.Addr) *Server { // long logged queries are kept (spec §6; the privacy default is 24 h).
func New(zone, nsName string, v4, v6 netip.Addr, retention time.Duration) *Server {
z := strings.ToLower(strings.TrimSuffix(zone, ".")) + "." z := strings.ToLower(strings.TrimSuffix(zone, ".")) + "."
return &Server{zone: z, nsName: strings.TrimSuffix(nsName, ".") + ".", primaryV4: v4, primaryV6: v6} return &Server{
zone: z, nsName: strings.TrimSuffix(nsName, ".") + ".",
primaryV4: v4, primaryV6: v6, retention: retention,
}
} }
// RecentForPrefix returns logged queries whose qname contains ".<prefix>." // RecentForPrefix returns logged queries whose qname contains ".<prefix>."
@@ -84,6 +88,7 @@ func New(zone, nsName string, v4, v6 netip.Addr) *Server {
func (s *Server) RecentForPrefix(prefix string) []Query { func (s *Server) RecentForPrefix(prefix string) []Query {
s.mu.Lock() s.mu.Lock()
defer s.mu.Unlock() defer s.mu.Unlock()
s.dropExpiredLocked(time.Now().UTC())
needle := "." + strings.ToLower(prefix) + "." needle := "." + strings.ToLower(prefix) + "."
var out []Query var out []Query
for _, q := range s.log { for _, q := range s.log {
@@ -97,12 +102,34 @@ func (s *Server) RecentForPrefix(prefix string) []Query {
func (s *Server) record(q Query) { func (s *Server) record(q Query) {
s.mu.Lock() s.mu.Lock()
defer s.mu.Unlock() defer s.mu.Unlock()
s.dropExpiredLocked(q.At)
if len(s.log) >= logCap { if len(s.log) >= logCap {
s.log = s.log[1:] s.log = s.log[1:]
} }
s.log = append(s.log, q) s.log = append(s.log, q)
} }
// dropExpiredLocked enforces the retention window on the query log.
//
// The 24-hour retention was advertised as the privacy default (spec §6/§7) and then not
// enforced: the ring only bounded *count*, so on a quiet server a resolver's queries could sit
// in memory for weeks. Aged out on every write and every read — whichever comes first — so an
// idle log still forgets on schedule the moment anyone looks. Entries are appended in time
// order, so expiry is always a prefix of the slice.
func (s *Server) dropExpiredLocked(now time.Time) {
if s.retention <= 0 {
return
}
cutoff := now.Add(-s.retention)
i := 0
for i < len(s.log) && s.log[i].At.Before(cutoff) {
i++
}
if i > 0 {
s.log = append([]Query(nil), s.log[i:]...) // reallocate so the old backing array frees
}
}
// ServeUDP / ServeTCP run read loops; call one per bound address. // ServeUDP / ServeTCP run read loops; call one per bound address.
func (s *Server) ServeUDP(conn *net.UDPConn) error { func (s *Server) ServeUDP(conn *net.UDPConn) error {
buf := make([]byte, 1500) buf := make([]byte, 1500)
+28 -1
View File
@@ -8,6 +8,7 @@ import (
"net" "net"
"net/netip" "net/netip"
"testing" "testing"
"time"
) )
// buildQuery makes a single-question DNS query, optionally with an EDNS OPT. // buildQuery makes a single-question DNS query, optionally with an EDNS OPT.
@@ -65,7 +66,7 @@ func parseResponse(t *testing.T, resp []byte) (flags uint16, answers []ans) {
} }
func newTestServer() *Server { func newTestServer() *Server {
return New("c.echo-lot.app", "fmr", netip.MustParseAddr("192.0.2.1"), netip.MustParseAddr("2001:db8::1")) return New("c.echo-lot.app", "fmr", netip.MustParseAddr("192.0.2.1"), netip.MustParseAddr("2001:db8::1"), 24*time.Hour)
} }
func TestReferenceRecords(t *testing.T) { func TestReferenceRecords(t *testing.T) {
@@ -159,3 +160,29 @@ func TestOutOfZoneNXDomain(t *testing.T) {
t.Fatalf("out-of-zone should be NXDOMAIN, flags=%#x", flags) t.Fatalf("out-of-zone should be NXDOMAIN, flags=%#x", flags)
} }
} }
func TestQueryLogRetentionForgetsOldEntries(t *testing.T) {
s := newTestServer() // 24 h retention
now := time.Now().UTC()
s.record(Query{QName: "old.sess1.c.echo-lot.app", At: now.Add(-25 * time.Hour)})
s.record(Query{QName: "fresh.sess1.c.echo-lot.app", At: now})
got := s.RecentForPrefix("sess1")
if len(got) != 1 || got[0].QName != "fresh.sess1.c.echo-lot.app" {
t.Fatalf("retention not enforced: %+v", got)
}
// Reads must age the log too: an idle server still has to forget on schedule.
s.log[0].At = now.Add(-25 * time.Hour)
if got := s.RecentForPrefix("sess1"); len(got) != 0 {
t.Fatalf("read path did not expire entries: %+v", got)
}
}
func TestZeroRetentionKeepsEverything(t *testing.T) {
s := New("c.echo-lot.app", "fmr", netip.MustParseAddr("192.0.2.1"), netip.MustParseAddr("2001:db8::1"), 0)
s.record(Query{QName: "ancient.sess1.c.echo-lot.app", At: time.Now().UTC().Add(-1000 * time.Hour)})
if got := s.RecentForPrefix("sess1"); len(got) != 1 {
t.Fatal("retention 0 must mean 'ring cap only', not 'keep nothing'")
}
}
+124
View File
@@ -0,0 +1,124 @@
// SPDX-FileCopyrightText: 2026 Echolot contributors
// SPDX-License-Identifier: GPL-3.0-or-later
// Package certreload serves a TLS certificate that can be replaced on disk without a restart.
//
// The hard part of TLS is never termination — the stdlib does that — it is renewal. A certificate
// obtained from an ACME client expires every sixty days, and the usual arrangement is a renewal
// hook that reloads or restarts the service. That hook is the part that quietly fails: it works
// when it is written and then, months later, does not, and nobody notices until the certificate
// has already expired.
//
// So the certificate is re-read when the file changes. There is no hook to forget, no reload to
// coordinate, and a renewal that drops new files in place is picked up on the next handshake.
//
// Deliberately not used for the control plane. Clients pin that certificate's public key
// (probe-protocol.md §1), so swapping it at runtime would silently break every enrolled device —
// there the operator *should* have to think, and a restart is the least of what a key rotation
// costs. Two listeners, two different right answers.
package certreload
import (
"crypto/tls"
"fmt"
"os"
"sync"
"time"
)
// Reloader holds a certificate and refreshes it when the files on disk change.
type Reloader struct {
certPath, keyPath string
mu sync.RWMutex
cert *tls.Certificate
certMod time.Time
keyMod time.Time
checked time.Time
interval time.Duration
}
// New loads the pair once so a bad path fails at startup rather than at the first handshake,
// when the only symptom is a connection error at the far end.
func New(certPath, keyPath string) (*Reloader, error) {
r := &Reloader{certPath: certPath, keyPath: keyPath, interval: 30 * time.Second}
if err := r.load(); err != nil {
return nil, err
}
return r, nil
}
// TLSConfig returns a config that asks this reloader for the certificate on every handshake.
func (r *Reloader) TLSConfig() *tls.Config {
return &tls.Config{
MinVersion: tls.VersionTLS12,
GetCertificate: r.getCertificate,
}
}
func (r *Reloader) getCertificate(*tls.ClientHelloInfo) (*tls.Certificate, error) {
r.maybeReload()
r.mu.RLock()
defer r.mu.RUnlock()
if r.cert == nil {
return nil, fmt.Errorf("no certificate loaded")
}
return r.cert, nil
}
// maybeReload stats the files at most once per interval.
//
// Rate-limited because this runs on every handshake: a busy listener would otherwise stat twice
// per connection, and a certificate that is thirty seconds stale has never mattered to anyone.
func (r *Reloader) maybeReload() {
r.mu.RLock()
fresh := time.Since(r.checked) < r.interval
r.mu.RUnlock()
if fresh {
return
}
r.mu.Lock()
r.checked = time.Now()
certMod, keyMod := modTime(r.certPath), modTime(r.keyPath)
unchanged := certMod.Equal(r.certMod) && keyMod.Equal(r.keyMod)
r.mu.Unlock()
if unchanged {
return
}
// A failed reload keeps the certificate already in memory. Renewal tools write the two files
// separately, so there is a window where the pair does not match; serving the previous
// certificate through that window is strictly better than serving none.
_ = r.load()
}
func (r *Reloader) load() error {
cert, err := tls.LoadX509KeyPair(r.certPath, r.keyPath)
if err != nil {
return fmt.Errorf("loading %s / %s: %w", r.certPath, r.keyPath, err)
}
r.mu.Lock()
defer r.mu.Unlock()
r.cert = &cert
r.certMod, r.keyMod = modTime(r.certPath), modTime(r.keyPath)
return nil
}
// NotAfter is when the loaded certificate expires, for the admin UI to show and for a startup
// warning. An expiry an operator can see is one they can act on before a browser tells them.
func (r *Reloader) NotAfter() time.Time {
r.mu.RLock()
defer r.mu.RUnlock()
if r.cert == nil || r.cert.Leaf == nil {
return time.Time{}
}
return r.cert.Leaf.NotAfter
}
func modTime(path string) time.Time {
fi, err := os.Stat(path)
if err != nil {
return time.Time{}
}
return fi.ModTime()
}
@@ -0,0 +1,119 @@
// SPDX-FileCopyrightText: 2026 Echolot contributors
// SPDX-License-Identifier: GPL-3.0-or-later
package certreload
import (
"crypto/ecdsa"
"crypto/elliptic"
"crypto/rand"
"crypto/x509"
"crypto/x509/pkix"
"encoding/pem"
"math/big"
"os"
"path/filepath"
"testing"
"time"
)
func writePair(t *testing.T, dir, cn string) (string, string) {
t.Helper()
key, err := ecdsa.GenerateKey(elliptic.P256(), rand.Reader)
if err != nil {
t.Fatal(err)
}
tmpl := x509.Certificate{
SerialNumber: big.NewInt(time.Now().UnixNano()),
Subject: pkix.Name{CommonName: cn},
NotBefore: time.Now().Add(-time.Hour),
NotAfter: time.Now().Add(24 * time.Hour),
}
der, err := x509.CreateCertificate(rand.Reader, &tmpl, &tmpl, &key.PublicKey, key)
if err != nil {
t.Fatal(err)
}
certPath := filepath.Join(dir, "cert.pem")
keyPath := filepath.Join(dir, "key.pem")
cb := pem.EncodeToMemory(&pem.Block{Type: "CERTIFICATE", Bytes: der})
kb, _ := x509.MarshalECPrivateKey(key)
if err := os.WriteFile(certPath, cb, 0o600); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(keyPath, pem.EncodeToMemory(&pem.Block{Type: "EC PRIVATE KEY", Bytes: kb}), 0o600); err != nil {
t.Fatal(err)
}
return certPath, keyPath
}
func TestBadPathsFailAtStartupNotAtHandshake(t *testing.T) {
if _, err := New("/nonexistent/cert.pem", "/nonexistent/key.pem"); err == nil {
t.Fatal("a missing certificate was accepted; the failure would surface as an " +
"unexplained connection error at the client instead")
}
}
// The whole point: a renewal that drops new files in place is picked up without a restart and
// without a reload hook that can silently stop working.
func TestANewCertificateOnDiskIsPickedUp(t *testing.T) {
dir := t.TempDir()
certPath, keyPath := writePair(t, dir, "first")
r, err := New(certPath, keyPath)
if err != nil {
t.Fatal(err)
}
r.interval = 0 // check on every handshake, rather than waiting out the rate limit
got, err := r.getCertificate(nil)
if err != nil {
t.Fatal(err)
}
first := got.Leaf
time.Sleep(10 * time.Millisecond) // ensure a distinct mtime
writePair(t, dir, "second")
got, err = r.getCertificate(nil)
if err != nil {
t.Fatal(err)
}
if got.Leaf != nil && first != nil && got.Leaf.SerialNumber.Cmp(first.SerialNumber) == 0 {
t.Fatal("the replaced certificate was not picked up")
}
}
// Renewal tools write the certificate and the key separately, so there is a window where the two
// do not match. Serving the previous certificate through it beats serving none.
func TestAHalfWrittenPairKeepsTheOldCertificate(t *testing.T) {
dir := t.TempDir()
certPath, keyPath := writePair(t, dir, "good")
r, err := New(certPath, keyPath)
if err != nil {
t.Fatal(err)
}
r.interval = 0
time.Sleep(10 * time.Millisecond)
if err := os.WriteFile(certPath, []byte("-----BEGIN CERTIFICATE-----\ntruncated\n"), 0o600); err != nil {
t.Fatal(err)
}
got, err := r.getCertificate(nil)
if err != nil {
t.Fatalf("a torn write took the listener down: %v", err)
}
if got == nil {
t.Fatal("no certificate served during a torn write")
}
}
func TestExpiryIsVisible(t *testing.T) {
dir := t.TempDir()
certPath, keyPath := writePair(t, dir, "x")
r, err := New(certPath, keyPath)
if err != nil {
t.Fatal(err)
}
if got := r.NotAfter(); got.IsZero() || time.Until(got) > 48*time.Hour {
t.Fatalf("expiry not reported sensibly: %v", got)
}
}
+56
View File
@@ -0,0 +1,56 @@
// SPDX-FileCopyrightText: 2026 Echolot contributors
// SPDX-License-Identifier: GPL-3.0-or-later
package config
import "testing"
// The fixture is fmr's real UDP listen spec, because the point of deriving these from the bound
// listeners is that they cannot disagree with what the server actually answers on.
const fmrUDP = "89.185.109.150:8442,89.185.109.151:8442," +
"[2001:1ad0:c4fe:6767::150]:8442,[2001:1ad0:c4fe:6767::151]:8442"
func TestMeasurementAddrsSplitsPrimaryFromReserved(t *testing.T) {
c := &Config{
UDPListen: fmrUDP,
ReservedAddrs: "89.185.109.151,2001:1ad0:c4fe:6767::151",
}
ip4, ip6, ip4Alt, ip6Alt := c.MeasurementAddrs()
for _, tc := range []struct{ got, want, name string }{
{ip4, "89.185.109.150", "ip4"},
{ip6, "2001:1ad0:c4fe:6767::150", "ip6"},
{ip4Alt, "89.185.109.151", "ip4_alt"},
{ip6Alt, "2001:1ad0:c4fe:6767::151", "ip6_alt"},
} {
if tc.got != tc.want {
t.Errorf("%s = %q, want %q", tc.name, tc.got, tc.want)
}
}
}
func TestMeasurementAddrsWithNothingReserved(t *testing.T) {
// No reservation means no alternate: reporting a second address as the RFC 5780 alternate
// when it was never set aside for that would tell a client to expect a redirect that the
// server has no intention of sending.
c := &Config{UDPListen: fmrUDP}
ip4, ip6, ip4Alt, ip6Alt := c.MeasurementAddrs()
if ip4 == "" || ip6 == "" {
t.Fatalf("primaries should still be found: ip4=%q ip6=%q", ip4, ip6)
}
if ip4Alt != "" || ip6Alt != "" {
t.Errorf("no address is reserved, so there is no alternate; got %q / %q", ip4Alt, ip6Alt)
}
}
func TestMeasurementAddrsIgnoresWhatItCannotRead(t *testing.T) {
// A wildcard bind names no address, and a hostname is not resolved here. Either would be a
// guess presented to clients as fact.
c := &Config{UDPListen: ":8442,probe.example.net:8442,89.185.109.150:8442"}
ip4, ip6, _, _ := c.MeasurementAddrs()
if ip4 != "89.185.109.150" {
t.Errorf("ip4 = %q, want the one address that was actually spelled out", ip4)
}
if ip6 != "" {
t.Errorf("ip6 = %q, want empty — none was configured", ip6)
}
}
+262 -1
View File
@@ -10,6 +10,8 @@ package config
import ( import (
"flag" "flag"
"fmt" "fmt"
"io"
"net"
"os" "os"
"strconv" "strconv"
"strings" "strings"
@@ -39,6 +41,24 @@ type Config struct {
// Admin UI / health listener (spec §7: localhost-only by default) // Admin UI / health listener (spec §7: localhost-only by default)
AdminListen string // ECHOLOT_ADMIN_LISTEN / --admin-listen AdminListen string // ECHOLOT_ADMIN_LISTEN / --admin-listen
// ReservedAddrs are IPs reserved for measurement: addresses whose listening state must stay
// known, so that "nothing answered on port 443" is a fact about the network rather than a
// fact about this server's configuration. Enforced by CheckReserved.
ReservedAddrs string // ECHOLOT_RESERVED_ADDRS / --reserved-addrs
// ControlHostname lets the control plane share port 443 with the admin UI.
//
// They cannot share a certificate: the control plane is trusted by SPKI pin and so uses a
// long-lived self-signed certificate, while a browser needs one a CA vouches for. One name on
// one port means one certificate, so sharing the port requires two names — this one selects
// the pinned certificate and the control-plane routes by SNI, everything else gets the admin
// UI. Empty leaves the control plane on its own listener only.
//
// Why bother: captive portals and corporate firewalls routinely permit only 80 and 443, which
// are exactly the networks this tool exists to diagnose. A control plane on 8443 is
// unreachable precisely when it matters most.
ControlHostname string // ECHOLOT_CONTROL_HOSTNAME / --control-hostname
// State directory: device store, generated TLS material. // State directory: device store, generated TLS material.
StateDir string // ECHOLOT_STATE_DIR / --state-dir StateDir string // ECHOLOT_STATE_DIR / --state-dir
@@ -50,6 +70,11 @@ type Config struct {
// e.g. https://git.example.net/api/v1/repos/owner/repo // e.g. https://git.example.net/api/v1/repos/owner/repo
SelfUpdateAPI string // ECHOLOT_SELF_UPDATE_API / --self-update-api SelfUpdateAPI string // ECHOLOT_SELF_UPDATE_API / --self-update-api
// SelfUpdatePubKey overrides the release-signing public key baked into the binary
// (selfupdate.DefaultPublicKeyB64) — for operators running their own release pipeline
// against their own Gitea. Empty = the built-in project key.
SelfUpdatePubKey string // ECHOLOT_SELF_UPDATE_PUBKEY / --self-update-pubkey
// Uploaded-run storage. The default is "anonymous": any enrolled device may upload, // Uploaded-run storage. The default is "anonymous": any enrolled device may upload,
// which is what a self-hosted server wants. Operators of shared servers turn it down. // which is what a self-hosted server wants. Operators of shared servers turn it down.
UploadsMode string // ECHOLOT_UPLOADS / --uploads (off|anonymous|account) UploadsMode string // ECHOLOT_UPLOADS / --uploads (off|anonymous|account)
@@ -58,6 +83,18 @@ type Config struct {
UploadMaxRuns int // ECHOLOT_UPLOAD_MAX_RUNS / --upload-max-runs (per device) UploadMaxRuns int // ECHOLOT_UPLOAD_MAX_RUNS / --upload-max-runs (per device)
UploadMinAnon string // ECHOLOT_UPLOAD_MIN_ANONYMIZATION / --upload-min-anonymization UploadMinAnon string // ECHOLOT_UPLOAD_MIN_ANONYMIZATION / --upload-min-anonymization
// Rate limits (spec §2.5), each applied per credential and per source IP. 0 disables a
// ceiling. The UDP ceilings are deliberately above the largest legitimate run (a 200 Mbps
// upstream throughput test is ~21k pps of 1472-byte packets), so they only ever catch abuse
// — a rate limit that clips a real measurement produces a confidently wrong number.
RateSessionsPerMin int // ECHOLOT_RATE_SESSIONS_PER_MIN / --rate-sessions-per-min
RateActionsPerMin int // ECHOLOT_RATE_ACTIONS_PER_MIN / --rate-actions-per-min
RateUDPPps int // ECHOLOT_RATE_UDP_PPS / --rate-udp-pps
RateUDPKbps int // ECHOLOT_RATE_UDP_KBPS / --rate-udp-kbps
// How long canary DNS query logs are kept, in hours (spec §6; privacy default 24).
DNSLogRetentionH int // ECHOLOT_DNS_LOG_RETENTION_H / --dns-log-retention-h
// Client compatibility window. Bounds are SemVer; an empty maximum means unbounded. The // Client compatibility window. Bounds are SemVer; an empty maximum means unbounded. The
// defaults sit at breaking boundaries, so shipping a patch never requires changing them. // defaults sit at breaking boundaries, so shipping a patch never requires changing them.
MinAppVersion string // ECHOLOT_MIN_APP_VERSION / --min-app-version MinAppVersion string // ECHOLOT_MIN_APP_VERSION / --min-app-version
@@ -67,10 +104,63 @@ type Config struct {
// first control listen address. // first control listen address.
PublicControlURL string // ECHOLOT_PUBLIC_URL / --public-url PublicControlURL string // ECHOLOT_PUBLIC_URL / --public-url
// Identity provider. Empty issuer disables sign-in entirely; the server is a relying
// party and never stores passwords.
OIDCIssuer string // ECHOLOT_OIDC_ISSUER / --oidc-issuer
OIDCClientID string // ECHOLOT_OIDC_CLIENT_ID / --oidc-client-id (confidential, admin UI)
OIDCAppClientID string // ECHOLOT_OIDC_APP_CLIENT_ID / --oidc-app-client-id (public, the phone app)
// Issuer for the app's client, when the IdP gives each application its own.
//
// Authentik derives the issuer from the application slug, so two applications mean two
// issuers — and a token's `iss` must match the one that minted it. Empty means both clients
// share ECHOLOT_OIDC_ISSUER, which is what IdPs with a single global issuer do.
OIDCAppIssuer string // ECHOLOT_OIDC_APP_ISSUER / --oidc-app-issuer
OIDCAdminGroup string // ECHOLOT_OIDC_ADMIN_GROUP / --oidc-admin-group
// Break-glass admin username; the password lives hashed in the state store.
AdminUser string // ECHOLOT_ADMIN_USER / --admin-user
// Secret for the *confidential* admin client. Prefer ECHOLOT_OIDC_CLIENT_SECRET_FILE: a path
// keeps the secret out of the environment, where it is readable by anything that can see
// /proc/<pid>/environ and lands in every dump of the unit's configuration.
OIDCClientSecret string // ECHOLOT_OIDC_CLIENT_SECRET / _FILE
// Where the admin UI is reachable, used to build the OIDC redirect URI. Must match what is
// registered at the IdP exactly.
AdminBaseURL string // ECHOLOT_ADMIN_BASE_URL / --admin-base-url
// TLS for the admin listener. Without these it serves plaintext, which is only acceptable on
// loopback — see checkAdminExposure.
AdminTLSCert string // ECHOLOT_ADMIN_TLS_CERT / --admin-tls-cert
AdminTLSKey string // ECHOLOT_ADMIN_TLS_KEY / --admin-tls-key
// Deliberate override for serving the admin UI in plaintext off loopback, so that decision
// is made rather than stumbled into.
AdminInsecure bool // ECHOLOT_ADMIN_INSECURE / --admin-insecure
// Port-80 listener that answers ACME HTTP-01 challenges and redirects everything else to
// the admin UI. Empty disables it. HTTP-01 always arrives on port 80 — the CA picks the
// port — so this never collides with the admin UI on 443.
ACMEHTTPListen string // ECHOLOT_ACME_HTTP_LISTEN / --acme-http-listen
// Directory an ACME client writes challenge tokens into. Defaults to <state-dir>/acme.
ACMEWebroot string // ECHOLOT_ACME_WEBROOT / --acme-webroot
// Mode // Mode
Docker bool // --docker (or autodetected; env ECHOLOT_DOCKER=1 forces) Docker bool // --docker (or autodetected; env ECHOLOT_DOCKER=1 forces)
} }
// secretOr reads ECHOLOT_<key>, or the contents of the file named by ECHOLOT_<key>_FILE.
//
// The file form exists because a secret in the environment is readable by anything that can see
// /proc/<pid>/environ and lands in every dump of the unit's configuration. A path costs nothing
// and keeps the value in one file whose permissions an operator can reason about.
func secretOr(key, def string) string {
if path := envOr(key+"_FILE", ""); path != "" {
if b, err := os.ReadFile(path); err == nil {
return strings.TrimSpace(string(b))
}
}
return envOr(key, def)
}
// envInt reads ECHOLOT_<key> as an integer with a fallback. // envInt reads ECHOLOT_<key> as an integer with a fallback.
func envInt(key string, def int) int { func envInt(key string, def int) int {
if v := envOr(key, ""); v != "" { if v := envOr(key, ""); v != "" {
@@ -107,21 +197,48 @@ func Load(args []string) (*Config, *Actions, error) {
fs.StringVar(&c.HTTPEchoListen, "http-echo-listen", envOr("HTTP_ECHO_LISTEN", ""), "optional CLEARTEXT http-echo listen address(es); empty disables (spec §4)") fs.StringVar(&c.HTTPEchoListen, "http-echo-listen", envOr("HTTP_ECHO_LISTEN", ""), "optional CLEARTEXT http-echo listen address(es); empty disables (spec §4)")
fs.StringVar(&c.MTUProbeTargets, "mtu-probe-targets", envOr("MTU_PROBE_TARGETS", "1.1.1.1,2606:4700:4700::1111"), "egress-MTU self-proof anchors, comma-separated") fs.StringVar(&c.MTUProbeTargets, "mtu-probe-targets", envOr("MTU_PROBE_TARGETS", "1.1.1.1,2606:4700:4700::1111"), "egress-MTU self-proof anchors, comma-separated")
fs.StringVar(&c.AdminListen, "admin-listen", envOr("ADMIN_LISTEN", "127.0.0.1:8444"), "admin/health listen address (keep localhost)") fs.StringVar(&c.AdminListen, "admin-listen", envOr("ADMIN_LISTEN", "127.0.0.1:8444"), "admin/health listen address (keep localhost)")
fs.StringVar(&c.ControlHostname, "control-hostname", envOr("CONTROL_HOSTNAME", ""), "hostname that selects the pinned control-plane certificate when sharing the admin UI's port")
fs.StringVar(&c.ReservedAddrs, "reserved-addrs", envOr("RESERVED_ADDRS", ""), "comma-separated IPs reserved for measurement; no listener but the STUN alternate may bind them")
fs.StringVar(&c.StateDir, "state-dir", envOr("STATE_DIR", defaultStateDir()), "state directory (device store, generated TLS)") fs.StringVar(&c.StateDir, "state-dir", envOr("STATE_DIR", defaultStateDir()), "state directory (device store, generated TLS)")
fs.StringVar(&c.Name, "name", envOr("NAME", "echolot"), "server profile name") fs.StringVar(&c.Name, "name", envOr("NAME", "echolot"), "server profile name")
fs.StringVar(&c.SelfUpdateAPI, "self-update-api", envOr("SELF_UPDATE_API", ""), "Gitea repo API base for self-update; empty disables") fs.StringVar(&c.SelfUpdateAPI, "self-update-api", envOr("SELF_UPDATE_API", ""), "Gitea repo API base for self-update; empty disables")
fs.StringVar(&c.SelfUpdatePubKey, "self-update-pubkey", envOr("SELF_UPDATE_PUBKEY", ""), "release-signing public key (base64 ed25519) self-update verifies against; empty uses the built-in project key")
fs.StringVar(&c.UploadsMode, "uploads", envOr("UPLOADS", "anonymous"), "who may upload measurement runs: off|anonymous|account") fs.StringVar(&c.UploadsMode, "uploads", envOr("UPLOADS", "anonymous"), "who may upload measurement runs: off|anonymous|account")
fs.Int64Var(&c.UploadMaxBytes, "upload-max-bytes", int64(envInt("UPLOAD_MAX_BYTES", 4<<20)), "largest accepted uploaded run, bytes") fs.Int64Var(&c.UploadMaxBytes, "upload-max-bytes", int64(envInt("UPLOAD_MAX_BYTES", 4<<20)), "largest accepted uploaded run, bytes")
fs.IntVar(&c.UploadRetentionDays, "upload-retention-days", envInt("UPLOAD_RETENTION_DAYS", 90), "delete uploaded runs older than this; 0 disables") fs.IntVar(&c.UploadRetentionDays, "upload-retention-days", envInt("UPLOAD_RETENTION_DAYS", 90), "delete uploaded runs older than this; 0 disables")
fs.IntVar(&c.UploadMaxRuns, "upload-max-runs", envInt("UPLOAD_MAX_RUNS", 200), "keep at most this many runs per device; 0 disables") fs.IntVar(&c.UploadMaxRuns, "upload-max-runs", envInt("UPLOAD_MAX_RUNS", 200), "keep at most this many runs per device; 0 disables")
fs.StringVar(&c.UploadMinAnon, "upload-min-anonymization", envOr("UPLOAD_MIN_ANONYMIZATION", "full"), "least anonymization accepted: full|balanced|strict") fs.StringVar(&c.UploadMinAnon, "upload-min-anonymization", envOr("UPLOAD_MIN_ANONYMIZATION", "full"), "least anonymization accepted: full|balanced|strict")
fs.StringVar(&c.OIDCIssuer, "oidc-issuer", envOr("OIDC_ISSUER", ""), "OpenID Connect issuer URL; empty disables sign-in")
fs.StringVar(&c.OIDCClientID, "oidc-client-id", envOr("OIDC_CLIENT_ID", ""), "confidential OIDC client id for the admin UI")
fs.StringVar(&c.OIDCAppClientID, "oidc-app-client-id", envOr("OIDC_APP_CLIENT_ID", ""), "public OIDC client id used by the Android app (PKCE)")
fs.StringVar(&c.OIDCAppIssuer, "oidc-app-issuer", envOr("OIDC_APP_ISSUER", ""), "issuer for the app client when the IdP uses per-application issuers; empty = same as --oidc-issuer")
fs.StringVar(&c.OIDCAdminGroup, "oidc-admin-group", envOr("OIDC_ADMIN_GROUP", ""), "group claim required for admin access; empty means nobody is an admin via OIDC")
fs.StringVar(&c.OIDCClientSecret, "oidc-client-secret", secretOr("OIDC_CLIENT_SECRET", ""), "secret for the confidential admin client; prefer ECHOLOT_OIDC_CLIENT_SECRET_FILE")
fs.StringVar(&c.AdminBaseURL, "admin-base-url", envOr("ADMIN_BASE_URL", ""), "public URL of the admin UI, for the OIDC redirect (e.g. https://admin.example.net)")
fs.StringVar(&c.AdminTLSCert, "admin-tls-cert", envOr("ADMIN_TLS_CERT", ""), "TLS certificate for the admin listener")
fs.StringVar(&c.AdminTLSKey, "admin-tls-key", envOr("ADMIN_TLS_KEY", ""), "TLS key for the admin listener")
fs.BoolVar(&c.AdminInsecure, "admin-insecure", envOr("ADMIN_INSECURE", "") == "1", "allow the admin UI in plaintext off loopback (you are on your own)")
fs.StringVar(&c.ACMEHTTPListen, "acme-http-listen", envOr("ACME_HTTP_LISTEN", ""), "port-80 listener for ACME HTTP-01 challenges and http->https redirects")
fs.StringVar(&c.ACMEWebroot, "acme-webroot", envOr("ACME_WEBROOT", ""), "directory an ACME client writes challenges into (default <state-dir>/acme)")
fs.StringVar(&c.PublicControlURL, "public-url", envOr("PUBLIC_URL", ""), "public control-plane URL for enrollment links, e.g. https://probe.example.net:8443") fs.StringVar(&c.PublicControlURL, "public-url", envOr("PUBLIC_URL", ""), "public control-plane URL for enrollment links, e.g. https://probe.example.net:8443")
fs.IntVar(&c.RateSessionsPerMin, "rate-sessions-per-min", envInt("RATE_SESSIONS_PER_MIN", 10), "per-credential and per-IP ceiling on session creation (spec §2.5); 0 disables")
fs.IntVar(&c.RateActionsPerMin, "rate-actions-per-min", envInt("RATE_ACTIONS_PER_MIN", 60), "per-credential and per-IP ceiling on §5 actions; 0 disables")
fs.IntVar(&c.RateUDPPps, "rate-udp-pps", envInt("RATE_UDP_PPS", 25_000), "per-credential and per-IP data-plane packet ceiling, packets/s, silent drop; 0 disables")
fs.IntVar(&c.RateUDPKbps, "rate-udp-kbps", envInt("RATE_UDP_KBPS", 250_000), "per-credential and per-IP data-plane byte ceiling, kbit/s, silent drop; 0 disables")
fs.IntVar(&c.DNSLogRetentionH, "dns-log-retention-h", envInt("DNS_LOG_RETENTION_H", 24), "hours canary DNS query logs are kept (spec §6 privacy default 24); 0 keeps until the ring overwrites")
fs.StringVar(&c.MinAppVersion, "min-app-version", envOr("MIN_APP_VERSION", "0.2.0"), "oldest app version this server will serve (SemVer, inclusive)") fs.StringVar(&c.MinAppVersion, "min-app-version", envOr("MIN_APP_VERSION", "0.2.0"), "oldest app version this server will serve (SemVer, inclusive)")
fs.StringVar(&c.MaxAppVersion, "max-app-version", envOr("MAX_APP_VERSION", "1.0.0"), "first app version this server will refuse (SemVer, exclusive); empty = unbounded") fs.StringVar(&c.MaxAppVersion, "max-app-version", envOr("MAX_APP_VERSION", "1.0.0"), "first app version this server will refuse (SemVer, exclusive); empty = unbounded")
fs.BoolVar(&c.Docker, "docker", envOr("DOCKER", "") == "1", "force container mode (config from env, no systemd/self-update)") fs.BoolVar(&c.Docker, "docker", envOr("DOCKER", "") == "1", "force container mode (config from env, no systemd/self-update)")
fs.BoolVar(&a.InstallSystemd, "install-systemd", false, "install a systemd unit for this binary and exit") fs.BoolVar(&a.InstallSystemd, "install-systemd", false, "install a systemd unit for this binary and exit")
fs.BoolVar(&a.UninstallSystemd, "uninstall-systemd", false, "remove the systemd unit and exit") fs.BoolVar(&a.UninstallSystemd, "uninstall-systemd", false, "remove the systemd unit and exit")
var daemon bool
fs.BoolVar(&a.Serve, "serve", false, "run the server (bind listeners and answer requests)")
fs.BoolVar(&daemon, "daemon", false, "alias for --serve")
fs.BoolVar(&a.SetAdminPassword, "set-admin-password", false,
"set the break-glass admin password (username as --admin-user, password read from stdin) and exit")
fs.StringVar(&c.AdminUser, "admin-user", envOr("ADMIN_USER", "admin"), "username for the break-glass admin")
fs.StringVar(&a.MintEnrollToken, "mint-enroll-token", "", "mint a single-use enrollment link (argument is a note for the audit log) and exit")
fs.BoolVar(&a.SelfUpdate, "self-update", false, "check for a newer release, replace this binary, and exit") fs.BoolVar(&a.SelfUpdate, "self-update", false, "check for a newer release, replace this binary, and exit")
fs.BoolVar(&a.Version, "version", false, "print version and exit") fs.BoolVar(&a.Version, "version", false, "print version and exit")
@@ -131,12 +248,141 @@ func Load(args []string) (*Config, *Actions, error) {
if !c.Docker { if !c.Docker {
c.Docker = inContainer() c.Docker = inContainer()
} }
a.Serve = a.Serve || daemon
// No verb at all means the caller has not said what they want. Usage is the answer, and it
// is a usage error rather than success — otherwise a service manager sees a clean exit and
// concludes the server ran and finished.
if !a.Serve && !a.InstallSystemd && !a.UninstallSystemd && !a.SelfUpdate &&
!a.SetAdminPassword && !a.Version && a.MintEnrollToken == "" {
a.Help = true
}
if a.Serve {
if err := c.checkAdminExposure(); err != nil {
return nil, nil, err
}
if err := c.CheckReserved(c.Listeners()); err != nil {
return nil, nil, err
}
}
if c.Docker && (a.InstallSystemd || a.UninstallSystemd || a.SelfUpdate) { if c.Docker && (a.InstallSystemd || a.UninstallSystemd || a.SelfUpdate) {
return nil, nil, fmt.Errorf("systemd/self-update actions are native-mode only (container detected; override with ECHOLOT_DOCKER=0 if this is wrong)") return nil, nil, fmt.Errorf("systemd/self-update actions are native-mode only (container detected; override with ECHOLOT_DOCKER=0 if this is wrong)")
} }
return c, a, nil return c, a, nil
} }
// Usage prints the verbs first and the tuning flags second, because the question someone has
// when they run this by name is "what does it do", not "what can I set".
func Usage(w io.Writer) {
fmt.Fprint(w, `echolot-server the Echolot probe server
USAGE
echolot-server --serve run the server
echolot-server --version print the version
echolot-server --install-systemd install and enable a systemd unit
echolot-server --uninstall-systemd remove it
echolot-server --self-update replace this binary with the latest release
echolot-server --set-admin-password set the break-glass admin password (stdin)
echolot-server --help full flag list
Every flag can also be set as an environment variable: --control-listen becomes
ECHOLOT_CONTROL_LISTEN. In a container, configuration comes from the environment.
Running with no verb prints this and exits non-zero: starting to serve the internet
should be something you asked for.
`)
}
// checkAdminExposure refuses to serve an unencrypted admin UI on a non-loopback address.
//
// The admin session cookie is a bearer credential for everything this server can do, and the OIDC
// authorization code arrives in a URL. In plaintext, both are readable by anyone on the path — and
// on a globally routable address "the path" means the internet. This is a hard stop rather than a
// warning because a warning in a log is not read by the person who most needs it, and because the
// two safe answers are cheap: bind to loopback and tunnel, or supply a certificate.
func (c *Config) checkAdminExposure() error {
if c.AdminTLSCert != "" || c.AdminInsecure {
return nil
}
for _, addr := range Addrs(c.AdminListen) {
host, _, err := net.SplitHostPort(addr)
if err != nil {
continue
}
ip := net.ParseIP(strings.Trim(host, "[]"))
if host == "" || ip == nil || ip.IsLoopback() {
continue // loopback, or a name we cannot judge; RFC 8252 blesses loopback plaintext
}
return fmt.Errorf(
"refusing to serve the admin UI in plaintext on %s: the session cookie and the OIDC "+
"authorization code would cross the network in the clear.\n"+
" Fix it one of three ways:\n"+
" - bind to 127.0.0.1 and reach it over an SSH tunnel (no certificate needed)\n"+
" - set ECHOLOT_ADMIN_TLS_CERT and ECHOLOT_ADMIN_TLS_KEY\n"+
" - set ECHOLOT_ADMIN_INSECURE=1 if you genuinely mean it", addr)
}
return nil
}
// Listeners enumerates every configured listen spec, for CheckReserved.
//
// Kept as one list here rather than checked at each call site, so a listener added later is
// caught by the compiler when this function is updated — and, more to the point, so that the
// person adding one sees the reserved-address rule exists at all.
func (c *Config) Listeners() []Listener {
return []Listener{
// The instrument: these belong on the reserved addresses as much as anywhere.
{Name: "control-listen", Spec: c.ControlListen, Measurement: true},
{Name: "udp-listen", Spec: c.UDPListen, Measurement: true},
{Name: "tcp-listen", Spec: c.TCPListen, Measurement: true},
{Name: "dns-listen", Spec: c.DNSListen, Measurement: true},
{Name: "stun-listen", Spec: c.StunListen, Measurement: true},
// http-echo is deliberately not marked as measurement: it is cleartext HTTP, so on a
// reserved address it would be the very listener that ruins the port-80 test.
{Name: "http-echo-listen", Spec: c.HTTPEchoListen},
// Services. These have no business on an address kept for measuring.
{Name: "admin-listen", Spec: c.AdminListen},
{Name: "acme-http-listen", Spec: c.ACMEHTTPListen},
}
}
// MeasurementAddrs picks out the addresses this server can be measured on, by family, splitting
// primaries from the reserved alternates.
//
// Derived from what is actually bound rather than configured separately: a second list of the
// server's own addresses is a second thing to keep in step, and the copy that drifts is the one
// clients are told about.
func (c *Config) MeasurementAddrs() (ip4, ip6, ip4Alt, ip6Alt string) {
reserved := map[string]bool{}
for _, ip := range c.ReservedIPs() {
reserved[ip.String()] = true
}
// The UDP data plane binds every address a client may be pointed at, which makes it the
// honest source for this.
for _, a := range Addrs(c.UDPListen) {
host, _, err := net.SplitHostPort(a)
if err != nil {
continue
}
host = strings.Trim(host, "[]")
ip := net.ParseIP(host)
if ip == nil {
continue
}
alt := reserved[ip.String()]
switch {
case ip.To4() != nil && alt && ip4Alt == "":
ip4Alt = host
case ip.To4() != nil && !alt && ip4 == "":
ip4 = host
case ip.To4() == nil && alt && ip6Alt == "":
ip6Alt = host
case ip.To4() == nil && !alt && ip6 == "":
ip6 = host
}
}
return
}
// Addrs splits a comma-separated listen spec into individual addresses. // Addrs splits a comma-separated listen spec into individual addresses.
// Explicit per-address binds matter on multi-IP hosts: a wildcard bind // Explicit per-address binds matter on multi-IP hosts: a wildcard bind
// (":8443") would also claim addresses reserved for other purposes (e.g. an // (":8443") would also claim addresses reserved for other purposes (e.g. an
@@ -151,11 +397,26 @@ func Addrs(spec string) []string {
return out return out
} }
// Actions are one-shot verbs that exit instead of serving. // Actions are the verbs. Serving is one of them, and it is explicit: running the binary with no
// arguments prints usage rather than binding a dozen ports and starting to answer the internet.
// Someone typing the name of an unfamiliar program on a terminal should be told what it does, not
// have it start doing it.
type Actions struct { type Actions struct {
Serve bool
// Help is set when there is nothing to do: no verb was given.
Help bool
InstallSystemd bool InstallSystemd bool
UninstallSystemd bool UninstallSystemd bool
SelfUpdate bool SelfUpdate bool
SetAdminPassword bool
// MintEnrollToken is the note to record against a freshly minted enrollment link.
//
// A local action rather than an HTTP endpoint: whoever can run this binary against the state
// directory already has every privilege the server has, so authenticating them to themselves
// would be theatre — and an unauthenticated endpoint on loopback is how the admin API was
// briefly reachable from the network by accident.
MintEnrollToken string
Version bool Version bool
} }
+123
View File
@@ -0,0 +1,123 @@
// SPDX-FileCopyrightText: 2026 Echolot contributors
// SPDX-License-Identifier: GPL-3.0-or-later
package config
import (
"fmt"
"net"
"sort"
"strings"
)
// Listener is one configured listen spec, named for the error message.
type Listener struct {
Name string // the flag/env this came from, e.g. "control-listen"
Spec string // comma-separated listen addresses
// Measurement marks a listener that is part of the instrument rather than a service on the
// host. Those belong on the reserved addresses — STUN's RFC 5780 alternate, the UDP data
// plane, the canary DNS — and reserving an address only to forbid the measurements that need
// it would defeat the purpose.
Measurement bool
}
// webPorts are the ports whose closed state on a reserved address is itself the measurement.
//
// A TLS handshake that completes on a port known not to be listening proves interception, with no
// competing explanation. That proof is the whole reason for reserving an address, and it survives
// exactly as long as nothing binds these two ports there.
var webPorts = map[string]bool{"80": true, "443": true}
// CheckReserved refuses to start when a listener would occupy an address reserved for measurement.
//
// The reserved addresses are the instrument, not the service. Their diagnostic value comes from
// their listening state being *known*: if nothing listens on port 443 there, then a TLS handshake
// that completes proves something on the path intercepted it, with no other explanation available.
// One stray listener silently converts that proof into an ambiguity.
//
// This is a hard stop rather than a warning for the same reason as [Config.checkAdminExposure]: the
// failure is invisible. A polluted reserved address does not crash, log, or behave oddly — it just
// quietly turns a conclusive test into an inconclusive one, and the first symptom is a measurement
// that says the network is clean when it is not. Nobody reads a warning for that.
//
// Wildcard binds are the realistic way this happens. Every listener defaults to ":port", and the
// next one added will be copied from an existing default; that binds every address on the host,
// reserved ones included, without anyone deciding to.
func (c *Config) CheckReserved(listeners []Listener) error {
reserved := c.ReservedIPs()
if len(reserved) == 0 {
return nil
}
var problems []string
for _, l := range listeners {
for _, addr := range Addrs(l.Spec) {
host, port, err := net.SplitHostPort(addr)
if err != nil {
// Not host:port — a bare port or something malformed. Leave it to the listener
// itself to complain; guessing here would produce a confusing error about the
// wrong problem.
continue
}
host = strings.Trim(host, "[]")
if host == "" || host == "0.0.0.0" || host == "::" {
problems = append(problems, fmt.Sprintf(
" --%s=%q binds every address on this host, including the reserved ones",
l.Name, addr))
continue
}
ip := net.ParseIP(host)
if ip == nil {
continue // a hostname; cannot resolve it here without lying about what we checked
}
for _, r := range reserved {
if !ip.Equal(r) {
continue
}
switch {
case webPorts[port]:
problems = append(problems, fmt.Sprintf(
" --%s=%q puts port %s on reserved address %s, which is the one thing "+
"that address exists to keep closed", l.Name, addr, port, r))
case !l.Measurement:
problems = append(problems, fmt.Sprintf(
" --%s=%q binds reserved address %s; only measurement listeners belong there",
l.Name, addr, r))
}
}
}
}
if len(problems) == 0 {
return nil
}
sort.Strings(problems)
return fmt.Errorf(
"refusing to start: these listeners would occupy addresses reserved for measurement\n%s\n"+
"\nReserved: %s\n"+
"Those addresses are the instrument. A test can only prove interception on a port that\n"+
"is known not to be listening, so anything bound there destroys the conclusion rather\n"+
"than merely sharing the address.\n"+
" Fix it one of three ways:\n"+
" - bind each listener to explicit service addresses instead of a wildcard\n"+
" - remove the address from ECHOLOT_RESERVED_ADDRS if it is no longer reserved\n"+
" - unset ECHOLOT_RESERVED_ADDRS if this host has no reserved addresses",
strings.Join(problems, "\n"), joinIPs(reserved))
}
// ReservedIPs parses the configured reserved addresses, ignoring anything unparseable.
func (c *Config) ReservedIPs() []net.IP {
var out []net.IP
for _, s := range Addrs(c.ReservedAddrs) {
if ip := net.ParseIP(strings.Trim(s, "[]")); ip != nil {
out = append(out, ip)
}
}
return out
}
func joinIPs(ips []net.IP) string {
s := make([]string, 0, len(ips))
for _, ip := range ips {
s = append(s, ip.String())
}
return strings.Join(s, ", ")
}
+151
View File
@@ -0,0 +1,151 @@
// SPDX-FileCopyrightText: 2026 Echolot contributors
// SPDX-License-Identifier: GPL-3.0-or-later
package config
import (
"strings"
"testing"
)
const (
svc4 = "89.185.109.150"
res4 = "89.185.109.151"
res6 = "2001:1ad0:c4fe:6767::151"
)
func withReserved(l ...Listener) error {
c := &Config{ReservedAddrs: res4 + "," + res6}
return c.CheckReserved(l)
}
func TestWildcardBindIsRefused(t *testing.T) {
// The realistic failure: every listener defaults to ":port", and the next one added gets
// copied from an existing default. Nobody decides to claim the reserved address; it just
// happens, and nothing looks wrong afterwards.
err := withReserved(Listener{Name: "control-listen", Spec: ":8443"})
if err == nil {
t.Fatal("a wildcard bind was allowed while addresses were reserved")
}
if !strings.Contains(err.Error(), "binds every address") {
t.Fatalf("the error should say why a wildcard is the problem, got: %v", err)
}
}
func TestWebPortsOnReservedAreRefusedEvenForMeasurement(t *testing.T) {
// The strictest rule, and the one carrying the diagnostic value: 80 and 443 must stay closed
// on a reserved address whatever wants them, because their closed state *is* the measurement.
for _, spec := range []string{res4 + ":443", "[" + res6 + "]:80"} {
err := withReserved(Listener{Name: "control-listen", Spec: spec, Measurement: true})
if err == nil {
t.Fatalf("port 80/443 on a reserved address was allowed: %q", spec)
}
if !strings.Contains(err.Error(), "keep closed") {
t.Errorf("the error should explain what is lost, got: %v", err)
}
}
}
func TestServiceOnReservedIsRefused(t *testing.T) {
if err := withReserved(Listener{Name: "admin-listen", Spec: res4 + ":8444"}); err == nil {
t.Fatal("a service was allowed onto a reserved address")
}
}
func TestMeasurementListenersBelongOnReserved(t *testing.T) {
// The live fmr config: the UDP data plane, canary DNS and STUN all bind the reserved pair on
// purpose. A guard that refused this would be describing a rule nobody wants.
err := withReserved(
Listener{Name: "udp-listen", Spec: res4 + ":8442,[" + res6 + "]:8442", Measurement: true},
Listener{Name: "dns-listen", Spec: res4 + ":53,[" + res6 + "]:53", Measurement: true},
Listener{Name: "stun-listen", Spec: res4 + ":3478", Measurement: true},
)
if err != nil {
t.Fatalf("measurement listeners must be allowed on reserved addresses: %v", err)
}
}
func TestServiceAddressesAreFine(t *testing.T) {
err := withReserved(
Listener{Name: "control-listen", Spec: svc4 + ":8443,[2001:1ad0:c4fe:6767::150]:8443"},
Listener{Name: "admin-listen", Spec: "127.0.0.1:8444"},
)
if err != nil {
t.Fatalf("service addresses should be allowed: %v", err)
}
}
func TestHttpEchoIsNotTreatedAsMeasurement(t *testing.T) {
// http-echo is cleartext HTTP. On a reserved address it is precisely the listener that would
// ruin the port-80 test, so it does not get the measurement exemption.
if err := withReserved(Listener{Name: "http-echo-listen", Spec: res4 + ":8080"}); err == nil {
t.Fatal("http-echo was allowed onto a reserved address")
}
}
func TestNoReservationMeansNoOpinion(t *testing.T) {
// A host with nothing reserved must keep working exactly as before, wildcards included.
c := &Config{}
if err := c.CheckReserved([]Listener{{Name: "control-listen", Spec: ":8443"}}); err != nil {
t.Fatalf("with no reserved addresses this must not interfere: %v", err)
}
}
func TestEveryOffenderIsNamed(t *testing.T) {
// Reporting one problem at a time turns a config fix into several restart cycles, and on a
// remote host each cycle is a chance to lock yourself out.
err := withReserved(
Listener{Name: "control-listen", Spec: ":8443"},
Listener{Name: "tcp-listen", Spec: res4 + ":8441"},
)
if err == nil {
t.Fatal("expected a refusal")
}
for _, want := range []string{"control-listen", "tcp-listen"} {
if !strings.Contains(err.Error(), want) {
t.Errorf("the error should name %s; got: %v", want, err)
}
}
}
func TestHostnamesAreNotGuessedAt(t *testing.T) {
// Resolving here would check a name against whatever DNS says at startup, which is not
// necessarily what it will say later — and a guard that is sometimes right is worse than one
// with a stated limit.
if err := withReserved(Listener{Name: "control-listen", Spec: "fmr-1.echo-lot.app:8443"}); err != nil {
t.Fatalf("a hostname must be left alone, not resolved: %v", err)
}
}
func TestListenersCoversEverySpec(t *testing.T) {
// A listener missing from Listeners() is invisible to the guard, which is the one way this
// protection fails silently. Fill every spec with the reserved address: each one that is
// actually enumerated produces a complaint naming it.
// Every spec on port 443 of the reserved address: the web-port rule applies to measurement
// listeners too, so each one that is genuinely enumerated must produce a complaint.
c := &Config{
ReservedAddrs: res4,
ControlListen: res4 + ":443",
UDPListen: res4 + ":443",
TCPListen: res4 + ":443",
DNSListen: res4 + ":443",
HTTPEchoListen: res4 + ":443",
AdminListen: res4 + ":443",
ACMEHTTPListen: res4 + ":443",
StunListen: res4 + ":443",
}
err := c.CheckReserved(c.Listeners())
if err == nil {
t.Fatal("expected a refusal")
}
// Every spec is on the reserved address; the web-port rule catches even the measurement ones,
// so anything missing from Listeners() is invisible here and that is what this asserts.
for _, want := range []string{
"control-listen", "udp-listen", "tcp-listen", "dns-listen",
"http-echo-listen", "admin-listen", "acme-http-listen", "stun-listen",
} {
if !strings.Contains(err.Error(), want) {
t.Errorf("%s is not enumerated in Listeners(), so the guard cannot see it", want)
}
}
}
+404 -30
View File
@@ -7,6 +7,7 @@
package control package control
import ( import (
"context"
"crypto/rand" "crypto/rand"
"crypto/sha256" "crypto/sha256"
"crypto/tls" "crypto/tls"
@@ -15,6 +16,7 @@ import (
"encoding/hex" "encoding/hex"
"encoding/json" "encoding/json"
"errors" "errors"
"fmt"
"io" "io"
"log/slog" "log/slog"
"net" "net"
@@ -27,6 +29,8 @@ import (
"echo-lot.app/server/internal/compat" "echo-lot.app/server/internal/compat"
"echo-lot.app/server/internal/dataplane" "echo-lot.app/server/internal/dataplane"
"echo-lot.app/server/internal/oidc"
"echo-lot.app/server/internal/ratelimit"
"echo-lot.app/server/internal/runs" "echo-lot.app/server/internal/runs"
"echo-lot.app/server/internal/session" "echo-lot.app/server/internal/session"
"echo-lot.app/server/internal/store" "echo-lot.app/server/internal/store"
@@ -55,13 +59,22 @@ type Server struct {
// DelayedEcho schedules/sends a DELAYED_ECHO for a session (may be nil). // DelayedEcho schedules/sends a DELAYED_ECHO for a session (may be nil).
DelayedEcho func(sess *session.Session, actionID string) error DelayedEcho func(sess *session.Session, actionID string) error
// Granted server->client sends (spec §5). Both consume an asymmetric grant. // Granted server->client sends (spec §5). Both consume an asymmetric grant.
DownTrain func(sess *session.Session, g *session.Grant, count, sizeBytes, intervalUs int) (int, error) // DownTrain's dscp is -1 for "leave the socket's default marking alone".
DownTrain func(sess *session.Session, g *session.Grant, count, sizeBytes, intervalUs, dscp int) (int, error)
BigSend func(sess *session.Session, g *session.Grant, sizes []int, df bool) ([]dataplane.BigSendResult, error) BigSend func(sess *session.Session, g *session.Grant, sizes []int, df bool) ([]dataplane.BigSendResult, error)
// OIDC verifies ID tokens presented by the *app* (may be nil).
OIDC *oidc.Verifier
// AdminOIDC verifies tokens from the admin UI's own client. Separate because an IdP may
// give each application its own issuer — Authentik derives it from the application slug —
// and a verifier pins exactly one issuer and the clients belonging to it.
AdminOIDC *oidc.Verifier
// Runs stores uploaded measurement documents (may be nil: uploads unsupported). // Runs stores uploaded measurement documents (may be nil: uploads unsupported).
Runs *runs.Store Runs *runs.Store
// FragSend emits one datagram as hand-built IP fragments in a chosen order (may be nil: // FragSend emits one datagram as hand-built IP fragments in a chosen order (may be nil:
// needs a raw socket, so it is unavailable to an unprivileged server). // needs a raw socket, so it is unavailable to an unprivileged server).
FragSend func(sess *session.Session, g *session.Grant, sizeBytes int, mode dataplane.FragMode, fragSize int) (dataplane.FragResult, error) FragSend func(sess *session.Session, g *session.Grant, sizeBytes int, mode dataplane.FragMode, fragSize int) (dataplane.FragResult, error)
// DownThroughput sends paced traffic toward the client for a bounded time (may be nil).
DownThroughput func(sess *session.Session, g *session.Grant, durationMs, kbps, sizeBytes int) (dataplane.ThroughputResult, error)
// EgressMTU reports the server's own measured egress path MTU (0 = unknown). With DF set // EgressMTU reports the server's own measured egress path MTU (0 = unknown). With DF set
// we cannot emit a datagram larger than this, so requested sizes above it are refused up // we cannot emit a datagram larger than this, so requested sizes above it are refused up
// front and reported as such — the client must not read that as a downstream path limit. // front and reported as such — the client must not read that as a downstream path limit.
@@ -70,6 +83,10 @@ type Server struct {
CanaryQueries func(sessionPrefix string) any CanaryQueries func(sessionPrefix string) any
// CanaryZone is surfaced in the profile so the app knows what to query. // CanaryZone is surfaced in the profile so the app knows what to query.
CanaryZone string CanaryZone string
// The addresses this server can be measured on. The "_alt" pair is the second address
// RFC 5780 behaviour discovery redirects to, and the one reserved from services so that
// nothing answering there is itself a measurement.
IP4, IP6, IP4Alt, IP6Alt string
// ProvenGood reports the server's self-test signal (may be nil). Surfaced // ProvenGood reports the server's self-test signal (may be nil). Surfaced
// in the profile so a client can trust — or skip — MTU tests: if the // in the profile so a client can trust — or skip — MTU tests: if the
// server's own egress isn't full-MTU, client MTU results measure the // server's own egress isn't full-MTU, client MTU results measure the
@@ -83,6 +100,11 @@ type Server struct {
// AppRange is the app-version window this server will serve. Zero value means the built-in // AppRange is the app-version window this server will serve. Zero value means the built-in
// default (see DefaultAppRange). // default (see DefaultAppRange).
AppRange compat.Range AppRange compat.Range
// Spec §2.5 token buckets, keyed per credential and per source IP inside the limiter.
// Nil disables a ceiling (config value 0).
RateSessions *ratelimit.Limiter // POST /v1/sessions
RateActions *ratelimit.Limiter // POST /v1/sessions/{id}/actions
} }
// AppVersionHeader is how a client states its version. A client too old to send it is treated as // AppVersionHeader is how a client states its version. A client too old to send it is treated as
@@ -92,7 +114,12 @@ const AppVersionHeader = "X-Echolot-App-Version"
// ProtocolVersion is the wire contract (probe-protocol.md) this build implements. It is what the // ProtocolVersion is the wire contract (probe-protocol.md) this build implements. It is what the
// version window is really about; the release version is only a proxy for it. // version window is really about; the release version is only a proxy for it.
const ProtocolVersion = "1.0.0" //
// 1.0.1: upstream trains (§3.2 types 0x03/0x04/0x05) and the action-id bytes at payload[8:16]
// of granted packets. Patch, not minor: both are additive — a client that never sends
// TRAIN_REPORT_REQ and never reads granted payloads (today's client reads only header fields)
// sees no difference, so the fleet must not be split over it (§8.1).
const ProtocolVersion = "1.0.1"
// SchemaVersion is the measurement-document format this server can store. // SchemaVersion is the measurement-document format this server can store.
const SchemaVersion = "1.0.0" const SchemaVersion = "1.0.0"
@@ -150,17 +177,21 @@ func (s *Server) Handler() http.Handler {
gate := s.requireCompatibleApp gate := s.requireCompatibleApp
mux.HandleFunc("POST /v1/enroll", gate(s.enroll)) mux.HandleFunc("POST /v1/enroll", gate(s.enroll))
mux.HandleFunc("POST /v1/sessions", gate(s.newSession)) // §2.5 buckets sit on the two endpoints that make the server DO things — create state,
// send traffic. GET /v1/profile stays ungated on every axis (see above).
mux.HandleFunc("POST /v1/sessions", gate(s.rateLimited(s.RateSessions, s.newSession)))
mux.HandleFunc("DELETE /v1/sessions/{id}", gate(s.deleteSession)) mux.HandleFunc("DELETE /v1/sessions/{id}", gate(s.deleteSession))
mux.HandleFunc("GET /v1/sessions/{id}/observations", gate(s.observations)) mux.HandleFunc("GET /v1/sessions/{id}/observations", gate(s.observations))
mux.HandleFunc("POST /v1/sessions/{id}/actions", gate(s.actions)) mux.HandleFunc("POST /v1/sessions/{id}/actions", gate(s.rateLimited(s.RateActions, s.actions)))
mux.HandleFunc("POST /v1/echo", gate(s.httpEcho)) mux.HandleFunc("POST /v1/echo", gate(s.httpEcho))
mux.HandleFunc("GET /v1/tls-reference", gate(s.tlsReference)) mux.HandleFunc("GET /v1/tls-reference", gate(s.tlsReference))
mux.HandleFunc("POST /v1/runs", gate(s.uploadRun)) mux.HandleFunc("POST /v1/runs", gate(s.uploadRun))
mux.HandleFunc("GET /v1/runs", gate(s.listRuns)) mux.HandleFunc("GET /v1/runs", gate(s.listRuns))
mux.HandleFunc("GET /v1/runs/{id}", gate(s.getRun)) mux.HandleFunc("GET /v1/runs/{id}", gate(s.getRun))
mux.HandleFunc("DELETE /v1/runs/{id}", gate(s.deleteRun)) mux.HandleFunc("DELETE /v1/runs/{id}", gate(s.deleteRun))
// TODO(spec §5): frag_send, throughput (both build on the same grant machinery) mux.HandleFunc("POST /v1/account/link", gate(s.linkAccount))
mux.HandleFunc("DELETE /v1/account/link", gate(s.unlinkAccount))
mux.HandleFunc("GET /v1/account", gate(s.accountStatus))
return mux return mux
} }
@@ -182,6 +213,37 @@ func selftestSignal(f func() (bool, bool)) map[string]any {
return map[string]any{"mtu_ok": mtuOK, "sysctl_ok": sysctlOK} return map[string]any{"mtu_ok": mtuOK, "sysctl_ok": sysctlOK}
} }
// rateLimited enforces one §2.5 bucket policy on an endpoint: a token per credential AND one per
// source IP. Two keys because each closes the other's hole — keyed only by credential, one
// address cycles through credentials; keyed only by address, one credential rides many
// addresses. The refusal is 429 with Retry-After, which is the whole point of a token bucket
// over a hard drop here: a well-behaved client is told when to come back.
func (s *Server) rateLimited(l *ratelimit.Limiter, next http.HandlerFunc) http.HandlerFunc {
return func(w http.ResponseWriter, r *http.Request) {
okCred, waitCred := l.Allow("cred:" + bearer(r))
okIP, waitIP := l.Allow("ip:" + remoteIP(r))
if !okCred || !okIP {
wait := max(waitCred, waitIP)
secs := int(wait/time.Second) + 1 // Retry-After is whole seconds, rounded up
w.Header().Set("Retry-After", strconv.Itoa(secs))
writeJSON(w, http.StatusTooManyRequests, map[string]any{
"error": "rate limited", "retry_after_s": secs,
})
return
}
next(w, r)
}
}
// remoteIP is the request's source address without the port, for rate-limit keys.
func remoteIP(r *http.Request) string {
host, _, err := net.SplitHostPort(r.RemoteAddr)
if err != nil {
return r.RemoteAddr
}
return strings.Trim(host, "[]")
}
// sessionAuth resolves {id} and requires the bearer to be the owning device. // sessionAuth resolves {id} and requires the bearer to be the owning device.
func (s *Server) sessionAuth(w http.ResponseWriter, r *http.Request) *session.Session { func (s *Server) sessionAuth(w http.ResponseWriter, r *http.Request) *session.Session {
dev := s.Store.DeviceByCredential(bearer(r)) dev := s.Store.DeviceByCredential(bearer(r))
@@ -218,10 +280,20 @@ func (s *Server) observations(w http.ResponseWriter, r *http.Request) {
dnsCanary = s.CanaryQueries(sess.ID[:16]) // the session's wire prefix dnsCanary = s.CanaryQueries(sess.ID[:16]) // the session's wire prefix
} }
writeJSON(w, http.StatusOK, map[string]any{ writeJSON(w, http.StatusOK, map[string]any{
"udp": map[string]any{"packets_seen": packetsSeen, "packets": udp}, "udp": map[string]any{
"tcp": tcp, "packets_seen": packetsSeen,
"packets": udp,
// The per-train received view (spec §6 "trains"), columnar like the wire report —
// the flat packet list above stays for clients that predate trains.
"trains": trainsJSON(sess.Trains()),
},
"tcp": tcp,
"connect_back": cb, "connect_back": cb,
"dns_canary": dnsCanary, // The sender's own count, which is what makes the receiver's count mean something.
"throughput": sess.ThroughputReports(),
// The receiver's count for upstream runs — same idea, other direction.
"throughput_up": upstreamJSON(sess),
"dns_canary": dnsCanary,
// TODO(spec §6): http echo records // TODO(spec §6): http echo records
}) })
} }
@@ -243,9 +315,14 @@ func (s *Server) actions(w http.ResponseWriter, r *http.Request) {
SizeBytes int `json:"size_bytes"` SizeBytes int `json:"size_bytes"`
IntervalUs int `json:"interval_us"` IntervalUs int `json:"interval_us"`
SizesBytes []int `json:"sizes_bytes"` SizesBytes []int `json:"sizes_bytes"`
DSCP *int `json:"dscp"`
DF *bool `json:"df"` DF *bool `json:"df"`
Mode string `json:"mode"` Mode string `json:"mode"`
FragBytes int `json:"frag_bytes"` FragBytes int `json:"frag_bytes"`
Direction string `json:"direction"`
DurationS int `json:"duration_s"`
Kbps int `json:"kbps"`
Streams int `json:"streams"`
} }
if err := json.NewDecoder(r.Body).Decode(&req); err != nil { if err := json.NewDecoder(r.Body).Decode(&req); err != nil {
writeJSON(w, http.StatusBadRequest, map[string]string{"error": "bad body"}) writeJSON(w, http.StatusBadRequest, map[string]string{"error": "bad body"})
@@ -301,6 +378,11 @@ func (s *Server) actions(w http.ResponseWriter, r *http.Request) {
writeJSON(w, http.StatusNotImplemented, map[string]string{"error": "downtrain not wired"}) writeJSON(w, http.StatusNotImplemented, map[string]string{"error": "downtrain not wired"})
return return
} }
dscp, err := dscpArg(req.DSCP)
if err != nil {
writeJSON(w, http.StatusBadRequest, map[string]string{"error": err.Error()})
return
}
// A downstream train sends far more than it receives, so it needs a grant (§3.4). // A downstream train sends far more than it receives, so it needs a grant (§3.4).
count := clamp(req.Count, 1, 5000) count := clamp(req.Count, 1, 5000)
size := clamp(req.SizeBytes, dataMinPacket, 1500) size := clamp(req.SizeBytes, dataMinPacket, 1500)
@@ -311,13 +393,20 @@ func (s *Server) actions(w http.ResponseWriter, r *http.Request) {
return return
} }
go func() { go func() {
sent, err := s.DownTrain(sess, g, count, size, interval) sent, err := s.DownTrain(sess, g, count, size, interval, dscp)
slog.Info("downtrain finished", "action", actionID, "sent", sent, "bytes", g.Sent(), "err", err) slog.Info("downtrain finished", "action", actionID, "sent", sent, "bytes", g.Sent(), "err", err)
}() }()
writeJSON(w, http.StatusAccepted, map[string]any{ resp := map[string]any{
"action_id": actionID, "count": count, "size_bytes": size, "interval_us": interval, "action_id": actionID, "count": count, "size_bytes": size, "interval_us": interval,
"grant": map[string]any{"max_bytes": g.MaxBytes, "max_kbps": g.MaxKbps}, "grant": map[string]any{"max_bytes": g.MaxBytes, "max_kbps": g.MaxKbps},
}) }
if dscp >= 0 {
resp["dscp"] = dscp
// Told up front, not discovered: a client measuring DSCP survival on a burst the
// server could not mark would conclude the network stripped it.
resp["dscp_applied"] = dataplane.TOSSupported
}
writeJSON(w, http.StatusAccepted, resp)
case "big_send": case "big_send":
if s.BigSend == nil { if s.BigSend == nil {
@@ -417,6 +506,66 @@ func (s *Server) actions(w http.ResponseWriter, r *http.Request) {
"grant": map[string]any{"max_bytes": g.MaxBytes, "max_kbps": g.MaxKbps}, "grant": map[string]any{"max_bytes": g.MaxBytes, "max_kbps": g.MaxKbps},
}) })
case "throughput":
if s.DownThroughput == nil {
writeJSON(w, http.StatusNotImplemented, map[string]string{"error": "throughput not wired"})
return
}
// Only the downstream direction needs the server to send. Upstream is the client
// sending and the server counting, which needs no action at all — so asking for it here
// is a client bug worth naming rather than silently doing the other thing.
if req.Direction == "up" {
// Upstream needs nothing sent from here — the client generates the traffic and the
// server counts it. The only thing an action can usefully do is zero the counter so
// the run measures itself rather than inheriting an earlier one.
sess.ResetUpstream()
writeJSON(w, http.StatusAccepted, map[string]any{
"action_id": actionID, "direction": "up", "reset": true,
"note": "send TYPE_THROUGHPUT_UP packets, then read observations.throughput_up",
})
return
}
if req.Direction != "" && req.Direction != "down" {
writeJSON(w, http.StatusBadRequest, map[string]string{
"error": "direction must be up or down",
})
return
}
// Planned once, here, so the response promises exactly what the run will do. A request
// that would outlast the server's byte cap comes back with a shorter duration rather
// than being truncated halfway.
durationMs, kbps := dataplane.ThroughputPlan(
clamp(req.DurationS, 1, 30)*1000, clamp(req.Kbps, 100, 200_000))
size := clamp(req.SizeBytes, dataMinPacket, 1472)
if req.SizeBytes == 0 {
size = 1200
}
g := sess.NewGrant(actionID, 0, kbps, dataplane.ThroughputLimits(durationMs, kbps))
if g == nil {
writeJSON(w, http.StatusConflict, noDataPlaneYet)
return
}
// Answered before the run so the client can start listening, then reported through the
// observations API. Doing it the other way round would have the client miss the first
// second of a ten-second test.
writeJSON(w, http.StatusAccepted, map[string]any{
"action_id": actionID, "direction": "down",
"duration_s": durationMs / 1000, "duration_ms": durationMs,
"requested_duration_s": clamp(req.DurationS, 1, 30),
"kbps": kbps, "size_bytes": size,
"grant": map[string]any{"max_bytes": g.MaxBytes, "max_kbps": g.MaxKbps},
})
if f, ok := w.(http.Flusher); ok {
f.Flush()
}
go func() {
result, err := s.DownThroughput(sess, g, durationMs, kbps, size)
slog.Info("throughput finished", "action", actionID, "packets", result.Packets,
"bytes", result.Bytes, "kbps", result.Kbps, "limited_by", result.LimitedBy, "err", err)
sess.RecordThroughput(actionID, result.Packets, result.Bytes, result.DurationMs,
result.Kbps, result.LimitedBy)
}()
default: default:
writeJSON(w, http.StatusBadRequest, map[string]string{"error": "unknown or unimplemented action"}) writeJSON(w, http.StatusBadRequest, map[string]string{"error": "unknown or unimplemented action"})
} }
@@ -440,8 +589,54 @@ func (s *Server) maxDFPayload(sess *session.Session) int {
return mtu - overhead return mtu - overhead
} }
// dataMinPacket is the smallest datagram that still carries a header + a little payload. // dscpArg validates the optional downtrain dscp parameter (spec §5). Absent means -1: leave the
const dataMinPacket = 40 // socket's default marking alone, which is different from asking for DSCP 0 (explicitly
// best-effort). Out-of-range values are refused rather than clamped — a clamped 46→63 would mark
// the burst with a class the client never asked for and silently change what the test measures.
func dscpArg(v *int) (int, error) {
if v == nil {
return -1, nil
}
if *v < 0 || *v > 63 {
return 0, fmt.Errorf("dscp %d is out of range: the field is 6 bits (0..63)", *v)
}
return *v, nil
}
// trainsJSON renders the per-train received view columnar — one array per field, matching the
// wire report and the schema's train-evidence shape — with []int for the byte-wide columns
// because encoding/json would base64 a []uint8.
func trainsJSON(trains []session.Train) []map[string]any {
out := make([]map[string]any, 0, len(trains))
for _, t := range trains {
n := len(t.Entries)
seq := make([]uint32, n)
trx := make([]int64, n)
size := make([]int, n)
ttl := make([]int, n)
dscp := make([]int, n)
ecn := make([]int, n)
for i, e := range t.Entries {
seq[i], trx[i], size[i] = e.Seq, e.TRxNs, int(e.Size)
ttl[i], dscp[i], ecn[i] = int(e.TTL), int(e.DSCP), int(e.ECN)
}
out = append(out, map[string]any{
"train_id": t.ID,
// The loss denominator: every packet counted, whether or not its row was kept.
"packets_received": t.Received,
"truncated": t.Truncated,
"seq": seq, "t_rx_ns": trx, "size": size,
"ttl": ttl, "dscp": dscp, "ecn": ecn,
})
}
return out
}
// dataMinPacket is the smallest granted datagram: header + 16 payload bytes, because [8:16] of
// every granted payload carries the action id. It must match what the senders raise short sizes
// to, or a minimum-size train's grant is budgeted for fewer bytes than actually leave and the
// train is cut short by its own arithmetic.
const dataMinPacket = dataplane.HeaderSize + 16
var noDataPlaneYet = map[string]string{ var noDataPlaneYet = map[string]string{
"error": "no data-plane traffic seen yet — send an ECHO first so the destination is verified", "error": "no data-plane traffic seen yet — send an ECHO first so the destination is verified",
@@ -533,13 +728,7 @@ func (s *Server) profile(w http.ResponseWriter, r *http.Request) {
// modified builds and gives clients provenance for the measurement. // modified builds and gives clients provenance for the measurement.
"source_url": "", // TODO: stamp from build metadata "source_url": "", // TODO: stamp from build metadata
"capabilities": s.Capabilities, "capabilities": s.Capabilities,
"targets": []map[string]any{{ "targets": []map[string]any{s.target(host)},
"id": s.Name,
"ip4": host, // TODO: explicit configured addresses, v6, second STUN addr
"udp_port": s.UDPPort,
"tcp_port": s.TCPPort,
"stun_port": s.StunPort,
}},
"pins": []string{"pin-sha256:" + s.PinB64}, "pins": []string{"pin-sha256:" + s.PinB64},
"next_pins": []string{}, "next_pins": []string{},
"canary_zone": s.CanaryZone, "canary_zone": s.CanaryZone,
@@ -548,6 +737,10 @@ func (s *Server) profile(w http.ResponseWriter, r *http.Request) {
// The app needs the upload rules before it offers the switch: whether uploads are // The app needs the upload rules before it offers the switch: whether uploads are
// accepted at all, and how much identifying detail it must strip first. // accepted at all, and how much identifying detail it must strip first.
"uploads": s.uploadPolicy(), "uploads": s.uploadPolicy(),
// What a client needs to start a sign-in, without hard-coding the operator's IdP into
// the app: where to authorize, which client id to use, and whether it is worth offering
// sign-in at all on this server.
"auth": s.authInfo(r.Context()),
// What this build speaks, and which app versions it will serve. A client checks the // What this build speaks, and which app versions it will serve. A client checks the
// server side of the same question against its own bounds. // server side of the same question against its own bounds.
"compat": map[string]any{ "compat": map[string]any{
@@ -644,7 +837,7 @@ func (s *Server) uploadRun(w http.ResponseWriter, r *http.Request) {
writeJSON(w, http.StatusBadRequest, map[string]string{"error": "read failed"}) writeJSON(w, http.StatusBadRequest, map[string]string{"error": "read failed"})
return return
} }
meta, err := s.Runs.Put(dev.ID, body) meta, err := s.Runs.Put(dev.ID, body, dev.LinkedToAccount())
switch { switch {
case err == nil: case err == nil:
slog.Info("run uploaded", "device", dev.ID, "run", meta.ID, slog.Info("run uploaded", "device", dev.ID, "run", meta.ID,
@@ -670,11 +863,19 @@ func (s *Server) listRuns(w http.ResponseWriter, r *http.Request) {
writeJSON(w, http.StatusUnauthorized, map[string]string{"error": "unknown credential"}) writeJSON(w, http.StatusUnauthorized, map[string]string{"error": "unknown credential"})
return return
} }
list := s.Runs.List(dev.ID) list := s.Runs.ListFor(s.visibleDevices(dev))
if list == nil { if list == nil {
list = []runs.Meta{} list = []runs.Meta{}
} }
writeJSON(w, http.StatusOK, map[string]any{"runs": list}) writeJSON(w, http.StatusOK, map[string]any{
"runs": list,
// Says whose history this is, so a client can show "3 devices" rather than leaving the
// user to wonder why runs from another phone appeared.
"scope": map[string]any{
"account_id": dev.AccountID,
"devices": len(s.visibleDevices(dev)),
},
})
} }
func (s *Server) getRun(w http.ResponseWriter, r *http.Request) { func (s *Server) getRun(w http.ResponseWriter, r *http.Request) {
@@ -683,9 +884,14 @@ func (s *Server) getRun(w http.ResponseWriter, r *http.Request) {
writeJSON(w, http.StatusUnauthorized, map[string]string{"error": "unknown credential"}) writeJSON(w, http.StatusUnauthorized, map[string]string{"error": "unknown credential"})
return return
} }
// Scoped to the calling device's own directory: one device cannot read another's runs by // Resolved against the caller's own devices only, so a run id from another account is not
// guessing a run id. // found rather than being fetched from wherever it happens to live.
b, err := s.Runs.Get(dev.ID, r.PathValue("id")) owner, ok := s.Runs.OwnerOf(s.visibleDevices(dev), r.PathValue("id"))
if !ok {
writeJSON(w, http.StatusNotFound, map[string]string{"error": "no such run"})
return
}
b, err := s.Runs.Get(owner, r.PathValue("id"))
if err != nil { if err != nil {
writeJSON(w, http.StatusNotFound, map[string]string{"error": "no such run"}) writeJSON(w, http.StatusNotFound, map[string]string{"error": "no such run"})
return return
@@ -700,7 +906,12 @@ func (s *Server) deleteRun(w http.ResponseWriter, r *http.Request) {
writeJSON(w, http.StatusUnauthorized, map[string]string{"error": "unknown credential"}) writeJSON(w, http.StatusUnauthorized, map[string]string{"error": "unknown credential"})
return return
} }
if err := s.Runs.Delete(dev.ID, r.PathValue("id")); err != nil { owner, ok := s.Runs.OwnerOf(s.visibleDevices(dev), r.PathValue("id"))
if !ok {
w.WriteHeader(http.StatusNoContent) // delete is idempotent; absent is the desired state
return
}
if err := s.Runs.Delete(owner, r.PathValue("id")); err != nil {
writeJSON(w, http.StatusInternalServerError, map[string]string{"error": err.Error()}) writeJSON(w, http.StatusInternalServerError, map[string]string{"error": err.Error()})
return return
} }
@@ -722,9 +933,172 @@ func maxOrEmpty(r compat.Range) string {
// three parts at once — its own URL, its own SPKI pin, and the token. An operator copying a pin // three parts at once — its own URL, its own SPKI pin, and the token. An operator copying a pin
// by hand is the step that goes wrong, and a pin wrong by one character does not fail loudly. // by hand is the step that goes wrong, and a pin wrong by one character does not fail loudly.
func (s *Server) EnrollmentLink(token string) string { func (s *Server) EnrollmentLink(token string) string {
u := s.PublicControlURL return EnrollmentURI(s.PublicControlURL, s.PinB64, token)
}
// EnrollmentURI is the same assembly without a running server, for the mint-a-link CLI action.
// Shared rather than reimplemented: two copies of this encoding would eventually disagree, and
// the failure mode is a pin that looks right and produces an inscrutable TLS error.
func EnrollmentURI(publicURL, pinB64, token string) string {
return "echolot://enroll?v=1" + return "echolot://enroll?v=1" +
"&u=" + url.QueryEscape(strings.TrimRight(u, "/")) + "&u=" + url.QueryEscape(strings.TrimRight(publicURL, "/")) +
"&p=" + url.QueryEscape("pin-sha256:"+s.PinB64) + "&p=" + url.QueryEscape("pin-sha256:"+pinB64) +
"&t=" + url.QueryEscape(token) "&t=" + url.QueryEscape(token)
} }
// target describes where this server can be measured, so a client can say which address a result
// came from instead of "the server".
//
// The alternates matter as much as the primaries: RFC 5780 behaviour discovery needs a second
// address to redirect to, and an operator reading a report needs to know which of their addresses
// a finding refers to. [fallback] is used only when nothing was configured explicitly, so a server
// that has not been told its own addresses still answers with something usable.
func (s *Server) target(fallback string) map[string]any {
t := map[string]any{
"id": s.Name,
"udp_port": s.UDPPort,
"tcp_port": s.TCPPort,
"stun_port": s.StunPort,
}
ip4 := s.IP4
if ip4 == "" {
ip4 = fallback
}
for k, v := range map[string]string{
"ip4": ip4, "ip6": s.IP6, "ip4_alt": s.IP4Alt, "ip6_alt": s.IP6Alt,
} {
if v != "" {
t[k] = v
}
}
return t
}
// upstreamJSON renders the upstream tally with the derived figures already computed, so every
// consumer does not have to repeat (and risk fumbling) the same arithmetic.
func upstreamJSON(sess *session.Session) map[string]any {
u := sess.Upstream()
return map[string]any{
"packets": u.Packets, "bytes": u.Bytes,
"span_ms": u.SpanMs(), "kbps": u.Kbps(),
}
}
// authInfo advertises the sign-in configuration, so the app can present a Sign in button only
// when there is something behind it, and can drive the flow without the user typing an issuer URL.
func (s *Server) authInfo(ctx context.Context) map[string]any {
if s.OIDC == nil || !s.OIDC.Config().Enabled() {
return map[string]any{"enabled": false}
}
cfg := s.OIDC.Config()
out := map[string]any{
"enabled": true,
"issuer": cfg.Issuer,
// The app's client, not the server's: this is what a phone should authorize as.
"client_id": cfg.AppClientID,
// The app is a public client on a phone: no secret can be kept, so PKCE is what
// protects the code exchange (RFC 7636), and the redirect comes back through the
// scheme the app already registers for enrollment links.
"flow": "authorization_code+pkce",
"redirect_uri": "echolot://auth",
"scopes": "openid profile email",
}
if d, err := s.OIDC.Discover(ctx); err == nil {
out["authorization_endpoint"] = d.AuthorizationEndpoint
out["token_endpoint"] = d.TokenEndpoint
out["end_session_endpoint"] = d.EndSessionEndpoint
} else {
// Reported rather than hidden: an unreachable IdP is the operator's problem to see, and
// a client that knows the difference can say "sign-in is configured but the provider is
// not answering" instead of failing obscurely.
out["discovery_error"] = err.Error()
}
return out
}
// linkAccount ties the calling device to the person whose ID token it presents.
//
// The device credential proves *which device*; the ID token proves *which person*. Both are
// required, and neither substitutes for the other: enrollment admits a device to the server,
// signing in attributes it to someone.
func (s *Server) linkAccount(w http.ResponseWriter, r *http.Request) {
dev := s.Store.DeviceByCredential(bearer(r))
if dev == nil {
writeJSON(w, http.StatusUnauthorized, map[string]string{"error": "unknown credential"})
return
}
if s.OIDC == nil || !s.OIDC.Config().Enabled() {
writeJSON(w, http.StatusNotImplemented, map[string]string{
"error": "this server has no identity provider configured, so there is nothing to sign in to",
})
return
}
var body struct {
IDToken string `json:"id_token"`
}
if err := json.NewDecoder(r.Body).Decode(&body); err != nil || body.IDToken == "" {
writeJSON(w, http.StatusBadRequest, map[string]string{"error": "expected an id_token"})
return
}
claims, err := s.OIDC.Verify(r.Context(), body.IDToken)
if err != nil {
// Deliberately terse to the client and detailed in the log: a caller probing token
// handling should not be told which check it failed.
slog.Info("rejected sign-in", "device", dev.ID, "err", err)
writeJSON(w, http.StatusForbidden, map[string]string{"error": "the identity token was not accepted"})
return
}
if err := s.Store.LinkAccount(dev.ID, claims.AccountID(), claims.Display()); err != nil {
writeJSON(w, http.StatusInternalServerError, map[string]string{"error": err.Error()})
return
}
slog.Info("device linked to account", "device", dev.ID, "account", claims.AccountID(),
"admin", s.OIDC.IsAdmin(claims))
writeJSON(w, http.StatusOK, map[string]any{
"account_id": claims.AccountID(), "display_name": claims.Display(),
"admin": s.OIDC.IsAdmin(claims),
})
}
// unlinkAccount signs out on this device. The device stays enrolled: signing out should not cost
// someone their enrollment, which an operator had to grant.
func (s *Server) unlinkAccount(w http.ResponseWriter, r *http.Request) {
dev := s.Store.DeviceByCredential(bearer(r))
if dev == nil {
writeJSON(w, http.StatusUnauthorized, map[string]string{"error": "unknown credential"})
return
}
if err := s.Store.LinkAccount(dev.ID, "", ""); err != nil {
writeJSON(w, http.StatusInternalServerError, map[string]string{"error": err.Error()})
return
}
w.WriteHeader(http.StatusNoContent)
}
func (s *Server) accountStatus(w http.ResponseWriter, r *http.Request) {
dev := s.Store.DeviceByCredential(bearer(r))
if dev == nil {
writeJSON(w, http.StatusUnauthorized, map[string]string{"error": "unknown credential"})
return
}
writeJSON(w, http.StatusOK, map[string]any{
"signed_in": dev.LinkedToAccount(),
"account_id": dev.AccountID,
"display_name": dev.AccountName,
"device_id": dev.ID,
})
}
// visibleDevices is the set of devices whose runs the caller may read.
//
// Signed in: every device on the same account, which is what an account is for. Not signed in:
// only itself — anonymous devices are not a group, and treating the absent account as a shared
// one would let any of them read all the others.
func (s *Server) visibleDevices(dev *store.Device) []string {
if dev.LinkedToAccount() {
if ids := s.Store.DeviceIDsForAccount(dev.AccountID); len(ids) > 0 {
return ids
}
}
return []string{dev.ID}
}
+27
View File
@@ -0,0 +1,27 @@
// SPDX-FileCopyrightText: 2026 Echolot contributors
// SPDX-License-Identifier: GPL-3.0-or-later
package control
import "testing"
func TestDscpArg(t *testing.T) {
ptr := func(v int) *int { return &v }
for _, tc := range []struct {
in *int
want int
wantErr bool
}{
{nil, -1, false}, // absent: leave the socket alone
{ptr(0), 0, false}, // explicit best-effort is not the same as absent
{ptr(46), 46, false}, // EF, the value people actually test with
{ptr(63), 63, false},
{ptr(64), 0, true}, // one past the 6-bit field
{ptr(-1), 0, true},
} {
got, err := dscpArg(tc.in)
if (err != nil) != tc.wantErr || got != tc.want {
t.Errorf("dscpArg(%v) = %d, err=%v; want %d, wantErr=%v", tc.in, got, err, tc.want, tc.wantErr)
}
}
}
+70
View File
@@ -0,0 +1,70 @@
// SPDX-FileCopyrightText: 2026 Echolot contributors
// SPDX-License-Identifier: GPL-3.0-or-later
//go:build linux
package dataplane
import (
"encoding/binary"
"net"
"syscall"
)
// Per-packet TTL and TOS/traffic-class arrive as control messages, and only if asked for at
// socket setup. These fill the spec §3.3 observation-block fields that were shipped as the 0xFF
// sentinel until now — received TTL is path-length evidence, the TOS byte is DSCP/ECN survival.
// enableRecvMeta asks the kernel to attach the cmsgs to every received datagram. Both the v4 and
// the v6 option sets are attempted on every socket: a dual-stack socket delivers v4-mapped
// traffic through the v6 fd, and the kernel refuses whichever set does not apply. Errors are
// dropped on purpose — a socket that cannot deliver metadata still serves probes, and the
// sentinel already says "not observed" for it.
func enableRecvMeta(conn *net.UDPConn) {
raw, err := conn.SyscallConn()
if err != nil {
return
}
_ = raw.Control(func(fd uintptr) {
_ = syscall.SetsockoptInt(int(fd), syscall.IPPROTO_IP, syscall.IP_RECVTTL, 1)
_ = syscall.SetsockoptInt(int(fd), syscall.IPPROTO_IP, syscall.IP_RECVTOS, 1)
_ = syscall.SetsockoptInt(int(fd), syscall.IPPROTO_IPV6, syscall.IPV6_RECVHOPLIMIT, 1)
_ = syscall.SetsockoptInt(int(fd), syscall.IPPROTO_IPV6, syscall.IPV6_RECVTCLASS, 1)
})
}
// parseMeta extracts TTL and the TOS byte from one datagram's control messages. Anything absent
// or unparseable keeps the sentinel — reported as unobserved, never guessed.
func parseMeta(oob []byte) pktMeta {
m := pktMeta{TTL: metaUnavailable, TOS: metaUnavailable}
if len(oob) == 0 {
return m
}
cmsgs, err := syscall.ParseSocketControlMessage(oob)
if err != nil {
return m
}
for _, c := range cmsgs {
switch {
case c.Header.Level == syscall.IPPROTO_IP && c.Header.Type == syscall.IP_TTL,
c.Header.Level == syscall.IPPROTO_IPV6 && c.Header.Type == syscall.IPV6_HOPLIMIT:
m.TTL = cmsgValue(c.Data)
case c.Header.Level == syscall.IPPROTO_IP && c.Header.Type == syscall.IP_TOS,
c.Header.Level == syscall.IPPROTO_IPV6 && c.Header.Type == syscall.IPV6_TCLASS:
m.TOS = cmsgValue(c.Data)
}
}
return m
}
// cmsgValue reads a cmsg the kernel encodes either as a native-endian int (IP_TTL,
// IPV6_HOPLIMIT, IPV6_TCLASS) or as a single byte (IP_TOS). Both fit a byte by definition.
func cmsgValue(data []byte) uint8 {
switch {
case len(data) >= 4:
return uint8(binary.NativeEndian.Uint32(data))
case len(data) >= 1:
return data[0]
}
return metaUnavailable
}
+15
View File
@@ -0,0 +1,15 @@
// SPDX-FileCopyrightText: 2026 Echolot contributors
// SPDX-License-Identifier: GPL-3.0-or-later
//go:build !linux
package dataplane
import "net"
// Per-packet TTL/TOS needs Linux's IP_RECVTTL-family cmsgs. Elsewhere the observation block and
// train buffers keep the spec §3.3 sentinel (0xFF = not observed) — absent is honest, a guess
// is not. Deployment targets are Linux; this build exists so the Windows dev loop compiles.
func enableRecvMeta(_ *net.UDPConn) {}
func parseMeta(_ []byte) pktMeta { return pktMeta{TTL: metaUnavailable, TOS: metaUnavailable} }
+6 -3
View File
@@ -104,8 +104,8 @@ func (s *Server) FragSend(
return res, fmt.Errorf("session has no recorded local address") return res, fmt.Errorf("session has no recorded local address")
} }
if sizeBytes < HeaderSize+8 { if sizeBytes < HeaderSize+32 {
sizeBytes = HeaderSize + 8 sizeBytes = HeaderSize + 32
} }
if sizeBytes > 8000 { if sizeBytes > 8000 {
sizeBytes = 8000 sizeBytes = 8000
@@ -117,7 +117,10 @@ func (s *Server) FragSend(
// The ELT1 packet, signed exactly as any other, then wrapped in UDP. // The ELT1 packet, signed exactly as any other, then wrapped in UDP.
payload := make([]byte, sizeBytes-HeaderSize) payload := make([]byte, sizeBytes-HeaderSize)
binary.BigEndian.PutUint32(payload[0:4], uint32(sizeBytes)) binary.BigEndian.PutUint32(payload[0:4], uint32(sizeBytes))
copy(payload[4:], mode) // [8:16]: the action id, as on every granted packet (spec §5 correlation); the mode string
// sits past the reserved slot.
putActionID(payload, g.ActionID)
copy(payload[16:], mode)
elt := s.buildPacket(sess, TypeFragData, 0, payload) elt := s.buildPacket(sess, TypeFragData, 0, payload)
udp := buildUDP(local, target, elt) udp := buildUDP(local, target, elt)
+39 -18
View File
@@ -21,7 +21,11 @@ import (
// client measures downstream loss, reordering and jitter from what arrives — the direction an // client measures downstream loss, reordering and jitter from what arrives — the direction an
// upstream-only train cannot see. Returns how many packets actually went out (the grant may cut // upstream-only train cannot see. Returns how many packets actually went out (the grant may cut
// it short, which is itself reportable). // it short, which is itself reportable).
func (s *Server) DownTrain(sess *session.Session, g *session.Grant, count, sizeBytes, intervalUs int) (int, error) { //
// dscp ≥ 0 marks the burst (spec §5 downtrain `dscp`): downstream DSCP survival is the half the
// client cannot produce itself. Best-effort off Linux — see withTOS/TOSSupported; the action
// response has already told the client whether the marking was applied.
func (s *Server) DownTrain(sess *session.Session, g *session.Grant, count, sizeBytes, intervalUs, dscp int) (int, error) {
target := sess.DataSource() target := sess.DataSource()
if !target.IsValid() { if !target.IsValid() {
return 0, fmt.Errorf("no observed data-plane source") return 0, fmt.Errorf("no observed data-plane source")
@@ -30,26 +34,41 @@ func (s *Server) DownTrain(sess *session.Session, g *session.Grant, count, sizeB
if conn == nil { if conn == nil {
return 0, fmt.Errorf("no data-plane socket matches target family") return 0, fmt.Errorf("no data-plane socket matches target family")
} }
if sizeBytes < HeaderSize+8 { if sizeBytes < HeaderSize+16 {
sizeBytes = HeaderSize + 8 sizeBytes = HeaderSize + 16
} }
payload := make([]byte, sizeBytes-HeaderSize) payload := make([]byte, sizeBytes-HeaderSize)
// Payload [8:16] carries the action id on every granted packet, so arriving traffic can be
// attributed to the action that caused it (spec §5/§9: test.params.action_id).
putActionID(payload, g.ActionID)
sent := 0 sent := 0
for i := 0; i < count; i++ { burst := func() error {
if !g.Allow(sizeBytes) { for i := 0; i < count; i++ {
break // budget or rate exhausted — stop, do not sleep it off if !g.Allow(sizeBytes) {
} break // budget or rate exhausted — stop, do not sleep it off
// Sequence + send timestamp in the payload head so the client can order and time them }
// even when packets arrive out of order. // Sequence + send timestamp in the payload head so the client can order and time them
binary.BigEndian.PutUint32(payload[0:4], uint32(i)) // even when packets arrive out of order.
binary.BigEndian.PutUint32(payload[4:8], uint32(time.Since(s.start).Microseconds())) binary.BigEndian.PutUint32(payload[0:4], uint32(i))
s.send(conn, target, sess, TypeDownTrainData, uint32(i), payload) binary.BigEndian.PutUint32(payload[4:8], uint32(time.Since(s.start).Microseconds()))
sent++ s.send(conn, target, sess, TypeDownTrainData, uint32(i), payload)
if intervalUs > 0 && i < count-1 { sent++
time.Sleep(time.Duration(intervalUs) * time.Microsecond) if intervalUs > 0 && i < count-1 {
time.Sleep(time.Duration(intervalUs) * time.Microsecond)
}
} }
return nil
} }
return sent, nil if dscp >= 0 && TOSSupported {
// Same shared-socket borrow as the DF window: dfMu keeps a concurrent burst from riding
// along with — or clearing — this marking.
s.dfMu.Lock()
defer s.dfMu.Unlock()
err := withTOS(conn, dscp, burst) // sent must be read after the burst ran, not before
return sent, err
}
err := burst()
return sent, err
} }
// BigSendResult records what happened to one requested size. `Sent` false with an EMSGSIZE-ish // BigSendResult records what happened to one requested size. `Sent` false with an EMSGSIZE-ish
@@ -83,8 +102,8 @@ func (s *Server) BigSend(sess *session.Session, g *session.Grant, sizes []int, d
results := make([]BigSendResult, 0, len(sizes)) results := make([]BigSendResult, 0, len(sizes))
burst := func() error { burst := func() error {
for i, size := range sizes { for i, size := range sizes {
if size < HeaderSize+8 { if size < HeaderSize+16 {
size = HeaderSize + 8 size = HeaderSize + 16
} }
if size > 9000 { // jumbo ceiling; beyond this the kernel will refuse anyway if size > 9000 { // jumbo ceiling; beyond this the kernel will refuse anyway
size = 9000 size = 9000
@@ -96,6 +115,8 @@ func (s *Server) BigSend(sess *session.Session, g *session.Grant, sizes []int, d
// Echo the intended size into the payload so a truncated/fragmented arrival is // Echo the intended size into the payload so a truncated/fragmented arrival is
// still attributable to the size we meant to send. // still attributable to the size we meant to send.
binary.BigEndian.PutUint32(payload[0:4], uint32(size)) binary.BigEndian.PutUint32(payload[0:4], uint32(size))
// [8:16]: the action id, as on every granted packet (spec §5 correlation).
putActionID(payload, g.ActionID)
err := s.sendErr(conn, target, sess, TypeBigSend, uint32(i), payload) err := s.sendErr(conn, target, sess, TypeBigSend, uint32(i), payload)
results = append(results, BigSendResult{ results = append(results, BigSendResult{
SizeBytes: size, Seq: i, Sent: err == nil, Err: errString(err), SizeBytes: size, Seq: i, Sent: err == nil, Err: errString(err),
+191
View File
@@ -0,0 +1,191 @@
// SPDX-FileCopyrightText: 2026 Echolot contributors
// SPDX-License-Identifier: GPL-3.0-or-later
package dataplane
import (
"encoding/binary"
"fmt"
"time"
"echo-lot.app/server/internal/session"
)
// Sustained-rate sending (spec §5 throughput).
//
// This is the most expensive thing the server will do on a client's say-so, so it is also the
// action where the §3.4 anti-amplification rules matter most. Three bounds apply, and all three
// are enforced here rather than trusted to the caller:
//
// - the destination is the session's *observed* data-plane source, verified by an HMAC-signed
// ECHO that arrived from that address, so this cannot be aimed at a third party;
// - the grant carries a byte budget and an average-rate ceiling, and the send stops the moment
// either is reached;
// - the duration is hard-capped, so a client that vanishes mid-test costs a bounded amount of
// traffic rather than an open-ended one.
//
// The measurement this produces is honest only if the client is told which limit it hit. A run
// that saturates the grant ceiling has measured *us*, not the network, and reporting that as
// throughput would be worse than not measuring at all — see ThroughputResult.LimitedBy.
// ThroughputResult is what the server actually managed to send.
type ThroughputResult struct {
Packets int `json:"packets"`
Bytes int64 `json:"bytes"`
DurationMs int64 `json:"duration_ms"`
Kbps int `json:"kbps"`
// LimitedBy says what stopped it: "duration" (ran the full time, so the rate is the path's
// or ours to give), "budget" (hit the grant's byte ceiling), or "rate" (the pacing ceiling
// held it back). Only "duration" makes the number a property of the network.
LimitedBy string `json:"limited_by"`
}
// ThroughputLimits derives a grant sized for one throughput run.
//
// The default 8 MiB action budget is deliberately far too small for this — ten seconds at
// 50 Mbps is 62 MB — so throughput gets its own budget computed from what it asked for, still
// clamped to a ceiling. Sizing the budget to the request (rather than raising the global default)
// keeps every *other* action bounded at 8 MiB.
func ThroughputLimits(durationMs, kbps int) session.GrantLimits {
durationMs, kbps = ThroughputPlan(durationMs, kbps)
// bytes = kbps * 1000 / 8 * seconds, with a little headroom so the byte budget is not what
// stops a run that was meant to be stopped by the clock.
budget := int64(kbps) * 1000 / 8 * int64(durationMs) / 1000
budget = budget * 11 / 10
if budget > maxThroughputBytes {
budget = maxThroughputBytes
}
return session.GrantLimits{
MaxBytes: budget,
// A little above the pacing target on purpose: the pacer should be what controls the
// rate, and the grant should be the safety net. If they are equal, ordinary scheduling
// jitter trips the grant and the run is cut short for no real reason.
MaxKbps: kbps * 12 / 10,
MaxHold: time.Duration(durationMs)*time.Millisecond + 5*time.Second,
}
}
// ThroughputPlan reduces a request to what this server will actually run, and is the single
// place that decides it.
//
// When the byte cap binds before the clock does, the *duration* is shortened rather than the run
// being cut off partway. Truncating mid-run is not wrong exactly — the rate is still computed
// over the elapsed time and limited_by says "budget" — but it means promising a client thirty
// seconds and giving it twenty-one. Saying "twenty-one seconds" up front is the same information
// without the surprise, and it keeps "the clock ended the run" as the normal case, which is the
// only case where the number is a clean property of the network.
func ThroughputPlan(durationMs, kbps int) (effectiveMs, effectiveKbps int) {
if durationMs <= 0 {
durationMs = 10_000
}
if durationMs > maxThroughputMs {
durationMs = maxThroughputMs
}
if kbps <= 0 || kbps > maxThroughputKbps {
kbps = maxThroughputKbps
}
bytesPerMs := int64(kbps) * 1000 / 8 / 1000
if bytesPerMs > 0 {
if maxMs := maxThroughputBytes / bytesPerMs; int64(durationMs) > maxMs {
durationMs = int(maxMs)
}
}
return durationMs, kbps
}
const (
maxThroughputMs = 30_000
maxThroughputKbps = 200_000
maxThroughputBytes = 256 << 20
)
// DownThroughput sends paced traffic toward the client for up to durationMs.
//
// Pacing is deliberate rather than "send as fast as possible": an unpaced burst measures the
// server's NIC and the first queue it meets, then collapses into loss that looks like a network
// fault. Spacing packets at the target rate makes loss mean what a reader will assume it means.
func (s *Server) DownThroughput(
sess *session.Session, g *session.Grant, durationMs, kbps, sizeBytes int,
) (ThroughputResult, error) {
res := ThroughputResult{}
target := sess.DataSource()
if !target.IsValid() {
return res, fmt.Errorf("no observed data-plane source")
}
conn := s.connFor(target, sess.DataLocal())
if conn == nil {
return res, fmt.Errorf("no data-plane socket matches target family")
}
// Same plan the grant was sized from, so the two cannot disagree.
durationMs, kbps = ThroughputPlan(durationMs, kbps)
if sizeBytes < HeaderSize+24 {
sizeBytes = 1200 // a size that survives every common path unfragmented
}
if sizeBytes > 1472 {
sizeBytes = 1472
}
// Nanoseconds between packets to hit the target rate.
perPacketNs := int64(sizeBytes) * 8 * 1_000_000 / int64(kbps)
if perPacketNs < 1_000 {
perPacketNs = 1_000
}
payload := make([]byte, sizeBytes-HeaderSize)
// [8:16]: the action id, as on every granted packet (spec §5 correlation). The send
// timestamp lives past it at [16:24]; the client reads only header fields today, so
// reserving the slot costs nothing and keeps one layout rule across granted types.
putActionID(payload, g.ActionID)
deadline := time.Now().Add(time.Duration(durationMs) * time.Millisecond)
start := time.Now()
next := start
var seq uint32
for time.Now().Before(deadline) {
ok, why := g.TryAllow(sizeBytes)
if !ok {
if why == session.RefusalRate {
// Transient: the bucket is momentarily empty. Wait for the next slot and carry
// on. Ending the run here would report a rate measured over a fraction of a
// second, which is worse than reporting no rate at all.
res.LimitedBy = "rate"
time.Sleep(time.Duration(perPacketNs))
continue
}
// Terminal: the budget is spent, or the grant expired.
res.LimitedBy = why
break
}
// Reaching here means the run is progressing normally; the clock will end it.
res.LimitedBy = "duration"
binary.BigEndian.PutUint32(payload[0:4], seq)
binary.BigEndian.PutUint64(payload[16:24], uint64(time.Since(s.start).Nanoseconds()))
if err := s.sendErr(conn, target, sess, TypeThroughputData, seq, payload); err != nil {
// A send error mid-run is a local condition (buffer full, route gone). Stop and
// report what got out rather than pretending the rest was lost on the path.
res.LimitedBy = "send_error"
break
}
res.Packets++
res.Bytes += int64(sizeBytes)
seq++
// Absolute schedule, not sleep-per-packet: sleeping a fixed interval accumulates the
// scheduler's error and drifts the achieved rate below the target over a 10-second run.
next = next.Add(time.Duration(perPacketNs))
if d := time.Until(next); d > 0 {
time.Sleep(d)
}
}
elapsed := time.Since(start)
res.DurationMs = elapsed.Milliseconds()
// bits per millisecond is kilobits per second, so no scaling constant is needed - and none
// can be got wrong. Guarded because a run that ends inside a millisecond has no rate.
if res.DurationMs > 0 {
res.Kbps = int(res.Bytes * 8 / res.DurationMs)
}
return res, nil
}
@@ -0,0 +1,99 @@
// SPDX-FileCopyrightText: 2026 Echolot contributors
// SPDX-License-Identifier: GPL-3.0-or-later
package dataplane
import (
"testing"
"time"
)
// The grant has to be big enough that the *clock* ends a throughput run, not the byte budget. Get
// this wrong and the test still "works": it stops early, reports a rate computed over a truncated
// window, and nothing anywhere says the number is meaningless. So the sizing is pinned.
func TestThroughputBudgetOutlastsTheRequestedRun(t *testing.T) {
cases := []struct{ durationMs, kbps int }{
{1_000, 1_000},
{10_000, 50_000},
{10_000, 200_000},
{30_000, 100_000},
}
for _, c := range cases {
// Against the *planned* duration, which is what will actually be run: a request the
// server shortens is answered with the shorter number, not truncated halfway.
planMs, planKbps := ThroughputPlan(c.durationMs, c.kbps)
lim := ThroughputLimits(c.durationMs, c.kbps)
needed := int64(planKbps) * 1000 / 8 * int64(planMs) / 1000
if lim.MaxBytes < needed {
t.Errorf("%d ms at %d kbps (planned %d ms) needs %d bytes, budget is %d - the run "+
"would stop early and report a rate over a truncated window",
c.durationMs, c.kbps, planMs, needed, lim.MaxBytes)
}
}
}
// The pacer should control the rate and the grant should be the safety net. If the grant's
// ceiling equals the pacing target, ordinary scheduling jitter trips it and cuts the run short
// for no real reason.
func TestGrantRateCeilingSitsAboveThePacingTarget(t *testing.T) {
lim := ThroughputLimits(10_000, 50_000)
if lim.MaxKbps <= 50_000 {
t.Fatalf("grant ceiling %d kbps is not above the 50000 kbps pacing target", lim.MaxKbps)
}
}
// A client asking for more than the server will do must get the server's number, not its own.
func TestThroughputRequestsAreClamped(t *testing.T) {
lim := ThroughputLimits(10*60*1000, 10_000_000) // ten minutes at 10 Gbps
if lim.MaxBytes > maxThroughputBytes {
t.Errorf("byte budget %d exceeds the hard cap %d", lim.MaxBytes, maxThroughputBytes)
}
if lim.MaxKbps > maxThroughputKbps*12/10 {
t.Errorf("rate ceiling %d exceeds the hard cap", lim.MaxKbps)
}
// The hold has to outlast the planned run, or the grant expires mid-send and the run is
// reported as rate-limited when it was really time-limited.
planMs, _ := ThroughputPlan(10*60*1000, 10_000_000)
if lim.MaxHold < time.Duration(planMs)*time.Millisecond {
t.Errorf("hold %v is shorter than the planned run of %d ms", lim.MaxHold, planMs)
}
}
// When the byte cap binds before the clock does, the server shortens the run and says so, rather
// than accepting thirty seconds and delivering twenty-one. Same information, no surprise - and it
// keeps "the clock ended the run" as the normal case, which is the only case where the resulting
// rate is a clean property of the network.
func TestAnOversizedRequestComesBackShorterRatherThanTruncated(t *testing.T) {
const kbps = 200_000
askedMs := 30_000
planMs, planKbps := ThroughputPlan(askedMs, kbps)
if planKbps != kbps {
t.Errorf("rate was reduced to %d; the duration should absorb the cap, not the rate", planKbps)
}
if planMs >= askedMs {
t.Fatalf("plan kept the full %d ms at %d kbps, which exceeds the %d byte cap",
askedMs, kbps, maxThroughputBytes)
}
// And what it does promise must fit.
if got := int64(planKbps) * 1000 / 8 * int64(planMs) / 1000; got > maxThroughputBytes {
t.Errorf("planned run needs %d bytes, over the %d cap", got, maxThroughputBytes)
}
}
// A short, ordinary request must come back untouched - the clamping only exists for the extremes.
func TestAnOrdinaryRequestIsNotRewritten(t *testing.T) {
planMs, planKbps := ThroughputPlan(10_000, 50_000)
if planMs != 10_000 || planKbps != 50_000 {
t.Errorf("10 s at 50 Mbps was rewritten to %d ms at %d kbps", planMs, planKbps)
}
}
// Every action other than throughput stays on the small default budget. Throughput needs a big
// one; raising the global default to suit it would quietly unbound everything else.
func TestOnlyThroughputGetsTheLargeBudget(t *testing.T) {
big := ThroughputLimits(10_000, 50_000)
if big.MaxBytes <= 8<<20 {
t.Fatalf("throughput budget %d is no larger than the default action budget", big.MaxBytes)
}
}
+55
View File
@@ -0,0 +1,55 @@
// SPDX-FileCopyrightText: 2026 Echolot contributors
// SPDX-License-Identifier: GPL-3.0-or-later
//go:build linux
package dataplane
import (
"net"
"syscall"
)
// withTOS runs fn with the socket's TOS/traffic class set to dscp<<2 (ECN bits left zero — the
// test is about DSCP survival, and claiming ECN capability we do not use would pollute it), then
// restores what was there before.
//
// Same borrow discipline as withDF: the socket is shared by every session on that family, so the
// caller must hold Server.dfMu for the whole window or a concurrent burst rides along with — or
// clears — someone else's marking.
func withTOS(conn *net.UDPConn, dscp int, fn func() error) error {
raw, err := conn.SyscallConn()
if err != nil {
return err
}
v4 := conn.LocalAddr().(*net.UDPAddr).IP.To4() != nil
level, opt := syscall.IPPROTO_IPV6, syscall.IPV6_TCLASS
if v4 {
level, opt = syscall.IPPROTO_IP, syscall.IP_TOS
}
var setErr error
prev := 0
if err := raw.Control(func(fd uintptr) {
if p, e := syscall.GetsockoptInt(int(fd), level, opt); e == nil {
prev = p
}
setErr = syscall.SetsockoptInt(int(fd), level, opt, dscp<<2)
}); err != nil {
return err
}
if setErr != nil {
return setErr
}
defer func() {
_ = raw.Control(func(fd uintptr) {
_ = syscall.SetsockoptInt(int(fd), level, opt, prev)
})
}()
return fn()
}
// TOSSupported reports whether withTOS can actually mark packets here. Exported so the control
// plane can tell the client up front that its dscp request will not be honored, instead of the
// client measuring an unmarked burst and concluding the network stripped the marking.
const TOSSupported = true
+16
View File
@@ -0,0 +1,16 @@
// SPDX-FileCopyrightText: 2026 Echolot contributors
// SPDX-License-Identifier: GPL-3.0-or-later
//go:build !linux
package dataplane
import "net"
// Setting DSCP per-burst uses IP_TOS/IPV6_TCLASS under the same fd-borrow pattern as withDF,
// which is only exercised on Linux deployments. Elsewhere the burst goes out with the default
// class and TOSSupported lets the action response say so — an unmarked burst reported as marked
// would read as "the network stripped DSCP", the exact wrong conclusion.
func withTOS(_ *net.UDPConn, _ int, fn func() error) error { return fn() }
const TOSSupported = false
+122
View File
@@ -0,0 +1,122 @@
// SPDX-FileCopyrightText: 2026 Echolot contributors
// SPDX-License-Identifier: GPL-3.0-or-later
package dataplane
// Upstream trains (spec §3.2, types 0x030x05). The client blasts TRAIN_DATA at the server and
// the server answers nothing per-packet — a reply would double the traffic and measure the
// return path at the same time. Afterwards the client asks for the server's received view with
// TRAIN_REPORT_REQ, and gets it back columnar, split across as many TRAIN_REPORT datagrams as
// it takes to stay under a safe size.
//
// Both TRAIN_DATA and TRAIN_REPORT_REQ carry the train id in payload[0:4]; the id is the
// client's to choose, unique within the session.
import (
"encoding/binary"
"net"
"net/netip"
"echo-lot.app/server/internal/session"
)
const (
// trainReportMaxDatagram caps one TRAIN_REPORT datagram at a size that survives every common
// path unfragmented. A report about loss must not itself be lost to MTU.
trainReportMaxDatagram = 1200
trainReportHeader = 16
trainReportRow = 17 // 4 seq + 8 t_rx_ns + 2 size + 1 ttl + 1 dscp + 1 ecn
)
// recordTrain buffers one TRAIN_DATA packet into its train (session-side, bounded — see
// session/train.go). A payload too short to carry the id is unreportable and stays only in the
// flat packet log, which already recorded it.
func recordTrain(sess *session.Session, payload []byte, seq uint32, size int, tRxNs int64, meta pktMeta) {
if len(payload) < 4 {
return
}
sess.RecordTrainPacket(binary.BigEndian.Uint32(payload[0:4]), session.TrainEntry{
Seq: seq, TRxNs: tRxNs, Size: uint16(min(size, 0xFFFF)),
TTL: meta.TTL, DSCP: meta.dscp(), ECN: meta.ecn(),
})
}
// trainReport answers one TRAIN_REPORT_REQ with the full columnar report.
//
// Grant-free on purpose. §3.4 caps ungranted responses at the request size, and a multi-part
// report is larger than the single REPORT_REQ that asked for it — but it cannot amplify: every
// 17-byte row accounts for one HMAC-valid TRAIN_DATA packet of at least HeaderSize+4 bytes this
// session already delivered here, so the whole report is a strict fraction of the traffic it
// describes, and it only ever goes to the session's verified source address. An unknown id gets
// a single zero-row report rather than silence — "nothing arrived" IS the measurement.
func (s *Server) trainReport(conn *net.UDPConn, raddr netip.AddrPort, sess *session.Session, payload []byte) {
if len(payload) < 4 {
return
}
id := binary.BigEndian.Uint32(payload[0:4])
train, _ := sess.TrainView(id)
train.ID = id
for i, part := range buildTrainReport(train) {
s.send(conn, raddr, sess, TypeTrainReport, uint32(i), part)
}
}
// buildTrainReport lays a train's received view out columnar and cuts it into datagram-sized
// payloads. Layout (mirrored by the client; all big-endian):
//
// 0 4 train_id
// 4 4 received (every packet counted, buffered or not — loss math uses this)
// 8 2 part (0-based)
// 10 2 parts
// 12 1 flags (bit0: buffer overflowed; rows beyond the cap were counted, not kept —
// the schema's evidence_truncated honesty, on the wire)
// 13 1 reserved
// 14 2 n (rows in this part)
// 16 n×4 seq, n×8 t_rx_ns, n×2 size, n×1 ttl, n×1 dscp, n×1 ecn (columns contiguous)
func buildTrainReport(t session.Train) [][]byte {
perPart := (trainReportMaxDatagram - HeaderSize - trainReportHeader) / trainReportRow
parts := (len(t.Entries) + perPart - 1) / perPart
if parts == 0 {
parts = 1 // an empty train still gets its "received: 0" answer
}
out := make([][]byte, 0, parts)
for p := 0; p < parts; p++ {
rows := t.Entries[p*perPart : min((p+1)*perPart, len(t.Entries))]
n := len(rows)
b := make([]byte, trainReportHeader+n*trainReportRow)
binary.BigEndian.PutUint32(b[0:4], t.ID)
binary.BigEndian.PutUint32(b[4:8], uint32(t.Received))
binary.BigEndian.PutUint16(b[8:10], uint16(p))
binary.BigEndian.PutUint16(b[10:12], uint16(parts))
if t.Truncated {
b[12] = 1
}
binary.BigEndian.PutUint16(b[14:16], uint16(n))
off := trainReportHeader
for i, r := range rows {
binary.BigEndian.PutUint32(b[off+i*4:], r.Seq)
}
off += n * 4
for i, r := range rows {
binary.BigEndian.PutUint64(b[off+i*8:], uint64(r.TRxNs))
}
off += n * 8
for i, r := range rows {
binary.BigEndian.PutUint16(b[off+i*2:], r.Size)
}
off += n * 2
for i, r := range rows {
b[off+i] = r.TTL
}
off += n
for i, r := range rows {
b[off+i] = r.DSCP
}
off += n
for i, r := range rows {
b[off+i] = r.ECN
}
out = append(out, b)
}
return out
}
+179
View File
@@ -0,0 +1,179 @@
// SPDX-FileCopyrightText: 2026 Echolot contributors
// SPDX-License-Identifier: GPL-3.0-or-later
package dataplane
import (
"encoding/binary"
"net"
"net/netip"
"testing"
"time"
"echo-lot.app/server/internal/session"
)
// parseReportPart decodes one TRAIN_REPORT payload back into rows, checking the header.
func parseReportPart(t *testing.T, b []byte) (id uint32, received int, part, parts int, truncated bool, rows []session.TrainEntry) {
t.Helper()
if len(b) < trainReportHeader {
t.Fatalf("report part shorter than its header: %d", len(b))
}
id = binary.BigEndian.Uint32(b[0:4])
received = int(binary.BigEndian.Uint32(b[4:8]))
part = int(binary.BigEndian.Uint16(b[8:10]))
parts = int(binary.BigEndian.Uint16(b[10:12]))
truncated = b[12]&1 != 0
n := int(binary.BigEndian.Uint16(b[14:16]))
if want := trainReportHeader + n*trainReportRow; len(b) != want {
t.Fatalf("part length %d, want %d for %d rows", len(b), want, n)
}
off := trainReportHeader
rows = make([]session.TrainEntry, n)
for i := range rows {
rows[i].Seq = binary.BigEndian.Uint32(b[off+i*4:])
}
off += n * 4
for i := range rows {
rows[i].TRxNs = int64(binary.BigEndian.Uint64(b[off+i*8:]))
}
off += n * 8
for i := range rows {
rows[i].Size = binary.BigEndian.Uint16(b[off+i*2:])
}
off += n * 2
for i := range rows {
rows[i].TTL = b[off+i]
}
off += n
for i := range rows {
rows[i].DSCP = b[off+i]
}
off += n
for i := range rows {
rows[i].ECN = b[off+i]
}
return
}
func TestBuildTrainReportSplitsAndRoundTrips(t *testing.T) {
const count = 250 // enough to need several parts
train := session.Train{ID: 42, Received: count, Truncated: true}
for i := 0; i < count; i++ {
train.Entries = append(train.Entries, session.TrainEntry{
Seq: uint32(i), TRxNs: int64(i) * 1_000_000, Size: uint16(100 + i),
TTL: 64, DSCP: 46, ECN: 1,
})
}
parts := buildTrainReport(train)
if len(parts) < 2 {
t.Fatalf("250 rows should not fit one ≤%d-byte datagram", trainReportMaxDatagram)
}
var got []session.TrainEntry
for i, p := range parts {
if HeaderSize+len(p) > trainReportMaxDatagram {
t.Fatalf("part %d would be a %d-byte datagram, cap is %d", i, HeaderSize+len(p), trainReportMaxDatagram)
}
id, received, part, total, truncated, rows := parseReportPart(t, p)
if id != 42 || received != count || part != i || total != len(parts) || !truncated {
t.Fatalf("part %d header: id=%d received=%d part=%d/%d truncated=%v",
i, id, received, part, total, truncated)
}
got = append(got, rows...)
}
if len(got) != count {
t.Fatalf("round-tripped %d rows, want %d", len(got), count)
}
for i, r := range got {
want := train.Entries[i]
if r != want {
t.Fatalf("row %d = %+v, want %+v", i, r, want)
}
}
}
func TestBuildTrainReportEmptyTrainStillAnswers(t *testing.T) {
parts := buildTrainReport(session.Train{ID: 9})
if len(parts) != 1 {
t.Fatalf("empty train: %d parts, want 1 — 'nothing arrived' is the answer, not silence", len(parts))
}
id, received, _, total, _, rows := parseReportPart(t, parts[0])
if id != 9 || received != 0 || total != 1 || len(rows) != 0 {
t.Fatalf("empty report: id=%d received=%d parts=%d rows=%d", id, received, total, len(rows))
}
}
func TestTrainDataThenReportOverTheWire(t *testing.T) {
mgr, addr := startServer(t)
sess, _, err := mgr.New("dev1", "credential-ikm", netip.MustParseAddr("127.0.0.1"))
if err != nil {
t.Fatal(err)
}
client, err := net.DialUDP("udp", nil, net.UDPAddrFromAddrPort(addr))
if err != nil {
t.Fatal(err)
}
defer client.Close()
client.SetDeadline(time.Now().Add(3 * time.Second))
// A short train: id 5 in payload[0:4], plus padding.
const trainID, count = 5, 4
for i := 0; i < count; i++ {
payload := make([]byte, 60)
binary.BigEndian.PutUint32(payload[0:4], trainID)
if _, err := client.Write(craft(t, sess, TypeTrainData, uint32(i+1), payload)); err != nil {
t.Fatal(err)
}
}
// TRAIN_DATA must be silent (spec §3.2).
client.SetReadDeadline(time.Now().Add(300 * time.Millisecond))
if _, err := client.Read(make([]byte, 1500)); err == nil {
t.Fatal("TRAIN_DATA got a response, want none")
}
// Ask for the report.
reqPayload := make([]byte, 4)
binary.BigEndian.PutUint32(reqPayload, trainID)
client.SetDeadline(time.Now().Add(3 * time.Second))
if _, err := client.Write(craft(t, sess, TypeTrainReportReq, 100, reqPayload)); err != nil {
t.Fatal(err)
}
buf := make([]byte, 2000)
n, err := client.Read(buf)
if err != nil {
t.Fatalf("no TRAIN_REPORT: %v", err)
}
if buf[4] != TypeTrainReport {
t.Fatalf("type = %#x, want TRAIN_REPORT", buf[4])
}
id, received, part, parts, truncated, rows := parseReportPart(t, buf[HeaderSize:n])
if id != trainID || received != count || part != 0 || parts != 1 || truncated {
t.Fatalf("report header: id=%d received=%d part=%d/%d truncated=%v", id, received, part, parts, truncated)
}
if len(rows) != count {
t.Fatalf("%d rows, want %d", len(rows), count)
}
for i, r := range rows {
if r.Seq != uint32(i+1) {
t.Fatalf("row %d seq = %d, want %d", i, r.Seq, i+1)
}
if r.Size != HeaderSize+60 {
t.Fatalf("row %d size = %d, want %d", i, r.Size, HeaderSize+60)
}
}
// Unknown train id: one zero-row report, not silence.
binary.BigEndian.PutUint32(reqPayload, 999)
if _, err := client.Write(craft(t, sess, TypeTrainReportReq, 101, reqPayload)); err != nil {
t.Fatal(err)
}
n, err = client.Read(buf)
if err != nil {
t.Fatalf("no report for unknown train: %v", err)
}
id, received, _, _, _, rows = parseReportPart(t, buf[HeaderSize:n])
if id != 999 || received != 0 || len(rows) != 0 {
t.Fatalf("unknown-train report: id=%d received=%d rows=%d", id, received, len(rows))
}
}
+109 -15
View File
@@ -2,15 +2,15 @@
// SPDX-License-Identifier: GPL-3.0-or-later // SPDX-License-Identifier: GPL-3.0-or-later
// Package dataplane implements the binary UDP probe protocol (spec §3): // Package dataplane implements the binary UDP probe protocol (spec §3):
// 32-byte header, HMAC gate, anti-replay, ECHO with observation block. // 32-byte header, HMAC gate, anti-replay, ECHO with observation block,
// Skeleton scope: ECHO_REQ/ECHO_RESP and TIMESYNC only; trains, MTU probes // upstream trains with columnar reports, and the granted server->client sends.
// and delayed echo land with the corresponding client tests.
package dataplane package dataplane
import ( import (
"crypto/hmac" "crypto/hmac"
"crypto/sha256" "crypto/sha256"
"encoding/binary" "encoding/binary"
"encoding/hex"
"fmt" "fmt"
"log/slog" "log/slog"
"net" "net"
@@ -18,6 +18,7 @@ import (
"sync" "sync"
"time" "time"
"echo-lot.app/server/internal/ratelimit"
"echo-lot.app/server/internal/session" "echo-lot.app/server/internal/session"
) )
@@ -25,9 +26,14 @@ const (
Magic = "ELT1" Magic = "ELT1"
HeaderSize = 32 HeaderSize = 32
TypeEchoReq = 0x01 TypeEchoReq = 0x01
TypeEchoResp = 0x02 TypeEchoResp = 0x02
TypeTimesyncReq = 0x07 // Upstream trains (spec §3.2): DATA gets no per-packet response; REPORT_REQ fetches the
// server's received view as one or more REPORT datagrams (train.go).
TypeTrainData = 0x03
TypeTrainReportReq = 0x04
TypeTrainReport = 0x05
TypeTimesyncReq = 0x07
TypeTimesyncRsp = 0x08 TypeTimesyncRsp = 0x08
TypeMtuProbe = 0x09 TypeMtuProbe = 0x09
TypeMtuAck = 0x0A TypeMtuAck = 0x0A
@@ -37,10 +43,22 @@ const (
TypeBigSend = 0x0C TypeBigSend = 0x0C
// TypeFragData is delivered only after IP reassembly, so its arrival IS the measurement. // TypeFragData is delivered only after IP reassembly, so its arrival IS the measurement.
TypeFragData = 0x0D TypeFragData = 0x0D
// TypeThroughputData is one packet of a sustained-rate downstream run.
TypeThroughputData = 0x0E
// TypeThroughputUp is one packet of a client-driven upstream run. The server counts it and
// deliberately does not answer: a reply would double the traffic and measure the return
// path at the same time, which is the one thing this test is trying not to do.
TypeThroughputUp = 0x0F
) )
type Server struct { type Server struct {
Sessions *session.Manager Sessions *session.Manager
// Spec §2.5 ceilings on verified traffic, silent-drop (nil = no ceiling). Charged after the
// HMAC gate so an unauthenticated flood cannot spend anyone's budget, keyed per source
// address AND per device credential so neither one hot address nor one hot credential can
// crowd out the rest.
PacketRate *ratelimit.Limiter // tokens are packets
ByteRate *ratelimit.Limiter // tokens are bytes
// Epoch for server-side t_rx/t_tx: process start; observation consumers // Epoch for server-side t_rx/t_tx: process start; observation consumers
// only need differences plus the timesync exchange, not absolute time. // only need differences plus the timesync exchange, not absolute time.
start time.Time start time.Time
@@ -53,6 +71,34 @@ type Server struct {
dfMu sync.Mutex dfMu sync.Mutex
} }
// pktMeta is what the kernel told us about one received datagram beyond its bytes (spec §3.3:
// received TTL, DSCP, ECN). 0xFF means "not observed": non-Linux hosts and datagrams whose
// cmsg never arrived keep the sentinel rather than inventing a value.
type pktMeta struct {
TTL uint8
TOS uint8 // the whole DSCP/ECN byte; DSCP = TOS>>2, ECN = TOS&3
}
const metaUnavailable = 0xFF
func (m pktMeta) dscp() uint8 {
if m.TOS == metaUnavailable {
return metaUnavailable
}
return m.TOS >> 2
}
func (m pktMeta) ecn() uint8 {
if m.TOS == metaUnavailable {
return metaUnavailable
}
return m.TOS & 0x3
}
// oobCap fits the two cmsgs (TTL + TOS, each ≤ CMSG_SPACE(4)) with headroom for whatever else
// the kernel decides to attach.
const oobCap = 64
// Serve runs the read loop for one socket; call once per bound address. // Serve runs the read loop for one socket; call once per bound address.
// The socket is retained so actions (delayed echo) can pick a family-matching // The socket is retained so actions (delayed echo) can pick a family-matching
// sender later. // sender later.
@@ -63,14 +109,16 @@ func (s *Server) Serve(conn *net.UDPConn) error {
} }
s.conns = append(s.conns, conn) s.conns = append(s.conns, conn)
s.mu.Unlock() s.mu.Unlock()
enableRecvMeta(conn)
buf := make([]byte, 65535) buf := make([]byte, 65535)
oob := make([]byte, oobCap)
for { for {
n, raddr, err := conn.ReadFromUDPAddrPort(buf) n, oobn, _, raddr, err := conn.ReadMsgUDPAddrPort(buf, oob)
if err != nil { if err != nil {
return err return err
} }
tRx := time.Since(s.start).Nanoseconds() tRx := time.Since(s.start).Nanoseconds()
s.handle(conn, raddr, buf[:n], tRx) s.handle(conn, raddr, buf[:n], tRx, parseMeta(oob[:oobn]))
} }
} }
@@ -120,7 +168,7 @@ func (s *Server) SendDelayedEcho(sess *session.Session, actionID string) error {
// handle enforces spec §3.1/§3.4: unknown prefix, bad HMAC, expired session, // handle enforces spec §3.1/§3.4: unknown prefix, bad HMAC, expired session,
// replayed seq → silent drop, never a response. // replayed seq → silent drop, never a response.
func (s *Server) handle(conn *net.UDPConn, raddr netip.AddrPort, pkt []byte, tRxNs int64) { func (s *Server) handle(conn *net.UDPConn, raddr netip.AddrPort, pkt []byte, tRxNs int64, meta pktMeta) {
if len(pkt) < HeaderSize || string(pkt[0:4]) != Magic { if len(pkt) < HeaderSize || string(pkt[0:4]) != Magic {
return return
} }
@@ -143,6 +191,12 @@ func (s *Server) handle(conn *net.UDPConn, raddr netip.AddrPort, pkt []byte, tRx
if !hmac.Equal(mac.Sum(nil)[:4], pkt[28:32]) { if !hmac.Equal(mac.Sum(nil)[:4], pkt[28:32]) {
return return
} }
// Spec §2.5: over-ceiling traffic is silently dropped (probes tolerate loss by design).
// After the HMAC gate so a spoofed flood cannot drain a victim's budget; before the replay
// window so a dropped packet's seq stays usable for a resend.
if !s.allowUDP(raddr, sess.Device, len(pkt)) {
return
}
if !sess.CheckSeq(seq) { if !sess.CheckSeq(seq) {
return return
} }
@@ -150,14 +204,29 @@ func (s *Server) handle(conn *net.UDPConn, raddr netip.AddrPort, pkt []byte, tRx
if la, ok := conn.LocalAddr().(*net.UDPAddr); ok { if la, ok := conn.LocalAddr().(*net.UDPAddr); ok {
sess.NoteDataLocal(la.AddrPort()) sess.NoteDataLocal(la.AddrPort())
} }
// Upstream throughput short-circuits before the observation log. Recording one struct per
// packet here would mean tens of thousands of allocations for a single run; the counter is
// all anyone needs, since the client holds the send-side record.
if typ == TypeThroughputUp {
sess.CountUpstream(len(pkt), tRxNs)
return
}
sess.RecordUDP(session.UDPObservation{ sess.RecordUDP(session.UDPObservation{
Seq: seq, TRxNs: tRxNs, TTxNs: time.Since(s.start).Nanoseconds(), Seq: seq, TRxNs: tRxNs, TTxNs: time.Since(s.start).Nanoseconds(),
Src: raddr.String(), Size: len(pkt), Type: typ, Src: raddr.String(), Size: len(pkt), Type: typ,
}) })
payload := pkt[HeaderSize : HeaderSize+int(payloadLen)]
switch typ { switch typ {
case TypeEchoReq: case TypeEchoReq:
s.echoResp(conn, raddr, sess, pkt, seq, tRxNs) s.echoResp(conn, raddr, sess, pkt, seq, tRxNs, meta)
case TypeTrainData:
// No response (spec §3.2): the train is upstream-only; its received view is fetched
// afterwards via TRAIN_REPORT_REQ or the observations API.
recordTrain(sess, payload, seq, len(pkt), tRxNs, meta)
case TypeTrainReportReq:
s.trainReport(conn, raddr, sess, payload)
case TypeTimesyncReq: case TypeTimesyncReq:
s.timesyncResp(conn, raddr, sess, pkt, seq, tRxNs) s.timesyncResp(conn, raddr, sess, pkt, seq, tRxNs)
case TypeMtuProbe: case TypeMtuProbe:
@@ -167,6 +236,16 @@ func (s *Server) handle(conn *net.UDPConn, raddr netip.AddrPort, pkt []byte, tRx
} }
} }
// allowUDP charges the §2.5 packet and byte buckets, per source address and per credential.
func (s *Server) allowUDP(raddr netip.AddrPort, device string, size int) bool {
ipKey, credKey := "ip:"+raddr.Addr().String(), "cred:"+device
okA, _ := s.PacketRate.Allow(ipKey)
okC, _ := s.PacketRate.Allow(credKey)
okAB, _ := s.ByteRate.AllowN(ipKey, float64(size))
okCB, _ := s.ByteRate.AllowN(credKey, float64(size))
return okA && okC && okAB && okCB
}
// mtuAck replies to an MTU_PROBE with a small MTU_ACK carrying the total // mtuAck replies to an MTU_PROBE with a small MTU_ACK carrying the total
// datagram size the server actually received (spec §3.2). The client sends // datagram size the server actually received (spec §3.2). The client sends
// DF-flagged probes of increasing size and binary-searches the path MTU / a // DF-flagged probes of increasing size and binary-searches the path MTU / a
@@ -184,28 +263,43 @@ func (s *Server) mtuAck(conn *net.UDPConn, raddr netip.AddrPort, sess *session.S
// 8 8 t_tx_ns // 8 8 t_tx_ns
// 16 16 observed source IP (v4-mapped when v4) // 16 16 observed source IP (v4-mapped when v4)
// 32 2 observed source port // 32 2 observed source port
// 34 1 received TTL (0xFF = not observed yet; needs recvmsg cmsgs) // 34 1 received TTL (0xFF = not observed; cmsgs unavailable on this host)
// 35 1 received DSCP/ECN byte (0xFF = not observed) // 35 1 received DSCP/ECN byte (0xFF = not observed)
// 36 4 received size // 36 4 received size
func observation(tRxNs, tTxNs int64, src netip.AddrPort, rcvd int) []byte { func observation(tRxNs, tTxNs int64, src netip.AddrPort, rcvd int, meta pktMeta) []byte {
b := make([]byte, 40) b := make([]byte, 40)
binary.BigEndian.PutUint64(b[0:8], uint64(tRxNs)) binary.BigEndian.PutUint64(b[0:8], uint64(tRxNs))
binary.BigEndian.PutUint64(b[8:16], uint64(tTxNs)) binary.BigEndian.PutUint64(b[8:16], uint64(tTxNs))
a16 := src.Addr().As16() a16 := src.Addr().As16()
copy(b[16:32], a16[:]) copy(b[16:32], a16[:])
binary.BigEndian.PutUint16(b[32:34], src.Port()) binary.BigEndian.PutUint16(b[32:34], src.Port())
b[34], b[35] = 0xFF, 0xFF b[34], b[35] = meta.TTL, meta.TOS
binary.BigEndian.PutUint32(b[36:40], uint32(rcvd)) binary.BigEndian.PutUint32(b[36:40], uint32(rcvd))
return b return b
} }
// putActionID writes a grant's action id into payload[8:16] — the correlation the spec promises
// (§5: "an action_id echoed in resulting data-plane packets"), consumed by the client as
// test.params.action_id (§9). Bytes [0:8] stay with the packet type; [8:16] is reserved for this
// across every granted type, so the client needs one rule, not five.
func putActionID(payload []byte, actionID string) {
if len(payload) < 16 {
return
}
raw, err := hex.DecodeString(actionID)
if err != nil || len(raw) != 8 {
return // a malformed id yields zero bytes, not a crash mid-burst
}
copy(payload[8:16], raw)
}
// echoResp mirrors the request header (type flipped), appends the observation // echoResp mirrors the request header (type flipped), appends the observation
// block, and re-HMACs with the session key. Anti-amplification: the response // block, and re-HMACs with the session key. Anti-amplification: the response
// is capped at the request size (spec §3.4) — the observation block replaces // is capped at the request size (spec §3.4) — the observation block replaces
// padding rather than growing the datagram; if the request was smaller than // padding rather than growing the datagram; if the request was smaller than
// header+observation, the block is truncated to fit. // header+observation, the block is truncated to fit.
func (s *Server) echoResp(conn *net.UDPConn, raddr netip.AddrPort, sess *session.Session, req []byte, seq uint32, tRxNs int64) { func (s *Server) echoResp(conn *net.UDPConn, raddr netip.AddrPort, sess *session.Session, req []byte, seq uint32, tRxNs int64, meta pktMeta) {
obs := observation(tRxNs, time.Since(s.start).Nanoseconds(), raddr, len(req)) obs := observation(tRxNs, time.Since(s.start).Nanoseconds(), raddr, len(req), meta)
max := len(req) max := len(req)
if max < HeaderSize { if max < HeaderSize {
return return
+504
View File
@@ -0,0 +1,504 @@
// SPDX-FileCopyrightText: 2026 Echolot contributors
// SPDX-License-Identifier: GPL-3.0-or-later
// Package oidc verifies OpenID Connect ID tokens against a configured issuer.
//
// Echolot is a *relying party*, never an identity provider. It delegates to whatever IdP the
// operator already runs and stores no passwords — no hashing, no reset flow, no lockout policy,
// and no credential database to leak. For a tool people self-host on a box they also use for
// other things, that is the difference between "one more service" and "one more thing that can
// lose your users' passwords".
//
// Verification is written against the stdlib rather than a JWT library, because the server has no
// external dependencies by design. That is a real constraint and it cuts both ways: the code below
// is longer than `jwt.Parse`, but it is also auditable in one sitting and cannot be broken by
// somebody else's release. The algorithm allow-list is the part that matters — accepting `alg`
// from the token itself is the classic JWT forgery, so it is fixed here and `none` can never
// appear.
package oidc
import (
"context"
"crypto"
"crypto/ecdsa"
"crypto/elliptic"
"crypto/rsa"
"crypto/sha256"
"crypto/sha512"
"encoding/base64"
"encoding/json"
"errors"
"fmt"
"math/big"
"net/http"
"strings"
"sync"
"time"
)
// Claims are the parts of an ID token Echolot acts on.
type Claims struct {
Issuer string `json:"iss"`
Subject string `json:"sub"`
Audience audience `json:"aud"`
Expiry int64 `json:"exp"`
IssuedAt int64 `json:"iat"`
Nonce string `json:"nonce"`
Email string `json:"email"`
Name string `json:"name"`
Username string `json:"preferred_username"`
Groups []string `json:"groups"`
}
// AccountID is the stable identity of a person: issuer plus subject.
//
// Subject alone is not enough — it is only unique within an issuer — and email is not stable,
// since people change them and IdPs allow reuse. Keying on iss+sub means an operator can switch
// IdPs and know that the accounts did not silently merge.
func (c Claims) AccountID() string { return c.Issuer + "#" + c.Subject }
// Display is the friendliest name available, for the admin UI.
func (c Claims) Display() string {
for _, s := range []string{c.Name, c.Username, c.Email} {
if s != "" {
return s
}
}
return c.Subject
}
// audience tolerates the spec's two shapes: a string or an array of strings.
type audience []string
func (a *audience) UnmarshalJSON(b []byte) error {
var one string
if err := json.Unmarshal(b, &one); err == nil {
*a = audience{one}
return nil
}
var many []string
if err := json.Unmarshal(b, &many); err != nil {
return err
}
*a = many
return nil
}
func (a audience) contains(s string) bool {
for _, v := range a {
if v == s {
return true
}
}
return false
}
// Config is what the operator supplies.
type Config struct {
// Issuer is the IdP's base URL, e.g. https://auth.example.net/application/o/echolot/
Issuer string
// ClientID is this server's own registered client — confidential, used for the admin UI's
// browser login, where a secret can genuinely be kept in the host's config.
ClientID string
// AppClientID is the mobile app's registered client. It is a separate, *public* client
// because an APK cannot keep a secret, so it uses PKCE instead.
//
// Both are accepted as audiences, and they must be listed rather than merged: a token is
// addressed to a specific client, and accepting "any client of this issuer" would let every
// other application registered with the same IdP authenticate here.
AppClientID string
// AdminGroup, when set, is the group claim a person must hold to reach the admin UI.
// Empty means no one is an admin via OIDC, which is the safe default: an operator who has
// not said who may administer the server has not said "everyone".
AdminGroup string
// Skew tolerated on exp/iat, for ordinary clock drift between the IdP and this server.
Skew time.Duration
}
func (c Config) Enabled() bool { return c.Issuer != "" && (c.ClientID != "" || c.AppClientID != "") }
// acceptedAudiences is every client id this server answers for.
func (v *Verifier) acceptedAudiences() []string {
out := make([]string, 0, 2)
for _, id := range []string{v.cfg.ClientID, v.cfg.AppClientID} {
if id != "" {
out = append(out, id)
}
}
return out
}
func (v *Verifier) audienceAccepted(aud audience) bool {
for _, id := range v.acceptedAudiences() {
if aud.contains(id) {
return true
}
}
return false
}
// Discovery is the subset of the provider metadata document that is used.
type Discovery struct {
Issuer string `json:"issuer"`
AuthorizationEndpoint string `json:"authorization_endpoint"`
TokenEndpoint string `json:"token_endpoint"`
JWKSURI string `json:"jwks_uri"`
UserinfoEndpoint string `json:"userinfo_endpoint"`
EndSessionEndpoint string `json:"end_session_endpoint"`
}
// Verifier fetches provider metadata and keys, and checks tokens against them.
type Verifier struct {
cfg Config
client *http.Client
mu sync.RWMutex
discovery *Discovery
keys map[string]crypto.PublicKey
keysAt time.Time
}
func New(cfg Config, client *http.Client) *Verifier {
if cfg.Skew == 0 {
cfg.Skew = 2 * time.Minute
}
if client == nil {
client = &http.Client{Timeout: 10 * time.Second}
}
return &Verifier{cfg: cfg, client: client, keys: map[string]crypto.PublicKey{}}
}
func (v *Verifier) Config() Config { return v.cfg }
var (
ErrDisabled = errors.New("no OIDC issuer is configured on this server")
ErrMalformed = errors.New("token is not a well-formed JWT")
ErrSignature = errors.New("token signature does not verify")
ErrClaims = errors.New("token claims are not acceptable")
)
// Discover fetches (and caches) the provider metadata.
func (v *Verifier) Discover(ctx context.Context) (*Discovery, error) {
if !v.cfg.Enabled() {
return nil, ErrDisabled
}
v.mu.RLock()
d := v.discovery
v.mu.RUnlock()
if d != nil {
return d, nil
}
url := strings.TrimRight(v.cfg.Issuer, "/") + "/.well-known/openid-configuration"
req, err := http.NewRequestWithContext(ctx, http.MethodGet, url, nil)
if err != nil {
return nil, err
}
resp, err := v.client.Do(req)
if err != nil {
return nil, fmt.Errorf("discovery: %w", err)
}
defer resp.Body.Close()
if resp.StatusCode != http.StatusOK {
return nil, fmt.Errorf("discovery: %s returned %d", url, resp.StatusCode)
}
var got Discovery
if err := json.NewDecoder(resp.Body).Decode(&got); err != nil {
return nil, fmt.Errorf("discovery: %w", err)
}
// The issuer in the document must match the one configured, or a redirect could point us at
// somebody else's keys while we keep believing we are talking to the configured provider.
if strings.TrimRight(got.Issuer, "/") != strings.TrimRight(v.cfg.Issuer, "/") {
return nil, fmt.Errorf("discovery: document says issuer %q, configured %q", got.Issuer, v.cfg.Issuer)
}
v.mu.Lock()
v.discovery = &got
v.mu.Unlock()
return &got, nil
}
// jwksTTL is how long keys are trusted before refetching. Short enough to pick up a rotation
// without an operator restarting anything; long enough that token checks are not IdP round trips.
const jwksTTL = 15 * time.Minute
func (v *Verifier) keyFor(ctx context.Context, kid string) (crypto.PublicKey, error) {
v.mu.RLock()
k, ok := v.keys[kid]
fresh := time.Since(v.keysAt) < jwksTTL
v.mu.RUnlock()
if ok && fresh {
return k, nil
}
if err := v.refreshKeys(ctx); err != nil {
return nil, err
}
v.mu.RLock()
defer v.mu.RUnlock()
if k, ok := v.keys[kid]; ok {
return k, nil
}
// A kid we have never seen, after a refresh, is a token from somewhere else.
return nil, fmt.Errorf("%w: no key %q at the issuer", ErrSignature, kid)
}
func (v *Verifier) refreshKeys(ctx context.Context) error {
d, err := v.Discover(ctx)
if err != nil {
return err
}
req, err := http.NewRequestWithContext(ctx, http.MethodGet, d.JWKSURI, nil)
if err != nil {
return err
}
resp, err := v.client.Do(req)
if err != nil {
return fmt.Errorf("jwks: %w", err)
}
defer resp.Body.Close()
if resp.StatusCode != http.StatusOK {
return fmt.Errorf("jwks: %s returned %d", d.JWKSURI, resp.StatusCode)
}
var set struct {
Keys []jwk `json:"keys"`
}
if err := json.NewDecoder(resp.Body).Decode(&set); err != nil {
return fmt.Errorf("jwks: %w", err)
}
parsed := make(map[string]crypto.PublicKey, len(set.Keys))
for _, k := range set.Keys {
if pub, err := k.publicKey(); err == nil {
parsed[k.Kid] = pub
}
}
if len(parsed) == 0 {
return errors.New("jwks: no usable keys")
}
v.mu.Lock()
v.keys = parsed
v.keysAt = time.Now()
v.mu.Unlock()
return nil
}
type jwk struct {
Kty string `json:"kty"`
Kid string `json:"kid"`
Alg string `json:"alg"`
Use string `json:"use"`
N string `json:"n"`
E string `json:"e"`
Crv string `json:"crv"`
X string `json:"x"`
Y string `json:"y"`
}
func (k jwk) publicKey() (crypto.PublicKey, error) {
switch k.Kty {
case "RSA":
n, err := b64uint(k.N)
if err != nil {
return nil, err
}
e, err := b64uint(k.E)
if err != nil {
return nil, err
}
if !e.IsInt64() || e.Int64() > 1<<31 {
return nil, errors.New("implausible RSA exponent")
}
return &rsa.PublicKey{N: n, E: int(e.Int64())}, nil
case "EC":
curve, err := curveFor(k.Crv)
if err != nil {
return nil, err
}
x, err := b64uint(k.X)
if err != nil {
return nil, err
}
y, err := b64uint(k.Y)
if err != nil {
return nil, err
}
return &ecdsa.PublicKey{Curve: curve, X: x, Y: y}, nil
}
return nil, fmt.Errorf("unsupported key type %q", k.Kty)
}
// Verify checks a serialized ID token and returns its claims.
//
// The order is deliberate: structure, then algorithm, then signature, then claims. Nothing about
// the token's contents is believed before its signature has been checked — reading `iss` or `aud`
// out of an unverified token and acting on it is how "verified" tokens turn out not to be.
func (v *Verifier) Verify(ctx context.Context, token string) (*Claims, error) {
if !v.cfg.Enabled() {
return nil, ErrDisabled
}
parts := strings.Split(token, ".")
if len(parts) != 3 {
return nil, ErrMalformed
}
headerJSON, err := b64(parts[0])
if err != nil {
return nil, ErrMalformed
}
var hdr struct {
Alg string `json:"alg"`
Kid string `json:"kid"`
Typ string `json:"typ"`
}
if err := json.Unmarshal(headerJSON, &hdr); err != nil {
return nil, ErrMalformed
}
// The allow-list is fixed here rather than taken from the token. Trusting the token's own
// `alg` is the classic JWT forgery: "none" turns any token into a valid one, and swapping RS256
// for HS256 lets an attacker sign with the public key. Neither is reachable from here.
if _, ok := allowedAlgs[hdr.Alg]; !ok {
return nil, fmt.Errorf("%w: algorithm %q is not accepted", ErrSignature, hdr.Alg)
}
pub, err := v.keyFor(ctx, hdr.Kid)
if err != nil {
return nil, err
}
sig, err := b64(parts[2])
if err != nil {
return nil, ErrMalformed
}
signed := parts[0] + "." + parts[1]
if err := verifySignature(hdr.Alg, pub, []byte(signed), sig); err != nil {
return nil, err
}
payload, err := b64(parts[1])
if err != nil {
return nil, ErrMalformed
}
var claims Claims
if err := json.Unmarshal(payload, &claims); err != nil {
return nil, ErrMalformed
}
if err := v.checkClaims(claims); err != nil {
return nil, err
}
return &claims, nil
}
func (v *Verifier) checkClaims(c Claims) error {
if strings.TrimRight(c.Issuer, "/") != strings.TrimRight(v.cfg.Issuer, "/") {
return fmt.Errorf("%w: issued by %q, expected %q", ErrClaims, c.Issuer, v.cfg.Issuer)
}
// A token addressed to a different client is a valid token that was not meant for us —
// accepting it lets any other client of the same IdP authenticate here.
if !v.audienceAccepted(c.Audience) {
return fmt.Errorf("%w: addressed to %v, not to %v", ErrClaims,
[]string(c.Audience), v.acceptedAudiences())
}
if c.Subject == "" {
return fmt.Errorf("%w: no subject", ErrClaims)
}
now := time.Now()
if c.Expiry == 0 || now.After(time.Unix(c.Expiry, 0).Add(v.cfg.Skew)) {
return fmt.Errorf("%w: expired", ErrClaims)
}
if c.IssuedAt != 0 && now.Add(v.cfg.Skew).Before(time.Unix(c.IssuedAt, 0)) {
return fmt.Errorf("%w: issued in the future", ErrClaims)
}
return nil
}
// IsAdmin reports whether these claims carry the configured admin group.
//
// With no group configured nobody is an admin: an operator who has not said who may administer
// the server has not thereby said "anyone who can log in".
func (v *Verifier) IsAdmin(c *Claims) bool {
if c == nil || v.cfg.AdminGroup == "" {
return false
}
for _, g := range c.Groups {
if g == v.cfg.AdminGroup {
return true
}
}
return false
}
var allowedAlgs = map[string]crypto.Hash{
"RS256": crypto.SHA256, "RS384": crypto.SHA384, "RS512": crypto.SHA512,
"ES256": crypto.SHA256, "ES384": crypto.SHA384, "ES512": crypto.SHA512,
}
func verifySignature(alg string, pub crypto.PublicKey, signed, sig []byte) error {
h := allowedAlgs[alg]
digest := hashOf(h, signed)
switch {
case strings.HasPrefix(alg, "RS"):
k, ok := pub.(*rsa.PublicKey)
if !ok {
return fmt.Errorf("%w: %s token against a non-RSA key", ErrSignature, alg)
}
if err := rsa.VerifyPKCS1v15(k, h, digest, sig); err != nil {
return ErrSignature
}
return nil
case strings.HasPrefix(alg, "ES"):
k, ok := pub.(*ecdsa.PublicKey)
if !ok {
return fmt.Errorf("%w: %s token against a non-EC key", ErrSignature, alg)
}
// JWS packs ECDSA signatures as r||s, fixed width — not the ASN.1 form ecdsa.Verify
// would otherwise expect.
if len(sig)%2 != 0 {
return ErrSignature
}
half := len(sig) / 2
r := new(big.Int).SetBytes(sig[:half])
s := new(big.Int).SetBytes(sig[half:])
if !ecdsa.Verify(k, digest, r, s) {
return ErrSignature
}
return nil
}
return ErrSignature
}
func hashOf(h crypto.Hash, b []byte) []byte {
switch h {
case crypto.SHA384:
d := sha512.Sum384(b)
return d[:]
case crypto.SHA512:
d := sha512.Sum512(b)
return d[:]
default:
d := sha256.Sum256(b)
return d[:]
}
}
func curveFor(crv string) (elliptic.Curve, error) {
switch crv {
case "P-256":
return elliptic.P256(), nil
case "P-384":
return elliptic.P384(), nil
case "P-521":
return elliptic.P521(), nil
}
return nil, fmt.Errorf("unsupported curve %q", crv)
}
// b64 decodes JWT base64url, which omits padding.
func b64(s string) ([]byte, error) { return base64.RawURLEncoding.DecodeString(s) }
func b64uint(s string) (*big.Int, error) {
b, err := b64(s)
if err != nil {
return nil, err
}
if len(b) == 0 {
return nil, errors.New("empty value")
}
return new(big.Int).SetBytes(b), nil
}
+325
View File
@@ -0,0 +1,325 @@
// SPDX-FileCopyrightText: 2026 Echolot contributors
// SPDX-License-Identifier: GPL-3.0-or-later
package oidc
import (
"context"
"crypto"
"crypto/ecdsa"
"crypto/elliptic"
"crypto/rand"
"crypto/rsa"
"crypto/sha256"
"encoding/base64"
"encoding/json"
"errors"
"math/big"
"net/http"
"net/http/httptest"
"testing"
"time"
)
// A self-contained IdP: real keys, real signatures, real discovery and JWKS documents. Testing
// token verification against anything less than a genuine signer proves nothing — the failure
// modes that matter here (accepting `none`, accepting another client's token, accepting an
// expired one) all look fine to a mock that just returns success.
type testIdP struct {
*httptest.Server
rsaKey *rsa.PrivateKey
ecKey *ecdsa.PrivateKey
}
func newIdP(t *testing.T) *testIdP {
t.Helper()
rk, err := rsa.GenerateKey(rand.Reader, 2048)
if err != nil {
t.Fatal(err)
}
ek, err := ecdsa.GenerateKey(elliptic.P256(), rand.Reader)
if err != nil {
t.Fatal(err)
}
idp := &testIdP{rsaKey: rk, ecKey: ek}
mux := http.NewServeMux()
mux.HandleFunc("/.well-known/openid-configuration", func(w http.ResponseWriter, r *http.Request) {
_ = json.NewEncoder(w).Encode(Discovery{
Issuer: idp.URL,
AuthorizationEndpoint: idp.URL + "/auth",
TokenEndpoint: idp.URL + "/token",
JWKSURI: idp.URL + "/jwks",
})
})
mux.HandleFunc("/jwks", func(w http.ResponseWriter, r *http.Request) {
_ = json.NewEncoder(w).Encode(map[string]any{"keys": []map[string]string{
{
"kty": "RSA", "kid": "rsa-1", "alg": "RS256", "use": "sig",
"n": raw(rk.N.Bytes()),
"e": raw(big.NewInt(int64(rk.E)).Bytes()),
},
{
"kty": "EC", "kid": "ec-1", "alg": "ES256", "use": "sig", "crv": "P-256",
"x": raw(ek.X.Bytes()), "y": raw(ek.Y.Bytes()),
},
}})
})
idp.Server = httptest.NewServer(mux)
t.Cleanup(idp.Close)
return idp
}
func raw(b []byte) string { return base64.RawURLEncoding.EncodeToString(b) }
func (i *testIdP) sign(t *testing.T, alg, kid string, claims map[string]any) string {
t.Helper()
h, _ := json.Marshal(map[string]string{"alg": alg, "kid": kid, "typ": "JWT"})
p, _ := json.Marshal(claims)
signing := raw(h) + "." + raw(p)
digest := sha256.Sum256([]byte(signing))
var sig []byte
switch alg {
case "RS256":
s, err := rsa.SignPKCS1v15(rand.Reader, i.rsaKey, crypto.SHA256, digest[:])
if err != nil {
t.Fatal(err)
}
sig = s
case "ES256":
r, s, err := ecdsa.Sign(rand.Reader, i.ecKey, digest[:])
if err != nil {
t.Fatal(err)
}
// JWS wants fixed-width r||s, not ASN.1.
sig = make([]byte, 64)
r.FillBytes(sig[:32])
s.FillBytes(sig[32:])
default:
t.Fatalf("unsupported test alg %q", alg)
}
return signing + "." + raw(sig)
}
func (i *testIdP) claims(extra map[string]any) map[string]any {
c := map[string]any{
"iss": i.URL, "sub": "user-1", "aud": "echolot",
"exp": time.Now().Add(time.Hour).Unix(), "iat": time.Now().Unix(),
"email": "someone@example.net", "groups": []string{"users"},
}
for k, v := range extra {
c[k] = v
}
return c
}
func verifier(i *testIdP, adminGroup string) *Verifier {
return New(Config{
Issuer: i.URL, ClientID: "echolot", AppClientID: "echolot-app", AdminGroup: adminGroup,
}, i.Client())
}
func TestAcceptsAGenuineToken(t *testing.T) {
idp := newIdP(t)
v := verifier(idp, "")
for _, tc := range []struct{ alg, kid string }{{"RS256", "rsa-1"}, {"ES256", "ec-1"}} {
got, err := v.Verify(context.Background(), idp.sign(t, tc.alg, tc.kid, idp.claims(nil)))
if err != nil {
t.Fatalf("%s: %v", tc.alg, err)
}
if got.Subject != "user-1" || got.Email != "someone@example.net" {
t.Fatalf("%s: claims not parsed: %+v", tc.alg, got)
}
if want := idp.URL + "#user-1"; got.AccountID() != want {
t.Errorf("AccountID = %q, want %q", got.AccountID(), want)
}
}
}
// "alg": "none" is the oldest JWT forgery there is: strip the signature, declare no algorithm,
// and a naive verifier accepts anything. It must not even reach the key lookup.
func TestRejectsAlgNone(t *testing.T) {
idp := newIdP(t)
v := verifier(idp, "")
h, _ := json.Marshal(map[string]string{"alg": "none", "kid": "rsa-1", "typ": "JWT"})
p, _ := json.Marshal(idp.claims(nil))
token := raw(h) + "." + raw(p) + "."
if _, err := v.Verify(context.Background(), token); !errors.Is(err, ErrSignature) {
t.Fatalf("alg=none was not refused as a signature failure: %v", err)
}
}
// The other classic: declare HS256 so the verifier treats the RSA *public* key as an HMAC secret,
// which the attacker also has. The allow-list has no symmetric algorithms at all.
func TestRejectsSymmetricAlgorithmConfusion(t *testing.T) {
idp := newIdP(t)
v := verifier(idp, "")
h, _ := json.Marshal(map[string]string{"alg": "HS256", "kid": "rsa-1", "typ": "JWT"})
p, _ := json.Marshal(idp.claims(nil))
token := raw(h) + "." + raw(p) + "." + raw([]byte("whatever"))
if _, err := v.Verify(context.Background(), token); !errors.Is(err, ErrSignature) {
t.Fatalf("HS256 confusion was not refused: %v", err)
}
}
func TestRejectsATamperedPayload(t *testing.T) {
idp := newIdP(t)
v := verifier(idp, "")
good := idp.sign(t, "RS256", "rsa-1", idp.claims(nil))
// Swap the payload for one claiming to be somebody else, keeping the valid signature.
forged, _ := json.Marshal(idp.claims(map[string]any{"sub": "admin"}))
parts := []byte(good)
dot1, dot2 := 0, 0
for i, c := range parts {
if c == '.' {
if dot1 == 0 {
dot1 = i
} else {
dot2 = i
}
}
}
token := string(parts[:dot1+1]) + raw(forged) + string(parts[dot2:])
if _, err := v.Verify(context.Background(), token); !errors.Is(err, ErrSignature) {
t.Fatalf("a swapped payload was not refused: %v", err)
}
}
// A token from the same IdP but issued to a different client is perfectly valid — just not for
// us. Accepting it would let any other client of the same provider authenticate here.
func TestRejectsAnotherClientsToken(t *testing.T) {
idp := newIdP(t)
v := verifier(idp, "")
tok := idp.sign(t, "RS256", "rsa-1", idp.claims(map[string]any{"aud": "some-other-app"}))
if _, err := v.Verify(context.Background(), tok); !errors.Is(err, ErrClaims) {
t.Fatalf("another client's token was accepted: %v", err)
}
}
func TestAcceptsAudienceArrayContainingUs(t *testing.T) {
idp := newIdP(t)
v := verifier(idp, "")
tok := idp.sign(t, "RS256", "rsa-1", idp.claims(map[string]any{"aud": []string{"other", "echolot"}}))
if _, err := v.Verify(context.Background(), tok); err != nil {
t.Fatalf("an audience array including us was refused: %v", err)
}
}
func TestRejectsExpiredAndFutureTokens(t *testing.T) {
idp := newIdP(t)
v := verifier(idp, "")
expired := idp.sign(t, "RS256", "rsa-1", idp.claims(map[string]any{
"exp": time.Now().Add(-time.Hour).Unix(),
}))
if _, err := v.Verify(context.Background(), expired); !errors.Is(err, ErrClaims) {
t.Errorf("expired token accepted: %v", err)
}
future := idp.sign(t, "RS256", "rsa-1", idp.claims(map[string]any{
"iat": time.Now().Add(time.Hour).Unix(),
}))
if _, err := v.Verify(context.Background(), future); !errors.Is(err, ErrClaims) {
t.Errorf("token issued in the future accepted: %v", err)
}
}
// A token signed by a completely different provider, with its own keys and its own kid.
func TestRejectsATokenFromAnotherIssuer(t *testing.T) {
ours, theirs := newIdP(t), newIdP(t)
v := verifier(ours, "")
tok := theirs.sign(t, "RS256", "rsa-1", theirs.claims(nil))
if _, err := v.Verify(context.Background(), tok); err == nil {
t.Fatal("a token from another issuer was accepted")
}
}
func TestRejectsMalformedTokens(t *testing.T) {
idp := newIdP(t)
v := verifier(idp, "")
for _, bad := range []string{"", "not-a-token", "a.b", "a.b.c.d", "...", "!!!.???.***"} {
if _, err := v.Verify(context.Background(), bad); err == nil {
t.Errorf("%q was accepted", bad)
}
}
}
// With no admin group configured, nobody is an admin. An operator who has not said who may
// administer the server has not thereby said "anyone who can log in".
func TestNobodyIsAdminUntilAGroupIsConfigured(t *testing.T) {
idp := newIdP(t)
claims := &Claims{Groups: []string{"users", "echolot-admins"}}
if verifier(idp, "").IsAdmin(claims) {
t.Error("someone was an admin with no admin group configured")
}
if !verifier(idp, "echolot-admins").IsAdmin(claims) {
t.Error("a member of the configured group was not an admin")
}
if verifier(idp, "other-group").IsAdmin(claims) {
t.Error("a non-member was an admin")
}
if verifier(idp, "echolot-admins").IsAdmin(nil) {
t.Error("an absent identity was an admin")
}
}
// A discovery document whose issuer disagrees with the configured one means we were redirected
// somewhere — and would otherwise have fetched that somewhere's signing keys while believing
// they belonged to the configured provider.
func TestRefusesDiscoveryThatRenamesTheIssuer(t *testing.T) {
mux := http.NewServeMux()
srv := httptest.NewServer(mux)
defer srv.Close()
mux.HandleFunc("/.well-known/openid-configuration", func(w http.ResponseWriter, r *http.Request) {
_ = json.NewEncoder(w).Encode(Discovery{Issuer: "https://somewhere.else", JWKSURI: srv.URL + "/jwks"})
})
v := New(Config{Issuer: srv.URL, ClientID: "echolot"}, srv.Client())
if _, err := v.Discover(context.Background()); err == nil {
t.Fatal("discovery accepted a document for a different issuer")
}
}
func TestDisabledWithoutConfiguration(t *testing.T) {
v := New(Config{}, nil)
if v.Config().Enabled() {
t.Fatal("an unconfigured verifier reports itself enabled")
}
if _, err := v.Verify(context.Background(), "x.y.z"); !errors.Is(err, ErrDisabled) {
t.Fatalf("want ErrDisabled, got %v", err)
}
}
// Two clients, because the phone and the admin UI have different properties: an APK cannot keep a
// secret (public + PKCE) while the server can (confidential). Both must be accepted — but only
// those two. "Any client of this issuer" would let every other application registered with the
// same IdP authenticate here, which is the whole reason the audience check exists.
func TestBothRegisteredClientsAreAccepted(t *testing.T) {
idp := newIdP(t)
v := verifier(idp, "")
for _, aud := range []any{"echolot", "echolot-app", []string{"echolot-app", "other"}} {
tok := idp.sign(t, "RS256", "rsa-1", idp.claims(map[string]any{"aud": aud}))
if _, err := v.Verify(context.Background(), tok); err != nil {
t.Errorf("aud %v was refused: %v", aud, err)
}
}
// A third application at the same issuer is still not us.
tok := idp.sign(t, "RS256", "rsa-1", idp.claims(map[string]any{"aud": "someone-elses-app"}))
if _, err := v.Verify(context.Background(), tok); !errors.Is(err, ErrClaims) {
t.Fatalf("a third client's token was accepted: %v", err)
}
}
// Either client id alone is enough to make sign-in usable: an operator may register only the app
// (no admin UI login) or only the server.
func TestEitherClientIDAloneEnablesSignIn(t *testing.T) {
if !(Config{Issuer: "https://i", ClientID: "a"}).Enabled() {
t.Error("a server-only configuration was reported disabled")
}
if !(Config{Issuer: "https://i", AppClientID: "b"}).Enabled() {
t.Error("an app-only configuration was reported disabled")
}
if (Config{Issuer: "https://i"}).Enabled() {
t.Error("an issuer with no client at all was reported enabled")
}
}
+100
View File
@@ -0,0 +1,100 @@
// SPDX-FileCopyrightText: 2026 Echolot contributors
// SPDX-License-Identifier: GPL-3.0-or-later
// Package ratelimit implements the spec §2.5 token buckets: per-credential and per-source-IP
// ceilings on session creation, actions, UDP packets and bytes. One Limiter holds one policy
// (rate + burst) and lazily creates a bucket per key; callers namespace their keys ("cred:…",
// "ip:…") so a single Limiter can enforce both axes of the same rule.
package ratelimit
import (
"sync"
"time"
)
// Limiter is a keyed set of token buckets sharing one rate and burst.
//
// A nil *Limiter allows everything: the ceilings are configurable down to "off" (config value 0),
// and a nil check in one place beats a sentinel policy that every call site must know about.
type Limiter struct {
rate float64 // tokens per second
burst float64
mu sync.Mutex
buckets map[string]*bucket
lastSweep time.Time
now func() time.Time // swappable so tests need no sleeping
}
type bucket struct {
tokens float64
last time.Time
}
// New creates a limiter granting ratePerSec tokens per second per key, holding at most burst.
func New(ratePerSec, burst float64) *Limiter {
return &Limiter{
rate: ratePerSec,
burst: burst,
buckets: map[string]*bucket{},
now: time.Now,
}
}
// Allow takes one token for key. See AllowN.
func (l *Limiter) Allow(key string) (bool, time.Duration) {
return l.AllowN(key, 1)
}
// AllowN takes n tokens for key, reporting whether they were available and — when they were
// not — how long until they will be, which is what a control-plane 429 puts in Retry-After.
// A refusal consumes nothing: the caller being told to wait must not itself push the wait out.
func (l *Limiter) AllowN(key string, n float64) (bool, time.Duration) {
if l == nil {
return true, 0
}
l.mu.Lock()
defer l.mu.Unlock()
now := l.now()
l.sweepLocked(now)
b := l.buckets[key]
if b == nil {
b = &bucket{tokens: l.burst, last: now}
l.buckets[key] = b
}
b.tokens += now.Sub(b.last).Seconds() * l.rate
if b.tokens > l.burst {
b.tokens = l.burst
}
b.last = now
if b.tokens >= n {
b.tokens -= n
return true, 0
}
return false, time.Duration((n - b.tokens) / l.rate * float64(time.Second))
}
// sweepEvery bounds how often the map is walked; the walk is cheap but there is no point doing
// it per packet on the data plane's hot path.
const sweepEvery = time.Minute
// sweepLocked drops buckets that have been idle long enough to be full again. A full bucket
// carries no state a fresh one would not, and without the sweep the map grows one entry per
// source address ever seen — an attacker-controlled key space must not be an unbounded one.
func (l *Limiter) sweepLocked(now time.Time) {
if now.Sub(l.lastSweep) < sweepEvery {
return
}
l.lastSweep = now
idle := sweepEvery
if l.rate > 0 {
if refill := time.Duration(l.burst / l.rate * float64(time.Second)); refill > idle {
idle = refill
}
}
for k, b := range l.buckets {
if now.Sub(b.last) > idle {
delete(l.buckets, k)
}
}
}
+105
View File
@@ -0,0 +1,105 @@
// SPDX-FileCopyrightText: 2026 Echolot contributors
// SPDX-License-Identifier: GPL-3.0-or-later
package ratelimit
import (
"testing"
"time"
)
// clockAt pins the limiter to a fake clock so refill is a function of arithmetic, not sleeping.
func clockAt(l *Limiter) *time.Time {
t := time.Unix(1000, 0)
l.now = func() time.Time { return t }
return &t
}
func TestBurstThenRefusalThenRefill(t *testing.T) {
l := New(1, 3) // 1 token/s, burst 3
now := clockAt(l)
for i := 0; i < 3; i++ {
if ok, _ := l.Allow("k"); !ok {
t.Fatalf("token %d of the burst refused", i)
}
}
ok, wait := l.Allow("k")
if ok {
t.Fatal("fourth token inside the same instant should be refused")
}
if wait <= 0 || wait > time.Second {
t.Fatalf("retry-after = %v, want (0, 1s]", wait)
}
*now = now.Add(2 * time.Second) // refills 2 tokens
if ok, _ := l.Allow("k"); !ok {
t.Fatal("refused after refill")
}
if ok, _ := l.Allow("k"); !ok {
t.Fatal("second refilled token refused")
}
if ok, _ := l.Allow("k"); ok {
t.Fatal("third token allowed but only two seconds elapsed")
}
}
func TestRefusalConsumesNothing(t *testing.T) {
l := New(1, 1)
now := clockAt(l)
l.Allow("k")
// Hammering while empty must not push the refill out.
for i := 0; i < 10; i++ {
if ok, _ := l.Allow("k"); ok {
t.Fatal("allowed while empty")
}
}
*now = now.Add(time.Second)
if ok, _ := l.Allow("k"); !ok {
t.Fatal("the refused attempts ate the refill")
}
}
func TestKeysAreIndependent(t *testing.T) {
l := New(1, 1)
clockAt(l)
l.Allow("a")
if ok, _ := l.Allow("b"); !ok {
t.Fatal("draining key a refused key b")
}
}
func TestAllowNChargesBytes(t *testing.T) {
l := New(1000, 1000) // e.g. bytes/s
clockAt(l)
if ok, _ := l.AllowN("k", 900); !ok {
t.Fatal("900 of 1000 refused")
}
if ok, _ := l.AllowN("k", 200); ok {
t.Fatal("1100 of 1000 allowed")
}
if ok, _ := l.AllowN("k", 100); !ok {
t.Fatal("the refused 200 consumed the remaining 100")
}
}
func TestNilLimiterAllowsEverything(t *testing.T) {
var l *Limiter
if ok, wait := l.AllowN("k", 1e12); !ok || wait != 0 {
t.Fatal("nil limiter must be a no-op")
}
}
func TestSweepDropsIdleBucketsOnly(t *testing.T) {
l := New(1, 3)
now := clockAt(l)
l.Allow("idle")
*now = now.Add(2 * time.Minute)
l.Allow("busy") // triggers the sweep; "idle" refilled long ago
if _, held := l.buckets["idle"]; held {
t.Fatal("idle bucket survived the sweep")
}
if _, held := l.buckets["busy"]; !held {
t.Fatal("active bucket was swept")
}
}
+66
View File
@@ -0,0 +1,66 @@
// SPDX-FileCopyrightText: 2026 Echolot contributors
// SPDX-License-Identifier: GPL-3.0-or-later
// Package relsign signs and verifies release manifests (detached ed25519 over SHA256SUMS).
//
// The checksum file alone protects download integrity, not authenticity: SHA256SUMS and the
// binaries come from the same Gitea release, so whoever can alter one can alter both. The
// signature is what separates "the file arrived intact" from "the project published this file" —
// its private key lives in the CI secret store, not on the release host, so a compromised Gitea
// can serve corrupted binaries but cannot make a self-updating server accept them.
//
// Formats, chosen to be reproducible with nothing but a stock library in any language:
// the private key is the base64 of the 32-byte ed25519 seed, the public key the base64 of the
// 32-byte public key, and the signature file the base64 of the 64-byte signature over the exact
// bytes of the signed file.
package relsign
import (
"crypto/ed25519"
"encoding/base64"
"fmt"
"strings"
)
// GenerateKey mints a fresh signing keypair.
func GenerateKey() (pubB64, seedB64 string, err error) {
pub, priv, err := ed25519.GenerateKey(nil)
if err != nil {
return "", "", err
}
return base64.StdEncoding.EncodeToString(pub),
base64.StdEncoding.EncodeToString(priv.Seed()), nil
}
// Sign produces the detached signature (base64) for data.
func Sign(seedB64 string, data []byte) (string, error) {
seed, err := base64.StdEncoding.DecodeString(strings.TrimSpace(seedB64))
if err != nil {
return "", fmt.Errorf("signing key is not valid base64: %w", err)
}
if len(seed) != ed25519.SeedSize {
return "", fmt.Errorf("signing key must be %d bytes, got %d", ed25519.SeedSize, len(seed))
}
priv := ed25519.NewKeyFromSeed(seed)
return base64.StdEncoding.EncodeToString(ed25519.Sign(priv, data)), nil
}
// Verify checks a detached signature. A nil error means the holder of the private key matching
// pubB64 signed exactly these bytes.
func Verify(pubB64 string, data []byte, sigB64 string) error {
pub, err := base64.StdEncoding.DecodeString(strings.TrimSpace(pubB64))
if err != nil {
return fmt.Errorf("public key is not valid base64: %w", err)
}
if len(pub) != ed25519.PublicKeySize {
return fmt.Errorf("public key must be %d bytes, got %d", ed25519.PublicKeySize, len(pub))
}
sig, err := base64.StdEncoding.DecodeString(strings.TrimSpace(sigB64))
if err != nil {
return fmt.Errorf("signature is not valid base64: %w", err)
}
if !ed25519.Verify(ed25519.PublicKey(pub), data, sig) {
return fmt.Errorf("signature does not verify: the file was not signed by this key, or was altered after signing")
}
return nil
}
+74
View File
@@ -0,0 +1,74 @@
// SPDX-FileCopyrightText: 2026 Echolot contributors
// SPDX-License-Identifier: GPL-3.0-or-later
package relsign
import (
"strings"
"testing"
)
func TestRoundTrip(t *testing.T) {
pub, seed, err := GenerateKey()
if err != nil {
t.Fatal(err)
}
data := []byte("abc123 echolot-server_linux_amd64\n")
sig, err := Sign(seed, data)
if err != nil {
t.Fatal(err)
}
if err := Verify(pub, data, sig); err != nil {
t.Fatalf("a signature this package just made must verify: %v", err)
}
}
func TestAlteredContentIsRefused(t *testing.T) {
// The attack this exists for: same length, one checksum swapped for another.
pub, seed, _ := GenerateKey()
sig, _ := Sign(seed, []byte("aaa echolot-server_linux_amd64\n"))
if err := Verify(pub, []byte("bbb echolot-server_linux_amd64\n"), sig); err == nil {
t.Fatal("altered content must not verify")
}
}
func TestWrongKeyIsRefused(t *testing.T) {
// A compromised release host can re-sign with its own key; only ours may pass.
pub1, _, _ := GenerateKey()
_, seed2, _ := GenerateKey()
data := []byte("payload")
sig, _ := Sign(seed2, data)
if err := Verify(pub1, data, sig); err == nil {
t.Fatal("a signature from a different key must not verify")
}
}
func TestSurroundingWhitespaceIsTolerated(t *testing.T) {
// Keys travel through env vars and files; a trailing newline must not break verification.
pub, seed, _ := GenerateKey()
data := []byte("data")
sig, err := Sign(" "+seed+"\n", data)
if err != nil {
t.Fatal(err)
}
if err := Verify(pub+"\n", data, "\t"+sig+"\n"); err != nil {
t.Fatalf("whitespace around base64 must be tolerated: %v", err)
}
}
func TestGarbageInputsFailCleanly(t *testing.T) {
pub, seed, _ := GenerateKey()
if _, err := Sign("not base64!!", []byte("x")); err == nil || !strings.Contains(err.Error(), "base64") {
t.Fatalf("bad seed must name the problem, got %v", err)
}
if _, err := Sign("c2hvcnQ=", []byte("x")); err == nil {
t.Fatal("short seed must be refused")
}
if err := Verify("c2hvcnQ=", []byte("x"), "AAAA"); err == nil {
t.Fatal("short public key must be refused")
}
if err := Verify(pub, []byte("x"), "not base64!!"); err == nil {
t.Fatal("bad signature encoding must be refused")
}
_ = seed
}
+78
View File
@@ -0,0 +1,78 @@
// SPDX-FileCopyrightText: 2026 Echolot contributors
// SPDX-License-Identifier: GPL-3.0-or-later
package runs
import (
"testing"
"time"
)
// Account scoping widens what a caller can read, so the test that matters is the one about what
// it must NOT widen: a run id from another account has to be invisible, not merely unlisted.
func TestAccountScopingDoesNotReachOtherAccounts(t *testing.T) {
s, _ := open(t, DefaultPolicy())
// Two devices on one account, one device belonging to somebody else.
mine := []string{"phone-a", "tablet-a"}
for i, d := range mine {
if _, err := s.Put(d, doc("run-"+d, AnonFull), true); err != nil {
t.Fatal(err)
}
_ = i
time.Sleep(2 * time.Millisecond)
}
if _, err := s.Put("phone-b", doc("run-secret", AnonFull), true); err != nil {
t.Fatal(err)
}
got := s.ListFor(mine)
if len(got) != 2 {
t.Fatalf("account history has %d runs, want 2", len(got))
}
for _, m := range got {
if m.ID == "run-secret" {
t.Fatal("another account's run appeared in the history")
}
}
// The decisive one: knowing the id is not enough.
if _, ok := s.OwnerOf(mine, "run-secret"); ok {
t.Fatal("a run id from another account resolved against this account's devices")
}
if owner, ok := s.OwnerOf(mine, "run-phone-a"); !ok || owner != "phone-a" {
t.Fatalf("own run did not resolve: owner=%q ok=%v", owner, ok)
}
// A sibling device's run must resolve — that is the point of the feature.
if owner, ok := s.OwnerOf(mine, "run-tablet-a"); !ok || owner != "tablet-a" {
t.Fatalf("sibling device's run did not resolve: owner=%q ok=%v", owner, ok)
}
}
func TestAccountHistoryIsNewestFirstAcrossDevices(t *testing.T) {
s, _ := open(t, DefaultPolicy())
if _, err := s.Put("phone", doc("older", AnonFull), true); err != nil {
t.Fatal(err)
}
time.Sleep(5 * time.Millisecond)
if _, err := s.Put("tablet", doc("newer", AnonFull), true); err != nil {
t.Fatal(err)
}
got := s.ListFor([]string{"phone", "tablet"})
if len(got) != 2 || got[0].ID != "newer" {
t.Fatalf("not merged newest-first: %+v", got)
}
}
func TestEmptyDeviceSetSeesNothing(t *testing.T) {
s, _ := open(t, DefaultPolicy())
if _, err := s.Put("someone", doc("run-1", AnonFull), true); err != nil {
t.Fatal(err)
}
if got := s.ListFor(nil); len(got) != 0 {
t.Fatalf("an empty device set returned %d runs", len(got))
}
if _, ok := s.OwnerOf(nil, "run-1"); ok {
t.Fatal("a run resolved against an empty device set")
}
}

Some files were not shown because too many files have changed in this diff Show More