Compare commits
24
Commits
@@ -43,3 +43,6 @@ web/.wrangler/
|
|||||||
|
|
||||||
# Eclipse/JDT output from the VSCodium Java extension — not a build artifact we own
|
# Eclipse/JDT output from the VSCodium Java extension — not a build artifact we own
|
||||||
echolot-app/*/bin/
|
echolot-app/*/bin/
|
||||||
|
|
||||||
|
# Kotlin compiler scratch/error logs
|
||||||
|
echolot-app/.kotlin/
|
||||||
|
|||||||
@@ -32,10 +32,25 @@ Keep prober result IDs aligned with the measurement-schema test-type registry.
|
|||||||
|
|
||||||
## Versioning
|
## Versioning
|
||||||
|
|
||||||
Prefer **patch** bumps (`server-v0.3.1`) for additive/incremental work; reserve **minor** bumps
|
Both artifacts are **SemVer**. Prefer **patch** bumps (`server-v0.3.1`) for additive/incremental
|
||||||
for real milestones. Don't burn through minor versions. Tags are namespaced: `server-v*` for the
|
work; reserve **minor** bumps for real milestones. Don't burn through minor versions. Tags are
|
||||||
Go server, `v*` for the app. Pushing a `server-v*` tag runs CI → binaries + Gitea release +
|
namespaced: `server-v*` for the Go server, `v*` for the app. Pushing a `server-v*` tag runs CI →
|
||||||
registry image; the server on fmr can `--self-update` from those releases.
|
binaries + Gitea release + registry image; the server on fmr can `--self-update` from those
|
||||||
|
releases.
|
||||||
|
|
||||||
|
The app's version lives once, as `appVersionName` in `app/build.gradle.kts`; **`versionCode` is
|
||||||
|
derived from it** (`major*1e6 + minor*1e4 + patch*10`). Never set it by hand — a second number a
|
||||||
|
human has to remember to bump eventually disagrees with the first.
|
||||||
|
|
||||||
|
**Versions are load-bearing** (probe-protocol.md §8): the server refuses apps outside its window
|
||||||
|
with `426`, and the app refuses servers outside its own. Two axes, kept separate:
|
||||||
|
- `protocol_version` — *can* they talk. The correctness axis; below 1.0.0 the **minor** is the
|
||||||
|
breaking axis.
|
||||||
|
- release-version window — *may* they, per policy. `[min, max)`, bounds at breaking boundaries so
|
||||||
|
a patch never strands a fleet. Client bounds: `Compat.kt`. Server: `ECHOLOT_MIN/MAX_APP_VERSION`.
|
||||||
|
|
||||||
|
Raise a minimum only when older peers are actively harmful, and say why in the constant's comment.
|
||||||
|
`GET /v1/profile` must stay ungated — it is how a refused client learns what it needs.
|
||||||
|
|
||||||
## Layout
|
## Layout
|
||||||
|
|
||||||
@@ -52,6 +67,29 @@ echolot-prober/ the capability prober (self-contained Gradle buil
|
|||||||
ui/ProberScreen.kt result cards colored by verdict
|
ui/ProberScreen.kt result cards colored by verdict
|
||||||
```
|
```
|
||||||
|
|
||||||
|
## Client modules (echolot-app/)
|
||||||
|
|
||||||
|
Pure Kotlin/JVM where possible, so the interesting logic is unit-testable without a device and can
|
||||||
|
be exercised against the live server from the PC:
|
||||||
|
|
||||||
|
- `core-protocol` — control plane (pinned TLS) + ELT1 UDP data plane. **One `ProbeSession` per
|
||||||
|
server session, for its whole lifetime**: a second one restarts sequence numbers, the server's
|
||||||
|
anti-replay window discards every packet, and granted sends then target the closed socket.
|
||||||
|
- `core-measurement` — the schema types. `core-engine` — composes probes into documents.
|
||||||
|
- `core-privacy` — the §8 anonymizer (`full` / `balanced` / `strict`). Field classification lives
|
||||||
|
in one table (`Classification.kt`); keep it there rather than annotating models.
|
||||||
|
- `core-archive` — on-device run storage + retention. `enabled` is separate from the three
|
||||||
|
ceilings: all-zeros means "no limits", not "keep nothing".
|
||||||
|
|
||||||
|
**The local archive keeps the unredacted document; anonymization happens per upload, on the way
|
||||||
|
out.** Never redact what is stored locally.
|
||||||
|
|
||||||
|
### Live testing without a device
|
||||||
|
`echolot-app/scripts/test-fmr.sh [gradle-task] [test-filter]` mints an enrollment token over SSH,
|
||||||
|
enrolls, computes the SPKI pin from the served cert and runs a `Live*Test` against fmr. This covers
|
||||||
|
the whole server-facing vertical (granted sends, downstream MTU, uploads) with no phone involved —
|
||||||
|
use it before asking the user to test on hardware.
|
||||||
|
|
||||||
## Conventions
|
## Conventions
|
||||||
|
|
||||||
- Every probe returns a `ProbeResult` — never throws to the caller (MainActivity wraps anyway).
|
- Every probe returns a `ProbeResult` — never throws to the caller (MainActivity wraps anyway).
|
||||||
@@ -107,6 +145,12 @@ First build downloads AGP/Compose/Shizuku from Google Maven + Maven Central.
|
|||||||
Shizuku, **toggle Wireless debugging off/on** — Shizuku keeps running (separate process), a fresh
|
Shizuku, **toggle Wireless debugging off/on** — Shizuku keeps running (separate process), a fresh
|
||||||
port + mDNS record appear, and the beacon/connector recover. Plan the Shizuku-tier dev loop
|
port + mDNS record appear, and the beacon/connector recover. Plan the Shizuku-tier dev loop
|
||||||
around this (or USB, if ever available).
|
around this (or USB, if ever available).
|
||||||
|
- **Empty-jar race with the IDE.** VSCodium's Java/Kotlin extension runs its own Gradle daemon on
|
||||||
|
the same project; when it overlaps a CLI build, a module's `build/libs/*.jar` can end up
|
||||||
|
containing only a manifest, and Gradle then considers `jar` up-to-date. Dependent modules fail
|
||||||
|
with "Unresolved reference" on symbols that plainly exist. Fix: `rm -f <module>/build/libs/*.jar`
|
||||||
|
and re-run the `jar` task. Suspect this whenever a reference resolves in one module but not in
|
||||||
|
its consumer.
|
||||||
- As of the AGP 9.2.0 / Gradle 9.6.0 / Kotlin 2.2.10 bump, JDK 17+ (including 25) works —
|
- As of the AGP 9.2.0 / Gradle 9.6.0 / Kotlin 2.2.10 bump, JDK 17+ (including 25) works —
|
||||||
AGP 9 requires Gradle 9.1.0+ and Kotlin 2.2.10+ as its minimum KGP version. On machines with
|
AGP 9 requires Gradle 9.1.0+ and Kotlin 2.2.10+ as its minimum KGP version. On machines with
|
||||||
Android Studio, its `jbr` directory works as `JAVA_HOME`.
|
Android Studio, its `jbr` directory works as `JAVA_HOME`.
|
||||||
@@ -118,3 +162,11 @@ First build downloads AGP/Compose/Shizuku from Google Maven + Maven Central.
|
|||||||
are SUPPORTED on both known devices via `Os.recvmsg` + `StructMsghdr` reflection.
|
are SUPPORTED on both known devices via `Os.recvmsg` + `StructMsghdr` reflection.
|
||||||
3. Fold the confirmed capabilities + Shizuku dump-format samples back into the production
|
3. Fold the confirmed capabilities + Shizuku dump-format samples back into the production
|
||||||
`core-probe` / `core-shizuku` modules.
|
`core-probe` / `core-shizuku` modules.
|
||||||
|
|
||||||
|
## Enrolling a device with a server
|
||||||
|
|
||||||
|
`echolot-app/scripts/enroll-link.sh [note]` mints a §2.1 bootstrap link on fmr over SSH and prints
|
||||||
|
it (plus a QR if `qrencode` is installed, plus the `adb shell am start -a …VIEW -d '<uri>'` command
|
||||||
|
when a device is attached). The link carries a single-use token — treat it as a secret until spent.
|
||||||
|
Never hand-assemble one: the base64 pin needs percent-encoding, and a pin wrong by one character
|
||||||
|
fails as an inscrutable TLS error rather than as a bad pin.
|
||||||
|
|||||||
@@ -543,3 +543,401 @@ Best available behavior, now implemented: the banner still opens Shizuku, but th
|
|||||||
exact steps there ("Pairing", then "Start"), and a second tap target opens **Developer options**
|
exact steps there ("Pairing", then "Start"), and a second tap target opens **Developer options**
|
||||||
(`Settings.ACTION_APPLICATION_DEVELOPMENT_SETTINGS` — public and exported) since Wireless
|
(`Settings.ACTION_APPLICATION_DEVELOPMENT_SETTINGS` — public and exported) since Wireless
|
||||||
debugging must be enabled first for Shizuku's wireless start to work at all.
|
debugging must be enabled first for Shizuku's wireless start to work at all.
|
||||||
|
|
||||||
|
### Downstream measurements: asymmetric grants, DF-mode big_send (server-v0.4.0 … v0.4.2, 2026-08-01)
|
||||||
|
The client can measure a round trip and the largest packet it can *send*. It cannot measure the
|
||||||
|
largest packet it can *receive*, or downstream-only loss — those need the server to push, which is
|
||||||
|
exactly what §3.4 gates behind an asymmetric grant. Implemented and verified live from the PC:
|
||||||
|
|
||||||
|
- **`session.Grant`** — created per action, bound at creation to the session's *observed*
|
||||||
|
data-plane source (no grant without a verified destination), clamped to server limits, with a
|
||||||
|
byte budget, an average-rate ceiling and an expiry. Unit-tested for each of those refusals.
|
||||||
|
- **`downtrain`** — N packets of size S every I µs; the client derives downstream loss,
|
||||||
|
reordering and inter-arrival spacing.
|
||||||
|
- **`big_send`** — one datagram per requested size. **DF is on by default**, so the largest size
|
||||||
|
that arrives *is* the downstream path MTU. Without DF the kernel fragments and the result only
|
||||||
|
says whether fragments get through — a different fact, and the reason the schema has both
|
||||||
|
`mtu.pmtud_down` and `mtu.frag_delivery`. Sizes above the server's own egress MTU (from the
|
||||||
|
startup self-test) are refused up front and reported as `max_df_bytes`, so an absence caused by
|
||||||
|
our kernel is never read as a limit of the client's path.
|
||||||
|
|
||||||
|
Live from the PC against fmr: downstream path MTU **1500** (1472 payload, DF), fragmented delivery
|
||||||
|
up to **4000**, downstream train **100/100, 0 % loss, 0 reordered**, inter-arrival 3.3 ms for a
|
||||||
|
3000 µs send interval.
|
||||||
|
|
||||||
|
#### Two bugs this shook out, both invisible in a single-homed lab
|
||||||
|
1. **Granted sends went out from the wrong local address** (fixed in server-v0.4.2). fmr binds two
|
||||||
|
IPv4 addresses; `connFor` returned whichever socket of the right family came first in the bind
|
||||||
|
list. A train for a session established on `.150` left from `.151` and every packet was dropped
|
||||||
|
by the client's NAT, which has no mapping for that pair. tcpdump showed all 50 leaving, the
|
||||||
|
client saw none — reported as *100 % downstream loss*, a confident measurement of something
|
||||||
|
that never happened. Sessions now record which of our own bound addresses received their
|
||||||
|
traffic and granted sends go back through that socket; `connfor_test.go` pins both that and the
|
||||||
|
family fallback.
|
||||||
|
2. **A second `ProbeSession` on one server session is silently dead.** Sequence numbers restart at
|
||||||
|
zero client-side while the server's anti-replay window keeps counting, so every packet is
|
||||||
|
discarded as a replay — and because the server then never records the new source, the grant
|
||||||
|
still targets the closed socket. `ServerMeasurement` now uses one ProbeSession for the whole
|
||||||
|
run; `ProbeSession`'s doc comment states the constraint.
|
||||||
|
|
||||||
|
### Run archive, anonymizer and uploads (2026-08-01)
|
||||||
|
Three pieces, deliberately separate:
|
||||||
|
|
||||||
|
- **`core-archive`** — one JSON file per run plus an index entry, in a plain directory the user can
|
||||||
|
inspect or delete with a file manager. Retention (max runs / max age / max total bytes) is
|
||||||
|
enforced on every save rather than by a sweeper. `enabled` is a separate flag from the three
|
||||||
|
ceilings because "no limits" and "keep nothing" are opposite intentions; collapsing them onto
|
||||||
|
all-zeros is how a user who turns the caps off ends up with an empty history. 13 tests.
|
||||||
|
- **`core-privacy`** — the schema §8 anonymizer, three levels. `full` (your own server) changes
|
||||||
|
nothing; `balanced` pseudonymizes SSIDs/hostnames, keeps the OUI half of a MAC and the /16 of a
|
||||||
|
public IP, keeps RFC1918 verbatim (it describes topology, not a person), and *drops* neighbour
|
||||||
|
inventories (SSDP/ARP/scan results) rather than mangling them; `strict` keeps only metrics,
|
||||||
|
statuses and finding codes. Pseudonyms are consistent within a document and — by default — not
|
||||||
|
across documents, so an upload endpoint cannot link a device's runs; a stable salt is opt-in for
|
||||||
|
people diffing their own history. Classification is one readable table, not annotations spread
|
||||||
|
across modules. 14 tests, each pinning a property someone's privacy depends on.
|
||||||
|
- **Server-side upload policy** — `off | anonymous | account`, plus max size, retention days, max
|
||||||
|
runs per device, and the *least* anonymization accepted. The profile advertises all of it so the
|
||||||
|
app presents the choice honestly instead of discovering the rules by being rejected. `account`
|
||||||
|
refuses today rather than falling back to anonymous: picking the strict setting before OIDC
|
||||||
|
lands must not silently mean the loose one.
|
||||||
|
|
||||||
|
**The archive holds the unredacted document; redaction happens on the way out, per upload.** The
|
||||||
|
local archive is the user's own data on their own device, and redacting it would destroy exactly
|
||||||
|
the detail that makes a week-old run worth keeping.
|
||||||
|
|
||||||
|
App-side: settings screen (archive limits, privacy level with a plain-language description of what
|
||||||
|
each keeps, auto-upload off by default, server URL/pin/credential), history screen showing whether
|
||||||
|
each run left the device, and a **preview of the exact bytes an upload would send** — an anonymizer
|
||||||
|
the user cannot inspect is only a promise.
|
||||||
|
|
||||||
|
Live round trip against fmr: uploaded a run, listed it, fetched it back and asserted the SSID, the
|
||||||
|
SSDP neighbour name and the free-text note are absent from what the server stores while the
|
||||||
|
finding code and the metrics survive, then deleted it.
|
||||||
|
|
||||||
|
### Still open
|
||||||
|
- `mtu.pmtud_up` (DF + errqueue), `frag_send`, `throughput`, TRAIN_REPORT retrieval.
|
||||||
|
- Enrollment UI in the app (server URL/pin/credential are typed in by hand today).
|
||||||
|
- Accounts/OIDC on the server, which is what `uploads=account` is waiting for.
|
||||||
|
- Nothing in this entry has been exercised on a phone yet — all of it was verified from the PC
|
||||||
|
against the live server. On-device verification is the next step.
|
||||||
|
|
||||||
|
### SemVer compatibility windows between app and server (server-v0.5.0 … v0.5.2, 2026-08-01)
|
||||||
|
Both artifacts are SemVer, and each now declares — and enforces — which peer versions it will talk
|
||||||
|
to. Spec: `docs/probe-protocol.md` §8.
|
||||||
|
|
||||||
|
**Two axes, deliberately not conflated.** Release versions are a *proxy* for what actually has to
|
||||||
|
match, so the real thing is checked first:
|
||||||
|
- `protocol_version` — **can** these builds talk. Advertised in the profile; a peer in a different
|
||||||
|
breaking series is refused whatever its release version says. Below 1.0.0 the **minor** is the
|
||||||
|
breaking axis (SemVer §4).
|
||||||
|
- release-version window — **may** they, per policy. `[min, max)`, min inclusive, max exclusive,
|
||||||
|
because the useful bound is always "the version that broke it".
|
||||||
|
|
||||||
|
Bounds sit at breaking boundaries, not at releases, so shipping a patch never requires editing a
|
||||||
|
range. The app requires server `>= 0.4.2` for a stated reason, not caution: earlier multi-homed
|
||||||
|
servers mis-addressed granted sends and the client measured 100 % downstream loss that never
|
||||||
|
happened. Operators override the server side with `ECHOLOT_MIN_APP_VERSION` /
|
||||||
|
`ECHOLOT_MAX_APP_VERSION`; a malformed bound is fatal at startup rather than ignored, so a typo
|
||||||
|
cannot silently disable a restriction.
|
||||||
|
|
||||||
|
Three rules that shaped the implementation:
|
||||||
|
1. **`GET /v1/profile` is never gated.** It is where a refused client learns which version it needs;
|
||||||
|
gating it leaves the user with a network error instead of an answer.
|
||||||
|
2. **An unparseable or absent version is `unknown`, and is allowed.** Dev builds report `dev`, and a
|
||||||
|
client too old to send the header cannot be identified anyway.
|
||||||
|
3. **Refusal is 426 with a body naming both versions and the window**, surfaced client-side as a
|
||||||
|
distinct `VersionRefused` rather than folded into "network error".
|
||||||
|
|
||||||
|
The app's `versionCode` is now derived from its SemVer (`major*1e6 + minor*1e4 + patch*10`) instead
|
||||||
|
of being a second number to remember.
|
||||||
|
|
||||||
|
Verified live against fmr (`LiveCompatTest`): profile advertises the window and stays readable for a
|
||||||
|
refused version; 0.1.0 and 99.0.0 are both refused with actionable messages; 0.2.0 and a missing
|
||||||
|
header are both served.
|
||||||
|
|
||||||
|
One user-visible bug caught in the process: Go's JSON encoder HTML-escapes `<`, `>` and `&` by
|
||||||
|
default, so the refusal reached the client as `needs \u003e= 0.2.0`. Disabled at the encoder (this
|
||||||
|
is an API, not a page), and the client now *parses* the error field instead of pattern-matching it,
|
||||||
|
so it survives whatever a future encoder decides to escape.
|
||||||
|
|
||||||
|
### Enrollment: the server mints the bootstrap link (server-v0.5.3 … v0.5.4, 2026-08-01)
|
||||||
|
Until now a device was configured by hand-typing a control URL, a base64 SPKI pin and a
|
||||||
|
credential. That is the step that goes wrong, and it goes wrong quietly: a pin off by one
|
||||||
|
character does not fail loudly, it just never matches, and surfaces days later as an inscrutable
|
||||||
|
TLS error.
|
||||||
|
|
||||||
|
`POST /admin/enroll-tokens` now returns the whole §2.1 bootstrap link alongside the token, because
|
||||||
|
the server is the only party holding all three parts at once. The app takes it from a paste or an
|
||||||
|
`echolot://enroll` deep link (so a QR scan configures a server in one action) and writes URL, pin
|
||||||
|
and credential **together or not at all** — a half-applied server fails later, somewhere else,
|
||||||
|
with an error pointing at the wrong thing.
|
||||||
|
|
||||||
|
The control URL comes from `ECHOLOT_PUBLIC_URL` (set on fmr to `https://fmr-1.echo-lot.app:8443`),
|
||||||
|
falling back to the first control listen address; a wildcard bind warns rather than emitting a
|
||||||
|
link to `0.0.0.0`.
|
||||||
|
|
||||||
|
**The encoding trap, which is the whole reason this is tested across both languages.** The pin is
|
||||||
|
base64, so it contains `+`, `/` and `=` — each of which means something else in a query string. An
|
||||||
|
unencoded `+` decodes to a space, leaving the pin wrong by exactly one character. Base64 has no
|
||||||
|
spaces, so the parser restores them; that cannot damage a correctly-encoded pin and it rescues
|
||||||
|
every hand-assembled link. `LiveEnrollmentTest` redeems a link the *server* produced, which is the
|
||||||
|
only way to catch a disagreement between the Go assembler and the Kotlin parser — a unit test on
|
||||||
|
either side alone cannot see it. It also asserts the token is refused the second time.
|
||||||
|
|
||||||
|
Also fixed a spec divergence found while reading §2.1: the spec names the field
|
||||||
|
`device_credential`, the first implementation shipped `credential`. The server now sends both and
|
||||||
|
the client prefers the spec's; the alias goes once nothing reads it.
|
||||||
|
|
||||||
|
Two process notes from this round:
|
||||||
|
- An edit to the admin handler silently failed to apply and the endpoint kept returning just the
|
||||||
|
token. Caught by deploying and *looking at the response*, not by trusting a green build.
|
||||||
|
- The live suite is now six tests (`LiveServerTest`, `LiveMeasurement`, `LiveGranted`,
|
||||||
|
`LiveUpload`, `LiveCompat`, `LiveEnrollment`), all green against fmr from the PC with no device.
|
||||||
|
|
||||||
|
### Directional loss: which way is the packet loss? (2026-08-01)
|
||||||
|
A round trip can only report that *something* was lost somewhere, which is the least useful form
|
||||||
|
of the answer — "3 % loss" sends an engineer looking in both directions at once. The server
|
||||||
|
already records every packet it received per sequence number (§6), so the two cases are actually
|
||||||
|
distinguishable, and `train.udp_updown` now reports them separately:
|
||||||
|
|
||||||
|
- sent, never seen by the server → **upstream** loss
|
||||||
|
- seen by the server, reply never arrived → **downstream** loss
|
||||||
|
|
||||||
|
Findings name the direction and say what is *not* implicated, which is half the value:
|
||||||
|
`connectivity.loss_upstream` ("the return path is not implicated: replies came back for everything
|
||||||
|
that arrived"), `connectivity.loss_downstream`, `nat.udp_unreachable_upstream`.
|
||||||
|
|
||||||
|
Two things the implementation gets deliberately right:
|
||||||
|
- **Downstream loss is measured against what reached the server**, not against what was sent.
|
||||||
|
Using "sent" as the denominator counts every upstream loss a second time and overstates the
|
||||||
|
return path. Pinned by a test with loss in both directions at once.
|
||||||
|
- **Per-direction jitter without synchronised clocks.** Absolute one-way delay would need clock
|
||||||
|
sync and we deliberately have none (the two-clock rule). But `server_rx − client_tx` carries a
|
||||||
|
constant unknown offset, and differencing successive samples cancels it — so RFC 3393 one-way
|
||||||
|
delay variation *is* honestly attributable to a direction even though latency is not. A test
|
||||||
|
pins that a 10-second clock offset changes nothing.
|
||||||
|
|
||||||
|
Correlation is by **wire sequence number**, which is not the loop index: the counter is shared
|
||||||
|
with every other packet type on the session, so "the nth echo" is not "sequence n". `ProbeSession`
|
||||||
|
now exposes `lastSeq`, including for a probe that was lost — a lost packet still has a sequence
|
||||||
|
number, and that number is exactly what tells you which way it was lost.
|
||||||
|
|
||||||
|
Live against fmr: 20/20 both ways, and jitter of **0.08 ms upstream vs 0.85 ms downstream** — a
|
||||||
|
tenfold asymmetry that a round-trip measurement cannot see at all.
|
||||||
|
|
||||||
|
10 unit tests on the arithmetic (a wrong denominator here does not crash, it produces a plausible
|
||||||
|
number pointing at the wrong half of the network) plus the live correlation check.
|
||||||
|
|
||||||
|
### frag_send: crafted IP fragments, so *ordering* is testable (server-v0.6.0, 2026-08-01)
|
||||||
|
`big_send` with `df=false` answers one question — do fragments get through. It cannot answer the
|
||||||
|
more interesting one, because the kernel always emits fragments in order, first one first.
|
||||||
|
|
||||||
|
The classic middlebox fault is exactly about that ordering. Only the **first** fragment carries the
|
||||||
|
UDP header, and therefore the ports; a stateful firewall or NAT that has not seen it has no flow to
|
||||||
|
match the rest against, and many simply drop them. That is invisible to every in-order test, and in
|
||||||
|
the field it looks like "large DNS answers fail on this network" or "the tunnel breaks when the MTU
|
||||||
|
drops" — it works until the network reorders, then fails intermittently, which is the hardest kind
|
||||||
|
of fault to chase.
|
||||||
|
|
||||||
|
So the server builds the fragments itself (raw socket, `IP_HDRINCL`) and controls their order:
|
||||||
|
`in_order` (baseline), `reversed` (last fragment first), `first_last` (first fragment held back
|
||||||
|
250 ms). The datagram is assembled and **signed whole** before being cut up, so what the client
|
||||||
|
reassembles is indistinguishable from an ordinary packet — otherwise the test would be measuring
|
||||||
|
our sender rather than the path. New test type `mtu.frag_ordering`; findings
|
||||||
|
`mtu.fragments_blocked` and `mtu.fragment_reorder_sensitive`.
|
||||||
|
|
||||||
|
Two details that would otherwise produce confidently wrong answers:
|
||||||
|
- **The UDP checksum is computed, not left zero.** Zero is legal in IPv4 and would be less code,
|
||||||
|
but zero-checksum datagrams are dropped by some middleboxes — and that drop would be recorded as
|
||||||
|
a fragmentation failure, which is the wrong conclusion entirely.
|
||||||
|
- **Fragment offsets are in 8-byte units**, so non-final fragments are rounded down to a multiple
|
||||||
|
of 8. A 100-byte fragment is not an error; it is a datagram no host will ever reassemble.
|
||||||
|
|
||||||
|
`frag-send` is advertised only when a raw socket can actually be opened — checked by opening one,
|
||||||
|
because a permission model has more ways to say no (userns, seccomp, LSM) than a capability bit has
|
||||||
|
to say yes. fmr runs as root with `cap_net_raw` in its bounding set, so it is available there.
|
||||||
|
|
||||||
|
Fragment ordering runs only after `mtu.frag_delivery` shows fragments arrive at all; otherwise the
|
||||||
|
three orderings would each report "not delivered" and read as three faults instead of one.
|
||||||
|
|
||||||
|
The header arithmetic is unit-tested (reassembly coverage with no gaps or double-delivery, MF
|
||||||
|
flags, shared IP ID, 8-byte offsets, checksum verification over odd and even lengths). Because the
|
||||||
|
code is `//go:build linux`, the tests are **cross-compiled and run on fmr** — there is no Go
|
||||||
|
toolchain there, so `go test -c` plus scp is the loop.
|
||||||
|
|
||||||
|
Live against fmr: 4 fragments per burst, and all three orderings reassembled — a healthy path, and
|
||||||
|
the baseline against which a mobile network will be interesting.
|
||||||
|
|
||||||
|
### Testing state (2026-08-01)
|
||||||
|
Six live tests against fmr, all green, no device involved: `LiveServerTest`, `LiveMeasurement`,
|
||||||
|
`LiveGranted`, `LiveDownstream`, `LiveUpload`, `LiveCompat`, `LiveEnrollment`. Plus 74 client unit
|
||||||
|
tests and the full Go suite. Everything in the last several entries is verified from the PC; the
|
||||||
|
app's UI (settings, history, deep-link enrollment) and `mtu.pmtud_up` remain device-only.
|
||||||
|
|
||||||
|
### throughput: a rate, plus the qualifier that makes it a measurement (server-v0.6.1 … v0.6.2)
|
||||||
|
A throughput test reports the *smallest* limit on the path — and the sender's own ceiling is one of
|
||||||
|
the candidates. If the server is asked for 50 Mbps and 50 Mbps arrives, the network was never the
|
||||||
|
constraint and "50 Mbps" says nothing about it. So `perf.throughput_udp` always carries
|
||||||
|
`limited_by` (duration | budget | rate | send_error) and `measures_network`, and a finding is
|
||||||
|
raised only when the path is actually implicated. The live run against fmr reports 20 Mbit/s with
|
||||||
|
`measures_network: false`, which is the correct and useful answer.
|
||||||
|
|
||||||
|
Loss is computed against the **sender's own count**, fetched from the observations API, not against
|
||||||
|
the requested rate. A receiver alone cannot tell "the network dropped it" from "the sender never
|
||||||
|
sent it", and guessing turns a healthy server-side limit into a phantom network fault. The server
|
||||||
|
keeps one summary per action rather than per-packet records — a ten-second run at 50 Mbps is half a
|
||||||
|
million packets, and a struct each would turn a measurement into memory exhaustion.
|
||||||
|
|
||||||
|
Sending is **paced**, on an absolute schedule. Unpaced would measure the server's NIC and the first
|
||||||
|
queue it meets, then collapse into loss that reads as a network fault; sleep-per-packet would
|
||||||
|
accumulate scheduler error and drift the rate down over a ten-second run.
|
||||||
|
|
||||||
|
Throughput gets its own grant budget sized from the request, so every *other* action stays bounded
|
||||||
|
at 8 MiB. When the byte cap binds before the clock does, the **duration is shortened and reported**
|
||||||
|
rather than the run being truncated: promising thirty seconds and delivering twenty-one is the same
|
||||||
|
information with a surprise attached, and it keeps "the clock ended the run" as the normal case —
|
||||||
|
the only case where the rate is a clean property of the path. That behaviour came out of a test
|
||||||
|
that failed honestly (30 s at 100 Mbps needs 375 MB against a 256 MB cap).
|
||||||
|
|
||||||
|
It is **opt-in** in the run config, default off. A 5-second run at 50 Mbps moves ~30 MB; on a
|
||||||
|
metered mobile connection that is the user's money, and a tool that spends it without being asked
|
||||||
|
is not one people keep installed.
|
||||||
|
|
||||||
|
#### The bug the live test found
|
||||||
|
The first live run delivered 104 packets and stopped after 50 ms. The grant's rate check exempted
|
||||||
|
the first 50 ms entirely, meaning to be lenient at startup — the effect was the opposite. A sender
|
||||||
|
could dump an unbounded burst into that free window, and the instant the check switched on it
|
||||||
|
compared those bytes against 50 ms worth of allowance and refused everything until real time caught
|
||||||
|
up. **Every short test passed** (downtrain sends 50 packets, big_send seven); every sustained send
|
||||||
|
died fifty milliseconds in.
|
||||||
|
|
||||||
|
Replaced with a token bucket (`allowance = burst + rate × elapsed`), which is smooth from t=0.
|
||||||
|
The burst is 100 ms of the allowed rate, floored at one ordinary datagram — deliberately one, since
|
||||||
|
at 8 kbps a 64 KB floor is sixty-four seconds' worth, exactly the instant dump the ceiling exists to
|
||||||
|
prevent. The pre-existing rate test caught that when I first tried the generous floor, and it was
|
||||||
|
right to. Second half of the same bug: callers treated *any* refusal as terminal, so `TryAllow` now
|
||||||
|
says why — a sender paces through a transient "too fast just now" and still stops dead on a spent
|
||||||
|
budget or an expired grant. Both halves are pinned by regression tests.
|
||||||
|
|
||||||
|
### Findings registry (2026-08-01)
|
||||||
|
Closes open item 1 of measurement-schema.md §9. A finding code is the stable, machine-readable half
|
||||||
|
of a result — what a dashboard groups by and what someone greps a year of archived runs for — and
|
||||||
|
that only holds if a code means exactly one thing forever. Ad-hoc string literals at fifteen call
|
||||||
|
sites cannot promise that, and by the time the registry was written the failure had already
|
||||||
|
happened.
|
||||||
|
|
||||||
|
**Two emitters had independently produced `connectivity.downstream_loss` and
|
||||||
|
`connectivity.loss_downstream` for the same claim**, and nothing anywhere objected. Anyone
|
||||||
|
aggregating either one would have silently seen half their data. Merged into
|
||||||
|
`connectivity.loss_downstream`, paired with `loss_upstream` so the two directions read as a set.
|
||||||
|
|
||||||
|
**Two codes were also renamed out of `nat.*`.** `nat.udp_unreachable` is not about NAT — it means
|
||||||
|
no replies came back — but the prefix determines the category, and the category determines which
|
||||||
|
verdict light the finding rolls up into (§7.3). A `nat.*` code landing under *connectivity* is not
|
||||||
|
a naming quibble; it changes which light turns red. Cheap to fix now, a breaking change later.
|
||||||
|
|
||||||
|
Codes are now declared as typed `FindingSpec`s carrying their category and default severity, and
|
||||||
|
emitters reference the spec instead of retyping the string — so a typo is a compile error and two
|
||||||
|
call sites cannot disagree about a finding's category.
|
||||||
|
|
||||||
|
`docs/findings-registry.md` is the contract, and a test reads it: it fails when the document and
|
||||||
|
the registry have codes the other lacks, or when a severity differs. Documentation that drifts from
|
||||||
|
its implementation is worse than none, because it still looks authoritative. The check scopes
|
||||||
|
itself to table rows, so the prose can keep explaining which codes were retired and why.
|
||||||
|
|
||||||
|
Six tests: uniqueness, declared-vs-listed, prefix↔category agreement, naming convention, a
|
||||||
|
word-order-anagram check (the shape the duplication actually took), and the document agreement.
|
||||||
|
|
||||||
|
### A real privacy leak, found by starting on the machine-readable schema (2026-08-01)
|
||||||
|
The intent was `measurement.schema.json` (§8's promised companion). The first step — checking
|
||||||
|
whether the anonymizer actually covers the fields the schema declares as sensitive — found that it
|
||||||
|
did not, so that became the work.
|
||||||
|
|
||||||
|
**At the `balanced` level, five identifying values were being uploaded verbatim:**
|
||||||
|
|
||||||
|
| value | field | why it matters |
|
||||||
|
|---|---|---|
|
||||||
|
| `2001:…::150` | `networks[].link.addresses[].addr` | the device's own global IPv6 address — a strong, geolocatable device identifier |
|
||||||
|
| `2a02:…::1` | `networks[].link.routes[].gateway` | identifies the ISP allocation |
|
||||||
|
| `203.0.113.77` | `networks[].link.dns.servers[]` | the configured resolver |
|
||||||
|
| `nas.example.lan` | `private_dns_hostname` | an internal hostname |
|
||||||
|
| `example.lan` | `search_domains[]` | the internal domain |
|
||||||
|
|
||||||
|
The settings screen describes that level as pseudonymizing addresses. It was not.
|
||||||
|
|
||||||
|
**Root cause:** classification keyed on field *names*, and the schema's actual names (`addr`,
|
||||||
|
`gateway`, `dst`, `servers`, `search_domains`, `private_dns_hostname`) had never been added to the
|
||||||
|
table. Not a subtle bug — just an unfalsifiable design. The existing tests all passed, because each
|
||||||
|
one checked a field somebody had remembered to write a case for.
|
||||||
|
|
||||||
|
**Two fixes, one of them structural:**
|
||||||
|
1. The missing names were added.
|
||||||
|
2. More importantly, a **shape-based backstop**: when a field name is unrecognised, the *value* is
|
||||||
|
inspected, and anything shaped like an IPv4/IPv6 address or a MAC is treated as one. A name
|
||||||
|
table can only protect fields someone thought of, which is precisely the wrong property for a
|
||||||
|
privacy control. Hostnames are deliberately *not* inferred by shape — `train.udp_updown` is
|
||||||
|
indistinguishable from a domain, and mangling a test type would corrupt the document to protect
|
||||||
|
nothing.
|
||||||
|
|
||||||
|
`LeakTest` is the new guard and is written to fail for fields nobody has considered: it plants
|
||||||
|
identifying values wherever one can actually occur and asserts none survive, rather than checking
|
||||||
|
a list of known cases. It also pins that RFC1918 addresses still come through readable, so the
|
||||||
|
test cannot pass by over-redacting everything.
|
||||||
|
|
||||||
|
Route prefixes and the unspecified address needed care in the transform: `0.0.0.0/0` and `::/0`
|
||||||
|
must stay themselves, or a routing table becomes unreadable for no privacy gain.
|
||||||
|
|
||||||
|
**Still outstanding:** `measurement.schema.json` itself. Worth noting what this episode implies for
|
||||||
|
it — much of a document's payload lives in `evidence`/`metrics`/`params`, which are per-test-type
|
||||||
|
`JsonObject` by design and therefore *outside* any schema. A schema-driven anonymizer would have
|
||||||
|
less coverage there than the name-plus-shape one now does, so the schema should be built for
|
||||||
|
validation and external tooling, not as a replacement for the classifier.
|
||||||
|
|
||||||
|
### ULA prefixes are pseudonymized whole (2026-08-01)
|
||||||
|
Spotted in a real uploaded run from the phone: the server had
|
||||||
|
`fda1:3fb1:ff92:6696::2662` for a DNS server. The general IPv6 path preserves the leading two
|
||||||
|
groups (deliberately — for a global address that keeps the ISP allocation, which is the
|
||||||
|
diagnostically useful part), and for a ULA that passed through **32 of the 40 random bits** of the
|
||||||
|
global ID.
|
||||||
|
|
||||||
|
ULA looks like the v6 equivalent of RFC1918 and the instinct is to treat it the same. That
|
||||||
|
reasoning does not carry over, and the difference is the whole point: an RFC1918 prefix is shared
|
||||||
|
by millions of networks and identifies none of them, while a ULA global ID is random and unique to
|
||||||
|
one network by construction (RFC 4193). The prefix *is* the identifier — it is a network
|
||||||
|
fingerprint that was surviving redaction.
|
||||||
|
|
||||||
|
Now pseudonymized as a unit, so two addresses on the same ULA subnet still land on the same
|
||||||
|
pseudonymous prefix: "these hosts are on one network" survives, "this is *that* network" does not.
|
||||||
|
Three tests, one of which uses the exact value observed on the wire.
|
||||||
|
|
||||||
|
Worth recording as a reasoning trap: I had originally raised this as "ULA should probably be kept
|
||||||
|
verbatim, like RFC1918, for consistency". The surface analogy pointed the wrong way, and the
|
||||||
|
correct answer was the opposite.
|
||||||
|
|
||||||
|
### Registry adopted everywhere; v6 findings renamed; Back works (2026-08-01)
|
||||||
|
The findings registry was only adopted in `core-engine`. The app module still emitted seven codes
|
||||||
|
as raw strings, so the registry test passed while codes existed outside it — including
|
||||||
|
`ipv6.broken`, which fired on a real network and was in no registry at all.
|
||||||
|
|
||||||
|
All seven now reference registry entries for code, category and severity, so those three cannot
|
||||||
|
disagree at a call site. A grep for `code = "…"` across the app, engine and probe modules returns
|
||||||
|
nothing.
|
||||||
|
|
||||||
|
**`ipv6.*` → `v6.*`.** The third instance of rule 1: they declared `Category.IPV6` while the prefix
|
||||||
|
map only knows `v6`, so `TestType.category("ipv6.broken")` fell through to *connectivity* and the
|
||||||
|
finding rolled up under the wrong verdict light. The test-type registry already used `v6.`.
|
||||||
|
|
||||||
|
Two severities reconciled while merging:
|
||||||
|
- `connectivity.captive_portal` is **medium**, not high. The registry had guessed high; the probe
|
||||||
|
that emits it had always said medium, and the probe was the considered value — a captive portal
|
||||||
|
on hotel wifi is what should be there, and logging in clears it. `connectivity.no_internet` is
|
||||||
|
the high one, because nothing the user does locally fixes that.
|
||||||
|
- `v6.not_offered` is **info, and the registry says it must stay info**. Most networks still do not
|
||||||
|
offer IPv6 and that is not a fault; a warning here lights a yellow verdict on a healthy network,
|
||||||
|
which teaches people to ignore the light.
|
||||||
|
|
||||||
|
Also: a `BackHandler` now returns from Settings/History to the run screen. The screen was a plain
|
||||||
|
state variable with nothing connecting it to the back stack, so the system Back gesture left the
|
||||||
|
app entirely. Enabled only when there is somewhere to go back to, so Back still exits from the run
|
||||||
|
screen.
|
||||||
|
|||||||
@@ -0,0 +1,106 @@
|
|||||||
|
<!--
|
||||||
|
SPDX-FileCopyrightText: 2026 Echolot contributors
|
||||||
|
SPDX-License-Identifier: CC-BY-4.0
|
||||||
|
-->
|
||||||
|
|
||||||
|
# Echolot findings registry
|
||||||
|
|
||||||
|
Closes open item 1 of `measurement-schema.md` §9.
|
||||||
|
|
||||||
|
A **finding code** is the stable, machine-readable half of a result. The prose around it changes
|
||||||
|
freely; the code is what a dashboard groups by, what a diff between two runs keys on, and what
|
||||||
|
someone greps a year of archived runs for. That only works if a code means exactly one thing,
|
||||||
|
forever.
|
||||||
|
|
||||||
|
This document is the contract. It is kept in step with
|
||||||
|
`echolot-app/core-measurement/.../FindingRegistry.kt` by a test that fails when either side has a
|
||||||
|
code the other does not — a registry that drifts from its documentation is worse than none,
|
||||||
|
because it looks authoritative.
|
||||||
|
|
||||||
|
## Rules
|
||||||
|
|
||||||
|
1. **The prefix determines the category**, and the category determines which verdict light the
|
||||||
|
finding rolls up into (§7.3). A `nat.*` code appearing under *connectivity* is not a naming
|
||||||
|
quibble; it changes which light turns red. Two codes were renamed from `nat.*` to
|
||||||
|
`connectivity.*` for exactly this reason.
|
||||||
|
2. **One code per concept.** Two emitters independently produced `connectivity.downstream_loss`
|
||||||
|
and `connectivity.loss_downstream` for the same claim before this registry existed. Anyone
|
||||||
|
aggregating either would have silently seen half their data.
|
||||||
|
3. **Codes are declared, not typed.** Emitters reference a `FindingSpec`, so a typo is a compile
|
||||||
|
error and no two call sites can disagree about a finding's category or default severity.
|
||||||
|
4. **Severity in the registry is the default.** An emitter may escalate for a specific run; it may
|
||||||
|
not quietly reclassify the finding in general.
|
||||||
|
5. **Say what is ruled out**, where that is the useful half. "Loss upstream" is worth far more
|
||||||
|
when it also states that the return path is clean, because that halves where to look next.
|
||||||
|
6. **Renaming a code is a breaking change** once runs are archived at scale. Before 1.0 it is
|
||||||
|
cheap; after, it needs an alias and a deprecation window.
|
||||||
|
|
||||||
|
## Registry
|
||||||
|
|
||||||
|
### connectivity
|
||||||
|
|
||||||
|
| code | severity | means | rules out |
|
||||||
|
|---|---|---|---|
|
||||||
|
| `connectivity.udp_unreachable` | high | No UDP echo replies came back from the server at all. | — |
|
||||||
|
| `connectivity.udp_unreachable_upstream` | high | The server received none of the probes, so traffic is dropped on the way out. | The return path: nothing arrived to be replied to. |
|
||||||
|
| `connectivity.udp_loss` | medium | A large fraction of round-trip probes were lost, direction unknown. | — |
|
||||||
|
| `connectivity.loss_upstream` | medium | Probes were lost on the way to the server. | The return path: replies came back for everything that arrived. |
|
||||||
|
| `connectivity.loss_downstream` | medium | Packets were lost on the way back from the server. | The outbound path: the server received what it was answering. |
|
||||||
|
| `connectivity.downstream_blocked` | high | Server-initiated packets never arrive, although round trips work. | Basic reachability: the path forwards replies, just not unsolicited traffic. |
|
||||||
|
| `connectivity.downstream_reorder` | low | Downstream packets arrive in a different order than they were sent. | — |
|
||||||
|
| `connectivity.captive_portal` | medium | A captive portal is intercepting connectivity checks. | — |
|
||||||
|
| `connectivity.no_internet` | high | Android's own connectivity checks fail on this network. | — |
|
||||||
|
|
||||||
|
### mtu
|
||||||
|
|
||||||
|
| code | severity | means | rules out |
|
||||||
|
|---|---|---|---|
|
||||||
|
| `mtu.reduced_downstream` | low | The downstream path MTU is below the usual 1500 bytes. | — |
|
||||||
|
| `mtu.downstream_blackhole` | medium | Datagrams above the path MTU are dropped downstream, fragmented or not. | — |
|
||||||
|
| `mtu.fragments_blocked` | medium | IP fragments do not reach this device even when sent in order. | — |
|
||||||
|
| `mtu.fragment_reorder_sensitive` | low | Fragments are delivered in order but dropped when reordered or delayed. | Fragmentation itself: in-order fragments arrive fine. |
|
||||||
|
|
||||||
|
### nat
|
||||||
|
|
||||||
|
| code | severity | means | rules out |
|
||||||
|
|---|---|---|---|
|
||||||
|
| `nat.udp_rebinding` | medium | A NAT remapped the UDP source port mid-flow. | — |
|
||||||
|
| `nat.symmetric` | medium | The NAT assigns a different external port per destination. | — |
|
||||||
|
|
||||||
|
### perf
|
||||||
|
|
||||||
|
| code | severity | means | rules out |
|
||||||
|
|---|---|---|---|
|
||||||
|
| `perf.throughput_no_delivery` | high | No throughput traffic arrived, although the server sent it. | — |
|
||||||
|
| `perf.throughput_below_offered` | low | Less throughput arrived than the server sent for the whole run. | — |
|
||||||
|
|
||||||
|
### dns
|
||||||
|
|
||||||
|
| code | severity | means | rules out |
|
||||||
|
|---|---|---|---|
|
||||||
|
| `dns.answer_rewritten` | high | A resolver returned an answer that differs from the authoritative record. | — |
|
||||||
|
| `dns.authoritative_unreachable` | medium | The canary zone's authoritative server could not be reached. | — |
|
||||||
|
|
||||||
|
### v6
|
||||||
|
|
||||||
|
The prefix is `v6.`, matching the test-type registry (`v6.brokenness`, `v6.happy_eyeballs`, …).
|
||||||
|
These were `ipv6.*` while declaring `Category.IPV6`; since the prefix map only knows `v6`, they
|
||||||
|
rolled up under *connectivity* instead — the third occurrence of rule 1 being broken.
|
||||||
|
|
||||||
|
| code | severity | means | rules out |
|
||||||
|
|---|---|---|---|
|
||||||
|
| `v6.broken` | medium | IPv6 is configured on this network but does not work. | Absence of IPv6: it is provisioned, it simply fails. |
|
||||||
|
| `v6.not_offered` | info | This network does not offer IPv6. | — |
|
||||||
|
|
||||||
|
`v6.not_offered` is **info and must stay info**. Most networks still do not offer IPv6 and that is
|
||||||
|
not a fault; reporting it as a warning lights a yellow verdict on a healthy network, which teaches
|
||||||
|
people to ignore the light — the one thing a diagnostic must never do.
|
||||||
|
|
||||||
|
## Adding a finding
|
||||||
|
|
||||||
|
1. Add a `FindingSpec` to `FindingRegistry`, and to its `all` list.
|
||||||
|
2. Add the row here, under the section its prefix names.
|
||||||
|
3. Emit it with `finding(FindingRegistry.YOUR_CODE, …)`.
|
||||||
|
|
||||||
|
The registry test checks 1 and 2 agree, that every prefix maps to the category it claims, and that
|
||||||
|
no two entries share a code.
|
||||||
@@ -269,7 +269,7 @@ The JSON Schema (machine-readable companion, `measurement.schema.json`, generate
|
|||||||
|
|
||||||
| type | example fields | v2 anonymizer transform |
|
| type | example fields | v2 anonymizer transform |
|
||||||
|---|---|---|
|
|---|---|---|
|
||||||
| `ip4`, `ip6` | addresses, routes, hops, DNS answers | prefix-preserving pseudonymization, consistent per document; well-known/reserved ranges kept verbatim |
|
| `ip4`, `ip6` | addresses, routes, hops, DNS answers | prefix-preserving pseudonymization, consistent per document; well-known/reserved ranges kept verbatim. **Exception: ULA (`fc00::/7`) has its whole prefix pseudonymized as a unit.** It resembles RFC1918 but is not analogous: a ULA global ID is 40 random bits, unique to one network by construction (RFC 4193), so the prefix *is* the identifier, whereas `192.168.0.0/16` is shared by millions of networks and identifies none. Pseudonymizing it as a unit keeps "these hosts are on one subnet" while dropping "this is that subnet". |
|
||||||
| `mac`, `bssid` | wifi, arp_watch | OUI kept, NIC part pseudonymized |
|
| `mac`, `bssid` | wifi, arp_watch | OUI kept, NIC part pseudonymized |
|
||||||
| `fqdn` | DNS names, reverse lookups | per-label pseudonyms, public-suffix kept |
|
| `fqdn` | DNS names, reverse lookups | per-label pseudonyms, public-suffix kept |
|
||||||
| `ssid` | wifi | pseudonym |
|
| `ssid` | wifi | pseudonym |
|
||||||
@@ -279,7 +279,8 @@ Free-text fields (`notes`, `error.detail`, dump excerpts from Shizuku parsers) c
|
|||||||
|
|
||||||
## 9. Open items
|
## 9. Open items
|
||||||
|
|
||||||
1. Findings registry document — start alongside the first implemented tests.
|
1. ~~Findings registry document~~ — done: `findings-registry.md`, kept in step with
|
||||||
|
`FindingRegistry.kt` by a test that fails when the two disagree.
|
||||||
2. Whether Shizuku raw-dump excerpts (dumpsys/ip output) are embedded in `evidence` verbatim (auditable, but large and hard to anonymize) or parsed-only with an optional "attach raw dumps" toggle. Proposal: toggle, default on for local archive, default off for export.
|
2. Whether Shizuku raw-dump excerpts (dumpsys/ip output) are embedded in `evidence` verbatim (auditable, but large and hard to anonymize) or parsed-only with an optional "attach raw dumps" toggle. Proposal: toggle, default on for local archive, default off for export.
|
||||||
3. Peer-mode documents: each device produces its own run; the coordinator embeds the peer's findings summary and cross-references by `run.id`. Full merge format deferred.
|
3. Peer-mode documents: each device produces its own run; the coordinator embeds the peer's findings summary and cross-references by `run.id`. Full merge format deferred.
|
||||||
4. Size guardrails: soft cap 20 MB uncompressed per run; trains beyond that downsample evidence (keep aggregates + first/last N + all anomalies) and record `"evidence_truncated": true`.
|
4. Size guardrails: soft cap 20 MB uncompressed per run; trains beyond that downsample evidence (keep aggregates + first/last N + all anomalies) and record `"evidence_truncated": true`.
|
||||||
|
|||||||
+91
-5
@@ -24,11 +24,34 @@ echolot://enroll?v=1&u=<control-URL, urlencoded>&p=pin-sha256:<b64 SPKI hash>&t=
|
|||||||
|
|
||||||
```
|
```
|
||||||
POST /v1/enroll Authorization: Bearer <enrollment-token>
|
POST /v1/enroll Authorization: Bearer <enrollment-token>
|
||||||
→ 200 { "device_credential": "<random 256-bit, b64url>",
|
→ 201 { "device_credential": "<random 256-bit, b64url>",
|
||||||
"device_id": "uuid",
|
"device_id": "uuid" }
|
||||||
"profile": { ... §2.2 ... } }
|
|
||||||
```
|
```
|
||||||
|
|
||||||
|
The **server assembles the bootstrap link**, because it is the only party holding all three parts
|
||||||
|
at once, and the part an operator gets wrong by hand is the base64 pin — which does not fail
|
||||||
|
loudly, it just never matches, and surfaces later as an inscrutable TLS error:
|
||||||
|
|
||||||
|
```
|
||||||
|
POST /admin/enroll-tokens
|
||||||
|
→ { "token": "…", "expires_in_s": 86400,
|
||||||
|
"enroll_uri": "echolot://enroll?v=1&u=…&p=…&t=…" }
|
||||||
|
```
|
||||||
|
|
||||||
|
The control URL in the link comes from `ECHOLOT_PUBLIC_URL`, falling back to the first control
|
||||||
|
listen address. A wildcard bind has no single right answer, so it warns rather than guessing.
|
||||||
|
|
||||||
|
Encoding notes that matter in practice:
|
||||||
|
- `u`, `p` and `t` are **percent-encoded**. The pin is base64, so it contains `+`, `/` and `=`,
|
||||||
|
every one of which means something else in a query string.
|
||||||
|
- A `+` that was *not* encoded decodes to a space. Base64 contains no spaces, so a parser SHOULD
|
||||||
|
restore them — the alternative is a pin wrong by one character and a failure that points nowhere
|
||||||
|
near the cause.
|
||||||
|
- The control URL MUST be `https://`. The pin only protects a TLS connection; a cleartext URL
|
||||||
|
would hand the token to anyone on the path.
|
||||||
|
- **The link is a secret** while it is live: it carries a bearer token, so anyone who sees it
|
||||||
|
before the device does can enroll instead.
|
||||||
|
|
||||||
Enrollment tokens are single-use with expiry, created in the admin UI, scoped `enroll`. The device credential is a long-lived bearer secret, scoped `run-tests`; it is also the HKDF input for session keys. Revocation = deleting the device in the admin UI.
|
Enrollment tokens are single-use with expiry, created in the admin UI, scoped `enroll`. The device credential is a long-lived bearer secret, scoped `run-tests`; it is also the HKDF input for session keys. Revocation = deleting the device in the admin UI.
|
||||||
|
|
||||||
### 2.2 Profile
|
### 2.2 Profile
|
||||||
@@ -192,14 +215,77 @@ Note: exact RDATA constants to be frozen in the implementation's `dns_reference.
|
|||||||
|
|
||||||
Same daemon, separate listener (default localhost-only): health + self-test (are both IPs live, is the canary zone delegated correctly, is UDP reachable from outside — tested via a public echolot "mirror" if configured), enrollment token management (create/expire/scope), device list + revocation, retention settings, QR rendering (client-side JS). Out of scope for this spec beyond the endpoints above.
|
Same daemon, separate listener (default localhost-only): health + self-test (are both IPs live, is the canary zone delegated correctly, is UDP reachable from outside — tested via a public echolot "mirror" if configured), enrollment token management (create/expire/scope), device list + revocation, retention settings, QR rendering (client-side JS). Out of scope for this spec beyond the endpoints above.
|
||||||
|
|
||||||
## 8. Cross-references to the measurement schema
|
## 8. Version compatibility
|
||||||
|
|
||||||
|
Both artifacts are versioned with **SemVer**: the Go server (`server-vX.Y.Z` tags) and the Android
|
||||||
|
app (`versionName`; `versionCode` is derived from it, never maintained separately). Two independent
|
||||||
|
things are checked, and conflating them is the mistake this section exists to prevent.
|
||||||
|
|
||||||
|
### 8.1 Protocol version — *can* these builds talk?
|
||||||
|
|
||||||
|
`protocol_version` is the version of **this document**. It is advertised in the profile
|
||||||
|
(`compat.protocol_version`) and is the correctness axis: a peer in a different breaking series
|
||||||
|
cannot be talked to, whatever its release version says. Below `1.0.0` the **minor** is the breaking
|
||||||
|
axis (SemVer §4); at and above it, the major is. A patch bump of the protocol never splits a fleet.
|
||||||
|
|
||||||
|
### 8.2 Release-version window — *should* they, per policy?
|
||||||
|
|
||||||
|
Each side declares the range of peer release versions it will work with, as `[min, max)` —
|
||||||
|
**minimum inclusive, maximum exclusive**, because the useful bound is always "the version that
|
||||||
|
broke it" and writing that literally is unambiguous. An empty maximum means unbounded.
|
||||||
|
|
||||||
|
The server advertises its window and enforces it:
|
||||||
|
|
||||||
|
```jsonc
|
||||||
|
"compat": {
|
||||||
|
"protocol_version": "1.0.0",
|
||||||
|
"schema_version": "1.0.0",
|
||||||
|
"app_min": "0.2.0",
|
||||||
|
"app_max": "1.0.0" // exclusive; "" = no upper bound
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
Operators override it with `ECHOLOT_MIN_APP_VERSION` / `ECHOLOT_MAX_APP_VERSION` (or
|
||||||
|
`--min-app-version` / `--max-app-version`). A malformed bound is **fatal at startup**, not ignored:
|
||||||
|
a typo must not silently disable a restriction the operator meant to set.
|
||||||
|
|
||||||
|
The app sends its version on every control-plane request:
|
||||||
|
|
||||||
|
```
|
||||||
|
X-Echolot-App-Version: 0.2.0
|
||||||
|
```
|
||||||
|
|
||||||
|
and carries its own bounds for the server (`MIN_SERVER` / `MAX_SERVER` in `Compat.kt`). It checks
|
||||||
|
the profile in **both** directions — is the server in our range, and are we in the server's — so a
|
||||||
|
mismatch is reported before a run starts rather than discovered halfway through one.
|
||||||
|
|
||||||
|
### 8.3 Rules
|
||||||
|
|
||||||
|
1. **`GET /v1/profile` is never gated.** It is where a refused client learns which version it needs.
|
||||||
|
Gating it leaves the user with a network error instead of an answer, defeating the check.
|
||||||
|
2. **Refusal is `426 Upgrade Required`**, with a body naming both versions and the accepted window:
|
||||||
|
```json
|
||||||
|
{ "error": "app 0.1.0 is older than this build supports (needs >= 0.2.0, < 1.0.0). Update the app.",
|
||||||
|
"app_version": "0.1.0", "accepts_app": ">= 0.2.0, < 1.0.0",
|
||||||
|
"server_version": "0.5.0", "protocol_version": "1.0.0" }
|
||||||
|
```
|
||||||
|
3. **An unparseable or absent version is `unknown`, and is allowed.** Development builds report
|
||||||
|
`dev`, and a client too old to send the header cannot be identified anyway. The check exists to
|
||||||
|
turn confusing failures into clear ones; refusing what it cannot identify does the opposite.
|
||||||
|
4. **Bounds move at breaking boundaries, not at releases.** Shipping a patch must never require
|
||||||
|
editing a range. A minimum is raised only when older peers are actually harmful — e.g. the app
|
||||||
|
requires server `>= 0.4.2` because earlier multi-homed servers sent granted traffic from an
|
||||||
|
address the session never used, which the client measured as 100 % downstream loss. A
|
||||||
|
confidently wrong measurement is worse than a refused one.
|
||||||
|
|
||||||
|
## 9. Cross-references to the measurement schema
|
||||||
|
|
||||||
- Observation-block fields (§3.3) appear as `*_seen_by_server` columns in `train` evidence (schema §6.2).
|
- Observation-block fields (§3.3) appear as `*_seen_by_server` columns in `train` evidence (schema §6.2).
|
||||||
- `TIMESYNC` (§3.2) produces the `time.server_offset` test; without it, cross-clock fields must not be compared (schema §6.2 note).
|
- `TIMESYNC` (§3.2) produces the `time.server_offset` test; without it, cross-clock fields must not be compared (schema §6.2 note).
|
||||||
- Capability strings (§2.3) are copied verbatim into `server_sessions[].capabilities` (schema §5); tests skipped for missing capability get `status: "unsupported"`.
|
- Capability strings (§2.3) are copied verbatim into `server_sessions[].capabilities` (schema §5); tests skipped for missing capability get `status: "unsupported"`.
|
||||||
- `session_id` maps to `server_sessions[].session_id`; `action_id`s appear in test `params`.
|
- `session_id` maps to `server_sessions[].session_id`; `action_id`s appear in test `params`.
|
||||||
|
|
||||||
## 9. Open items
|
## 10. Open items
|
||||||
|
|
||||||
1. Whether TRAIN_REPORT should also stream during long trains (partial reports every N packets) for live UI feedback — leaning yes, same type with a `flags` bit.
|
1. Whether TRAIN_REPORT should also stream during long trains (partial reports every N packets) for live UI feedback — leaning yes, same type with a `flags` bit.
|
||||||
2. Throughput methodology (fixed streams vs BBR-style ramp) — decide with `perf.*` test design.
|
2. Throughput methodology (fixed streams vs BBR-style ramp) — decide with `perf.*` test design.
|
||||||
|
|||||||
@@ -8,6 +8,20 @@ plugins {
|
|||||||
alias(libs.plugins.kotlin.serialization)
|
alias(libs.plugins.kotlin.serialization)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// The app's version is SemVer and lives here, once. versionCode is derived from it rather than
|
||||||
|
// maintained alongside: Play/F-Droid need a monotonically increasing integer, but a second number
|
||||||
|
// that a human has to remember to bump is a number that eventually disagrees with the first — and
|
||||||
|
// the version is now load-bearing, since the server decides whether to serve us by it.
|
||||||
|
//
|
||||||
|
// major*1_000_000 + minor*10_000 + patch*10 leaves room for 9 patch-level rebuilds (the trailing
|
||||||
|
// digit) without disturbing the mapping, and stays inside the 2_100_000_000 ceiling until major 2100.
|
||||||
|
val appVersionName = "0.2.0"
|
||||||
|
|
||||||
|
fun versionCodeOf(semver: String): Int {
|
||||||
|
val (major, minor, patch) = semver.substringBefore('-').split(".").map(String::toInt)
|
||||||
|
return major * 1_000_000 + minor * 10_000 + patch * 10
|
||||||
|
}
|
||||||
|
|
||||||
android {
|
android {
|
||||||
namespace = "app.echo_lot.app"
|
namespace = "app.echo_lot.app"
|
||||||
compileSdk = 36
|
compileSdk = 36
|
||||||
@@ -16,12 +30,15 @@ android {
|
|||||||
applicationId = "app.echo_lot.app"
|
applicationId = "app.echo_lot.app"
|
||||||
minSdk = 26
|
minSdk = 26
|
||||||
targetSdk = 36
|
targetSdk = 36
|
||||||
versionCode = 2
|
versionCode = versionCodeOf(appVersionName)
|
||||||
versionName = "0.2.0"
|
versionName = appVersionName
|
||||||
// Automation: `adb shell am start -n app.echo_lot.app/.MainActivity --ez autorun true`
|
// Automation: `adb shell am start -n app.echo_lot.app/.MainActivity --ez autorun true`
|
||||||
// runs a measurement immediately and POSTs the report here (dev collection endpoint).
|
// runs a measurement immediately and POSTs the report here (dev collection endpoint).
|
||||||
buildConfigField("String", "REPORT_UPLOAD_URL", "\"http://89.185.109.150:443/report\"")
|
buildConfigField("String", "REPORT_UPLOAD_URL", "\"http://89.185.109.150:443/report\"")
|
||||||
buildConfigField("String", "REPORT_UPLOAD_SECRET", "\"D4OmG5gGJsElqVVbtYIZbR\"")
|
buildConfigField("String", "REPORT_UPLOAD_SECRET", "\"D4OmG5gGJsElqVVbtYIZbR\"")
|
||||||
|
// The bare SemVer, without the debug build's "-dev" suffix stripped away by the server's
|
||||||
|
// parser anyway — sent to servers so they can apply their compatibility window.
|
||||||
|
buildConfigField("String", "APP_SEMVER", "\"$appVersionName\"")
|
||||||
}
|
}
|
||||||
buildTypes {
|
buildTypes {
|
||||||
release { isMinifyEnabled = false }
|
release { isMinifyEnabled = false }
|
||||||
|
|||||||
@@ -27,6 +27,18 @@
|
|||||||
<action android:name="android.intent.action.MAIN" />
|
<action android:name="android.intent.action.MAIN" />
|
||||||
<category android:name="android.intent.category.LAUNCHER" />
|
<category android:name="android.intent.category.LAUNCHER" />
|
||||||
</intent-filter>
|
</intent-filter>
|
||||||
|
<!--
|
||||||
|
Enrollment bootstrap (probe-protocol.md §2.1): echolot://enroll?v=1&u=…&p=…&t=…
|
||||||
|
Scanning a QR or tapping a link the operator sent configures the server in one
|
||||||
|
action, instead of transcribing a URL, a base64 pin and a token by hand — the pin
|
||||||
|
in particular fails silently when it is wrong by one character.
|
||||||
|
-->
|
||||||
|
<intent-filter android:autoVerify="false">
|
||||||
|
<action android:name="android.intent.action.VIEW" />
|
||||||
|
<category android:name="android.intent.category.DEFAULT" />
|
||||||
|
<category android:name="android.intent.category.BROWSABLE" />
|
||||||
|
<data android:scheme="echolot" android:host="enroll" />
|
||||||
|
</intent-filter>
|
||||||
</activity>
|
</activity>
|
||||||
|
|
||||||
<provider
|
<provider
|
||||||
|
|||||||
@@ -4,6 +4,7 @@
|
|||||||
package app.echo_lot.app
|
package app.echo_lot.app
|
||||||
|
|
||||||
import androidx.compose.foundation.layout.Arrangement
|
import androidx.compose.foundation.layout.Arrangement
|
||||||
|
import androidx.compose.foundation.layout.safeDrawingPadding
|
||||||
import androidx.compose.foundation.layout.Column
|
import androidx.compose.foundation.layout.Column
|
||||||
import androidx.compose.foundation.layout.Row
|
import androidx.compose.foundation.layout.Row
|
||||||
import androidx.compose.foundation.layout.fillMaxWidth
|
import androidx.compose.foundation.layout.fillMaxWidth
|
||||||
@@ -39,7 +40,7 @@ fun HistoryScreen(
|
|||||||
onDelete: (String) -> Unit,
|
onDelete: (String) -> Unit,
|
||||||
onBack: () -> Unit,
|
onBack: () -> Unit,
|
||||||
) {
|
) {
|
||||||
Column(Modifier.fillMaxWidth().padding(16.dp), verticalArrangement = Arrangement.spacedBy(10.dp)) {
|
Column(Modifier.fillMaxWidth().safeDrawingPadding().padding(16.dp), verticalArrangement = Arrangement.spacedBy(10.dp)) {
|
||||||
Row(verticalAlignment = Alignment.CenterVertically) {
|
Row(verticalAlignment = Alignment.CenterVertically) {
|
||||||
TextButton(onClick = onBack) { Text("‹ Back") }
|
TextButton(onClick = onBack) { Text("‹ Back") }
|
||||||
Text("History", style = MaterialTheme.typography.titleLarge)
|
Text("History", style = MaterialTheme.typography.titleLarge)
|
||||||
@@ -71,12 +72,20 @@ fun HistoryScreen(
|
|||||||
)
|
)
|
||||||
}
|
}
|
||||||
Text(
|
Text(
|
||||||
"${r.findingCount} finding(s) · ${r.sizeBytes / 1024} kB · ${r.anonymization}",
|
"${r.findingCount} finding(s) · ${r.sizeBytes / 1024} kB · " +
|
||||||
|
"kept complete on this device",
|
||||||
style = MaterialTheme.typography.bodySmall,
|
style = MaterialTheme.typography.bodySmall,
|
||||||
)
|
)
|
||||||
|
// The upload line names the level the upload was made at, not the
|
||||||
|
// archive's. They describe different documents, and showing the archive's
|
||||||
|
// level here claimed more had left the device than actually did.
|
||||||
Text(
|
Text(
|
||||||
if (r.uploaded) "uploaded to ${r.uploadedTo ?: "a server"}"
|
if (r.uploaded) {
|
||||||
else "on this device only",
|
"uploaded to ${r.uploadedTo ?: "a server"}" +
|
||||||
|
(r.uploadedAs?.let { " as $it" } ?: "")
|
||||||
|
} else {
|
||||||
|
"on this device only"
|
||||||
|
},
|
||||||
style = MaterialTheme.typography.bodySmall,
|
style = MaterialTheme.typography.bodySmall,
|
||||||
color = if (r.uploaded) Color(0xFF7FD17F) else Color(0xFFBBBBBB),
|
color = if (r.uploaded) Color(0xFF7FD17F) else Color(0xFFBBBBBB),
|
||||||
)
|
)
|
||||||
|
|||||||
@@ -59,6 +59,17 @@ class MainActivity : ComponentActivity() {
|
|||||||
// starts a run immediately and uploads the report, so an unattended
|
// starts a run immediately and uploads the report, so an unattended
|
||||||
// measurement needs no UI tapping and no adb round-trip to collect.
|
// measurement needs no UI tapping and no adb round-trip to collect.
|
||||||
val autorun = intent?.getBooleanExtra("autorun", false) == true
|
val autorun = intent?.getBooleanExtra("autorun", false) == true
|
||||||
|
|
||||||
|
// An echolot://enroll link (QR scan, or a link the operator sent) opens the
|
||||||
|
// app straight into settings with the enrollment already done, so the user
|
||||||
|
// sees the result rather than a form they still have to fill in.
|
||||||
|
val enrollUri = intent?.takeIf { it.action == Intent.ACTION_VIEW }?.dataString
|
||||||
|
androidx.compose.runtime.LaunchedEffect(enrollUri) {
|
||||||
|
if (enrollUri != null) {
|
||||||
|
vm.enroll(enrollUri)
|
||||||
|
screen = Screen.SETTINGS
|
||||||
|
}
|
||||||
|
}
|
||||||
androidx.compose.runtime.LaunchedEffect(autorun) {
|
androidx.compose.runtime.LaunchedEffect(autorun) {
|
||||||
if (autorun) vm.run(devUpload = true)
|
if (autorun) vm.run(devUpload = true)
|
||||||
}
|
}
|
||||||
@@ -74,10 +85,18 @@ class MainActivity : ComponentActivity() {
|
|||||||
finish()
|
finish()
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
// Without this, the system Back gesture leaves the activity from Settings or
|
||||||
|
// History instead of returning to the run screen — the screen is a plain state
|
||||||
|
// variable, so nothing connects it to the back stack. Registered only when
|
||||||
|
// there is somewhere to go back to, so Back still exits from the run screen.
|
||||||
|
androidx.activity.compose.BackHandler(enabled = screen != Screen.RUN) {
|
||||||
|
screen = Screen.RUN
|
||||||
|
}
|
||||||
|
|
||||||
when (screen) {
|
when (screen) {
|
||||||
Screen.SETTINGS -> SettingsScreen(
|
Screen.SETTINGS -> SettingsScreen(
|
||||||
settings = vm.settings,
|
settings = vm.settings,
|
||||||
archivedRuns = vm.state.history.size,
|
archivedRuns = vm.archivedRunCount(),
|
||||||
archivedBytes = vm.archivedBytes(),
|
archivedBytes = vm.archivedBytes(),
|
||||||
onApplyRetention = vm::applyRetention,
|
onApplyRetention = vm::applyRetention,
|
||||||
onDeleteAll = vm::deleteAllRuns,
|
onDeleteAll = vm::deleteAllRuns,
|
||||||
@@ -88,6 +107,9 @@ class MainActivity : ComponentActivity() {
|
|||||||
lifecycleScope.launch { preview = vm.uploadPreview(r.id) }
|
lifecycleScope.launch { preview = vm.uploadPreview(r.id) }
|
||||||
}
|
}
|
||||||
},
|
},
|
||||||
|
onCheckServer = vm::checkServer,
|
||||||
|
onEnroll = vm::enroll,
|
||||||
|
serverStatus = vm.state.archiveStatus,
|
||||||
onBack = { screen = Screen.RUN },
|
onBack = { screen = Screen.RUN },
|
||||||
)
|
)
|
||||||
Screen.HISTORY -> HistoryScreen(
|
Screen.HISTORY -> HistoryScreen(
|
||||||
|
|||||||
@@ -10,8 +10,10 @@ import app.echo_lot.measurement.MeasurementDocument
|
|||||||
import app.echo_lot.privacy.Anonymizer
|
import app.echo_lot.privacy.Anonymizer
|
||||||
import app.echo_lot.privacy.PrivacyLevel
|
import app.echo_lot.privacy.PrivacyLevel
|
||||||
import app.echo_lot.privacy.Salt
|
import app.echo_lot.privacy.Salt
|
||||||
|
import app.echo_lot.protocol.Compat
|
||||||
import app.echo_lot.protocol.ControlClient
|
import app.echo_lot.protocol.ControlClient
|
||||||
import app.echo_lot.protocol.UploadRefused
|
import app.echo_lot.protocol.UploadRefused
|
||||||
|
import app.echo_lot.protocol.VersionRefused
|
||||||
import kotlinx.serialization.json.Json
|
import kotlinx.serialization.json.Json
|
||||||
import kotlinx.serialization.json.JsonObject
|
import kotlinx.serialization.json.JsonObject
|
||||||
import kotlinx.serialization.json.jsonObject
|
import kotlinx.serialization.json.jsonObject
|
||||||
@@ -66,10 +68,75 @@ class RunStore(context: Context, private val settings: Settings) {
|
|||||||
data class Sent(val serverName: String, val detail: String) : UploadOutcome
|
data class Sent(val serverName: String, val detail: String) : UploadOutcome
|
||||||
/** The operator's policy says no. Not retryable, and not the user's fault. */
|
/** The operator's policy says no. Not retryable, and not the user's fault. */
|
||||||
data class Refused(val reason: String) : UploadOutcome
|
data class Refused(val reason: String) : UploadOutcome
|
||||||
|
/**
|
||||||
|
* The two builds do not go together. Kept apart from [Refused] and [Failed] because the
|
||||||
|
* remedy is different and specific — install a particular version — and a message that
|
||||||
|
* says so is worth more than one that says "upload failed".
|
||||||
|
*/
|
||||||
|
data class Incompatible(val reason: String) : UploadOutcome
|
||||||
data class Failed(val detail: String) : UploadOutcome
|
data class Failed(val detail: String) : UploadOutcome
|
||||||
data object NotConfigured : UploadOutcome
|
data object NotConfigured : UploadOutcome
|
||||||
}
|
}
|
||||||
|
|
||||||
|
private fun client() = ControlClient(
|
||||||
|
settings.serverUrl, setOf(settings.serverPin), BuildConfig.APP_SEMVER,
|
||||||
|
)
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Checks the configured server without uploading anything: reachable, pinned, compatible, and
|
||||||
|
* willing to accept runs. Lets the user find out in settings rather than from a failed run.
|
||||||
|
*/
|
||||||
|
fun checkServer(): String {
|
||||||
|
if (!settings.serverConfigured) return "Fill in the server URL, pin and credential first."
|
||||||
|
return try {
|
||||||
|
val profile = client().profile(settings.serverCredential)
|
||||||
|
val compat = Compat.check(profile, BuildConfig.APP_SEMVER)
|
||||||
|
val head = "${profile.name} · server ${profile.serverVersion} · " +
|
||||||
|
"protocol ${profile.compat.protocolVersion.ifBlank { "unstated" }}"
|
||||||
|
when {
|
||||||
|
compat.message != null -> head + "\n" + compat.message
|
||||||
|
else -> {
|
||||||
|
val uploads = profile.uploads.refusalReason()
|
||||||
|
?: "uploads accepted (min anonymization: ${profile.uploads.minAnonymization})"
|
||||||
|
head + "\nCompatible. " + uploads
|
||||||
|
}
|
||||||
|
}
|
||||||
|
} catch (e: VersionRefused) {
|
||||||
|
"This server will not serve this app: ${e.message}"
|
||||||
|
} catch (t: Throwable) {
|
||||||
|
"Could not reach the server: ${t.message ?: t.javaClass.simpleName}"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Redeems an enrollment link and stores the resulting server configuration (§2.1).
|
||||||
|
*
|
||||||
|
* Everything is written at once or not at all: a half-applied server — say a URL and pin with
|
||||||
|
* no credential — fails later, somewhere else, with an error that points at the wrong thing.
|
||||||
|
* Blocking; callers run it off the main thread.
|
||||||
|
*/
|
||||||
|
fun enroll(link: String, deviceName: String?): String {
|
||||||
|
val parsed = app.echo_lot.protocol.EnrollmentLink.parse(link)
|
||||||
|
?: return "That does not look like an Echolot enrollment link. It should start with " +
|
||||||
|
"echolot://enroll and carry a URL, a pin and a token."
|
||||||
|
return try {
|
||||||
|
val enrolled = parsed.redeem(deviceName, BuildConfig.APP_SEMVER)
|
||||||
|
val compat = Compat.check(enrolled.profile, BuildConfig.APP_SEMVER)
|
||||||
|
settings.serverUrl = enrolled.controlUrl
|
||||||
|
settings.serverPin = enrolled.pin
|
||||||
|
settings.serverCredential = enrolled.credential
|
||||||
|
val head = "Enrolled with ${enrolled.profile.name} " +
|
||||||
|
"(server ${enrolled.profile.serverVersion})."
|
||||||
|
if (compat.message != null) head + " " + compat.message else head
|
||||||
|
} catch (e: VersionRefused) {
|
||||||
|
"That server will not serve this app: ${e.message}"
|
||||||
|
} catch (t: Throwable) {
|
||||||
|
// The commonest causes are a spent token and a wrong pin, and they look nothing alike
|
||||||
|
// in the message — so pass it through rather than flattening it to "enrollment failed".
|
||||||
|
"Enrollment failed: ${t.message ?: t.javaClass.simpleName}"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* Uploads one archived run to the configured server, redacting first.
|
* Uploads one archived run to the configured server, redacting first.
|
||||||
*
|
*
|
||||||
@@ -81,8 +148,14 @@ class RunStore(context: Context, private val settings: Settings) {
|
|||||||
if (!settings.serverConfigured) return UploadOutcome.NotConfigured
|
if (!settings.serverConfigured) return UploadOutcome.NotConfigured
|
||||||
val docJson = read(runId) ?: return UploadOutcome.Failed("run $runId is not in the archive")
|
val docJson = read(runId) ?: return UploadOutcome.Failed("run $runId is not in the archive")
|
||||||
return try {
|
return try {
|
||||||
val client = ControlClient(settings.serverUrl, setOf(settings.serverPin))
|
val client = client()
|
||||||
val profile = client.profile(settings.serverCredential)
|
val profile = client.profile(settings.serverCredential)
|
||||||
|
|
||||||
|
// Compatibility before policy: an incompatible server may well advertise an upload
|
||||||
|
// policy it would never actually apply to us.
|
||||||
|
val compat = Compat.check(profile, BuildConfig.APP_SEMVER)
|
||||||
|
if (!compat.usable) return UploadOutcome.Incompatible(compat.message ?: "incompatible versions")
|
||||||
|
|
||||||
profile.uploads.refusalReason()?.let { return UploadOutcome.Refused(it) }
|
profile.uploads.refusalReason()?.let { return UploadOutcome.Refused(it) }
|
||||||
|
|
||||||
val level = PrivacyLevel.max(
|
val level = PrivacyLevel.max(
|
||||||
@@ -91,8 +164,12 @@ class RunStore(context: Context, private val settings: Settings) {
|
|||||||
)
|
)
|
||||||
val body = redactedForUpload(docJson, level)
|
val body = redactedForUpload(docJson, level)
|
||||||
val reply = client.uploadRun(settings.serverCredential, body)
|
val reply = client.uploadRun(settings.serverCredential, body)
|
||||||
archive.markUploaded(runId, profile.name)
|
archive.markUploaded(runId, profile.name, level.wire)
|
||||||
UploadOutcome.Sent(profile.name, "as $level, ${body.toByteArray().size} bytes: ${reply.take(120)}")
|
// Deliberately not echoing `reply`: it is the server's index entry as raw JSON, and
|
||||||
|
// it ended up rendered verbatim in the UI. Size and level are what a person wants.
|
||||||
|
UploadOutcome.Sent(profile.name, "as $level, ${body.toByteArray().size} bytes")
|
||||||
|
} catch (e: VersionRefused) {
|
||||||
|
UploadOutcome.Incompatible(e.message ?: "the server refused this app's version")
|
||||||
} catch (e: UploadRefused) {
|
} catch (e: UploadRefused) {
|
||||||
UploadOutcome.Refused(e.message ?: "refused by the server")
|
UploadOutcome.Refused(e.message ?: "refused by the server")
|
||||||
} catch (t: Throwable) {
|
} catch (t: Throwable) {
|
||||||
|
|||||||
@@ -144,6 +144,7 @@ class RunViewModel(app: Application) : AndroidViewModel(app) {
|
|||||||
private fun describe(o: RunStore.UploadOutcome): String = when (o) {
|
private fun describe(o: RunStore.UploadOutcome): String = when (o) {
|
||||||
is RunStore.UploadOutcome.Sent -> "uploaded to ${o.serverName} (${o.detail})"
|
is RunStore.UploadOutcome.Sent -> "uploaded to ${o.serverName} (${o.detail})"
|
||||||
is RunStore.UploadOutcome.Refused -> "server refused the upload: ${o.reason}"
|
is RunStore.UploadOutcome.Refused -> "server refused the upload: ${o.reason}"
|
||||||
|
is RunStore.UploadOutcome.Incompatible -> "version mismatch: ${o.reason}"
|
||||||
is RunStore.UploadOutcome.Failed -> "upload failed: ${o.detail}"
|
is RunStore.UploadOutcome.Failed -> "upload failed: ${o.detail}"
|
||||||
RunStore.UploadOutcome.NotConfigured -> "no server configured, so nothing was uploaded"
|
RunStore.UploadOutcome.NotConfigured -> "no server configured, so nothing was uploaded"
|
||||||
}
|
}
|
||||||
@@ -195,6 +196,26 @@ class RunViewModel(app: Application) : AndroidViewModel(app) {
|
|||||||
|
|
||||||
fun archivedBytes(): Long = store.totalBytes()
|
fun archivedBytes(): Long = store.totalBytes()
|
||||||
|
|
||||||
|
/** Counted from the archive itself, not from [UiState.history], which is empty until the
|
||||||
|
* history screen has been opened - the two disagreeing read as data loss. */
|
||||||
|
fun archivedRunCount(): Int = store.list().size
|
||||||
|
|
||||||
|
/** Redeems an enrollment link, from a paste or from an echolot:// deep link. */
|
||||||
|
fun enroll(link: String, deviceName: String? = android.os.Build.MODEL) {
|
||||||
|
viewModelScope.launch {
|
||||||
|
state = state.copy(archiveStatus = "enrolling …")
|
||||||
|
state = state.copy(archiveStatus = withContext(Dispatchers.IO) { store.enroll(link, deviceName) })
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Settings-screen action: report what the configured server is and whether we can use it. */
|
||||||
|
fun checkServer() {
|
||||||
|
viewModelScope.launch {
|
||||||
|
state = state.copy(archiveStatus = "checking server …")
|
||||||
|
state = state.copy(archiveStatus = withContext(Dispatchers.IO) { store.checkServer() })
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
/** Re-applies retention after the user changes the limits. */
|
/** Re-applies retention after the user changes the limits. */
|
||||||
fun applyRetention() {
|
fun applyRetention() {
|
||||||
viewModelScope.launch {
|
viewModelScope.launch {
|
||||||
@@ -335,8 +356,9 @@ class RunViewModel(app: Application) : AndroidViewModel(app) {
|
|||||||
when {
|
when {
|
||||||
ev.contains("\"captive_portal\"") -> out.add(
|
ev.contains("\"captive_portal\"") -> out.add(
|
||||||
Finding(
|
Finding(
|
||||||
id = ids.uuid(), code = "connectivity.captive_portal", category = Category.CONNECTIVITY,
|
id = ids.uuid(), code = FindingRegistry.CAPTIVE_PORTAL.code,
|
||||||
severity = Severity.MEDIUM, confidence = Confidence.HIGH,
|
category = FindingRegistry.CAPTIVE_PORTAL.category,
|
||||||
|
severity = FindingRegistry.CAPTIVE_PORTAL.severity, confidence = Confidence.HIGH,
|
||||||
title = "Captive portal intercepting connections",
|
title = "Captive portal intercepting connections",
|
||||||
description = "The generate_204 check returned a redirect or a page instead of HTTP 204 — a captive portal (login/splash page) is intercepting traffic on this network.",
|
description = "The generate_204 check returned a redirect or a page instead of HTTP 204 — a captive portal (login/splash page) is intercepting traffic on this network.",
|
||||||
evidenceRefs = listOf(EvidenceRef(t.id)),
|
evidenceRefs = listOf(EvidenceRef(t.id)),
|
||||||
@@ -344,8 +366,9 @@ class RunViewModel(app: Application) : AndroidViewModel(app) {
|
|||||||
)
|
)
|
||||||
t.status == TestStatus.FAILED -> out.add(
|
t.status == TestStatus.FAILED -> out.add(
|
||||||
Finding(
|
Finding(
|
||||||
id = ids.uuid(), code = "connectivity.no_internet", category = Category.CONNECTIVITY,
|
id = ids.uuid(), code = FindingRegistry.NO_INTERNET.code,
|
||||||
severity = Severity.HIGH, confidence = Confidence.HIGH,
|
category = FindingRegistry.NO_INTERNET.category,
|
||||||
|
severity = FindingRegistry.NO_INTERNET.severity, confidence = Confidence.HIGH,
|
||||||
title = "No working internet on any network",
|
title = "No working internet on any network",
|
||||||
description = "Android's own generate_204 connectivity checks failed on every active network (no HTTP 204) — this device has no validated internet path.",
|
description = "Android's own generate_204 connectivity checks failed on every active network (no HTTP 204) — this device has no validated internet path.",
|
||||||
evidenceRefs = listOf(EvidenceRef(t.id)),
|
evidenceRefs = listOf(EvidenceRef(t.id)),
|
||||||
@@ -358,8 +381,9 @@ class RunViewModel(app: Application) : AndroidViewModel(app) {
|
|||||||
if (ev.contains("MISMATCH")) {
|
if (ev.contains("MISMATCH")) {
|
||||||
out.add(
|
out.add(
|
||||||
Finding(
|
Finding(
|
||||||
id = ids.uuid(), code = "dns.answer_rewritten", category = Category.DNS,
|
id = ids.uuid(), code = FindingRegistry.DNS_ANSWER_REWRITTEN.code,
|
||||||
severity = Severity.HIGH, confidence = Confidence.HIGH,
|
category = FindingRegistry.DNS_ANSWER_REWRITTEN.category,
|
||||||
|
severity = FindingRegistry.DNS_ANSWER_REWRITTEN.severity, confidence = Confidence.HIGH,
|
||||||
title = "DNS answers are being rewritten",
|
title = "DNS answers are being rewritten",
|
||||||
description = "A canary reference record returned different RDATA than the spec-defined ground truth — something on the path is rewriting DNS answers (interception, filtering, or a middlebox).",
|
description = "A canary reference record returned different RDATA than the spec-defined ground truth — something on the path is rewriting DNS answers (interception, filtering, or a middlebox).",
|
||||||
evidenceRefs = listOf(EvidenceRef(t.id)),
|
evidenceRefs = listOf(EvidenceRef(t.id)),
|
||||||
@@ -368,8 +392,9 @@ class RunViewModel(app: Application) : AndroidViewModel(app) {
|
|||||||
} else if (ev.contains("\"reached_authoritative\":false")) {
|
} else if (ev.contains("\"reached_authoritative\":false")) {
|
||||||
out.add(
|
out.add(
|
||||||
Finding(
|
Finding(
|
||||||
id = ids.uuid(), code = "dns.authoritative_unreachable", category = Category.DNS,
|
id = ids.uuid(), code = FindingRegistry.DNS_AUTHORITATIVE_UNREACHABLE.code,
|
||||||
severity = Severity.MEDIUM, confidence = Confidence.MEDIUM,
|
category = FindingRegistry.DNS_AUTHORITATIVE_UNREACHABLE.category,
|
||||||
|
severity = FindingRegistry.DNS_AUTHORITATIVE_UNREACHABLE.severity, confidence = Confidence.MEDIUM,
|
||||||
title = "Canary queries don't reach the authoritative server",
|
title = "Canary queries don't reach the authoritative server",
|
||||||
description = "A per-run nonce name (which cannot be cached) was not answered by the canary server — the resolver is intercepting or failing to reach it.",
|
description = "A per-run nonce name (which cannot be cached) was not answered by the canary server — the resolver is intercepting or failing to reach it.",
|
||||||
evidenceRefs = listOf(EvidenceRef(t.id)),
|
evidenceRefs = listOf(EvidenceRef(t.id)),
|
||||||
@@ -382,8 +407,9 @@ class RunViewModel(app: Application) : AndroidViewModel(app) {
|
|||||||
if (ev.contains("address/port-dependent (symmetric NAT")) {
|
if (ev.contains("address/port-dependent (symmetric NAT")) {
|
||||||
out.add(
|
out.add(
|
||||||
Finding(
|
Finding(
|
||||||
id = ids.uuid(), code = "nat.symmetric", category = Category.NAT,
|
id = ids.uuid(), code = FindingRegistry.NAT_SYMMETRIC.code,
|
||||||
severity = Severity.MEDIUM, confidence = Confidence.HIGH,
|
category = FindingRegistry.NAT_SYMMETRIC.category,
|
||||||
|
severity = FindingRegistry.NAT_SYMMETRIC.severity, confidence = Confidence.HIGH,
|
||||||
title = "Symmetric NAT — peer-to-peer connections need a relay",
|
title = "Symmetric NAT — peer-to-peer connections need a relay",
|
||||||
description = "The NAT assigns a different external port per destination (address/port-dependent mapping). Direct peer-to-peer connections (calls, games, file transfer) will usually fail and fall back to relays.",
|
description = "The NAT assigns a different external port per destination (address/port-dependent mapping). Direct peer-to-peer connections (calls, games, file transfer) will usually fail and fall back to relays.",
|
||||||
evidenceRefs = listOf(EvidenceRef(t.id)),
|
evidenceRefs = listOf(EvidenceRef(t.id)),
|
||||||
@@ -400,8 +426,9 @@ class RunViewModel(app: Application) : AndroidViewModel(app) {
|
|||||||
if (ipv6Provisioned(networks)) {
|
if (ipv6Provisioned(networks)) {
|
||||||
out.add(
|
out.add(
|
||||||
Finding(
|
Finding(
|
||||||
id = ids.uuid(), code = "ipv6.broken", category = Category.IPV6,
|
id = ids.uuid(), code = FindingRegistry.V6_BROKEN.code,
|
||||||
severity = Severity.MEDIUM, confidence = Confidence.HIGH,
|
category = FindingRegistry.V6_BROKEN.category,
|
||||||
|
severity = FindingRegistry.V6_BROKEN.severity, confidence = Confidence.HIGH,
|
||||||
title = "IPv6 is configured but not working",
|
title = "IPv6 is configured but not working",
|
||||||
description = "This network advertises IPv6 (a global address and/or a default route), but ICMPv6 got no reply on any network. Half-configured IPv6 is worse than none: connections try IPv6 first and stall before falling back.",
|
description = "This network advertises IPv6 (a global address and/or a default route), but ICMPv6 got no reply on any network. Half-configured IPv6 is worse than none: connections try IPv6 first and stall before falling back.",
|
||||||
evidenceRefs = listOf(EvidenceRef(t.id)),
|
evidenceRefs = listOf(EvidenceRef(t.id)),
|
||||||
@@ -410,8 +437,9 @@ class RunViewModel(app: Application) : AndroidViewModel(app) {
|
|||||||
} else {
|
} else {
|
||||||
out.add(
|
out.add(
|
||||||
Finding(
|
Finding(
|
||||||
id = ids.uuid(), code = "ipv6.not_offered", category = Category.IPV6,
|
id = ids.uuid(), code = FindingRegistry.V6_NOT_OFFERED.code,
|
||||||
severity = Severity.INFO, confidence = Confidence.HIGH,
|
category = FindingRegistry.V6_NOT_OFFERED.category,
|
||||||
|
severity = FindingRegistry.V6_NOT_OFFERED.severity, confidence = Confidence.HIGH,
|
||||||
title = "IPv4-only network (no IPv6 offered)",
|
title = "IPv4-only network (no IPv6 offered)",
|
||||||
description = "No IPv6 address or default route was provisioned, so IPv6 tests could not run. This is normal — many networks are still IPv4-only and it is not a fault.",
|
description = "No IPv6 address or default route was provisioned, so IPv6 tests could not run. This is normal — many networks are still IPv4-only and it is not a fault.",
|
||||||
evidenceRefs = listOf(EvidenceRef(t.id)),
|
evidenceRefs = listOf(EvidenceRef(t.id)),
|
||||||
|
|||||||
@@ -4,6 +4,7 @@
|
|||||||
package app.echo_lot.app
|
package app.echo_lot.app
|
||||||
|
|
||||||
import androidx.compose.foundation.layout.Arrangement
|
import androidx.compose.foundation.layout.Arrangement
|
||||||
|
import androidx.compose.foundation.layout.safeDrawingPadding
|
||||||
import androidx.compose.foundation.layout.Column
|
import androidx.compose.foundation.layout.Column
|
||||||
import androidx.compose.foundation.layout.Row
|
import androidx.compose.foundation.layout.Row
|
||||||
import androidx.compose.foundation.layout.Spacer
|
import androidx.compose.foundation.layout.Spacer
|
||||||
@@ -46,6 +47,9 @@ fun SettingsScreen(
|
|||||||
onApplyRetention: () -> Unit,
|
onApplyRetention: () -> Unit,
|
||||||
onDeleteAll: () -> Unit,
|
onDeleteAll: () -> Unit,
|
||||||
onPreviewUpload: () -> Unit,
|
onPreviewUpload: () -> Unit,
|
||||||
|
onCheckServer: () -> Unit,
|
||||||
|
onEnroll: (String) -> Unit,
|
||||||
|
serverStatus: String?,
|
||||||
onBack: () -> Unit,
|
onBack: () -> Unit,
|
||||||
) {
|
) {
|
||||||
// SharedPreferences is not observable, so mirror each value into Compose state and write
|
// SharedPreferences is not observable, so mirror each value into Compose state and write
|
||||||
@@ -57,12 +61,13 @@ fun SettingsScreen(
|
|||||||
var autoUpload by remember { mutableStateOf(settings.autoUpload) }
|
var autoUpload by remember { mutableStateOf(settings.autoUpload) }
|
||||||
var privacy by remember { mutableStateOf(settings.privacyLevel) }
|
var privacy by remember { mutableStateOf(settings.privacyLevel) }
|
||||||
var stableSalt by remember { mutableStateOf(settings.stableSalt) }
|
var stableSalt by remember { mutableStateOf(settings.stableSalt) }
|
||||||
|
var enrollLink by remember { mutableStateOf("") }
|
||||||
var serverUrl by remember { mutableStateOf(settings.serverUrl) }
|
var serverUrl by remember { mutableStateOf(settings.serverUrl) }
|
||||||
var serverPin by remember { mutableStateOf(settings.serverPin) }
|
var serverPin by remember { mutableStateOf(settings.serverPin) }
|
||||||
var serverCred by remember { mutableStateOf(settings.serverCredential) }
|
var serverCred by remember { mutableStateOf(settings.serverCredential) }
|
||||||
|
|
||||||
Column(
|
Column(
|
||||||
Modifier.fillMaxWidth().verticalScroll(rememberScrollState()).padding(16.dp),
|
Modifier.fillMaxWidth().safeDrawingPadding().verticalScroll(rememberScrollState()).padding(16.dp),
|
||||||
verticalArrangement = Arrangement.spacedBy(12.dp),
|
verticalArrangement = Arrangement.spacedBy(12.dp),
|
||||||
) {
|
) {
|
||||||
Row(verticalAlignment = Alignment.CenterVertically) {
|
Row(verticalAlignment = Alignment.CenterVertically) {
|
||||||
@@ -149,6 +154,32 @@ fun SettingsScreen(
|
|||||||
checked = autoUpload,
|
checked = autoUpload,
|
||||||
) { autoUpload = it; settings.autoUpload = it }
|
) { autoUpload = it; settings.autoUpload = it }
|
||||||
|
|
||||||
|
// Enrollment first, because it is the path that works: one link carries the
|
||||||
|
// URL, the pin and a single-use token. The three fields below exist for when
|
||||||
|
// someone has to reconstruct a configuration by hand, not as the normal route.
|
||||||
|
Text(
|
||||||
|
"Paste an enrollment link from your server operator, or scan its QR code. " +
|
||||||
|
"It fills in all three fields below. The link contains a one-time token — " +
|
||||||
|
"treat it like a password until it is used.",
|
||||||
|
style = MaterialTheme.typography.bodySmall,
|
||||||
|
)
|
||||||
|
OutlinedTextField(
|
||||||
|
value = enrollLink, onValueChange = { enrollLink = it },
|
||||||
|
label = { Text("echolot://enroll?…") }, singleLine = true,
|
||||||
|
textStyle = MaterialTheme.typography.bodySmall.copy(fontFamily = FontFamily.Monospace),
|
||||||
|
modifier = Modifier.fillMaxWidth(),
|
||||||
|
)
|
||||||
|
Button(
|
||||||
|
onClick = {
|
||||||
|
onEnroll(enrollLink)
|
||||||
|
enrollLink = "" // spent either way; leaving it around invites a retry
|
||||||
|
serverUrl = settings.serverUrl
|
||||||
|
serverPin = settings.serverPin
|
||||||
|
serverCred = settings.serverCredential
|
||||||
|
},
|
||||||
|
enabled = enrollLink.isNotBlank(),
|
||||||
|
) { Text("Enroll") }
|
||||||
|
|
||||||
OutlinedTextField(
|
OutlinedTextField(
|
||||||
value = serverUrl, onValueChange = { serverUrl = it; settings.serverUrl = it },
|
value = serverUrl, onValueChange = { serverUrl = it; settings.serverUrl = it },
|
||||||
label = { Text("Server URL") }, singleLine = true, modifier = Modifier.fillMaxWidth(),
|
label = { Text("Server URL") }, singleLine = true, modifier = Modifier.fillMaxWidth(),
|
||||||
@@ -166,12 +197,26 @@ fun SettingsScreen(
|
|||||||
textStyle = MaterialTheme.typography.bodySmall.copy(fontFamily = FontFamily.Monospace),
|
textStyle = MaterialTheme.typography.bodySmall.copy(fontFamily = FontFamily.Monospace),
|
||||||
modifier = Modifier.fillMaxWidth(),
|
modifier = Modifier.fillMaxWidth(),
|
||||||
)
|
)
|
||||||
|
Row(verticalAlignment = Alignment.CenterVertically) {
|
||||||
|
Button(onClick = onCheckServer) { Text("Check server") }
|
||||||
Text(
|
Text(
|
||||||
if (settings.serverConfigured) "Server configured."
|
" " + if (settings.serverConfigured) "Configured."
|
||||||
else "Uploads stay off until all three fields are set.",
|
else "Uploads stay off until all three fields are set.",
|
||||||
style = MaterialTheme.typography.bodySmall,
|
style = MaterialTheme.typography.bodySmall,
|
||||||
)
|
)
|
||||||
}
|
}
|
||||||
|
// Version compatibility is checked here rather than discovered mid-run: a server
|
||||||
|
// that will refuse this build should say so before a measurement is wasted.
|
||||||
|
serverStatus?.let {
|
||||||
|
Text(it, style = MaterialTheme.typography.bodySmall)
|
||||||
|
}
|
||||||
|
Text(
|
||||||
|
"This app is ${BuildConfig.APP_SEMVER} and speaks probe protocol " +
|
||||||
|
"${app.echo_lot.protocol.Compat.PROTOCOL_VERSION}. It works with servers " +
|
||||||
|
"${app.echo_lot.protocol.Compat.serverRange}.",
|
||||||
|
style = MaterialTheme.typography.bodySmall,
|
||||||
|
)
|
||||||
|
}
|
||||||
}
|
}
|
||||||
Spacer(Modifier.height(24.dp))
|
Spacer(Modifier.height(24.dp))
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -31,10 +31,23 @@ data class ArchivedRun(
|
|||||||
val verdict: String? = null,
|
val verdict: String? = null,
|
||||||
@SerialName("finding_count") val findingCount: Int = 0,
|
@SerialName("finding_count") val findingCount: Int = 0,
|
||||||
@SerialName("size_bytes") val sizeBytes: Long = 0,
|
@SerialName("size_bytes") val sizeBytes: Long = 0,
|
||||||
|
/**
|
||||||
|
* How the *archived* document is redacted. Always "full" in practice, because the archive
|
||||||
|
* deliberately keeps the unredacted run - see the package doc. This is not what was uploaded.
|
||||||
|
*/
|
||||||
val anonymization: String = "full",
|
val anonymization: String = "full",
|
||||||
/** Whether this run has been accepted by a server, so history can show what is backed up. */
|
/** Whether this run has been accepted by a server, so history can show what is backed up. */
|
||||||
val uploaded: Boolean = false,
|
val uploaded: Boolean = false,
|
||||||
@SerialName("uploaded_to") val uploadedTo: String? = null,
|
@SerialName("uploaded_to") val uploadedTo: String? = null,
|
||||||
|
/**
|
||||||
|
* The level the run was *uploaded* at, which is a different document from the archived one.
|
||||||
|
*
|
||||||
|
* Kept separately because conflating the two is actively misleading: the history row showed
|
||||||
|
* the archive's own level ("full") directly beneath "uploaded to fmr", which reads as "the
|
||||||
|
* complete data was uploaded" when a redacted copy had been sent. A privacy display that
|
||||||
|
* overstates what left the device is worse than none.
|
||||||
|
*/
|
||||||
|
@SerialName("uploaded_as") val uploadedAs: String? = null,
|
||||||
)
|
)
|
||||||
|
|
||||||
/**
|
/**
|
||||||
@@ -119,7 +132,7 @@ class RunArchive(private val dir: File, private val now: () -> Long = System::cu
|
|||||||
fun deleteAll(): Int = list().count { delete(it.id) }
|
fun deleteAll(): Int = list().count { delete(it.id) }
|
||||||
|
|
||||||
/** Records that a server accepted this run, so history can distinguish backed-up from local. */
|
/** Records that a server accepted this run, so history can distinguish backed-up from local. */
|
||||||
fun markUploaded(id: String, serverName: String) {
|
fun markUploaded(id: String, serverName: String, uploadedAs: String? = null) {
|
||||||
val f = File(dir, safe(id) + META_EXT)
|
val f = File(dir, safe(id) + META_EXT)
|
||||||
val meta = runCatching { json.decodeFromString(ArchivedRun.serializer(), f.readText()) }.getOrNull()
|
val meta = runCatching { json.decodeFromString(ArchivedRun.serializer(), f.readText()) }.getOrNull()
|
||||||
?: return
|
?: return
|
||||||
@@ -127,7 +140,7 @@ class RunArchive(private val dir: File, private val now: () -> Long = System::cu
|
|||||||
f,
|
f,
|
||||||
json.encodeToString(
|
json.encodeToString(
|
||||||
ArchivedRun.serializer(),
|
ArchivedRun.serializer(),
|
||||||
meta.copy(uploaded = true, uploadedTo = serverName),
|
meta.copy(uploaded = true, uploadedTo = serverName, uploadedAs = uploadedAs),
|
||||||
),
|
),
|
||||||
)
|
)
|
||||||
}
|
}
|
||||||
@@ -181,7 +194,10 @@ class RunArchive(private val dir: File, private val now: () -> Long = System::cu
|
|||||||
id = id,
|
id = id,
|
||||||
savedAtEpochMs = now(),
|
savedAtEpochMs = now(),
|
||||||
startedAt = run["started_at"]?.jsonPrimitive?.content,
|
startedAt = run["started_at"]?.jsonPrimitive?.content,
|
||||||
verdict = doc["summary"]?.jsonObject?.get("verdict")?.jsonPrimitive?.content,
|
// The schema calls it `overall` (Summary.overall); reading `verdict` here silently
|
||||||
|
// yielded null for every run, so the history list's most prominent element - the
|
||||||
|
// coloured verdict - was blank on every row.
|
||||||
|
verdict = doc["summary"]?.jsonObject?.get("overall")?.jsonPrimitive?.content,
|
||||||
findingCount = (doc["findings"] as? kotlinx.serialization.json.JsonArray)?.size ?: 0,
|
findingCount = (doc["findings"] as? kotlinx.serialization.json.JsonArray)?.size ?: 0,
|
||||||
sizeBytes = size,
|
sizeBytes = size,
|
||||||
anonymization = run["privacy"]?.jsonObject?.get("anonymization")?.jsonPrimitive?.content ?: "full",
|
anonymization = run["privacy"]?.jsonObject?.get("anonymization")?.jsonPrimitive?.content ?: "full",
|
||||||
|
|||||||
@@ -25,7 +25,7 @@ class RunArchiveTest {
|
|||||||
private fun doc(id: String, findings: Int = 1, pad: Int = 0): String {
|
private fun doc(id: String, findings: Int = 1, pad: Int = 0): String {
|
||||||
val f = (1..findings).joinToString(",") { """{"id":"f$it"}""" }
|
val f = (1..findings).joinToString(",") { """{"id":"f$it"}""" }
|
||||||
return """{"run":{"id":"$id","started_at":"2026-08-01T10:00:00Z","privacy":{"anonymization":"balanced"}},""" +
|
return """{"run":{"id":"$id","started_at":"2026-08-01T10:00:00Z","privacy":{"anonymization":"balanced"}},""" +
|
||||||
""""findings":[$f],"summary":{"verdict":"warn"},"pad":"${"x".repeat(pad)}"}"""
|
""""findings":[$f],"summary":{"overall":"warn"},"pad":"${"x".repeat(pad)}"}"""
|
||||||
}
|
}
|
||||||
|
|
||||||
@Test
|
@Test
|
||||||
@@ -120,13 +120,40 @@ class RunArchiveTest {
|
|||||||
fun uploadStateIsRecorded() {
|
fun uploadStateIsRecorded() {
|
||||||
val a = archive()
|
val a = archive()
|
||||||
a.save(doc("run-1"))
|
a.save(doc("run-1"))
|
||||||
a.markUploaded("run-1", "fmr")
|
a.markUploaded("run-1", "fmr", "balanced")
|
||||||
val meta = a.list().single()
|
val meta = a.list().single()
|
||||||
assertTrue(meta.uploaded)
|
assertTrue(meta.uploaded)
|
||||||
assertEquals("fmr", meta.uploadedTo)
|
assertEquals("fmr", meta.uploadedTo)
|
||||||
assertEquals("run-1", meta.id, "marking upload must not disturb the rest of the entry")
|
assertEquals("run-1", meta.id, "marking upload must not disturb the rest of the entry")
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// The archive's own level and the level a run was uploaded at describe *different documents*.
|
||||||
|
// Showing the archive's ("full", because the archive is deliberately unredacted) next to
|
||||||
|
// "uploaded to fmr" reads as "the complete data was uploaded" when a redacted copy was sent —
|
||||||
|
// a privacy display that overstates what left the device is worse than none.
|
||||||
|
@Test
|
||||||
|
fun theUploadedLevelIsRecordedSeparatelyFromTheArchivedOne() {
|
||||||
|
val a = archive()
|
||||||
|
// A real archived document carries no privacy stamp: the anonymizer never runs on the
|
||||||
|
// archive. The shared doc() fixture has one, which is exactly the unrealism that let this
|
||||||
|
// confusion through in the first place.
|
||||||
|
a.save("""{"run":{"id":"run-1"},"findings":[],"summary":{"overall":"green"}}""")
|
||||||
|
a.markUploaded("run-1", "fmr", "balanced")
|
||||||
|
val meta = a.list().single()
|
||||||
|
assertEquals("full", meta.anonymization, "the archived copy is unredacted, by design")
|
||||||
|
assertEquals("balanced", meta.uploadedAs, "the uploaded copy was redacted, and must say so")
|
||||||
|
}
|
||||||
|
|
||||||
|
// The verdict is read from `summary.overall` — the schema's actual field name. Reading
|
||||||
|
// `summary.verdict` silently yielded null for every run, so the history list's most prominent
|
||||||
|
// element was blank on every row while everything else looked fine.
|
||||||
|
@Test
|
||||||
|
fun theVerdictComesFromTheSchemasOverallField() {
|
||||||
|
val a = archive()
|
||||||
|
a.save("""{"run":{"id":"r1"},"findings":[],"summary":{"overall":"yellow"}}""")
|
||||||
|
assertEquals("yellow", a.list().single().verdict)
|
||||||
|
}
|
||||||
|
|
||||||
@Test
|
@Test
|
||||||
fun deleteRemovesBothFiles() {
|
fun deleteRemovesBothFiles() {
|
||||||
val a = archive()
|
val a = archive()
|
||||||
|
|||||||
@@ -25,6 +25,6 @@ java { sourceCompatibility = JavaVersion.VERSION_17; targetCompatibility = JavaV
|
|||||||
|
|
||||||
tasks.test {
|
tasks.test {
|
||||||
useJUnitPlatform()
|
useJUnitPlatform()
|
||||||
listOf("ECHOLOT_LIVE_URL","ECHOLOT_LIVE_PIN","ECHOLOT_LIVE_CRED","ECHOLOT_LIVE_UDP","ECHOLOT_LIVE_TARGET")
|
listOf("ECHOLOT_LIVE_URL","ECHOLOT_LIVE_PIN","ECHOLOT_LIVE_CRED","ECHOLOT_LIVE_UDP","ECHOLOT_LIVE_TARGET","ECHOLOT_ENROLL_URI")
|
||||||
.forEach { k -> System.getenv(k)?.let { environment(k, it) } }
|
.forEach { k -> System.getenv(k)?.let { environment(k, it) } }
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -0,0 +1,107 @@
|
|||||||
|
// SPDX-FileCopyrightText: 2026 Echolot contributors
|
||||||
|
// SPDX-License-Identifier: GPL-3.0-or-later
|
||||||
|
|
||||||
|
package app.echo_lot.engine
|
||||||
|
|
||||||
|
import kotlinx.serialization.SerialName
|
||||||
|
import kotlinx.serialization.Serializable
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Splits a round-trip train into its two directions using what the server witnessed.
|
||||||
|
*
|
||||||
|
* A round trip can only report that *something* was lost somewhere. That is the least useful form
|
||||||
|
* of the answer: "3 % loss" sends an engineer looking in both directions at once. The server
|
||||||
|
* records every packet it received, per sequence number (probe-protocol.md §6), so the two cases
|
||||||
|
* are actually distinguishable:
|
||||||
|
*
|
||||||
|
* - sent, never seen by the server → **upstream** loss
|
||||||
|
* - seen by the server, reply never arrived → **downstream** loss
|
||||||
|
*
|
||||||
|
* The same records give one-way delay *variation* per direction. Absolute one-way delay would
|
||||||
|
* need synchronised clocks and we deliberately have none (measurement-schema.md's two-clock rule),
|
||||||
|
* but the variation does not: (server_rx − client_tx) contains an unknown constant clock offset,
|
||||||
|
* and differencing successive samples cancels it. So jitter is honestly attributable to a
|
||||||
|
* direction even though latency is not.
|
||||||
|
*/
|
||||||
|
object Directional {
|
||||||
|
|
||||||
|
/** One probe as the client saw it. [tRxNs] null means no reply came back. */
|
||||||
|
data class Sample(val seq: Int, val tTxNs: Long, val tRxNs: Long?)
|
||||||
|
|
||||||
|
/** One probe as the server saw it: its own receive and transmit stamps, on its own clock. */
|
||||||
|
data class ServerSighting(val seq: Int, val tRxNs: Long, val tTxNs: Long)
|
||||||
|
|
||||||
|
fun analyse(sent: List<Sample>, seen: List<ServerSighting>): DirectionalMetrics {
|
||||||
|
val byServerSeq = seen.associateBy { it.seq }
|
||||||
|
// Only sequences we actually sent count. A server record for a sequence we have no note
|
||||||
|
// of is not evidence about this train — it is a bug or a stray, and silently folding it
|
||||||
|
// in would produce loss percentages above 100 or below zero.
|
||||||
|
val relevant = sent.filter { byServerSeq.containsKey(it.seq) }
|
||||||
|
|
||||||
|
val nSent = sent.size
|
||||||
|
val nSeen = relevant.size
|
||||||
|
val nReplied = sent.count { it.tRxNs != null }
|
||||||
|
|
||||||
|
// A reply can only exist if the request arrived, so downstream loss is measured against
|
||||||
|
// what the server saw, not against what we sent — otherwise upstream loss is counted twice.
|
||||||
|
val lostUp = nSent - nSeen
|
||||||
|
val lostDown = (nSeen - nReplied).coerceAtLeast(0)
|
||||||
|
|
||||||
|
val upDeltas = relevant.sortedBy { it.seq }
|
||||||
|
.map { byServerSeq.getValue(it.seq).tRxNs - it.tTxNs }
|
||||||
|
val downDeltas = sent.filter { it.tRxNs != null && byServerSeq.containsKey(it.seq) }
|
||||||
|
.sortedBy { it.seq }
|
||||||
|
.map { it.tRxNs!! - byServerSeq.getValue(it.seq).tTxNs }
|
||||||
|
|
||||||
|
return DirectionalMetrics(
|
||||||
|
sent = nSent,
|
||||||
|
seenByServer = nSeen,
|
||||||
|
repliesReceived = nReplied,
|
||||||
|
lostUpstream = lostUp,
|
||||||
|
lostDownstream = lostDown,
|
||||||
|
lossUpstreamPct = pct(lostUp, nSent),
|
||||||
|
// Denominator is what reached the server: of the packets that got there, how many
|
||||||
|
// replies came back.
|
||||||
|
lossDownstreamPct = pct(lostDown, nSeen),
|
||||||
|
jitterUpstreamMs = jitterMs(upDeltas),
|
||||||
|
jitterDownstreamMs = jitterMs(downDeltas),
|
||||||
|
/** True when the server saw nothing at all, which is a different fault from loss. */
|
||||||
|
noneReachedServer = nSent > 0 && nSeen == 0,
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Mean absolute difference between consecutive one-way samples (RFC 3393 IPDV, averaged).
|
||||||
|
*
|
||||||
|
* Differencing is what makes this legitimate without synchronised clocks: each sample carries
|
||||||
|
* the same unknown offset between the two clocks, and the difference cancels it. Fewer than
|
||||||
|
* two samples yields null rather than zero — "no jitter" and "not enough data to say" are
|
||||||
|
* different claims and only one of them is true here.
|
||||||
|
*/
|
||||||
|
private fun jitterMs(oneWayNs: List<Long>): Double? {
|
||||||
|
if (oneWayNs.size < 2) return null
|
||||||
|
val deltas = oneWayNs.zipWithNext { a, b -> kotlin.math.abs(b - a) }
|
||||||
|
return round2(deltas.average() / 1_000_000.0)
|
||||||
|
}
|
||||||
|
|
||||||
|
private fun pct(part: Int, whole: Int): Double =
|
||||||
|
if (whole <= 0) 0.0 else round2(part * 100.0 / whole)
|
||||||
|
|
||||||
|
private fun round2(v: Double) = Math.round(v * 100.0) / 100.0
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Directional metrics for train.udp_updown; recomputable from the columnar evidence. */
|
||||||
|
@Serializable
|
||||||
|
data class DirectionalMetrics(
|
||||||
|
val sent: Int,
|
||||||
|
@SerialName("seen_by_server") val seenByServer: Int,
|
||||||
|
@SerialName("replies_received") val repliesReceived: Int,
|
||||||
|
@SerialName("lost_upstream") val lostUpstream: Int,
|
||||||
|
@SerialName("lost_downstream") val lostDownstream: Int,
|
||||||
|
@SerialName("loss_upstream_pct") val lossUpstreamPct: Double,
|
||||||
|
@SerialName("loss_downstream_pct") val lossDownstreamPct: Double,
|
||||||
|
/** One-way delay variation (RFC 3393), per direction. Null when there were too few samples. */
|
||||||
|
@SerialName("jitter_upstream_ms") val jitterUpstreamMs: Double? = null,
|
||||||
|
@SerialName("jitter_downstream_ms") val jitterDownstreamMs: Double? = null,
|
||||||
|
@SerialName("none_reached_server") val noneReachedServer: Boolean = false,
|
||||||
|
)
|
||||||
@@ -0,0 +1,493 @@
|
|||||||
|
// SPDX-FileCopyrightText: 2026 Echolot contributors
|
||||||
|
// SPDX-License-Identifier: GPL-3.0-or-later
|
||||||
|
|
||||||
|
package app.echo_lot.engine
|
||||||
|
|
||||||
|
import app.echo_lot.measurement.*
|
||||||
|
import app.echo_lot.protocol.ControlClient
|
||||||
|
import app.echo_lot.protocol.ProbeSession
|
||||||
|
import app.echo_lot.protocol.Wire
|
||||||
|
import kotlinx.serialization.SerialName
|
||||||
|
import kotlinx.serialization.Serializable
|
||||||
|
import kotlinx.serialization.json.Json
|
||||||
|
import kotlinx.serialization.json.JsonObject
|
||||||
|
import kotlinx.serialization.json.encodeToJsonElement
|
||||||
|
|
||||||
|
/**
|
||||||
|
* The measurements only the far end can make: what the *downstream* path does to traffic the
|
||||||
|
* client never asked for packet-by-packet.
|
||||||
|
*
|
||||||
|
* A client alone can measure a round trip, and it can find the largest packet it can *send*. It
|
||||||
|
* cannot find the largest packet it can *receive*, or whether the network drops downstream
|
||||||
|
* packets independently of upstream ones — those need a server willing to push, which is why the
|
||||||
|
* protocol gates them behind an asymmetric grant (probe-protocol.md §3.4).
|
||||||
|
*
|
||||||
|
* Three separate facts come out, and keeping them separate is the point:
|
||||||
|
* - `mtu.pmtud_down` — the largest datagram that arrives *unfragmented*. This is the number
|
||||||
|
* that matters for anything setting DF, and it is only meaningful because the server sets DF.
|
||||||
|
* - `mtu.frag_delivery` — whether larger datagrams arrive once the network is allowed to
|
||||||
|
* fragment them. A path can be fine for one and broken for the other; conflating them is how
|
||||||
|
* you get "MTU is 4000" on a link that drops every DF packet over 1400.
|
||||||
|
* - `train.udp_downstream` — loss, reordering and arrival spacing in the download direction.
|
||||||
|
*/
|
||||||
|
class DownstreamMeasurement(private val ids: IdSource) {
|
||||||
|
|
||||||
|
private val json = Json { encodeDefaults = true; explicitNulls = true }
|
||||||
|
|
||||||
|
/** How long to wait for a granted burst after the server accepts the action. */
|
||||||
|
private val collectWindowMs = 4_000L
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Shorter, but long enough to cover the first_last mode's deliberate 250 ms hold plus a
|
||||||
|
* reassembly. A fragment burst is one datagram: it is here quickly or not at all.
|
||||||
|
*/
|
||||||
|
private val fragWindowMs = 1_500L
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Asks the server to send one deliberately-fragmented datagram per ordering, and reports
|
||||||
|
* which orderings survive the path.
|
||||||
|
*
|
||||||
|
* Kernel fragmentation always emits fragments in order, first one first, so an oversized
|
||||||
|
* datagram can only answer "do fragments get through at all". The interesting fault is about
|
||||||
|
* ordering: only the *first* fragment carries the UDP ports, so a stateful firewall that has
|
||||||
|
* not seen it has nothing to match the rest against, and many drop them. That failure is
|
||||||
|
* invisible to every in-order test and shows up in the field as "large DNS answers fail here"
|
||||||
|
* or "the tunnel breaks when the MTU drops".
|
||||||
|
*/
|
||||||
|
fun fragmentOrdering(
|
||||||
|
credential: String,
|
||||||
|
sessionId: String,
|
||||||
|
control: ControlClient,
|
||||||
|
probe: ProbeSession,
|
||||||
|
sessionRef: String,
|
||||||
|
sizeBytes: Int = 2000,
|
||||||
|
fragBytes: Int = 576,
|
||||||
|
): Pair<Test, List<Finding>> {
|
||||||
|
val testId = ids.uuid()
|
||||||
|
val started = ids.monoNs()
|
||||||
|
val delivered = LinkedHashMap<String, Boolean>()
|
||||||
|
val fragmentCounts = LinkedHashMap<String, Int>()
|
||||||
|
var unsupported = false
|
||||||
|
|
||||||
|
for (mode in FRAG_MODES) {
|
||||||
|
val reply = runCatching {
|
||||||
|
control.action(
|
||||||
|
credential, sessionId,
|
||||||
|
"""{"action":"frag_send","size_bytes":$sizeBytes,"mode":"$mode","frag_bytes":$fragBytes}""",
|
||||||
|
)
|
||||||
|
}
|
||||||
|
if (reply.isFailure) {
|
||||||
|
// A server without a raw socket says so; that is a missing capability, not a
|
||||||
|
// property of the network, and must not be recorded as a failed delivery.
|
||||||
|
unsupported = true
|
||||||
|
break
|
||||||
|
}
|
||||||
|
parseInt(reply.getOrNull(), "fragments")?.let { fragmentCounts[mode] = it }
|
||||||
|
// The burst is already on the wire when the action returns (it is sent
|
||||||
|
// synchronously), so anything that survived is either here or lost.
|
||||||
|
val got = probe.collectGranted(fragWindowMs).any { it.type == Wire.TYPE_FRAG_DATA }
|
||||||
|
delivered[mode] = got
|
||||||
|
}
|
||||||
|
|
||||||
|
if (unsupported) {
|
||||||
|
return Test(
|
||||||
|
id = testId, type = TestType.MTU_FRAG_ORDERING, sessionRef = sessionRef, tier = Tier.APP,
|
||||||
|
startedMonoNs = started, endedMonoNs = ids.monoNs(),
|
||||||
|
status = TestStatus.UNSUPPORTED,
|
||||||
|
error = TestError("no_raw_socket", "this server cannot craft fragments"),
|
||||||
|
) to emptyList()
|
||||||
|
}
|
||||||
|
|
||||||
|
val metrics = json.encodeToJsonElement(
|
||||||
|
FragOrderingMetrics(
|
||||||
|
sizeBytes = sizeBytes,
|
||||||
|
fragBytes = fragBytes,
|
||||||
|
fragmentsPerBurst = fragmentCounts,
|
||||||
|
deliveredByMode = delivered,
|
||||||
|
inOrderDelivered = delivered[FRAG_IN_ORDER] == true,
|
||||||
|
reorderedDelivered = delivered[FRAG_REVERSED] == true,
|
||||||
|
delayedFirstDelivered = delivered[FRAG_FIRST_LAST] == true,
|
||||||
|
),
|
||||||
|
) as JsonObject
|
||||||
|
|
||||||
|
val findings = ArrayList<Finding>()
|
||||||
|
val inOrder = delivered[FRAG_IN_ORDER] == true
|
||||||
|
val reversed = delivered[FRAG_REVERSED] == true
|
||||||
|
val firstLast = delivered[FRAG_FIRST_LAST] == true
|
||||||
|
|
||||||
|
if (!inOrder) {
|
||||||
|
findings.add(
|
||||||
|
finding(
|
||||||
|
FindingRegistry.FRAGMENTS_BLOCKED, testId,
|
||||||
|
"IP fragments do not reach this device",
|
||||||
|
"A fragmented datagram sent in the normal order never arrived. Anything that " +
|
||||||
|
"relies on fragmentation — large DNS answers over UDP, some VPN traffic — " +
|
||||||
|
"will fail here rather than slow down.",
|
||||||
|
),
|
||||||
|
)
|
||||||
|
} else if (!reversed || !firstLast) {
|
||||||
|
// The precise and useful finding: fragments work, but only if they arrive tidily.
|
||||||
|
val which = buildList {
|
||||||
|
if (!reversed) add("out of order")
|
||||||
|
if (!firstLast) add("with the first fragment delayed")
|
||||||
|
}.joinToString(" or ")
|
||||||
|
findings.add(
|
||||||
|
finding(
|
||||||
|
FindingRegistry.FRAGMENT_REORDER_SENSITIVE, testId,
|
||||||
|
"Fragments are dropped when they arrive $which",
|
||||||
|
"In-order fragments are delivered, but the same datagram sent $which is not. " +
|
||||||
|
"Something on the path only reassembles when the first fragment (the one " +
|
||||||
|
"carrying the UDP ports) arrives first — typical of a stateful firewall " +
|
||||||
|
"or NAT. It works until the network reorders, then fails intermittently, " +
|
||||||
|
"which is the hardest kind of fault to chase.",
|
||||||
|
),
|
||||||
|
)
|
||||||
|
}
|
||||||
|
return Test(
|
||||||
|
id = testId, type = TestType.MTU_FRAG_ORDERING, sessionRef = sessionRef, tier = Tier.APP,
|
||||||
|
startedMonoNs = started, endedMonoNs = ids.monoNs(),
|
||||||
|
status = if (inOrder) TestStatus.OK else TestStatus.PARTIAL,
|
||||||
|
metrics = metrics,
|
||||||
|
) to findings
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Runs all three against an already-primed session.
|
||||||
|
*
|
||||||
|
* [session] must already have sent at least one ECHO: the grant is bound to the source the
|
||||||
|
* server has actually observed, so an unprimed session gets a 409 rather than a grant. That
|
||||||
|
* is the anti-amplification rule doing its job, not an error to work around.
|
||||||
|
*/
|
||||||
|
fun run(
|
||||||
|
credential: String,
|
||||||
|
sessionId: String,
|
||||||
|
control: ControlClient,
|
||||||
|
probe: ProbeSession,
|
||||||
|
sessionRef: String,
|
||||||
|
sizes: List<Int> = DEFAULT_SIZES,
|
||||||
|
trainCount: Int = 100,
|
||||||
|
trainSizeBytes: Int = 300,
|
||||||
|
trainIntervalUs: Int = 3_000,
|
||||||
|
): Pair<List<Test>, List<Finding>> {
|
||||||
|
val tests = ArrayList<Test>()
|
||||||
|
val findings = ArrayList<Finding>()
|
||||||
|
|
||||||
|
val df = bigSend(credential, sessionId, control, probe, sessionRef, sizes, df = true)
|
||||||
|
val frag = bigSend(credential, sessionId, control, probe, sessionRef, sizes, df = false)
|
||||||
|
val train = downTrain(
|
||||||
|
credential, sessionId, control, probe, sessionRef, trainCount, trainSizeBytes, trainIntervalUs,
|
||||||
|
)
|
||||||
|
|
||||||
|
tests.add(df.test); tests.add(frag.test); tests.add(train.test)
|
||||||
|
|
||||||
|
// Fragment ordering only makes sense once we know fragments arrive at all; when they do
|
||||||
|
// not, the ordering variants would all report "not delivered" and read as three faults
|
||||||
|
// instead of one.
|
||||||
|
if (frag.largestDelivered != null) {
|
||||||
|
val (fragTest, fragFindings) =
|
||||||
|
fragmentOrdering(credential, sessionId, control, probe, sessionRef)
|
||||||
|
tests.add(fragTest)
|
||||||
|
findings.addAll(fragFindings)
|
||||||
|
}
|
||||||
|
|
||||||
|
// A downstream MTU below the classic 1500-byte Ethernet payload is worth saying out loud:
|
||||||
|
// it is the usual cause of "small requests work, large responses hang".
|
||||||
|
val pathMtu = df.largestDelivered
|
||||||
|
if (pathMtu != null && pathMtu > 0) {
|
||||||
|
val ipMtu = pathMtu + IP_UDP_OVERHEAD4
|
||||||
|
if (ipMtu < 1500) {
|
||||||
|
findings.add(
|
||||||
|
finding(
|
||||||
|
FindingRegistry.MTU_REDUCED_DOWNSTREAM, df.test.id,
|
||||||
|
"Downstream path MTU is $ipMtu bytes, below 1500",
|
||||||
|
"The largest datagram that reached this device without fragmenting was " +
|
||||||
|
"$pathMtu bytes of payload ($ipMtu on the wire). Tunnels (PPPoE, VPN, " +
|
||||||
|
"IPv6-in-IPv4) commonly do this; it is only a fault when something " +
|
||||||
|
"on the path also blocks the ICMP messages that let senders discover it.",
|
||||||
|
),
|
||||||
|
)
|
||||||
|
}
|
||||||
|
// The dangerous combination: unfragmented large packets vanish AND fragments do too,
|
||||||
|
// so a sender that never gets told will retransmit into a black hole.
|
||||||
|
val fragLargest = frag.largestDelivered ?: 0
|
||||||
|
if (fragLargest <= pathMtu && sizes.any { it > pathMtu }) {
|
||||||
|
findings.add(
|
||||||
|
finding(
|
||||||
|
FindingRegistry.MTU_DOWNSTREAM_BLACKHOLE, frag.test.id,
|
||||||
|
"Datagrams above $pathMtu bytes are dropped downstream, fragmented or not",
|
||||||
|
"Nothing larger than $pathMtu bytes arrived, even when the network was " +
|
||||||
|
"free to fragment it. Traffic that relies on large responses will " +
|
||||||
|
"stall rather than fail cleanly.",
|
||||||
|
),
|
||||||
|
)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
if (train.received == 0) {
|
||||||
|
findings.add(
|
||||||
|
finding(
|
||||||
|
FindingRegistry.DOWNSTREAM_BLOCKED, train.test.id,
|
||||||
|
"No server-initiated packets arrived",
|
||||||
|
"The server sent ${train.sent} packets toward this device and none arrived, " +
|
||||||
|
"while the round-trip echo worked. Something on the path forwards replies " +
|
||||||
|
"but drops traffic the device did not individually solicit.",
|
||||||
|
),
|
||||||
|
)
|
||||||
|
} else if (train.lossPct >= 5.0) {
|
||||||
|
findings.add(
|
||||||
|
finding(
|
||||||
|
FindingRegistry.LOSS_DOWNSTREAM, train.test.id,
|
||||||
|
"Downstream loss of ${round1(train.lossPct)}%",
|
||||||
|
"${train.sent - train.received} of ${train.sent} packets sent toward this " +
|
||||||
|
"device were lost. Downstream loss is invisible to a round-trip test, " +
|
||||||
|
"which reports only that *something* was lost somewhere.",
|
||||||
|
),
|
||||||
|
)
|
||||||
|
}
|
||||||
|
if (train.reordered > 0) {
|
||||||
|
findings.add(
|
||||||
|
finding(
|
||||||
|
FindingRegistry.DOWNSTREAM_REORDER, train.test.id,
|
||||||
|
"${train.reordered} downstream packet(s) arrived out of order",
|
||||||
|
"Packets arrived in a different order than they were sent. Usually per-packet " +
|
||||||
|
"load balancing across links; harmless for most traffic, not for all of it.",
|
||||||
|
),
|
||||||
|
)
|
||||||
|
}
|
||||||
|
return tests to findings
|
||||||
|
}
|
||||||
|
|
||||||
|
// ---- big_send ---------------------------------------------------------------------
|
||||||
|
|
||||||
|
private class SizeResult(val test: Test, val largestDelivered: Int?)
|
||||||
|
|
||||||
|
private fun bigSend(
|
||||||
|
credential: String, sessionId: String, control: ControlClient, probe: ProbeSession,
|
||||||
|
sessionRef: String, sizes: List<Int>, df: Boolean,
|
||||||
|
): SizeResult {
|
||||||
|
val testId = ids.uuid()
|
||||||
|
val started = ids.monoNs()
|
||||||
|
val requested = sizes.joinToString(",")
|
||||||
|
|
||||||
|
val reply = runCatching {
|
||||||
|
control.action(
|
||||||
|
credential, sessionId,
|
||||||
|
"""{"action":"big_send","df":$df,"sizes_bytes":[$requested]}""",
|
||||||
|
)
|
||||||
|
}
|
||||||
|
if (reply.isFailure) {
|
||||||
|
return SizeResult(
|
||||||
|
Test(
|
||||||
|
id = testId, type = if (df) TestType.MTU_PMTUD_DOWN else TestType.MTU_FRAG_DELIVERY,
|
||||||
|
sessionRef = sessionRef, tier = Tier.APP,
|
||||||
|
startedMonoNs = started, endedMonoNs = ids.monoNs(),
|
||||||
|
status = TestStatus.UNSUPPORTED,
|
||||||
|
error = TestError("action_refused", reply.exceptionOrNull()?.message ?: "big_send refused"),
|
||||||
|
),
|
||||||
|
null,
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
// The server tells us which sizes it actually put on the wire. With DF it refuses
|
||||||
|
// anything above its own egress MTU, and treating those as "lost downstream" would
|
||||||
|
// blame the client's network for our own limit.
|
||||||
|
val accepted = parseIntArray(reply.getOrNull(), "sizes_bytes").ifEmpty { sizes }
|
||||||
|
val serverMaxDf = parseInt(reply.getOrNull(), "max_df_bytes")
|
||||||
|
|
||||||
|
val arrived = probe.collectGranted(collectWindowMs)
|
||||||
|
.filter { it.type == Wire.TYPE_BIG_SEND }
|
||||||
|
.map { it.sizeBytes }
|
||||||
|
.distinct()
|
||||||
|
.sorted()
|
||||||
|
val largest = arrived.maxOrNull()
|
||||||
|
|
||||||
|
val metrics = json.encodeToJsonElement(
|
||||||
|
BigSendMetrics(
|
||||||
|
requestedBytes = sizes,
|
||||||
|
sentBytes = accepted,
|
||||||
|
deliveredBytes = arrived,
|
||||||
|
largestDeliveredBytes = largest,
|
||||||
|
dontFragment = df,
|
||||||
|
serverMaxDfBytes = serverMaxDf,
|
||||||
|
// Only meaningful for the DF run; the IP-level MTU is the payload plus headers.
|
||||||
|
pathMtuBytes = if (df && largest != null) largest + IP_UDP_OVERHEAD4 else null,
|
||||||
|
),
|
||||||
|
) as JsonObject
|
||||||
|
|
||||||
|
val status = when {
|
||||||
|
arrived.isEmpty() -> TestStatus.FAILED
|
||||||
|
arrived.size < accepted.size -> TestStatus.PARTIAL
|
||||||
|
else -> TestStatus.OK
|
||||||
|
}
|
||||||
|
return SizeResult(
|
||||||
|
Test(
|
||||||
|
id = testId, type = if (df) TestType.MTU_PMTUD_DOWN else TestType.MTU_FRAG_DELIVERY,
|
||||||
|
sessionRef = sessionRef, tier = Tier.APP,
|
||||||
|
startedMonoNs = started, endedMonoNs = ids.monoNs(),
|
||||||
|
status = status, metrics = metrics,
|
||||||
|
),
|
||||||
|
largest,
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
// ---- downtrain --------------------------------------------------------------------
|
||||||
|
|
||||||
|
private class TrainResult(
|
||||||
|
val test: Test, val sent: Int, val received: Int, val lossPct: Double, val reordered: Int,
|
||||||
|
)
|
||||||
|
|
||||||
|
private fun downTrain(
|
||||||
|
credential: String, sessionId: String, control: ControlClient, probe: ProbeSession,
|
||||||
|
sessionRef: String, count: Int, sizeBytes: Int, intervalUs: Int,
|
||||||
|
): TrainResult {
|
||||||
|
val testId = ids.uuid()
|
||||||
|
val started = ids.monoNs()
|
||||||
|
|
||||||
|
val reply = runCatching {
|
||||||
|
control.action(
|
||||||
|
credential, sessionId,
|
||||||
|
"""{"action":"downtrain","count":$count,"size_bytes":$sizeBytes,"interval_us":$intervalUs}""",
|
||||||
|
)
|
||||||
|
}
|
||||||
|
if (reply.isFailure) {
|
||||||
|
return TrainResult(
|
||||||
|
Test(
|
||||||
|
id = testId, type = TestType.TRAIN_UDP_DOWNSTREAM, sessionRef = sessionRef,
|
||||||
|
tier = Tier.APP, startedMonoNs = started, endedMonoNs = ids.monoNs(),
|
||||||
|
status = TestStatus.UNSUPPORTED,
|
||||||
|
error = TestError("action_refused", reply.exceptionOrNull()?.message ?: "downtrain refused"),
|
||||||
|
),
|
||||||
|
0, 0, 0.0, 0,
|
||||||
|
)
|
||||||
|
}
|
||||||
|
val sent = parseInt(reply.getOrNull(), "count") ?: count
|
||||||
|
|
||||||
|
val got = probe.collectGranted(collectWindowMs).filter { it.type == Wire.TYPE_DOWNTRAIN_DATA }
|
||||||
|
val received = got.size
|
||||||
|
val lossPct = if (sent == 0) 0.0 else (sent - received) * 100.0 / sent
|
||||||
|
|
||||||
|
// Reordering: a packet whose sequence is below the highest already seen. Counting
|
||||||
|
// inversions rather than "not sorted" keeps one late packet from being reported as
|
||||||
|
// dozens of reorder events.
|
||||||
|
var highest = -1
|
||||||
|
var reordered = 0
|
||||||
|
for (p in got) {
|
||||||
|
if (p.seq < highest) reordered++ else highest = p.seq
|
||||||
|
}
|
||||||
|
|
||||||
|
// Columnar evidence per the schema: what arrived, when, and how big — so every metric
|
||||||
|
// above is recomputable by a reader who does not trust our arithmetic.
|
||||||
|
val evidence = TrainEvidence(
|
||||||
|
epochMonoNs = started,
|
||||||
|
seq = got.map { it.seq },
|
||||||
|
tTxNs = got.map { null },
|
||||||
|
tRxNs = got.map { it.tRxNs },
|
||||||
|
sizeBytes = got.map { it.sizeBytes },
|
||||||
|
).toEvidence()
|
||||||
|
|
||||||
|
val interArrival = got.zipWithNext { a, b -> (b.tRxNs - a.tRxNs) / 1_000_000.0 }
|
||||||
|
val metrics = json.encodeToJsonElement(
|
||||||
|
DownTrainMetrics(
|
||||||
|
sent = sent, received = received, lossPct = round1(lossPct),
|
||||||
|
reorderedPackets = reordered,
|
||||||
|
sizeBytes = sizeBytes,
|
||||||
|
interArrivalMsAvg = interArrival.average().takeIf { interArrival.isNotEmpty() }?.let(::round1),
|
||||||
|
interArrivalMsMax = interArrival.maxOrNull()?.let(::round1),
|
||||||
|
sendIntervalUs = intervalUs,
|
||||||
|
),
|
||||||
|
) as JsonObject
|
||||||
|
|
||||||
|
val status = when {
|
||||||
|
received == 0 -> TestStatus.FAILED
|
||||||
|
received < sent -> TestStatus.PARTIAL
|
||||||
|
else -> TestStatus.OK
|
||||||
|
}
|
||||||
|
return TrainResult(
|
||||||
|
Test(
|
||||||
|
id = testId, type = TestType.TRAIN_UDP_DOWNSTREAM, sessionRef = sessionRef, tier = Tier.APP,
|
||||||
|
startedMonoNs = started, endedMonoNs = ids.monoNs(),
|
||||||
|
status = status, evidence = evidence, metrics = metrics,
|
||||||
|
),
|
||||||
|
sent, received, lossPct, reordered,
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
// ---- helpers ----------------------------------------------------------------------
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Builds a finding from a registry entry, which supplies the code, category and severity.
|
||||||
|
*
|
||||||
|
* Taking a [FindingSpec] rather than three loose values is the point: a typo becomes a
|
||||||
|
* compile error, and two call sites cannot disagree about which category a finding belongs
|
||||||
|
* to - a disagreement that would split one fault across two verdict lights.
|
||||||
|
*/
|
||||||
|
private fun finding(spec: FindingSpec, testId: String, title: String, desc: String) =
|
||||||
|
Finding(
|
||||||
|
id = ids.uuid(), code = spec.code, category = spec.category, severity = spec.severity,
|
||||||
|
confidence = Confidence.HIGH,
|
||||||
|
title = title, description = desc, evidenceRefs = listOf(EvidenceRef(testId)),
|
||||||
|
)
|
||||||
|
|
||||||
|
/** Minimal scalar extraction from the action reply; the shape is small and server-owned. */
|
||||||
|
private fun parseInt(body: String?, key: String): Int? =
|
||||||
|
body?.let { Regex("\"$key\"\\s*:\\s*(-?\\d+)").find(it)?.groupValues?.get(1)?.toIntOrNull() }
|
||||||
|
|
||||||
|
private fun parseIntArray(body: String?, key: String): List<Int> =
|
||||||
|
body?.let { b ->
|
||||||
|
Regex("\"$key\"\\s*:\\s*\\[([^\\]]*)\\]").find(b)?.groupValues?.get(1)
|
||||||
|
?.split(",")?.mapNotNull { it.trim().toIntOrNull() }
|
||||||
|
} ?: emptyList()
|
||||||
|
|
||||||
|
private companion object {
|
||||||
|
/** IPv4 (20) + UDP (8). The v6 case is 48; reported per-family once v6 sessions land. */
|
||||||
|
const val IP_UDP_OVERHEAD4 = 28
|
||||||
|
|
||||||
|
const val FRAG_IN_ORDER = "in_order"
|
||||||
|
const val FRAG_REVERSED = "reversed"
|
||||||
|
const val FRAG_FIRST_LAST = "first_last"
|
||||||
|
val FRAG_MODES = listOf(FRAG_IN_ORDER, FRAG_REVERSED, FRAG_FIRST_LAST)
|
||||||
|
|
||||||
|
/** Straddles the usual suspects: 1500 Ethernet, 1492 PPPoE, 1400-ish tunnels. */
|
||||||
|
val DEFAULT_SIZES = listOf(600, 1200, 1372, 1400, 1450, 1472, 1500, 2000, 4000)
|
||||||
|
|
||||||
|
fun round1(v: Double) = Math.round(v * 10.0) / 10.0
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Metrics for mtu.pmtud_down / mtu.frag_delivery. */
|
||||||
|
@Serializable
|
||||||
|
data class BigSendMetrics(
|
||||||
|
@SerialName("requested_bytes") val requestedBytes: List<Int>,
|
||||||
|
@SerialName("sent_bytes") val sentBytes: List<Int>,
|
||||||
|
@SerialName("delivered_bytes") val deliveredBytes: List<Int>,
|
||||||
|
@SerialName("largest_delivered_bytes") val largestDeliveredBytes: Int? = null,
|
||||||
|
@SerialName("dont_fragment") val dontFragment: Boolean,
|
||||||
|
/** The server's own DF ceiling; sizes above it were never sent and are not path evidence. */
|
||||||
|
@SerialName("server_max_df_bytes") val serverMaxDfBytes: Int? = null,
|
||||||
|
@SerialName("path_mtu_bytes") val pathMtuBytes: Int? = null,
|
||||||
|
)
|
||||||
|
|
||||||
|
/** Metrics for mtu.frag_ordering. */
|
||||||
|
@Serializable
|
||||||
|
data class FragOrderingMetrics(
|
||||||
|
@SerialName("size_bytes") val sizeBytes: Int,
|
||||||
|
@SerialName("frag_bytes") val fragBytes: Int,
|
||||||
|
@SerialName("fragments_per_burst") val fragmentsPerBurst: Map<String, Int>,
|
||||||
|
@SerialName("delivered_by_mode") val deliveredByMode: Map<String, Boolean>,
|
||||||
|
@SerialName("in_order_delivered") val inOrderDelivered: Boolean,
|
||||||
|
@SerialName("reordered_delivered") val reorderedDelivered: Boolean,
|
||||||
|
@SerialName("delayed_first_delivered") val delayedFirstDelivered: Boolean,
|
||||||
|
)
|
||||||
|
|
||||||
|
/** Metrics for train.udp_downstream. */
|
||||||
|
@Serializable
|
||||||
|
data class DownTrainMetrics(
|
||||||
|
val sent: Int,
|
||||||
|
val received: Int,
|
||||||
|
@SerialName("loss_pct") val lossPct: Double,
|
||||||
|
@SerialName("reordered_packets") val reorderedPackets: Int,
|
||||||
|
@SerialName("size_bytes") val sizeBytes: Int,
|
||||||
|
@SerialName("inter_arrival_ms_avg") val interArrivalMsAvg: Double? = null,
|
||||||
|
@SerialName("inter_arrival_ms_max") val interArrivalMsMax: Double? = null,
|
||||||
|
@SerialName("send_interval_us") val sendIntervalUs: Int,
|
||||||
|
)
|
||||||
@@ -10,8 +10,13 @@ import app.echo_lot.measurement.*
|
|||||||
import app.echo_lot.protocol.ControlClient
|
import app.echo_lot.protocol.ControlClient
|
||||||
import app.echo_lot.protocol.ProbeSession
|
import app.echo_lot.protocol.ProbeSession
|
||||||
import kotlinx.serialization.json.Json
|
import kotlinx.serialization.json.Json
|
||||||
|
import kotlinx.serialization.json.JsonArray
|
||||||
import kotlinx.serialization.json.JsonObject
|
import kotlinx.serialization.json.JsonObject
|
||||||
import kotlinx.serialization.json.encodeToJsonElement
|
import kotlinx.serialization.json.encodeToJsonElement
|
||||||
|
import kotlinx.serialization.json.intOrNull
|
||||||
|
import kotlinx.serialization.json.jsonObject
|
||||||
|
import kotlinx.serialization.json.jsonPrimitive
|
||||||
|
import kotlinx.serialization.json.longOrNull
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* Runs the server-facing measurements against one target and assembles a [MeasurementDocument]:
|
* Runs the server-facing measurements against one target and assembles a [MeasurementDocument]:
|
||||||
@@ -36,6 +41,20 @@ class ServerMeasurement(
|
|||||||
val udpPort: Int,
|
val udpPort: Int,
|
||||||
val echoCount: Int = 20,
|
val echoCount: Int = 20,
|
||||||
val echoPaddingBytes: Int = 64,
|
val echoPaddingBytes: Int = 64,
|
||||||
|
/**
|
||||||
|
* Whether to ask the server to push traffic back (downstream MTU and downstream train).
|
||||||
|
* Costs a few hundred kB of download and needs a server that advertises the grants, so
|
||||||
|
* it is a flag rather than an assumption.
|
||||||
|
*/
|
||||||
|
val downstream: Boolean = true,
|
||||||
|
/**
|
||||||
|
* Throughput moves real data — a 5-second run at 50 Mbps is about 30 MB — so it is off
|
||||||
|
* unless asked for. On a metered mobile connection that is the user's money, and a
|
||||||
|
* measurement tool that spends it without being told to is not one people keep installed.
|
||||||
|
*/
|
||||||
|
val throughput: Boolean = false,
|
||||||
|
@Suppress("unused") val throughputSeconds: Int = 5,
|
||||||
|
@Suppress("unused") val throughputKbps: Int = 50_000,
|
||||||
)
|
)
|
||||||
|
|
||||||
fun run(cfg: Config): MeasurementDocument {
|
fun run(cfg: Config): MeasurementDocument {
|
||||||
@@ -57,11 +76,41 @@ class ServerMeasurement(
|
|||||||
target = SessionTarget(ip4 = cfg.udpHost, udpPort = cfg.udpPort),
|
target = SessionTarget(ip4 = cfg.udpHost, udpPort = cfg.udpPort),
|
||||||
)
|
)
|
||||||
|
|
||||||
val (test, findings) = echoTrain(cfg, control, session, startMono)
|
val tests = ArrayList<Test>()
|
||||||
|
val allFindings = ArrayList<Finding>()
|
||||||
|
|
||||||
|
// One ProbeSession for the whole run. A second one would open a new socket and restart
|
||||||
|
// the sequence counter, which the server's anti-replay window correctly rejects — so the
|
||||||
|
// re-primed source is never recorded and every granted send goes to the old, closed port.
|
||||||
|
// Session identity lives on the server; the socket must live as long as it does.
|
||||||
|
ProbeSession(cfg.credential, session, cfg.udpHost, cfg.udpPort).use { ps ->
|
||||||
|
val (test, findings) = echoTrain(cfg, ps, startMono, control, session.sessionId)
|
||||||
|
tests.add(test)
|
||||||
|
allFindings.addAll(findings)
|
||||||
|
|
||||||
|
// Downstream needs a session the server has already seen traffic from — the echo
|
||||||
|
// train just provided that — and a server that advertises the grants. Skipped
|
||||||
|
// quietly against an older server rather than reported as a failure of the network.
|
||||||
|
if (cfg.downstream && profile.supports("downtrain") && profile.supports("big-send")) {
|
||||||
|
val (dsTests, dsFindings) = DownstreamMeasurement(ids)
|
||||||
|
.run(cfg.credential, session.sessionId, control, ps, sessionRef = "sess-1")
|
||||||
|
tests.addAll(dsTests)
|
||||||
|
allFindings.addAll(dsFindings)
|
||||||
|
}
|
||||||
|
|
||||||
|
if (cfg.throughput && profile.supports("throughput")) {
|
||||||
|
val (tpTest, tpFindings) = ThroughputMeasurement(ids).run(
|
||||||
|
cfg.credential, session.sessionId, control, ps, sessionRef = "sess-1",
|
||||||
|
durationS = cfg.throughputSeconds, kbps = cfg.throughputKbps,
|
||||||
|
)
|
||||||
|
tests.add(tpTest)
|
||||||
|
allFindings.addAll(tpFindings)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
control.deleteSession(cfg.credential, session.sessionId)
|
control.deleteSession(cfg.credential, session.sessionId)
|
||||||
|
|
||||||
val summary = Verdicts.derive(listOf(test), findings)
|
val summary = Verdicts.derive(tests, allFindings)
|
||||||
return MeasurementDocument(
|
return MeasurementDocument(
|
||||||
run = Run(
|
run = Run(
|
||||||
id = runId, trigger = Trigger.MANUAL, startedAt = startWall, endedAt = ids.nowWall(),
|
id = runId, trigger = Trigger.MANUAL, startedAt = startWall, endedAt = ids.nowWall(),
|
||||||
@@ -70,14 +119,15 @@ class ServerMeasurement(
|
|||||||
tiers = Tiers(app = true),
|
tiers = Tiers(app = true),
|
||||||
),
|
),
|
||||||
serverSessions = listOf(serverSession),
|
serverSessions = listOf(serverSession),
|
||||||
tests = listOf(test),
|
tests = tests,
|
||||||
findings = findings,
|
findings = allFindings,
|
||||||
summary = summary,
|
summary = summary,
|
||||||
)
|
)
|
||||||
}
|
}
|
||||||
|
|
||||||
private fun echoTrain(
|
private fun echoTrain(
|
||||||
cfg: Config, control: ControlClient, session: app.echo_lot.protocol.SessionResponse, startMono: Long,
|
cfg: Config, ps: ProbeSession, startMono: Long,
|
||||||
|
control: ControlClient? = null, sessionId: String? = null,
|
||||||
): Pair<Test, List<Finding>> {
|
): Pair<Test, List<Finding>> {
|
||||||
val testId = ids.uuid()
|
val testId = ids.uuid()
|
||||||
val seqs = ArrayList<Int>()
|
val seqs = ArrayList<Int>()
|
||||||
@@ -87,10 +137,15 @@ class ServerMeasurement(
|
|||||||
val rtts = ArrayList<Double>()
|
val rtts = ArrayList<Double>()
|
||||||
val observedPorts = LinkedHashSet<Int>()
|
val observedPorts = LinkedHashSet<Int>()
|
||||||
|
|
||||||
ProbeSession(cfg.credential, session, cfg.udpHost, cfg.udpPort).use { ps ->
|
// Wire sequence numbers, kept so the server's observations can be correlated packet by
|
||||||
|
// packet. They are not 0..n-1: the counter is shared with every other packet type on the
|
||||||
|
// session, so "the nth echo" is not "sequence n".
|
||||||
|
val wireSeqs = ArrayList<Int>()
|
||||||
|
|
||||||
for (i in 0 until cfg.echoCount) {
|
for (i in 0 until cfg.echoCount) {
|
||||||
val txMono = ids.monoNs() - startMono
|
val txMono = ids.monoNs() - startMono
|
||||||
val r = ps.echo(cfg.echoPaddingBytes)
|
val r = ps.echo(cfg.echoPaddingBytes)
|
||||||
|
wireSeqs.add(ps.lastSeq)
|
||||||
seqs.add(i)
|
seqs.add(i)
|
||||||
tTx.add(txMono)
|
tTx.add(txMono)
|
||||||
sizes.add(Wire_HEADER + cfg.echoPaddingBytes)
|
sizes.add(Wire_HEADER + cfg.echoPaddingBytes)
|
||||||
@@ -102,6 +157,19 @@ class ServerMeasurement(
|
|||||||
tRx.add(null)
|
tRx.add(null)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// Ask the server what it actually received. This is what turns "3 % loss somewhere" into
|
||||||
|
// "3 % loss upstream" - the least useful form of the answer into a usable one.
|
||||||
|
val directional: DirectionalMetrics? =
|
||||||
|
if (control != null && sessionId != null) {
|
||||||
|
runCatching {
|
||||||
|
val samples = wireSeqs.indices.map {
|
||||||
|
Directional.Sample(wireSeqs[it], tTx[it] ?: 0L, tRx[it])
|
||||||
|
}
|
||||||
|
Directional.analyse(samples, serverSightings(control, cfg, sessionId))
|
||||||
|
}.getOrNull() // an older server without the endpoint simply yields no split
|
||||||
|
} else {
|
||||||
|
null
|
||||||
}
|
}
|
||||||
|
|
||||||
val sent = cfg.echoCount
|
val sent = cfg.echoCount
|
||||||
@@ -113,6 +181,9 @@ class ServerMeasurement(
|
|||||||
epochMonoNs = startMono, seq = seqs, tTxNs = tTx, tRxNs = tRx, sizeBytes = sizes,
|
epochMonoNs = startMono, seq = seqs, tTxNs = tTx, tRxNs = tRx, sizeBytes = sizes,
|
||||||
).toEvidence()
|
).toEvidence()
|
||||||
|
|
||||||
|
val directionalJson = directional?.let {
|
||||||
|
json.encodeToJsonElement(DirectionalMetrics.serializer(), it) as JsonObject
|
||||||
|
}
|
||||||
val metrics: JsonObject = json.encodeToJsonElement(
|
val metrics: JsonObject = json.encodeToJsonElement(
|
||||||
EchoMetrics(
|
EchoMetrics(
|
||||||
sent = sent, received = received, lossPct = round1(lossPct),
|
sent = sent, received = received, lossPct = round1(lossPct),
|
||||||
@@ -122,7 +193,7 @@ class ServerMeasurement(
|
|||||||
observedPorts = observedPorts.toList(),
|
observedPorts = observedPorts.toList(),
|
||||||
natRebindingDetected = natRebinding,
|
natRebindingDetected = natRebinding,
|
||||||
)
|
)
|
||||||
) as JsonObject
|
).let { base -> JsonObject((base as JsonObject) + (directionalJson ?: JsonObject(emptyMap()))) }
|
||||||
|
|
||||||
val status = when {
|
val status = when {
|
||||||
received == 0 -> TestStatus.FAILED
|
received == 0 -> TestStatus.FAILED
|
||||||
@@ -137,30 +208,91 @@ class ServerMeasurement(
|
|||||||
|
|
||||||
val findings = ArrayList<Finding>()
|
val findings = ArrayList<Finding>()
|
||||||
if (received == 0) {
|
if (received == 0) {
|
||||||
findings.add(finding("nat.udp_unreachable", Category.CONNECTIVITY, Severity.HIGH, testId,
|
findings.add(finding(FindingRegistry.UDP_UNREACHABLE, testId,
|
||||||
"No UDP echo replies from the server",
|
"No UDP echo replies from the server",
|
||||||
"Every ECHO probe to the server's UDP data plane was lost — the path blocks or drops the session's UDP traffic."))
|
"Every ECHO probe to the server's UDP data plane was lost — the path blocks or drops the session's UDP traffic."))
|
||||||
} else if (lossPct >= 20.0) {
|
} else if (lossPct >= 20.0) {
|
||||||
findings.add(finding("connectivity.udp_loss", Category.CONNECTIVITY, Severity.MEDIUM, testId,
|
findings.add(finding(FindingRegistry.UDP_LOSS, testId,
|
||||||
"High UDP loss to the server (${round1(lossPct)}%)",
|
"High UDP loss to the server (${round1(lossPct)}%)",
|
||||||
"A large fraction of ECHO probes were lost, indicating an unreliable UDP path."))
|
"A large fraction of ECHO probes were lost, indicating an unreliable UDP path."))
|
||||||
}
|
}
|
||||||
|
// Naming the direction is the entire value of the split, so the findings do.
|
||||||
|
directional?.let { d ->
|
||||||
|
when {
|
||||||
|
d.noneReachedServer && received == 0 -> findings.add(
|
||||||
|
finding(FindingRegistry.UDP_UNREACHABLE_UPSTREAM, testId,
|
||||||
|
"Nothing reached the server",
|
||||||
|
"The server received none of the ${d.sent} probes, so the traffic is being " +
|
||||||
|
"dropped on the way out, not on the way back. A firewall or NAT on " +
|
||||||
|
"this side of the path is the place to look."),
|
||||||
|
)
|
||||||
|
d.lossUpstreamPct >= 2.0 -> findings.add(
|
||||||
|
finding(FindingRegistry.LOSS_UPSTREAM, testId,
|
||||||
|
"${d.lossUpstreamPct} % of probes were lost on the way to the server",
|
||||||
|
"${d.lostUpstream} of ${d.sent} probes never reached the server. The " +
|
||||||
|
"return path is not implicated: replies came back for everything that " +
|
||||||
|
"arrived."),
|
||||||
|
)
|
||||||
|
}
|
||||||
|
if (d.lossDownstreamPct >= 2.0) {
|
||||||
|
findings.add(
|
||||||
|
finding(FindingRegistry.LOSS_DOWNSTREAM, testId,
|
||||||
|
"${d.lossDownstreamPct} % of replies were lost on the way back",
|
||||||
|
"The server received ${d.seenByServer} probes and answered them, but " +
|
||||||
|
"${d.lostDownstream} of those replies never arrived. The outbound path " +
|
||||||
|
"is fine; the fault is on the return leg."),
|
||||||
|
)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
if (natRebinding) {
|
if (natRebinding) {
|
||||||
findings.add(finding("nat.udp_rebinding", Category.NAT, Severity.MEDIUM, testId,
|
findings.add(finding(FindingRegistry.NAT_UDP_REBINDING, testId,
|
||||||
"NAT remapped the UDP source port mid-flow",
|
"NAT remapped the UDP source port mid-flow",
|
||||||
"The server observed more than one source port for this session (${observedPorts.joinToString()}), i.e. a NAT with a short UDP mapping or per-packet remapping."))
|
"The server observed more than one source port for this session (${observedPorts.joinToString()}), i.e. a NAT with a short UDP mapping or per-packet remapping."))
|
||||||
}
|
}
|
||||||
return test to findings
|
return test to findings
|
||||||
}
|
}
|
||||||
|
|
||||||
private fun finding(code: String, cat: Category, sev: Severity, testId: String, title: String, desc: String) =
|
/**
|
||||||
|
* The server's per-packet record of this session's echoes (spec section 6). Filtered to
|
||||||
|
* ECHO_REQ, because the observation list also holds MTU probes and anything else we sent -
|
||||||
|
* counting those as train packets would invent loss that is not there.
|
||||||
|
*/
|
||||||
|
private fun serverSightings(
|
||||||
|
control: ControlClient, cfg: Config, sessionId: String,
|
||||||
|
): List<Directional.ServerSighting> {
|
||||||
|
val body = control.observations(cfg.credential, sessionId)
|
||||||
|
val packets = Json.parseToJsonElement(body).jsonObject["udp"]
|
||||||
|
?.jsonObject?.get("packets") as? JsonArray ?: return emptyList()
|
||||||
|
return packets.mapNotNull { el ->
|
||||||
|
val o = el as? JsonObject ?: return@mapNotNull null
|
||||||
|
val type = o["type"]?.jsonPrimitive?.intOrNull ?: return@mapNotNull null
|
||||||
|
if (type != ECHO_REQ_TYPE) return@mapNotNull null
|
||||||
|
Directional.ServerSighting(
|
||||||
|
seq = o["seq"]?.jsonPrimitive?.intOrNull ?: return@mapNotNull null,
|
||||||
|
tRxNs = o["t_rx_ns"]?.jsonPrimitive?.longOrNull ?: return@mapNotNull null,
|
||||||
|
tTxNs = o["t_tx_ns"]?.jsonPrimitive?.longOrNull ?: return@mapNotNull null,
|
||||||
|
)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Builds a finding from a registry entry, which supplies the code, category and severity.
|
||||||
|
*
|
||||||
|
* Taking a [FindingSpec] rather than three loose values is the point: a typo becomes a
|
||||||
|
* compile error, and two call sites cannot disagree about which category a finding belongs
|
||||||
|
* to - a disagreement that would split one fault across two verdict lights.
|
||||||
|
*/
|
||||||
|
private fun finding(spec: FindingSpec, testId: String, title: String, desc: String) =
|
||||||
Finding(
|
Finding(
|
||||||
id = ids.uuid(), code = code, category = cat, severity = sev, confidence = Confidence.HIGH,
|
id = ids.uuid(), code = spec.code, category = spec.category, severity = spec.severity,
|
||||||
|
confidence = Confidence.HIGH,
|
||||||
title = title, description = desc, evidenceRefs = listOf(EvidenceRef(testId)),
|
title = title, description = desc, evidenceRefs = listOf(EvidenceRef(testId)),
|
||||||
)
|
)
|
||||||
|
|
||||||
private companion object {
|
private companion object {
|
||||||
const val Wire_HEADER = 32
|
const val Wire_HEADER = 32
|
||||||
|
const val ECHO_REQ_TYPE = 0x01
|
||||||
fun round1(v: Double) = Math.round(v * 10.0) / 10.0
|
fun round1(v: Double) = Math.round(v * 10.0) / 10.0
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -0,0 +1,338 @@
|
|||||||
|
// SPDX-FileCopyrightText: 2026 Echolot contributors
|
||||||
|
// SPDX-License-Identifier: GPL-3.0-or-later
|
||||||
|
|
||||||
|
package app.echo_lot.engine
|
||||||
|
|
||||||
|
import app.echo_lot.measurement.*
|
||||||
|
import app.echo_lot.protocol.ControlClient
|
||||||
|
import app.echo_lot.protocol.ProbeSession
|
||||||
|
import app.echo_lot.protocol.Wire
|
||||||
|
import kotlinx.serialization.SerialName
|
||||||
|
import kotlinx.serialization.Serializable
|
||||||
|
import kotlinx.serialization.json.Json
|
||||||
|
import kotlinx.serialization.json.JsonObject
|
||||||
|
import kotlinx.serialization.json.encodeToJsonElement
|
||||||
|
import kotlinx.serialization.json.jsonArray
|
||||||
|
import kotlinx.serialization.json.jsonObject
|
||||||
|
import kotlinx.serialization.json.jsonPrimitive
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Downstream throughput: the server sends at a paced rate for a bounded time and the client
|
||||||
|
* measures what arrives (`perf.throughput_udp`).
|
||||||
|
*
|
||||||
|
* The number this produces is only meaningful with a qualifier attached, and getting that
|
||||||
|
* qualifier right is most of the work here. A throughput test reports the *smallest* limit on the
|
||||||
|
* path, and the sender's own ceiling is one of the candidates: if the server was asked for 50 Mbps
|
||||||
|
* and 50 Mbps arrived, the network was never the constraint and "50 Mbps" says nothing about it.
|
||||||
|
* Reporting that as a capacity measurement would be a confident lie, so the result always carries
|
||||||
|
* [ThroughputMetrics.limitedBy] and a finding is only raised when the network is actually
|
||||||
|
* implicated.
|
||||||
|
*
|
||||||
|
* Comparing against the *sender's* count rather than the requested rate is the other half: the
|
||||||
|
* server reports how much it actually put on the wire, and the gap between that and what arrived
|
||||||
|
* is the loss. A receiver alone cannot tell "the network dropped it" from "the sender never sent
|
||||||
|
* it", and guessing turns a healthy server-side limit into a phantom network fault.
|
||||||
|
*/
|
||||||
|
class ThroughputMeasurement(private val ids: IdSource) {
|
||||||
|
|
||||||
|
private val json = Json { encodeDefaults = true; explicitNulls = true }
|
||||||
|
|
||||||
|
fun run(
|
||||||
|
credential: String,
|
||||||
|
sessionId: String,
|
||||||
|
control: ControlClient,
|
||||||
|
probe: ProbeSession,
|
||||||
|
sessionRef: String,
|
||||||
|
durationS: Int = 5,
|
||||||
|
kbps: Int = 50_000,
|
||||||
|
sizeBytes: Int = 1200,
|
||||||
|
): Pair<Test, List<Finding>> {
|
||||||
|
val testId = ids.uuid()
|
||||||
|
val started = ids.monoNs()
|
||||||
|
|
||||||
|
val reply = runCatching {
|
||||||
|
control.action(
|
||||||
|
credential, sessionId,
|
||||||
|
"""{"action":"throughput","direction":"down","duration_s":$durationS,""" +
|
||||||
|
""""kbps":$kbps,"size_bytes":$sizeBytes}""",
|
||||||
|
)
|
||||||
|
}
|
||||||
|
if (reply.isFailure) {
|
||||||
|
return Test(
|
||||||
|
id = testId, type = TestType.PERF_THROUGHPUT_UDP, sessionRef = sessionRef, tier = Tier.APP,
|
||||||
|
startedMonoNs = started, endedMonoNs = ids.monoNs(),
|
||||||
|
status = TestStatus.UNSUPPORTED,
|
||||||
|
error = TestError("action_refused", reply.exceptionOrNull()?.message ?: "throughput refused"),
|
||||||
|
) to emptyList()
|
||||||
|
}
|
||||||
|
|
||||||
|
// The server may have shortened the run to fit its own byte budget; listen for what it
|
||||||
|
// actually promised, not for what we asked.
|
||||||
|
val plannedMs = parseInt(reply.getOrNull(), "duration_ms") ?: (durationS * 1000)
|
||||||
|
|
||||||
|
// A margin past the planned end so the tail of the run is not counted as loss: packets
|
||||||
|
// still in flight when we stop listening were not dropped, they were merely late.
|
||||||
|
val received = probe.collectGranted(plannedMs + 1_500L)
|
||||||
|
.filter { it.type == Wire.TYPE_THROUGHPUT_DATA }
|
||||||
|
|
||||||
|
val bytes = received.sumOf { it.sizeBytes.toLong() }
|
||||||
|
val spanNs = if (received.size >= 2) {
|
||||||
|
received.maxOf { it.tRxNs } - received.minOf { it.tRxNs }
|
||||||
|
} else {
|
||||||
|
0L
|
||||||
|
}
|
||||||
|
// Measured over the arrival span rather than our listening window, which includes the
|
||||||
|
// request round trip and the trailing margin and would understate the rate.
|
||||||
|
val receivedKbps = if (spanNs > 0) (bytes * 8 * 1_000_000 / spanNs).toInt() else 0
|
||||||
|
|
||||||
|
val sender = senderReport(control, credential, sessionId)
|
||||||
|
val sentPackets = sender?.packets ?: 0
|
||||||
|
val lossPct = if (sentPackets > 0) {
|
||||||
|
round2((sentPackets - received.size).coerceAtLeast(0) * 100.0 / sentPackets)
|
||||||
|
} else {
|
||||||
|
null
|
||||||
|
}
|
||||||
|
|
||||||
|
// Only a run the *clock* ended measured the network. One stopped by our own byte budget
|
||||||
|
// or rate ceiling measured this server.
|
||||||
|
val limitedBy = sender?.limitedBy ?: "unknown"
|
||||||
|
val networkLimited = limitedBy == "duration" &&
|
||||||
|
sender != null && receivedKbps > 0 && receivedKbps < sender.kbps * 9 / 10
|
||||||
|
|
||||||
|
val metrics = json.encodeToJsonElement(
|
||||||
|
ThroughputMetrics(
|
||||||
|
requestedKbps = kbps,
|
||||||
|
plannedDurationMs = plannedMs,
|
||||||
|
packetsReceived = received.size,
|
||||||
|
bytesReceived = bytes,
|
||||||
|
receivedKbps = receivedKbps,
|
||||||
|
senderPackets = sender?.packets,
|
||||||
|
senderBytes = sender?.bytes,
|
||||||
|
senderKbps = sender?.kbps,
|
||||||
|
lossPct = lossPct,
|
||||||
|
limitedBy = limitedBy,
|
||||||
|
measuresNetwork = networkLimited,
|
||||||
|
),
|
||||||
|
) as JsonObject
|
||||||
|
|
||||||
|
val findings = ArrayList<Finding>()
|
||||||
|
when {
|
||||||
|
sender == null -> Unit // no sender report: nothing can be concluded, so nothing is
|
||||||
|
received.isEmpty() -> findings.add(
|
||||||
|
finding(
|
||||||
|
FindingRegistry.THROUGHPUT_NO_DELIVERY, testId,
|
||||||
|
"No throughput traffic arrived",
|
||||||
|
"The server sent ${sender.packets} packets and none arrived. This is a " +
|
||||||
|
"connectivity fault rather than a slow link.",
|
||||||
|
),
|
||||||
|
)
|
||||||
|
networkLimited -> findings.add(
|
||||||
|
finding(
|
||||||
|
FindingRegistry.THROUGHPUT_BELOW_OFFERED, testId,
|
||||||
|
"Downstream throughput ${receivedKbps / 1000} Mbit/s, below the " +
|
||||||
|
"${sender.kbps / 1000} Mbit/s offered",
|
||||||
|
"The server sent at ${sender.kbps / 1000} Mbit/s for the full run and " +
|
||||||
|
"${receivedKbps / 1000} Mbit/s arrived" +
|
||||||
|
(lossPct?.let { ", losing $it % of packets" } ?: "") +
|
||||||
|
". The path could not carry what was offered.",
|
||||||
|
),
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
return Test(
|
||||||
|
id = testId, type = TestType.PERF_THROUGHPUT_UDP, sessionRef = sessionRef, tier = Tier.APP,
|
||||||
|
startedMonoNs = started, endedMonoNs = ids.monoNs(),
|
||||||
|
status = if (received.isEmpty()) TestStatus.FAILED else TestStatus.OK,
|
||||||
|
metrics = metrics,
|
||||||
|
) to findings
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Upstream throughput: the client sends, the server counts.
|
||||||
|
*
|
||||||
|
* The mirror image of the downstream case, and it needs no grant — the client is generating
|
||||||
|
* its own traffic, so there is no amplification to gate. What it does need is the server's
|
||||||
|
* count: only the far end knows how much arrived, and without that number a sender can
|
||||||
|
* measure how fast it can *transmit*, which is not the same question and is usually just the
|
||||||
|
* speed of the local NIC.
|
||||||
|
*/
|
||||||
|
fun runUpstream(
|
||||||
|
credential: String,
|
||||||
|
sessionId: String,
|
||||||
|
control: ControlClient,
|
||||||
|
probe: ProbeSession,
|
||||||
|
sessionRef: String,
|
||||||
|
durationS: Int = 5,
|
||||||
|
kbps: Int = 20_000,
|
||||||
|
sizeBytes: Int = 1200,
|
||||||
|
): Pair<Test, List<Finding>> {
|
||||||
|
val testId = ids.uuid()
|
||||||
|
val started = ids.monoNs()
|
||||||
|
|
||||||
|
// Zeroes the server's counter so this run measures itself rather than inheriting the
|
||||||
|
// packets of an earlier one on the same session.
|
||||||
|
val reply = runCatching {
|
||||||
|
control.action(credential, sessionId, """{"action":"throughput","direction":"up"}""")
|
||||||
|
}
|
||||||
|
if (reply.isFailure) {
|
||||||
|
return Test(
|
||||||
|
id = testId, type = TestType.PERF_THROUGHPUT_UDP, sessionRef = sessionRef, tier = Tier.APP,
|
||||||
|
startedMonoNs = started, endedMonoNs = ids.monoNs(),
|
||||||
|
status = TestStatus.UNSUPPORTED,
|
||||||
|
error = TestError("action_refused", reply.exceptionOrNull()?.message ?: "refused"),
|
||||||
|
) to emptyList()
|
||||||
|
}
|
||||||
|
|
||||||
|
val sent = probe.sendThroughput(durationS * 1000L, kbps, sizeBytes)
|
||||||
|
// A moment for the tail of the run to arrive; counting still-in-flight packets as lost
|
||||||
|
// would inflate the loss figure by whatever the path's delay happens to be.
|
||||||
|
Thread.sleep(500)
|
||||||
|
val seen = upstreamCount(control, credential, sessionId)
|
||||||
|
|
||||||
|
val lossPct = if (sent.packets > 0 && seen != null) {
|
||||||
|
round2((sent.packets - seen.packets).coerceAtLeast(0) * 100.0 / sent.packets)
|
||||||
|
} else {
|
||||||
|
null
|
||||||
|
}
|
||||||
|
// The receiver's rate is the measurement. The sender's is what we managed to emit, which
|
||||||
|
// is a property of this phone and its radio, not of the network.
|
||||||
|
val achievedKbps = seen?.kbps ?: 0
|
||||||
|
|
||||||
|
val metrics = json.encodeToJsonElement(
|
||||||
|
UpstreamThroughputMetrics(
|
||||||
|
requestedKbps = kbps,
|
||||||
|
sentPackets = sent.packets,
|
||||||
|
sentBytes = sent.bytes,
|
||||||
|
sentKbps = sent.kbps,
|
||||||
|
receivedPackets = seen?.packets,
|
||||||
|
receivedBytes = seen?.bytes,
|
||||||
|
receivedKbps = achievedKbps,
|
||||||
|
lossPct = lossPct,
|
||||||
|
// Same honesty rule as downstream: if what arrived matches what we offered, the
|
||||||
|
// path was never the constraint and this number says nothing about it.
|
||||||
|
measuresNetwork = seen != null && achievedKbps > 0 && achievedKbps < sent.kbps * 9 / 10,
|
||||||
|
),
|
||||||
|
) as JsonObject
|
||||||
|
|
||||||
|
val findings = ArrayList<Finding>()
|
||||||
|
if (seen != null && seen.packets == 0 && sent.packets > 0) {
|
||||||
|
findings.add(
|
||||||
|
finding(
|
||||||
|
FindingRegistry.THROUGHPUT_NO_DELIVERY, testId,
|
||||||
|
"No upstream traffic reached the server",
|
||||||
|
"This device sent ${sent.packets} packets and the server received none. " +
|
||||||
|
"That is a connectivity fault on the outbound path rather than a slow link.",
|
||||||
|
),
|
||||||
|
)
|
||||||
|
} else if (lossPct != null && lossPct >= 2.0) {
|
||||||
|
findings.add(
|
||||||
|
finding(
|
||||||
|
FindingRegistry.THROUGHPUT_BELOW_OFFERED, testId,
|
||||||
|
"Upstream loss of $lossPct % at ${sent.kbps / 1000} Mbit/s",
|
||||||
|
"The server received ${seen?.packets} of the ${sent.packets} packets this " +
|
||||||
|
"device sent. The outbound path could not carry what was offered.",
|
||||||
|
),
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
return Test(
|
||||||
|
id = testId, type = TestType.PERF_THROUGHPUT_UDP, sessionRef = sessionRef, tier = Tier.APP,
|
||||||
|
startedMonoNs = started, endedMonoNs = ids.monoNs(),
|
||||||
|
status = if (seen == null || seen.packets == 0) TestStatus.FAILED else TestStatus.OK,
|
||||||
|
metrics = metrics,
|
||||||
|
) to findings
|
||||||
|
}
|
||||||
|
|
||||||
|
private data class UpstreamCount(val packets: Int, val bytes: Long, val kbps: Int)
|
||||||
|
|
||||||
|
/** The server's tally for this session's upstream run. */
|
||||||
|
private fun upstreamCount(
|
||||||
|
control: ControlClient, credential: String, sessionId: String,
|
||||||
|
): UpstreamCount? = runCatching {
|
||||||
|
val o = Json.parseToJsonElement(control.observations(credential, sessionId))
|
||||||
|
.jsonObject["throughput_up"]?.jsonObject ?: return null
|
||||||
|
UpstreamCount(
|
||||||
|
packets = o["packets"]?.jsonPrimitive?.content?.toIntOrNull() ?: 0,
|
||||||
|
bytes = o["bytes"]?.jsonPrimitive?.content?.toLongOrNull() ?: 0,
|
||||||
|
kbps = o["kbps"]?.jsonPrimitive?.content?.toIntOrNull() ?: 0,
|
||||||
|
)
|
||||||
|
}.getOrNull()
|
||||||
|
|
||||||
|
private data class SenderReport(
|
||||||
|
val packets: Int, val bytes: Long, val kbps: Int, val limitedBy: String,
|
||||||
|
)
|
||||||
|
|
||||||
|
/** The server's own account of the run, from the observations API. */
|
||||||
|
private fun senderReport(
|
||||||
|
control: ControlClient, credential: String, sessionId: String,
|
||||||
|
): SenderReport? = runCatching {
|
||||||
|
val arr = Json.parseToJsonElement(control.observations(credential, sessionId))
|
||||||
|
.jsonObject["throughput"]?.jsonArray ?: return null
|
||||||
|
val last = arr.lastOrNull()?.jsonObject ?: return null
|
||||||
|
SenderReport(
|
||||||
|
packets = last["packets"]?.jsonPrimitive?.content?.toIntOrNull() ?: 0,
|
||||||
|
bytes = last["bytes"]?.jsonPrimitive?.content?.toLongOrNull() ?: 0,
|
||||||
|
kbps = last["kbps"]?.jsonPrimitive?.content?.toIntOrNull() ?: 0,
|
||||||
|
limitedBy = last["limited_by"]?.jsonPrimitive?.content ?: "unknown",
|
||||||
|
)
|
||||||
|
}.getOrNull()
|
||||||
|
|
||||||
|
private fun parseInt(body: String?, key: String): Int? =
|
||||||
|
body?.let { Regex("\"$key\"\\s*:\\s*(-?\\d+)").find(it)?.groupValues?.get(1)?.toIntOrNull() }
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Builds a finding from a registry entry, which supplies the code, category and severity.
|
||||||
|
*
|
||||||
|
* Taking a [FindingSpec] rather than three loose values is the point: a typo becomes a
|
||||||
|
* compile error, and two call sites cannot disagree about which category a finding belongs
|
||||||
|
* to - a disagreement that would split one fault across two verdict lights.
|
||||||
|
*/
|
||||||
|
private fun finding(spec: FindingSpec, testId: String, title: String, desc: String) =
|
||||||
|
Finding(
|
||||||
|
id = ids.uuid(), code = spec.code, category = spec.category, severity = spec.severity,
|
||||||
|
confidence = Confidence.HIGH,
|
||||||
|
title = title, description = desc, evidenceRefs = listOf(EvidenceRef(testId)),
|
||||||
|
)
|
||||||
|
|
||||||
|
private fun round2(v: Double) = Math.round(v * 100.0) / 100.0
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Metrics for perf.throughput_udp in the upstream direction. */
|
||||||
|
@Serializable
|
||||||
|
data class UpstreamThroughputMetrics(
|
||||||
|
val direction: String = "up",
|
||||||
|
@SerialName("requested_kbps") val requestedKbps: Int,
|
||||||
|
@SerialName("sent_packets") val sentPackets: Int,
|
||||||
|
@SerialName("sent_bytes") val sentBytes: Long,
|
||||||
|
/** What this device managed to emit — a property of the phone and its radio, not the path. */
|
||||||
|
@SerialName("sent_kbps") val sentKbps: Int,
|
||||||
|
@SerialName("received_packets") val receivedPackets: Int? = null,
|
||||||
|
@SerialName("received_bytes") val receivedBytes: Long? = null,
|
||||||
|
/** What arrived, measured by the only party that can measure it. This is the result. */
|
||||||
|
@SerialName("received_kbps") val receivedKbps: Int,
|
||||||
|
@SerialName("loss_pct") val lossPct: Double? = null,
|
||||||
|
@SerialName("measures_network") val measuresNetwork: Boolean,
|
||||||
|
)
|
||||||
|
|
||||||
|
/** Metrics for perf.throughput_udp. */
|
||||||
|
@Serializable
|
||||||
|
data class ThroughputMetrics(
|
||||||
|
val direction: String = "down",
|
||||||
|
@SerialName("requested_kbps") val requestedKbps: Int,
|
||||||
|
@SerialName("planned_duration_ms") val plannedDurationMs: Int,
|
||||||
|
@SerialName("packets_received") val packetsReceived: Int,
|
||||||
|
@SerialName("bytes_received") val bytesReceived: Long,
|
||||||
|
@SerialName("received_kbps") val receivedKbps: Int,
|
||||||
|
@SerialName("sender_packets") val senderPackets: Int? = null,
|
||||||
|
@SerialName("sender_bytes") val senderBytes: Long? = null,
|
||||||
|
@SerialName("sender_kbps") val senderKbps: Int? = null,
|
||||||
|
/** Against the sender's count, so a server-side limit is never counted as network loss. */
|
||||||
|
@SerialName("loss_pct") val lossPct: Double? = null,
|
||||||
|
/** What ended the run: duration | budget | rate | send_error | unknown. */
|
||||||
|
@SerialName("limited_by") val limitedBy: String,
|
||||||
|
/**
|
||||||
|
* Whether this number says anything about the network. False when the sender's own ceiling
|
||||||
|
* was the binding constraint — in which case the rate is a property of the test, not the path.
|
||||||
|
*/
|
||||||
|
@SerialName("measures_network") val measuresNetwork: Boolean,
|
||||||
|
)
|
||||||
@@ -0,0 +1,155 @@
|
|||||||
|
// SPDX-FileCopyrightText: 2026 Echolot contributors
|
||||||
|
// SPDX-License-Identifier: GPL-3.0-or-later
|
||||||
|
|
||||||
|
package app.echo_lot.engine
|
||||||
|
|
||||||
|
import app.echo_lot.engine.Directional.Sample
|
||||||
|
import app.echo_lot.engine.Directional.ServerSighting
|
||||||
|
import kotlin.test.Test
|
||||||
|
import kotlin.test.assertEquals
|
||||||
|
import kotlin.test.assertFalse
|
||||||
|
import kotlin.test.assertNotNull
|
||||||
|
import kotlin.test.assertNull
|
||||||
|
import kotlin.test.assertTrue
|
||||||
|
|
||||||
|
/**
|
||||||
|
* The arithmetic that turns "3 % loss somewhere" into "3 % loss upstream". Getting a denominator
|
||||||
|
* wrong here does not crash anything — it produces a plausible number pointing at the wrong half
|
||||||
|
* of the network, which is worse than no number at all. Hence a test per claim.
|
||||||
|
*/
|
||||||
|
class DirectionalTest {
|
||||||
|
|
||||||
|
/** A clean train: every packet sent, seen and answered. Server clock offset by a constant. */
|
||||||
|
private fun clean(n: Int, offsetNs: Long = 5_000_000_000L): Pair<List<Sample>, List<ServerSighting>> {
|
||||||
|
val sent = (1..n).map { Sample(it, tTxNs = it * 10_000_000L, tRxNs = it * 10_000_000L + 4_000_000L) }
|
||||||
|
val seen = (1..n).map {
|
||||||
|
ServerSighting(it, tRxNs = offsetNs + it * 10_000_000L + 2_000_000L,
|
||||||
|
tTxNs = offsetNs + it * 10_000_000L + 2_100_000L)
|
||||||
|
}
|
||||||
|
return sent to seen
|
||||||
|
}
|
||||||
|
|
||||||
|
@Test
|
||||||
|
fun aCleanTrainReportsNoLossInEitherDirection() {
|
||||||
|
val (sent, seen) = clean(10)
|
||||||
|
val m = Directional.analyse(sent, seen)
|
||||||
|
assertEquals(10, m.sent)
|
||||||
|
assertEquals(10, m.seenByServer)
|
||||||
|
assertEquals(10, m.repliesReceived)
|
||||||
|
assertEquals(0.0, m.lossUpstreamPct)
|
||||||
|
assertEquals(0.0, m.lossDownstreamPct)
|
||||||
|
assertFalse(m.noneReachedServer)
|
||||||
|
}
|
||||||
|
|
||||||
|
// The whole point: a packet the server never saw was lost on the way there.
|
||||||
|
@Test
|
||||||
|
fun packetsTheServerNeverSawAreUpstreamLoss() {
|
||||||
|
val (sent, seen) = clean(10)
|
||||||
|
val m = Directional.analyse(sent, seen.filter { it.seq !in setOf(3, 7) })
|
||||||
|
assertEquals(2, m.lostUpstream)
|
||||||
|
assertEquals(0, m.lostDownstream)
|
||||||
|
assertEquals(20.0, m.lossUpstreamPct)
|
||||||
|
assertEquals(0.0, m.lossDownstreamPct, "a packet that never arrived cannot be lost coming back")
|
||||||
|
}
|
||||||
|
|
||||||
|
@Test
|
||||||
|
fun repliesThatNeverArrivedAreDownstreamLoss() {
|
||||||
|
val (sent, seen) = clean(10)
|
||||||
|
val withHoles = sent.map { if (it.seq in setOf(2, 5)) it.copy(tRxNs = null) else it }
|
||||||
|
val m = Directional.analyse(withHoles, seen)
|
||||||
|
assertEquals(0, m.lostUpstream)
|
||||||
|
assertEquals(2, m.lostDownstream)
|
||||||
|
assertEquals(20.0, m.lossDownstreamPct)
|
||||||
|
}
|
||||||
|
|
||||||
|
// Downstream loss is measured against what actually reached the server. Using "sent" as the
|
||||||
|
// denominator would count every upstream loss a second time and overstate the return path.
|
||||||
|
@Test
|
||||||
|
fun downstreamLossIsRelativeToWhatReachedTheServer() {
|
||||||
|
val (sent, seen) = clean(10)
|
||||||
|
// 5 lost on the way there; of the 5 that arrived, 1 reply is lost coming back.
|
||||||
|
val seenPartial = seen.filter { it.seq > 5 }
|
||||||
|
val withHole = sent.map {
|
||||||
|
when {
|
||||||
|
it.seq <= 5 -> it.copy(tRxNs = null) // never got there, so never came back
|
||||||
|
it.seq == 6 -> it.copy(tRxNs = null) // arrived, reply lost
|
||||||
|
else -> it
|
||||||
|
}
|
||||||
|
}
|
||||||
|
val m = Directional.analyse(withHole, seenPartial)
|
||||||
|
assertEquals(5, m.lostUpstream)
|
||||||
|
assertEquals(50.0, m.lossUpstreamPct)
|
||||||
|
assertEquals(1, m.lostDownstream)
|
||||||
|
assertEquals(20.0, m.lossDownstreamPct, "1 of the 5 that arrived, not 1 of 10")
|
||||||
|
}
|
||||||
|
|
||||||
|
@Test
|
||||||
|
fun aServerThatSawNothingIsCalledOutSeparately() {
|
||||||
|
val (sent, _) = clean(6)
|
||||||
|
val m = Directional.analyse(sent.map { it.copy(tRxNs = null) }, emptyList())
|
||||||
|
assertTrue(m.noneReachedServer)
|
||||||
|
assertEquals(100.0, m.lossUpstreamPct)
|
||||||
|
assertEquals(0.0, m.lossDownstreamPct, "with nothing arriving there is no return path to blame")
|
||||||
|
}
|
||||||
|
|
||||||
|
// Jitter is legitimate without synchronised clocks because the offset cancels when successive
|
||||||
|
// one-way samples are differenced. This pins that: a huge constant offset must not show up.
|
||||||
|
@Test
|
||||||
|
fun jitterIsUnaffectedByTheClockOffsetBetweenTheTwoMachines() {
|
||||||
|
val (sent, near) = clean(10, offsetNs = 0)
|
||||||
|
val (_, far) = clean(10, offsetNs = 9_999_999_999L)
|
||||||
|
val a = Directional.analyse(sent, near)
|
||||||
|
val b = Directional.analyse(sent, far)
|
||||||
|
assertEquals(a.jitterUpstreamMs, b.jitterUpstreamMs,
|
||||||
|
"a constant clock offset must cancel when consecutive samples are differenced")
|
||||||
|
assertEquals(0.0, assertNotNull(a.jitterUpstreamMs), "an evenly spaced train has no jitter")
|
||||||
|
}
|
||||||
|
|
||||||
|
@Test
|
||||||
|
fun jitterReflectsUnevenArrival() {
|
||||||
|
val sent = listOf(
|
||||||
|
Sample(1, 0, 10_000_000),
|
||||||
|
Sample(2, 10_000_000, 20_000_000),
|
||||||
|
Sample(3, 20_000_000, 30_000_000),
|
||||||
|
)
|
||||||
|
// Server receive times drift: +2ms, +7ms, +3ms relative to send.
|
||||||
|
val seen = listOf(
|
||||||
|
ServerSighting(1, 2_000_000, 2_100_000),
|
||||||
|
ServerSighting(2, 17_000_000, 17_100_000),
|
||||||
|
ServerSighting(3, 23_000_000, 23_100_000),
|
||||||
|
)
|
||||||
|
val m = Directional.analyse(sent, seen)
|
||||||
|
// one-way samples: 2ms, 7ms, 3ms → |7-2| and |3-7| → mean 4.5ms
|
||||||
|
assertEquals(4.5, assertNotNull(m.jitterUpstreamMs))
|
||||||
|
}
|
||||||
|
|
||||||
|
// "No jitter" and "not enough data to say" are different claims, and only one is true here.
|
||||||
|
@Test
|
||||||
|
fun tooFewSamplesReportsNoJitterRatherThanZero() {
|
||||||
|
val m = Directional.analyse(
|
||||||
|
listOf(Sample(1, 0, 10_000_000)),
|
||||||
|
listOf(ServerSighting(1, 2_000_000, 2_100_000)),
|
||||||
|
)
|
||||||
|
assertNull(m.jitterUpstreamMs)
|
||||||
|
assertNull(m.jitterDownstreamMs)
|
||||||
|
}
|
||||||
|
|
||||||
|
// A server record for a sequence we never sent is not evidence about this train; folding it
|
||||||
|
// in would yield loss percentages outside 0–100.
|
||||||
|
@Test
|
||||||
|
fun strayServerRecordsAreIgnored() {
|
||||||
|
val (sent, seen) = clean(5)
|
||||||
|
val m = Directional.analyse(sent, seen + ServerSighting(99, 1, 2) + ServerSighting(100, 3, 4))
|
||||||
|
assertEquals(5, m.seenByServer)
|
||||||
|
assertEquals(0.0, m.lossUpstreamPct)
|
||||||
|
assertTrue(m.lossDownstreamPct in 0.0..100.0)
|
||||||
|
}
|
||||||
|
|
||||||
|
@Test
|
||||||
|
fun anEmptyTrainDoesNotDivideByZero() {
|
||||||
|
val m = Directional.analyse(emptyList(), emptyList())
|
||||||
|
assertEquals(0.0, m.lossUpstreamPct)
|
||||||
|
assertEquals(0.0, m.lossDownstreamPct)
|
||||||
|
assertFalse(m.noneReachedServer, "nothing sent is not the same as nothing arriving")
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,71 @@
|
|||||||
|
// SPDX-FileCopyrightText: 2026 Echolot contributors
|
||||||
|
// SPDX-License-Identifier: GPL-3.0-or-later
|
||||||
|
|
||||||
|
package app.echo_lot.engine
|
||||||
|
|
||||||
|
import app.echo_lot.protocol.Compat
|
||||||
|
import app.echo_lot.protocol.ControlClient
|
||||||
|
import app.echo_lot.protocol.VersionRefused
|
||||||
|
import kotlin.test.Test
|
||||||
|
import kotlin.test.assertEquals
|
||||||
|
import kotlin.test.assertNotNull
|
||||||
|
import kotlin.test.assertTrue
|
||||||
|
import kotlin.test.fail
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Checks the version gate against a LIVE server — the half that unit tests cannot reach, because
|
||||||
|
* the whole point is that two independently-built artifacts agree. Self-skips without
|
||||||
|
* ECHOLOT_LIVE_*.
|
||||||
|
*/
|
||||||
|
class LiveCompatTest {
|
||||||
|
|
||||||
|
private val url = System.getenv("ECHOLOT_LIVE_URL")
|
||||||
|
private val pin = System.getenv("ECHOLOT_LIVE_PIN")
|
||||||
|
private val cred = System.getenv("ECHOLOT_LIVE_CRED")
|
||||||
|
|
||||||
|
private fun clientAs(version: String) = ControlClient(url!!, setOf(pin!!), version)
|
||||||
|
|
||||||
|
@Test
|
||||||
|
fun theServerAdvertisesAndEnforcesItsWindow() {
|
||||||
|
if (url == null || pin == null || cred == null) {
|
||||||
|
println("LiveCompatTest skipped (no ECHOLOT_LIVE_* env)"); return
|
||||||
|
}
|
||||||
|
|
||||||
|
// The profile must state the window — without it the app cannot pre-empt a refusal.
|
||||||
|
val profile = clientAs("0.2.0").profile(cred)
|
||||||
|
println("server ${profile.serverVersion} protocol=${profile.compat.protocolVersion} " +
|
||||||
|
"accepts app [${profile.compat.appMin}, ${profile.compat.appMax})")
|
||||||
|
assertTrue(profile.compat.protocolVersion.isNotBlank(), "profile omits protocol_version")
|
||||||
|
assertTrue(profile.compat.appMin.isNotBlank(), "profile omits app_min")
|
||||||
|
|
||||||
|
// This build must be inside it, or every other live test here is meaningless.
|
||||||
|
val verdict = Compat.check(profile, "0.2.0")
|
||||||
|
assertEquals(Compat.Verdict.OK, verdict.verdict, verdict.message ?: "")
|
||||||
|
|
||||||
|
// The profile stays reachable for a version the server would otherwise refuse: that is
|
||||||
|
// how a refused client discovers what it needs.
|
||||||
|
val ancient = clientAs("0.1.0")
|
||||||
|
val stillReadable = ancient.profile(cred)
|
||||||
|
assertEquals(profile.serverVersion, stillReadable.serverVersion,
|
||||||
|
"the profile endpoint must never be gated on app version")
|
||||||
|
|
||||||
|
// And a gated endpoint refuses it, with a message naming the window.
|
||||||
|
try {
|
||||||
|
ancient.createSession(cred, System.getenv("ECHOLOT_LIVE_TARGET") ?: "fmr")
|
||||||
|
fail("server accepted a session from an out-of-window app")
|
||||||
|
} catch (e: VersionRefused) {
|
||||||
|
val msg = assertNotNull(e.message)
|
||||||
|
println("refused as expected: $msg")
|
||||||
|
assertTrue(msg.contains("0.1.0"), "refusal should name the offending version: $msg")
|
||||||
|
assertTrue(msg.contains(profile.compat.appMin), "refusal should name the window: $msg")
|
||||||
|
}
|
||||||
|
|
||||||
|
// Too new is refused the same way — the window is a range, not a floor.
|
||||||
|
try {
|
||||||
|
clientAs("99.0.0").createSession(cred, System.getenv("ECHOLOT_LIVE_TARGET") ?: "fmr")
|
||||||
|
fail("server accepted a session from an app above its window")
|
||||||
|
} catch (e: VersionRefused) {
|
||||||
|
println("too-new refused as expected: ${e.message}")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,81 @@
|
|||||||
|
// SPDX-FileCopyrightText: 2026 Echolot contributors
|
||||||
|
// SPDX-License-Identifier: GPL-3.0-or-later
|
||||||
|
|
||||||
|
package app.echo_lot.engine
|
||||||
|
|
||||||
|
import app.echo_lot.measurement.TestStatus
|
||||||
|
import app.echo_lot.measurement.TestType
|
||||||
|
import app.echo_lot.protocol.ControlClient
|
||||||
|
import app.echo_lot.protocol.ProbeSession
|
||||||
|
import kotlin.test.Test as JTest
|
||||||
|
import kotlin.test.assertEquals
|
||||||
|
import kotlin.test.assertNotNull
|
||||||
|
import kotlin.test.assertTrue
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Runs DownstreamMeasurement against a LIVE server and checks the *documents* it produces, not
|
||||||
|
* just that packets moved: the tests must carry recomputable metrics and land on the right test
|
||||||
|
* types, because that is what an archived run is read back as. Self-skips without ECHOLOT_LIVE_*.
|
||||||
|
*/
|
||||||
|
class LiveDownstreamTest {
|
||||||
|
|
||||||
|
private val url = System.getenv("ECHOLOT_LIVE_URL")
|
||||||
|
private val pin = System.getenv("ECHOLOT_LIVE_PIN")
|
||||||
|
private val cred = System.getenv("ECHOLOT_LIVE_CRED")
|
||||||
|
private val udp = System.getenv("ECHOLOT_LIVE_UDP")
|
||||||
|
private val target = System.getenv("ECHOLOT_LIVE_TARGET") ?: "fmr"
|
||||||
|
|
||||||
|
@JTest
|
||||||
|
fun producesDownstreamTestsAndFindings() {
|
||||||
|
if (url == null || pin == null || cred == null || udp == null) {
|
||||||
|
println("LiveDownstreamTest skipped (no ECHOLOT_LIVE_* env)"); return
|
||||||
|
}
|
||||||
|
val control = ControlClient(url, setOf(pin))
|
||||||
|
val session = control.createSession(cred, target)
|
||||||
|
val (host, port) = udp.split(":").let { it[0] to it[1].toInt() }
|
||||||
|
|
||||||
|
val (tests, findings) = ProbeSession(cred, session, host, port).use { ps ->
|
||||||
|
ps.echo() // prime: the grant binds to the source the server has actually observed
|
||||||
|
DownstreamMeasurement(SystemIdSource())
|
||||||
|
.run(cred, session.sessionId, control, ps, sessionRef = "sess-1")
|
||||||
|
}
|
||||||
|
control.deleteSession(cred, session.sessionId)
|
||||||
|
|
||||||
|
for (t in tests) println("${t.type} status=${t.status} metrics=${t.metrics}")
|
||||||
|
for (f in findings) println("finding ${f.code} [${f.severity}] ${f.title}")
|
||||||
|
|
||||||
|
// Assert on what is present, not on how many: adding a measurement should not be a
|
||||||
|
// test edit. (It was, once — hence the note.)
|
||||||
|
assertTrue(tests.size >= 3, "expected at least the three downstream tests, got ${tests.size}")
|
||||||
|
val byType = tests.associateBy { it.type }
|
||||||
|
|
||||||
|
val pmtud = assertNotNull(byType[TestType.MTU_PMTUD_DOWN], "no mtu.pmtud_down test")
|
||||||
|
assertTrue(pmtud.status == TestStatus.OK || pmtud.status == TestStatus.PARTIAL,
|
||||||
|
"DF probe did not deliver anything: ${pmtud.status}")
|
||||||
|
val pathMtu = pmtud.metrics?.get("path_mtu_bytes")?.toString()?.toIntOrNull()
|
||||||
|
assertNotNull(pathMtu, "pmtud_down must report a path MTU")
|
||||||
|
assertTrue(pathMtu in 576..9000, "implausible downstream path MTU: $pathMtu")
|
||||||
|
println("downstream path MTU = $pathMtu bytes")
|
||||||
|
|
||||||
|
val frag = assertNotNull(byType[TestType.MTU_FRAG_DELIVERY], "no mtu.frag_delivery test")
|
||||||
|
assertNotNull(frag.metrics?.get("largest_delivered_bytes"))
|
||||||
|
|
||||||
|
// Fragment ordering runs only when fragments arrive at all, and only against a server
|
||||||
|
// that can craft them — so it is checked when present rather than required.
|
||||||
|
byType[TestType.MTU_FRAG_ORDERING]?.let { fo ->
|
||||||
|
val m = fo.metrics?.toString() ?: ""
|
||||||
|
println("fragment ordering: ${fo.status} $m")
|
||||||
|
if (fo.status != TestStatus.UNSUPPORTED) {
|
||||||
|
assertTrue(m.contains("in_order"), "no per-ordering result: $m")
|
||||||
|
assertTrue(m.contains("reversed"), "reversed ordering was never attempted: $m")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
val train = assertNotNull(byType[TestType.TRAIN_UDP_DOWNSTREAM], "no downstream train")
|
||||||
|
assertNotNull(train.evidence, "a train without columnar evidence is not recomputable")
|
||||||
|
val received = train.metrics?.get("received")?.toString()?.toIntOrNull() ?: 0
|
||||||
|
assertTrue(received > 0, "no downstream train packets arrived")
|
||||||
|
println("downstream train: $received received, loss=${train.metrics?.get("loss_pct")}, " +
|
||||||
|
"reordered=${train.metrics?.get("reordered_packets")}")
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,63 @@
|
|||||||
|
// SPDX-FileCopyrightText: 2026 Echolot contributors
|
||||||
|
// SPDX-License-Identifier: GPL-3.0-or-later
|
||||||
|
|
||||||
|
package app.echo_lot.engine
|
||||||
|
|
||||||
|
import app.echo_lot.protocol.EnrollmentLink
|
||||||
|
import kotlin.test.Test
|
||||||
|
import kotlin.test.assertEquals
|
||||||
|
import kotlin.test.assertNotNull
|
||||||
|
import kotlin.test.assertTrue
|
||||||
|
import kotlin.test.fail
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Enrolls against a LIVE server using the link the server itself minted (probe-protocol.md §2.1).
|
||||||
|
*
|
||||||
|
* This is the test that matters for enrollment, because the failure mode it guards against is a
|
||||||
|
* *disagreement* between two programs: the Go side assembles the link, the Kotlin side takes it
|
||||||
|
* apart, and if they differ by one percent-encoding the pin is wrong by one character — which
|
||||||
|
* does not fail loudly, it fails as an inscrutable TLS error days later. A unit test on either
|
||||||
|
* side alone cannot see that.
|
||||||
|
*
|
||||||
|
* Needs ECHOLOT_ENROLL_URI (minted over SSH by scripts/test-fmr.sh); self-skips without it.
|
||||||
|
*/
|
||||||
|
class LiveEnrollmentTest {
|
||||||
|
|
||||||
|
private val enrollUri = System.getenv("ECHOLOT_ENROLL_URI")
|
||||||
|
|
||||||
|
@Test
|
||||||
|
fun enrollsFromTheServersOwnLink() {
|
||||||
|
if (enrollUri.isNullOrBlank()) {
|
||||||
|
println("LiveEnrollmentTest skipped (no ECHOLOT_ENROLL_URI)"); return
|
||||||
|
}
|
||||||
|
println("link: ${enrollUri.take(60)}…")
|
||||||
|
|
||||||
|
val link = assertNotNull(
|
||||||
|
EnrollmentLink.parse(enrollUri),
|
||||||
|
"the client could not parse a link the server produced — the two sides disagree",
|
||||||
|
)
|
||||||
|
println("parsed: url=${link.controlUrl} pin=${link.pin.take(12)}… token=${link.token.take(8)}…")
|
||||||
|
|
||||||
|
// Redeeming applies the pin to the very request that spends the token, so a wrong pin
|
||||||
|
// fails here at the handshake rather than after the token is gone.
|
||||||
|
val enrolled = link.redeem(deviceName = "live-test", appVersion = "0.2.0")
|
||||||
|
assertTrue(enrolled.credential.isNotBlank(), "no credential came back")
|
||||||
|
assertTrue(enrolled.deviceId.isNotBlank(), "no device id came back")
|
||||||
|
println("enrolled: device=${enrolled.deviceId} server=${enrolled.profile.name} " +
|
||||||
|
"${enrolled.profile.serverVersion}")
|
||||||
|
|
||||||
|
// The credential must actually work, and the pin from the link must be the one that
|
||||||
|
// verifies the server — that is the whole claim the link is making.
|
||||||
|
assertEquals(link.controlUrl, enrolled.controlUrl)
|
||||||
|
assertTrue(enrolled.profile.capabilities.contains("udp-probe"),
|
||||||
|
"profile fetched with the new credential looks wrong: ${enrolled.profile.capabilities}")
|
||||||
|
|
||||||
|
// Single-use: a token that still works after redemption is a token an attacker can reuse.
|
||||||
|
try {
|
||||||
|
link.redeem(deviceName = "should-not-happen", appVersion = "0.2.0")
|
||||||
|
fail("the enrollment token was accepted twice — it must be single-use")
|
||||||
|
} catch (t: Throwable) {
|
||||||
|
println("second redemption correctly refused: ${t.message?.take(120)}")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -7,6 +7,7 @@ import app.echo_lot.measurement.*
|
|||||||
import kotlinx.serialization.json.Json
|
import kotlinx.serialization.json.Json
|
||||||
import kotlin.test.Test
|
import kotlin.test.Test
|
||||||
import kotlin.test.assertEquals
|
import kotlin.test.assertEquals
|
||||||
|
import kotlin.test.assertNotNull
|
||||||
import kotlin.test.assertTrue
|
import kotlin.test.assertTrue
|
||||||
|
|
||||||
/**
|
/**
|
||||||
@@ -47,8 +48,11 @@ class LiveMeasurementTest {
|
|||||||
|
|
||||||
assertEquals(1, doc.serverSessions.size)
|
assertEquals(1, doc.serverSessions.size)
|
||||||
assertTrue(doc.serverSessions[0].capabilities.contains("udp-probe"))
|
assertTrue(doc.serverSessions[0].capabilities.contains("udp-probe"))
|
||||||
val test = doc.tests.single()
|
// A full run is the echo train plus the three downstream tests; assert on the one this
|
||||||
assertEquals(TestType.TRAIN_UDP_UPDOWN, test.type)
|
// test is about rather than on the count, so adding a measurement is not a test edit.
|
||||||
|
for (t in doc.tests) println(" ${t.type} → ${t.status}")
|
||||||
|
for (f in doc.findings) println(" finding ${f.code} [${f.severity}] ${f.title}")
|
||||||
|
val test = doc.tests.first { it.type == TestType.TRAIN_UDP_UPDOWN }
|
||||||
assertTrue(test.status == TestStatus.OK || test.status == TestStatus.PARTIAL,
|
assertTrue(test.status == TestStatus.OK || test.status == TestStatus.PARTIAL,
|
||||||
"expected replies from live server, got ${test.status}")
|
"expected replies from live server, got ${test.status}")
|
||||||
|
|
||||||
@@ -56,6 +60,18 @@ class LiveMeasurementTest {
|
|||||||
println("metrics: $metrics")
|
println("metrics: $metrics")
|
||||||
assertTrue(metrics.toString().contains("rtt_ms_avg"))
|
assertTrue(metrics.toString().contains("rtt_ms_avg"))
|
||||||
|
|
||||||
|
// The directional split is the point of asking the server what it saw: without it a
|
||||||
|
// lossy path is reported as "loss" with no direction, which sends an engineer looking
|
||||||
|
// in both at once. Correlation is by wire sequence number, so a mismatch here means the
|
||||||
|
// two sides disagree about which packet is which.
|
||||||
|
val m = metrics.toString()
|
||||||
|
assertTrue(m.contains("seen_by_server"), "no directional split in the metrics: $m")
|
||||||
|
val seen = Regex(""""seen_by_server":(\d+)""").find(m)?.groupValues?.get(1)?.toInt()
|
||||||
|
assertNotNull(seen, "seen_by_server missing")
|
||||||
|
assertEquals(20, seen, "the server should have seen every probe on a healthy path")
|
||||||
|
assertTrue(m.contains("jitter_upstream_ms"), "no per-direction jitter: $m")
|
||||||
|
println("directional: $m")
|
||||||
|
|
||||||
assertTrue(doc.summary != null)
|
assertTrue(doc.summary != null)
|
||||||
// A healthy local->fmr path should be green (no loss, no rebinding) or yellow.
|
// A healthy local->fmr path should be green (no loss, no rebinding) or yellow.
|
||||||
println("summary: ${doc.summary}")
|
println("summary: ${doc.summary}")
|
||||||
|
|||||||
@@ -0,0 +1,68 @@
|
|||||||
|
// SPDX-FileCopyrightText: 2026 Echolot contributors
|
||||||
|
// SPDX-License-Identifier: GPL-3.0-or-later
|
||||||
|
|
||||||
|
package app.echo_lot.engine
|
||||||
|
|
||||||
|
import app.echo_lot.measurement.TestStatus
|
||||||
|
import app.echo_lot.protocol.ControlClient
|
||||||
|
import app.echo_lot.protocol.ProbeSession
|
||||||
|
import kotlin.test.Test
|
||||||
|
import kotlin.test.assertEquals
|
||||||
|
import kotlin.test.assertNotNull
|
||||||
|
import kotlin.test.assertTrue
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Downstream throughput against a LIVE server. Self-skips without ECHOLOT_LIVE_*.
|
||||||
|
*
|
||||||
|
* The assertions are about *honesty* rather than speed: a rate is only a measurement if the run
|
||||||
|
* was ended by the clock and the sender's own count backs it up. A test that just asserted "some
|
||||||
|
* Mbps arrived" would pass equally well against a broken implementation.
|
||||||
|
*/
|
||||||
|
class LiveThroughputTest {
|
||||||
|
|
||||||
|
private val url = System.getenv("ECHOLOT_LIVE_URL")
|
||||||
|
private val pin = System.getenv("ECHOLOT_LIVE_PIN")
|
||||||
|
private val cred = System.getenv("ECHOLOT_LIVE_CRED")
|
||||||
|
private val udp = System.getenv("ECHOLOT_LIVE_UDP")
|
||||||
|
private val target = System.getenv("ECHOLOT_LIVE_TARGET") ?: "fmr"
|
||||||
|
|
||||||
|
@Test
|
||||||
|
fun measuresDownstreamRateAndSaysWhatLimitedIt() {
|
||||||
|
if (url == null || pin == null || cred == null || udp == null) {
|
||||||
|
println("LiveThroughputTest skipped (no ECHOLOT_LIVE_* env)"); return
|
||||||
|
}
|
||||||
|
val control = ControlClient(url, setOf(pin), "0.2.0")
|
||||||
|
val session = control.createSession(cred, target)
|
||||||
|
val (host, port) = udp.split(":").let { it[0] to it[1].toInt() }
|
||||||
|
|
||||||
|
val (test, findings) = ProbeSession(cred, session, host, port).use { ps ->
|
||||||
|
ps.echo() // prime: the grant binds to the observed source
|
||||||
|
ThroughputMeasurement(SystemIdSource()).run(
|
||||||
|
cred, session.sessionId, control, ps, sessionRef = "sess-1",
|
||||||
|
durationS = 3, kbps = 20_000,
|
||||||
|
)
|
||||||
|
}
|
||||||
|
control.deleteSession(cred, session.sessionId)
|
||||||
|
|
||||||
|
val m = assertNotNull(test.metrics).toString()
|
||||||
|
println("throughput: ${test.status} $m")
|
||||||
|
for (f in findings) println("finding ${f.code} [${f.severity}] ${f.title}")
|
||||||
|
|
||||||
|
assertEquals(TestStatus.OK, test.status, "no throughput traffic arrived: $m")
|
||||||
|
|
||||||
|
// The sender's own count must be present — without it, loss cannot be attributed and the
|
||||||
|
// number is not a measurement.
|
||||||
|
assertTrue(m.contains("sender_packets"), "no sender report to compare against: $m")
|
||||||
|
assertTrue(m.contains("limited_by"), "the result must say what ended the run: $m")
|
||||||
|
|
||||||
|
val received = Regex(""""received_kbps":(\d+)""").find(m)?.groupValues?.get(1)?.toInt()
|
||||||
|
assertNotNull(received)
|
||||||
|
assertTrue(received > 0, "measured 0 kbps: $m")
|
||||||
|
println("received ${received / 1000} Mbit/s")
|
||||||
|
|
||||||
|
// A run this short and this far below the ceiling should end on the clock. Anything else
|
||||||
|
// means the grant was the constraint, and then the rate says nothing about the path.
|
||||||
|
assertTrue(m.contains(""""limited_by":"duration""""),
|
||||||
|
"the run did not end on the clock, so the rate measures the server, not the path: $m")
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -1,100 +0,0 @@
|
|||||||
// SPDX-FileCopyrightText: 2026 Echolot contributors
|
|
||||||
// SPDX-License-Identifier: GPL-3.0-or-later
|
|
||||||
|
|
||||||
// Package measurement models one measurement run (measurement-schema.md) — the archived,
|
|
||||||
// diffable, exportable unit. Design rules honored in the types: observation/interpretation
|
|
||||||
// separated (tests[] vs findings[]), two clocks (wall RFC3339 for humans, *_mono_ns for math),
|
|
||||||
// units in field names, columnar trains. params/evidence/metrics are per-test-type, so they are
|
|
||||||
// carried as JsonObject (the probe engine fills them; consumers ignore unknown fields).
|
|
||||||
package app.echo_lot.measurement
|
|
||||||
|
|
||||||
import kotlinx.serialization.SerialName
|
|
||||||
import kotlinx.serialization.Serializable
|
|
||||||
import kotlinx.serialization.json.JsonObject
|
|
||||||
|
|
||||||
@Serializable
|
|
||||||
data class MeasurementDocument(
|
|
||||||
val schema: String = "echolot/measurement",
|
|
||||||
@SerialName("schema_version") val schemaVersion: String = "1.0.0",
|
|
||||||
val run: Run,
|
|
||||||
val networks: List<Network> = emptyList(),
|
|
||||||
@SerialName("server_sessions") val serverSessions: List<ServerSession> = emptyList(),
|
|
||||||
val tests: List<Test> = emptyList(),
|
|
||||||
val findings: List<Finding> = emptyList(),
|
|
||||||
val summary: Summary? = null,
|
|
||||||
)
|
|
||||||
|
|
||||||
@Serializable
|
|
||||||
data class Run(
|
|
||||||
val id: String, // UUIDv7
|
|
||||||
val trigger: Trigger,
|
|
||||||
@SerialName("started_at") val startedAt: String, // RFC3339 UTC, human correlation only
|
|
||||||
@SerialName("ended_at") val endedAt: String? = null,
|
|
||||||
val clock: Clock,
|
|
||||||
val app: AppInfo,
|
|
||||||
val device: DeviceInfo,
|
|
||||||
val tiers: Tiers,
|
|
||||||
@SerialName("profiles_used") val profilesUsed: List<String> = emptyList(),
|
|
||||||
val notes: String? = null,
|
|
||||||
)
|
|
||||||
|
|
||||||
@Serializable
|
|
||||||
enum class Trigger {
|
|
||||||
@SerialName("manual") MANUAL,
|
|
||||||
@SerialName("scheduled") SCHEDULED,
|
|
||||||
@SerialName("monitor") MONITOR,
|
|
||||||
@SerialName("peer") PEER,
|
|
||||||
}
|
|
||||||
|
|
||||||
/** The two-clock anchor: mono_origin_wall maps the monotonic epoch to a wall time for humans;
|
|
||||||
* all math uses *_mono_ns relative to that monotonic origin. */
|
|
||||||
@Serializable
|
|
||||||
data class Clock(
|
|
||||||
@SerialName("mono_origin_wall") val monoOriginWall: String,
|
|
||||||
@SerialName("ntp_offset_ms") val ntpOffsetMs: Double? = null,
|
|
||||||
@SerialName("ntp_offset_source") val ntpOffsetSource: String? = null,
|
|
||||||
)
|
|
||||||
|
|
||||||
@Serializable
|
|
||||||
data class AppInfo(
|
|
||||||
val version: String,
|
|
||||||
val build: Int,
|
|
||||||
val git: String? = null,
|
|
||||||
val flavor: String? = null,
|
|
||||||
)
|
|
||||||
|
|
||||||
@Serializable
|
|
||||||
data class DeviceInfo(
|
|
||||||
val manufacturer: String,
|
|
||||||
val model: String,
|
|
||||||
@SerialName("android_sdk") val androidSdk: Int,
|
|
||||||
@SerialName("android_release") val androidRelease: String,
|
|
||||||
@SerialName("security_patch") val securityPatch: String? = null,
|
|
||||||
)
|
|
||||||
|
|
||||||
/** What each tier was *available*; each test records what it *used*. */
|
|
||||||
@Serializable
|
|
||||||
data class Tiers(
|
|
||||||
val app: Boolean = true,
|
|
||||||
val shizuku: Boolean = false,
|
|
||||||
val root: Boolean = false,
|
|
||||||
)
|
|
||||||
|
|
||||||
@Serializable
|
|
||||||
data class ServerSession(
|
|
||||||
val id: String,
|
|
||||||
@SerialName("profile_id") val profileId: String? = null,
|
|
||||||
@SerialName("profile_name") val profileName: String? = null,
|
|
||||||
@SerialName("control_url") val controlUrl: String,
|
|
||||||
@SerialName("server_version") val serverVersion: String? = null,
|
|
||||||
val capabilities: List<String> = emptyList(),
|
|
||||||
@SerialName("session_id") val sessionId: String,
|
|
||||||
val target: SessionTarget,
|
|
||||||
)
|
|
||||||
|
|
||||||
@Serializable
|
|
||||||
data class SessionTarget(
|
|
||||||
val ip4: String? = null,
|
|
||||||
val ip6: String? = null,
|
|
||||||
@SerialName("udp_port") val udpPort: Int = 0,
|
|
||||||
)
|
|
||||||
@@ -1,85 +0,0 @@
|
|||||||
// SPDX-FileCopyrightText: 2026 Echolot contributors
|
|
||||||
// SPDX-License-Identifier: GPL-3.0-or-later
|
|
||||||
|
|
||||||
package app.echo_lot.measurement
|
|
||||||
|
|
||||||
import kotlinx.serialization.SerialName
|
|
||||||
import kotlinx.serialization.Serializable
|
|
||||||
import kotlinx.serialization.json.Json
|
|
||||||
import kotlinx.serialization.json.JsonObject
|
|
||||||
import kotlinx.serialization.json.encodeToJsonElement
|
|
||||||
|
|
||||||
/**
|
|
||||||
* Typed builders for the per-test-type evidence shapes the schema fixes (§6.2/§6.3/§6.4). Probe
|
|
||||||
* code fills these and folds them into [Test.evidence] via [toEvidence]; keeping them typed here
|
|
||||||
* means the columnar/traceroute/resolver contracts live in one place.
|
|
||||||
*/
|
|
||||||
|
|
||||||
@PublishedApi
|
|
||||||
internal val evidenceJson = Json { encodeDefaults = true; explicitNulls = true }
|
|
||||||
|
|
||||||
/** Serialize any typed evidence object into the JsonObject the Test envelope carries. */
|
|
||||||
inline fun <reified T> T.toEvidence(): JsonObject =
|
|
||||||
evidenceJson.encodeToJsonElement(this) as JsonObject
|
|
||||||
|
|
||||||
/**
|
|
||||||
* Packet-train evidence (§6.2): columnar parallel arrays, one index per probe packet. Missing
|
|
||||||
* observations are null at that index — a 10k-packet train stays in the hundreds of kB. Server
|
|
||||||
* columns use the server session epoch; only differences within one clock are meaningful unless a
|
|
||||||
* time.server_offset test maps them.
|
|
||||||
*/
|
|
||||||
@Serializable
|
|
||||||
data class TrainEvidence(
|
|
||||||
@SerialName("epoch_mono_ns") val epochMonoNs: Long,
|
|
||||||
val seq: List<Int>,
|
|
||||||
@SerialName("t_tx_ns") val tTxNs: List<Long?>,
|
|
||||||
@SerialName("t_srv_rx_ns") val tSrvRxNs: List<Long?> = emptyList(),
|
|
||||||
@SerialName("t_srv_tx_ns") val tSrvTxNs: List<Long?> = emptyList(),
|
|
||||||
@SerialName("t_rx_ns") val tRxNs: List<Long?>,
|
|
||||||
@SerialName("size_bytes") val sizeBytes: List<Int>,
|
|
||||||
@SerialName("dscp_sent") val dscpSent: Int? = null,
|
|
||||||
@SerialName("dscp_seen_by_server") val dscpSeenByServer: List<Int?> = emptyList(),
|
|
||||||
@SerialName("ecn_sent") val ecnSent: Int? = null,
|
|
||||||
@SerialName("ecn_seen_by_server") val ecnSeenByServer: List<Int?> = emptyList(),
|
|
||||||
@SerialName("ttl_seen_by_server") val ttlSeenByServer: List<Int?> = emptyList(),
|
|
||||||
@SerialName("evidence_truncated") val evidenceTruncated: Boolean = false,
|
|
||||||
)
|
|
||||||
|
|
||||||
/** Traceroute evidence (§6.3): fixed-tuple flow + per-TTL probe replies. */
|
|
||||||
@Serializable
|
|
||||||
data class TracerouteEvidence(val flow: Flow, val hops: List<Hop>)
|
|
||||||
|
|
||||||
@Serializable
|
|
||||||
data class Flow(
|
|
||||||
@SerialName("src_port") val srcPort: Int,
|
|
||||||
@SerialName("dst_port") val dstPort: Int,
|
|
||||||
@SerialName("fixed_tuple") val fixedTuple: Boolean = true,
|
|
||||||
)
|
|
||||||
|
|
||||||
@Serializable
|
|
||||||
data class Hop(val ttl: Int, val probes: List<HopProbe>)
|
|
||||||
|
|
||||||
@Serializable
|
|
||||||
data class HopProbe(
|
|
||||||
@SerialName("reply_from") val replyFrom: String? = null,
|
|
||||||
@SerialName("rtt_ns") val rttNs: Long? = null,
|
|
||||||
val icmp: String? = null,
|
|
||||||
@SerialName("reply_ttl") val replyTtl: Int? = null,
|
|
||||||
)
|
|
||||||
|
|
||||||
/** Resolver under test (§6.4); every dns.* test carries this in params. */
|
|
||||||
@Serializable
|
|
||||||
data class ResolverSpec(
|
|
||||||
val source: ResolverSource,
|
|
||||||
val address: String? = null,
|
|
||||||
val port: Int = 53,
|
|
||||||
val transport: String, // do53-udp | do53-tcp | dot | doh
|
|
||||||
@SerialName("doh_url") val dohUrl: String? = null,
|
|
||||||
)
|
|
||||||
|
|
||||||
@Serializable
|
|
||||||
enum class ResolverSource {
|
|
||||||
@SerialName("system") SYSTEM,
|
|
||||||
@SerialName("manual") MANUAL,
|
|
||||||
@SerialName("server-recursive") SERVER_RECURSIVE,
|
|
||||||
}
|
|
||||||
@@ -1,68 +0,0 @@
|
|||||||
// SPDX-FileCopyrightText: 2026 Echolot contributors
|
|
||||||
// SPDX-License-Identifier: GPL-3.0-or-later
|
|
||||||
|
|
||||||
package app.echo_lot.measurement
|
|
||||||
|
|
||||||
import kotlinx.serialization.SerialName
|
|
||||||
import kotlinx.serialization.Serializable
|
|
||||||
|
|
||||||
/** Interpretation with references back to evidence (measurement-schema.md §7.1). A finding with
|
|
||||||
* no evidence_refs is invalid — every finding must be re-derivable from the evidence alone. */
|
|
||||||
@Serializable
|
|
||||||
data class Finding(
|
|
||||||
val id: String, // UUIDv7
|
|
||||||
val code: String, // stable registry (findings-registry.md), lint-rule style
|
|
||||||
val category: Category,
|
|
||||||
val severity: Severity,
|
|
||||||
val confidence: Confidence,
|
|
||||||
@SerialName("network_ref") val networkRef: String? = null,
|
|
||||||
val title: String,
|
|
||||||
val description: String,
|
|
||||||
@SerialName("evidence_refs") val evidenceRefs: List<EvidenceRef>,
|
|
||||||
val recommendation: String? = null,
|
|
||||||
) {
|
|
||||||
init {
|
|
||||||
require(evidenceRefs.isNotEmpty()) { "a finding must reference at least one piece of evidence" }
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
@Serializable
|
|
||||||
data class EvidenceRef(val test: String, val pointer: String? = null)
|
|
||||||
|
|
||||||
/** Fixed §7.2 categories; each maps to one traffic light. */
|
|
||||||
@Serializable
|
|
||||||
enum class Category {
|
|
||||||
@SerialName("connectivity") CONNECTIVITY,
|
|
||||||
@SerialName("dns") DNS,
|
|
||||||
@SerialName("nat") NAT,
|
|
||||||
@SerialName("mtu") MTU,
|
|
||||||
@SerialName("ipv6") IPV6,
|
|
||||||
@SerialName("security") SECURITY,
|
|
||||||
@SerialName("performance") PERFORMANCE,
|
|
||||||
@SerialName("local") LOCAL,
|
|
||||||
@SerialName("wifi") WIFI,
|
|
||||||
}
|
|
||||||
|
|
||||||
/** Ordered worst→best via [rank]; drives the §7.3 light mapping. */
|
|
||||||
@Serializable
|
|
||||||
enum class Severity(val rank: Int) {
|
|
||||||
@SerialName("critical") CRITICAL(4),
|
|
||||||
@SerialName("high") HIGH(3),
|
|
||||||
@SerialName("medium") MEDIUM(2),
|
|
||||||
@SerialName("low") LOW(1),
|
|
||||||
@SerialName("info") INFO(0);
|
|
||||||
|
|
||||||
/** §7.3: critical|high → red, medium|low → yellow, info → green. */
|
|
||||||
fun toLight(): Verdict = when (this) {
|
|
||||||
CRITICAL, HIGH -> Verdict.RED
|
|
||||||
MEDIUM, LOW -> Verdict.YELLOW
|
|
||||||
INFO -> Verdict.GREEN
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
@Serializable
|
|
||||||
enum class Confidence {
|
|
||||||
@SerialName("high") HIGH,
|
|
||||||
@SerialName("medium") MEDIUM,
|
|
||||||
@SerialName("low") LOW,
|
|
||||||
}
|
|
||||||
@@ -1,113 +0,0 @@
|
|||||||
// SPDX-FileCopyrightText: 2026 Echolot contributors
|
|
||||||
// SPDX-License-Identifier: GPL-3.0-or-later
|
|
||||||
|
|
||||||
package app.echo_lot.measurement
|
|
||||||
|
|
||||||
import kotlinx.serialization.SerialName
|
|
||||||
import kotlinx.serialization.Serializable
|
|
||||||
import kotlinx.serialization.json.JsonObject
|
|
||||||
|
|
||||||
/** One Android Network in play (measurement-schema.md §4). Shizuku-tier fields (route proto,
|
|
||||||
* lifetimes) are absent at app tier — absence means "not observed", never "not present". */
|
|
||||||
@Serializable
|
|
||||||
data class Network(
|
|
||||||
val id: String,
|
|
||||||
val transport: Transport,
|
|
||||||
@SerialName("interface") val iface: String? = null,
|
|
||||||
val link: Link,
|
|
||||||
val wifi: Wifi? = null,
|
|
||||||
val cellular: Cellular? = null,
|
|
||||||
val changes: List<NetworkChange> = emptyList(),
|
|
||||||
)
|
|
||||||
|
|
||||||
@Serializable
|
|
||||||
enum class Transport {
|
|
||||||
@SerialName("wifi") WIFI,
|
|
||||||
@SerialName("cellular") CELLULAR,
|
|
||||||
@SerialName("ethernet") ETHERNET,
|
|
||||||
@SerialName("vpn") VPN,
|
|
||||||
@SerialName("other") OTHER,
|
|
||||||
}
|
|
||||||
|
|
||||||
@Serializable
|
|
||||||
data class Link(
|
|
||||||
val mtu: Int? = null,
|
|
||||||
val addresses: List<Address> = emptyList(),
|
|
||||||
val routes: List<Route> = emptyList(),
|
|
||||||
val dns: DnsConfig? = null,
|
|
||||||
val dhcp: Dhcp? = null,
|
|
||||||
@SerialName("captive_portal") val captivePortal: CaptivePortal? = null,
|
|
||||||
)
|
|
||||||
|
|
||||||
@Serializable
|
|
||||||
data class Address(
|
|
||||||
val addr: String, // ip4 | ip6 (logical type, §8)
|
|
||||||
@SerialName("prefix_len") val prefixLen: Int,
|
|
||||||
val scope: String? = null,
|
|
||||||
val flags: List<String> = emptyList(),
|
|
||||||
@SerialName("valid_lft_s") val validLftS: Long? = null,
|
|
||||||
@SerialName("pref_lft_s") val prefLftS: Long? = null,
|
|
||||||
)
|
|
||||||
|
|
||||||
@Serializable
|
|
||||||
data class Route(
|
|
||||||
val dst: String,
|
|
||||||
val gateway: String? = null,
|
|
||||||
val iface: String? = null,
|
|
||||||
val proto: RouteProto? = null, // shizuku tier; null = not observed
|
|
||||||
@SerialName("expires_s") val expiresS: Long? = null,
|
|
||||||
)
|
|
||||||
|
|
||||||
@Serializable
|
|
||||||
enum class RouteProto {
|
|
||||||
@SerialName("dhcp") DHCP,
|
|
||||||
@SerialName("ra") RA,
|
|
||||||
@SerialName("static") STATIC,
|
|
||||||
@SerialName("unknown") UNKNOWN,
|
|
||||||
}
|
|
||||||
|
|
||||||
@Serializable
|
|
||||||
data class DnsConfig(
|
|
||||||
val servers: List<String> = emptyList(),
|
|
||||||
@SerialName("private_dns_mode") val privateDnsMode: String? = null,
|
|
||||||
@SerialName("private_dns_hostname") val privateDnsHostname: String? = null,
|
|
||||||
@SerialName("search_domains") val searchDomains: List<String> = emptyList(),
|
|
||||||
@SerialName("nat64_prefix") val nat64Prefix: String? = null,
|
|
||||||
)
|
|
||||||
|
|
||||||
@Serializable
|
|
||||||
data class Dhcp(val server: String? = null, @SerialName("lease_s") val leaseS: Long? = null)
|
|
||||||
|
|
||||||
@Serializable
|
|
||||||
data class CaptivePortal(
|
|
||||||
val detected: Boolean = false,
|
|
||||||
@SerialName("api_url") val apiUrl: String? = null,
|
|
||||||
@SerialName("venue_url") val venueUrl: String? = null,
|
|
||||||
)
|
|
||||||
|
|
||||||
@Serializable
|
|
||||||
data class Wifi(
|
|
||||||
val ssid: String? = null, // ssid (logical type)
|
|
||||||
val bssid: String? = null, // bssid (logical type)
|
|
||||||
@SerialName("rssi_dbm") val rssiDbm: Int? = null,
|
|
||||||
@SerialName("link_speed_mbps") val linkSpeedMbps: Int? = null,
|
|
||||||
@SerialName("frequency_mhz") val frequencyMhz: Int? = null,
|
|
||||||
@SerialName("channel_width_mhz") val channelWidthMhz: Int? = null,
|
|
||||||
val standard: String? = null,
|
|
||||||
val security: String? = null,
|
|
||||||
@SerialName("mac_randomization") val macRandomization: Boolean? = null,
|
|
||||||
)
|
|
||||||
|
|
||||||
@Serializable
|
|
||||||
data class Cellular(
|
|
||||||
val rat: String? = null,
|
|
||||||
val operator: String? = null,
|
|
||||||
val band: String? = null,
|
|
||||||
)
|
|
||||||
|
|
||||||
@Serializable
|
|
||||||
data class NetworkChange(
|
|
||||||
@SerialName("at_mono_ns") val atMonoNs: Long,
|
|
||||||
val kind: String, // lost | gained | link_changed
|
|
||||||
val detail: JsonObject? = null,
|
|
||||||
)
|
|
||||||
@@ -1,90 +0,0 @@
|
|||||||
// SPDX-FileCopyrightText: 2026 Echolot contributors
|
|
||||||
// SPDX-License-Identifier: GPL-3.0-or-later
|
|
||||||
|
|
||||||
package app.echo_lot.measurement
|
|
||||||
|
|
||||||
import kotlinx.serialization.SerialName
|
|
||||||
import kotlinx.serialization.Serializable
|
|
||||||
|
|
||||||
@Serializable
|
|
||||||
enum class Verdict {
|
|
||||||
@SerialName("green") GREEN,
|
|
||||||
@SerialName("yellow") YELLOW,
|
|
||||||
@SerialName("red") RED,
|
|
||||||
@SerialName("inconclusive") INCONCLUSIVE,
|
|
||||||
}
|
|
||||||
|
|
||||||
@Serializable
|
|
||||||
data class Summary(
|
|
||||||
val overall: Verdict,
|
|
||||||
val categories: Map<String, CategorySummary>,
|
|
||||||
)
|
|
||||||
|
|
||||||
@Serializable
|
|
||||||
data class CategorySummary(
|
|
||||||
val verdict: Verdict,
|
|
||||||
@SerialName("worst_finding") val worstFinding: String? = null,
|
|
||||||
@SerialName("tests_run") val testsRun: Int,
|
|
||||||
@SerialName("tests_failed") val testsFailed: Int,
|
|
||||||
)
|
|
||||||
|
|
||||||
/**
|
|
||||||
* Deterministic verdict derivation, fixed by measurement-schema.md §7.3:
|
|
||||||
*
|
|
||||||
* - A category's verdict = the light of its worst-severity finding
|
|
||||||
* (critical|high → red, medium|low → yellow, info/none → green).
|
|
||||||
* - A category is `inconclusive` when > 50% of its tests are failed/unsupported.
|
|
||||||
* - Overall = the worst category light; `inconclusive` only when ALL categories are.
|
|
||||||
*
|
|
||||||
* The mapping test-type → category comes from [TestType.category]. Only categories that have
|
|
||||||
* findings or tests appear in the summary.
|
|
||||||
*/
|
|
||||||
object Verdicts {
|
|
||||||
|
|
||||||
private fun isInconclusiveTest(s: TestStatus) =
|
|
||||||
s == TestStatus.FAILED || s == TestStatus.UNSUPPORTED
|
|
||||||
|
|
||||||
fun derive(tests: List<Test>, findings: List<Finding>): Summary {
|
|
||||||
val testsByCat = tests.groupBy { TestType.category(it.type) }
|
|
||||||
val findingsByCat = findings.groupBy { it.category }
|
|
||||||
val categories = (testsByCat.keys + findingsByCat.keys)
|
|
||||||
|
|
||||||
val perCat = LinkedHashMap<String, CategorySummary>()
|
|
||||||
for (cat in Category.entries) {
|
|
||||||
if (cat !in categories) continue
|
|
||||||
val catTests = testsByCat[cat].orEmpty()
|
|
||||||
val catFindings = findingsByCat[cat].orEmpty()
|
|
||||||
|
|
||||||
val failed = catTests.count { isInconclusiveTest(it.status) }
|
|
||||||
val inconclusive = catTests.isNotEmpty() && failed * 2 > catTests.size
|
|
||||||
|
|
||||||
val worst = catFindings.maxByOrNull { it.severity.rank }
|
|
||||||
val verdict = when {
|
|
||||||
inconclusive -> Verdict.INCONCLUSIVE
|
|
||||||
worst == null -> Verdict.GREEN
|
|
||||||
else -> worst.severity.toLight()
|
|
||||||
}
|
|
||||||
perCat[serialName(cat)] = CategorySummary(
|
|
||||||
verdict = verdict,
|
|
||||||
worstFinding = worst?.id,
|
|
||||||
testsRun = catTests.size,
|
|
||||||
testsFailed = failed,
|
|
||||||
)
|
|
||||||
}
|
|
||||||
|
|
||||||
val overall = deriveOverall(perCat.values)
|
|
||||||
return Summary(overall = overall, categories = perCat)
|
|
||||||
}
|
|
||||||
|
|
||||||
/** Overall = worst light; inconclusive only if every category is inconclusive. */
|
|
||||||
private fun deriveOverall(cats: Collection<CategorySummary>): Verdict {
|
|
||||||
if (cats.isEmpty()) return Verdict.INCONCLUSIVE
|
|
||||||
if (cats.all { it.verdict == Verdict.INCONCLUSIVE }) return Verdict.INCONCLUSIVE
|
|
||||||
val rank = mapOf(Verdict.GREEN to 0, Verdict.YELLOW to 1, Verdict.RED to 2)
|
|
||||||
// Non-inconclusive categories decide the overall light.
|
|
||||||
return cats.filter { it.verdict != Verdict.INCONCLUSIVE }
|
|
||||||
.maxByOrNull { rank.getValue(it.verdict) }!!.verdict
|
|
||||||
}
|
|
||||||
|
|
||||||
private fun serialName(cat: Category): String = cat.name.lowercase()
|
|
||||||
}
|
|
||||||
@@ -1,149 +0,0 @@
|
|||||||
// SPDX-FileCopyrightText: 2026 Echolot contributors
|
|
||||||
// SPDX-License-Identifier: GPL-3.0-or-later
|
|
||||||
|
|
||||||
package app.echo_lot.measurement
|
|
||||||
|
|
||||||
import kotlinx.serialization.SerialName
|
|
||||||
import kotlinx.serialization.Serializable
|
|
||||||
import kotlinx.serialization.json.JsonObject
|
|
||||||
|
|
||||||
/** The generic test envelope (measurement-schema.md §6). params must fully reproduce the test;
|
|
||||||
* evidence is append-only raw truth; metrics must be recomputable from evidence. All three are
|
|
||||||
* per-test-type JSON, so they are carried as JsonObject. */
|
|
||||||
@Serializable
|
|
||||||
data class Test(
|
|
||||||
val id: String, // UUIDv7
|
|
||||||
val type: String, // TestType registry (§6.1)
|
|
||||||
@SerialName("network_ref") val networkRef: String? = null,
|
|
||||||
@SerialName("session_ref") val sessionRef: String? = null, // null for local-only tests
|
|
||||||
val tier: Tier,
|
|
||||||
@SerialName("started_mono_ns") val startedMonoNs: Long,
|
|
||||||
@SerialName("ended_mono_ns") val endedMonoNs: Long,
|
|
||||||
val status: TestStatus,
|
|
||||||
val error: TestError? = null,
|
|
||||||
val params: JsonObject? = null,
|
|
||||||
val evidence: JsonObject? = null,
|
|
||||||
val metrics: JsonObject? = null,
|
|
||||||
)
|
|
||||||
|
|
||||||
@Serializable
|
|
||||||
enum class Tier {
|
|
||||||
@SerialName("app") APP,
|
|
||||||
@SerialName("shizuku") SHIZUKU,
|
|
||||||
@SerialName("root") ROOT,
|
|
||||||
}
|
|
||||||
|
|
||||||
@Serializable
|
|
||||||
enum class TestStatus {
|
|
||||||
@SerialName("ok") OK,
|
|
||||||
@SerialName("failed") FAILED,
|
|
||||||
@SerialName("unsupported") UNSUPPORTED,
|
|
||||||
@SerialName("skipped") SKIPPED,
|
|
||||||
@SerialName("partial") PARTIAL,
|
|
||||||
}
|
|
||||||
|
|
||||||
@Serializable
|
|
||||||
data class TestError(val code: String, val detail: String? = null)
|
|
||||||
|
|
||||||
/**
|
|
||||||
* The v1 test-type registry (§6.1). String constants (dotted, family-first) so probe code and the
|
|
||||||
* server's measurement-schema test-type registry stay aligned. [category] maps a type to one of
|
|
||||||
* the fixed §7.2 categories for verdict rollup.
|
|
||||||
*/
|
|
||||||
object TestType {
|
|
||||||
// link
|
|
||||||
const val LINK_SNAPSHOT = "link.snapshot"
|
|
||||||
const val LINK_DHCP_RENEWAL_WATCH = "link.dhcp_renewal_watch"
|
|
||||||
const val LINK_IP_MONITOR = "link.ip_monitor"
|
|
||||||
/** Who advertises IPv6 on this link (+ gateway identity). Registry addition, v1.1. */
|
|
||||||
const val LINK_RA_SOURCE = "link.ra_source"
|
|
||||||
// net — connectivity validation (reproduces Android's NetworkMonitor generate_204 checks)
|
|
||||||
const val NET_CAPTIVE_PORTAL = "net.captive_portal"
|
|
||||||
// icmp
|
|
||||||
const val ICMP_PING4 = "icmp.ping4"
|
|
||||||
const val ICMP_PING6 = "icmp.ping6"
|
|
||||||
// trace
|
|
||||||
const val TRACEROUTE_UDP4 = "traceroute.udp4"
|
|
||||||
const val TRACEROUTE_UDP6 = "traceroute.udp6"
|
|
||||||
const val TRACEROUTE_ICMP4 = "traceroute.icmp4"
|
|
||||||
const val TRACEROUTE_ICMP6 = "traceroute.icmp6"
|
|
||||||
// train
|
|
||||||
const val TRAIN_UDP_UPDOWN = "train.udp_updown"
|
|
||||||
// mtu
|
|
||||||
const val MTU_PMTUD_UP = "mtu.pmtud_up"
|
|
||||||
const val MTU_PMTUD_DOWN = "mtu.pmtud_down"
|
|
||||||
const val MTU_BLACKHOLE = "mtu.blackhole"
|
|
||||||
const val MTU_MSS_OBSERVED = "mtu.mss_observed"
|
|
||||||
const val MTU_FRAG_DELIVERY = "mtu.frag_delivery"
|
|
||||||
// nat
|
|
||||||
const val NAT_STUN_5780 = "nat.stun_5780"
|
|
||||||
const val NAT_MAPPING_LIFETIME_UDP = "nat.mapping_lifetime_udp"
|
|
||||||
const val NAT_MAPPING_LIFETIME_TCP = "nat.mapping_lifetime_tcp"
|
|
||||||
const val NAT_HAIRPIN = "nat.hairpin"
|
|
||||||
const val NAT_CONNECT_BACK = "nat.connect_back"
|
|
||||||
const val NAT_CGNAT_DETECT = "nat.cgnat_detect"
|
|
||||||
// dns
|
|
||||||
const val DNS_RESOLVER_INVENTORY = "dns.resolver_inventory"
|
|
||||||
const val DNS_CANARY = "dns.canary"
|
|
||||||
const val DNS_INTERCEPTION = "dns.interception"
|
|
||||||
const val DNS_TTL_INTEGRITY = "dns.ttl_integrity"
|
|
||||||
const val DNS_ANSWER_INTEGRITY = "dns.answer_integrity"
|
|
||||||
const val DNS_DNSSEC = "dns.dnssec"
|
|
||||||
const val DNS_NXDOMAIN_WILDCARD = "dns.nxdomain_wildcard"
|
|
||||||
const val DNS_REBIND_FILTER = "dns.rebind_filter"
|
|
||||||
const val DNS_AAAA_FILTER = "dns.aaaa_filter"
|
|
||||||
const val DNS_DNS64 = "dns.dns64"
|
|
||||||
const val DNS_COMPARE = "dns.compare"
|
|
||||||
// sec
|
|
||||||
const val SEC_TLS_REFERENCE = "sec.tls_reference"
|
|
||||||
const val SEC_CLIENTHELLO_ECHO = "sec.clienthello_echo"
|
|
||||||
const val SEC_HTTP_ECHO = "sec.http_echo"
|
|
||||||
const val SEC_SNI_FILTER = "sec.sni_filter"
|
|
||||||
const val SEC_DSCP_ECN_SURVIVAL = "sec.dscp_ecn_survival"
|
|
||||||
const val SEC_ARP_WATCH = "sec.arp_watch"
|
|
||||||
// port
|
|
||||||
const val PORT_REACH_SWEEP = "port.reach_sweep"
|
|
||||||
const val PORT_UDP_USABILITY = "port.udp_usability"
|
|
||||||
// perf
|
|
||||||
const val PERF_THROUGHPUT_TCP = "perf.throughput_tcp"
|
|
||||||
const val PERF_THROUGHPUT_UDP = "perf.throughput_udp"
|
|
||||||
const val PERF_BUFFERBLOAT = "perf.bufferbloat"
|
|
||||||
const val PERF_RRC_LATENCY = "perf.rrc_latency"
|
|
||||||
// v6
|
|
||||||
const val V6_DUALSTACK_COMPARE = "v6.dualstack_compare"
|
|
||||||
const val V6_HAPPY_EYEBALLS = "v6.happy_eyeballs"
|
|
||||||
const val V6_BROKENNESS = "v6.brokenness"
|
|
||||||
const val V6_NAT64_CLAT = "v6.nat64_clat"
|
|
||||||
// wifi
|
|
||||||
const val WIFI_ENVIRONMENT_SCAN = "wifi.environment_scan"
|
|
||||||
const val WIFI_ROAM_LOG = "wifi.roam_log"
|
|
||||||
const val WIFI_SIGNAL_LOG = "wifi.signal_log"
|
|
||||||
// local
|
|
||||||
const val LOCAL_MDNS_INVENTORY = "local.mdns_inventory"
|
|
||||||
const val LOCAL_SSDP_INVENTORY = "local.ssdp_inventory"
|
|
||||||
const val LOCAL_LLMNR_INVENTORY = "local.llmnr_inventory"
|
|
||||||
const val LOCAL_GATEWAY_SERVICES = "local.gateway_services"
|
|
||||||
const val LOCAL_NTP = "local.ntp"
|
|
||||||
// peer
|
|
||||||
const val PEER_REACHABILITY = "peer.reachability"
|
|
||||||
const val PEER_ISOLATION = "peer.isolation"
|
|
||||||
const val PEER_MULTICAST = "peer.multicast"
|
|
||||||
const val PEER_LAN_TRAIN = "peer.lan_train"
|
|
||||||
const val PEER_LEASE_DIFF = "peer.lease_diff"
|
|
||||||
// time
|
|
||||||
const val TIME_SERVER_OFFSET = "time.server_offset"
|
|
||||||
|
|
||||||
/** Maps a dotted test type to its §7.2 category for verdict rollup. */
|
|
||||||
fun category(type: String): Category = when (type.substringBefore('.')) {
|
|
||||||
"link", "icmp", "trace", "traceroute", "train", "port", "time", "net" -> Category.CONNECTIVITY
|
|
||||||
"dns" -> Category.DNS
|
|
||||||
"nat" -> Category.NAT
|
|
||||||
"mtu" -> Category.MTU
|
|
||||||
"v6" -> Category.IPV6
|
|
||||||
"sec" -> Category.SECURITY
|
|
||||||
"perf" -> Category.PERFORMANCE
|
|
||||||
"local", "peer" -> Category.LOCAL
|
|
||||||
"wifi" -> Category.WIFI
|
|
||||||
else -> Category.CONNECTIVITY
|
|
||||||
}
|
|
||||||
}
|
|
||||||
@@ -1,71 +0,0 @@
|
|||||||
// SPDX-FileCopyrightText: 2026 Echolot contributors
|
|
||||||
// SPDX-License-Identifier: GPL-3.0-or-later
|
|
||||||
|
|
||||||
package app.echo_lot.measurement
|
|
||||||
|
|
||||||
import kotlinx.serialization.json.Json
|
|
||||||
import kotlin.test.Test as JTest
|
|
||||||
import kotlin.test.assertEquals
|
|
||||||
import kotlin.test.assertTrue
|
|
||||||
|
|
||||||
class SerializationTest {
|
|
||||||
|
|
||||||
private val json = Json { ignoreUnknownKeys = true; encodeDefaults = true }
|
|
||||||
|
|
||||||
@JTest
|
|
||||||
fun documentRoundTrips() {
|
|
||||||
val doc = MeasurementDocument(
|
|
||||||
run = Run(
|
|
||||||
id = "0198c5f2-0000-7000-8000-000000000000",
|
|
||||||
trigger = Trigger.MANUAL,
|
|
||||||
startedAt = "2026-07-31T14:03:21.114Z",
|
|
||||||
clock = Clock(monoOriginWall = "2026-07-31T14:03:21.114Z"),
|
|
||||||
app = AppInfo(version = "0.1.0", build = 1),
|
|
||||||
device = DeviceInfo("OnePlus", "CPH2747", 36, "16"),
|
|
||||||
tiers = Tiers(app = true, shizuku = true),
|
|
||||||
),
|
|
||||||
networks = listOf(
|
|
||||||
Network(
|
|
||||||
id = "net-1", transport = Transport.WIFI, iface = "wlan0",
|
|
||||||
link = Link(mtu = 1500, addresses = listOf(Address("192.0.2.23", 24, "global"))),
|
|
||||||
wifi = Wifi(ssid = "example", rssiDbm = -54),
|
|
||||||
),
|
|
||||||
),
|
|
||||||
tests = listOf(
|
|
||||||
Test(
|
|
||||||
id = "t-1", type = TestType.ICMP_PING4, networkRef = "net-1", tier = Tier.APP,
|
|
||||||
startedMonoNs = 0, endedMonoNs = 38_000_000, status = TestStatus.OK,
|
|
||||||
evidence = TrainEvidence(
|
|
||||||
epochMonoNs = 0, seq = listOf(0, 1), tTxNs = listOf(0L, 20_000_000L),
|
|
||||||
tRxNs = listOf(16_500_000L, null), sizeBytes = listOf(64, 64),
|
|
||||||
).toEvidence(),
|
|
||||||
),
|
|
||||||
),
|
|
||||||
)
|
|
||||||
|
|
||||||
val encoded = json.encodeToString(MeasurementDocument.serializer(), doc)
|
|
||||||
val decoded = json.decodeFromString(MeasurementDocument.serializer(), encoded)
|
|
||||||
assertEquals(doc.run.id, decoded.run.id)
|
|
||||||
assertEquals(Transport.WIFI, decoded.networks[0].transport)
|
|
||||||
assertEquals(TestType.ICMP_PING4, decoded.tests[0].type)
|
|
||||||
// snake_case field names on the wire
|
|
||||||
assertTrue(encoded.contains("\"schema_version\""))
|
|
||||||
assertTrue(encoded.contains("\"mono_origin_wall\""))
|
|
||||||
assertTrue(encoded.contains("\"t_tx_ns\""))
|
|
||||||
// null preserved at train index 1
|
|
||||||
assertTrue(encoded.contains("[16500000,null]"))
|
|
||||||
}
|
|
||||||
|
|
||||||
@JTest
|
|
||||||
fun findingRequiresEvidence() {
|
|
||||||
try {
|
|
||||||
Finding(
|
|
||||||
id = "f-1", code = "x", category = Category.DNS, severity = Severity.INFO,
|
|
||||||
confidence = Confidence.LOW, title = "t", description = "d", evidenceRefs = emptyList(),
|
|
||||||
)
|
|
||||||
throw AssertionError("expected IllegalArgumentException for empty evidence_refs")
|
|
||||||
} catch (e: IllegalArgumentException) {
|
|
||||||
// expected — a finding with no evidence is invalid (§7.1)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
@@ -1,109 +0,0 @@
|
|||||||
// SPDX-FileCopyrightText: 2026 Echolot contributors
|
|
||||||
// SPDX-License-Identifier: GPL-3.0-or-later
|
|
||||||
|
|
||||||
package app.echo_lot.measurement
|
|
||||||
|
|
||||||
import kotlin.test.Test as JTest
|
|
||||||
import kotlin.test.assertEquals
|
|
||||||
|
|
||||||
class VerdictsTest {
|
|
||||||
|
|
||||||
private fun test(type: String, status: TestStatus, id: String = type): Test =
|
|
||||||
Test(id = id, type = type, tier = Tier.APP, startedMonoNs = 0, endedMonoNs = 1, status = status)
|
|
||||||
|
|
||||||
private fun finding(cat: Category, sev: Severity, id: String = "f-$cat-$sev"): Finding =
|
|
||||||
Finding(
|
|
||||||
id = id, code = "x.$cat", category = cat, severity = sev, confidence = Confidence.HIGH,
|
|
||||||
title = "t", description = "d", evidenceRefs = listOf(EvidenceRef("some-test")),
|
|
||||||
)
|
|
||||||
|
|
||||||
@JTest
|
|
||||||
fun categoryLightFromWorstSeverity() {
|
|
||||||
val tests = listOf(test(TestType.DNS_CANARY, TestStatus.OK))
|
|
||||||
val findings = listOf(
|
|
||||||
finding(Category.DNS, Severity.LOW),
|
|
||||||
finding(Category.DNS, Severity.HIGH), // worst → red
|
|
||||||
finding(Category.DNS, Severity.INFO),
|
|
||||||
)
|
|
||||||
val s = Verdicts.derive(tests, findings)
|
|
||||||
assertEquals(Verdict.RED, s.categories["dns"]!!.verdict)
|
|
||||||
assertEquals("f-DNS-HIGH", s.categories["dns"]!!.worstFinding)
|
|
||||||
assertEquals(Verdict.RED, s.overall)
|
|
||||||
}
|
|
||||||
|
|
||||||
@JTest
|
|
||||||
fun noFindingsIsGreen() {
|
|
||||||
val s = Verdicts.derive(listOf(test(TestType.MTU_BLACKHOLE, TestStatus.OK)), emptyList())
|
|
||||||
assertEquals(Verdict.GREEN, s.categories["mtu"]!!.verdict)
|
|
||||||
assertEquals(Verdict.GREEN, s.overall)
|
|
||||||
}
|
|
||||||
|
|
||||||
@JTest
|
|
||||||
fun mediumAndLowAreYellow() {
|
|
||||||
val s = Verdicts.derive(
|
|
||||||
listOf(test(TestType.SEC_HTTP_ECHO, TestStatus.OK)),
|
|
||||||
listOf(finding(Category.SECURITY, Severity.MEDIUM)),
|
|
||||||
)
|
|
||||||
assertEquals(Verdict.YELLOW, s.categories["security"]!!.verdict)
|
|
||||||
}
|
|
||||||
|
|
||||||
@JTest
|
|
||||||
fun majorityFailedIsInconclusive() {
|
|
||||||
// 2 of 3 dns tests failed → > 50% → inconclusive, even with a finding present.
|
|
||||||
val tests = listOf(
|
|
||||||
test(TestType.DNS_CANARY, TestStatus.FAILED, "a"),
|
|
||||||
test(TestType.DNS_TTL_INTEGRITY, TestStatus.UNSUPPORTED, "b"),
|
|
||||||
test(TestType.DNS_COMPARE, TestStatus.OK, "c"),
|
|
||||||
)
|
|
||||||
val s = Verdicts.derive(tests, listOf(finding(Category.DNS, Severity.HIGH)))
|
|
||||||
assertEquals(Verdict.INCONCLUSIVE, s.categories["dns"]!!.verdict)
|
|
||||||
assertEquals(2, s.categories["dns"]!!.testsFailed)
|
|
||||||
assertEquals(3, s.categories["dns"]!!.testsRun)
|
|
||||||
}
|
|
||||||
|
|
||||||
@JTest
|
|
||||||
fun exactlyHalfFailedIsNotInconclusive() {
|
|
||||||
// 1 of 2 failed → not > 50% → the finding decides.
|
|
||||||
val tests = listOf(
|
|
||||||
test(TestType.NAT_HAIRPIN, TestStatus.FAILED, "a"),
|
|
||||||
test(TestType.NAT_CONNECT_BACK, TestStatus.OK, "b"),
|
|
||||||
)
|
|
||||||
val s = Verdicts.derive(tests, listOf(finding(Category.NAT, Severity.CRITICAL)))
|
|
||||||
assertEquals(Verdict.RED, s.categories["nat"]!!.verdict)
|
|
||||||
}
|
|
||||||
|
|
||||||
@JTest
|
|
||||||
fun overallIsWorstCategory() {
|
|
||||||
val tests = listOf(
|
|
||||||
test(TestType.DNS_CANARY, TestStatus.OK, "d"),
|
|
||||||
test(TestType.MTU_BLACKHOLE, TestStatus.OK, "m"),
|
|
||||||
)
|
|
||||||
val findings = listOf(
|
|
||||||
finding(Category.DNS, Severity.MEDIUM), // yellow
|
|
||||||
finding(Category.MTU, Severity.CRITICAL), // red
|
|
||||||
)
|
|
||||||
val s = Verdicts.derive(tests, findings)
|
|
||||||
assertEquals(Verdict.RED, s.overall)
|
|
||||||
}
|
|
||||||
|
|
||||||
@JTest
|
|
||||||
fun overallInconclusiveOnlyWhenAllAre() {
|
|
||||||
val tests = listOf(
|
|
||||||
test(TestType.DNS_CANARY, TestStatus.FAILED, "d"), // dns inconclusive
|
|
||||||
test(TestType.MTU_BLACKHOLE, TestStatus.OK, "m"), // mtu green
|
|
||||||
)
|
|
||||||
val s = Verdicts.derive(tests, emptyList())
|
|
||||||
assertEquals(Verdict.INCONCLUSIVE, s.categories["dns"]!!.verdict)
|
|
||||||
assertEquals(Verdict.GREEN, s.categories["mtu"]!!.verdict)
|
|
||||||
assertEquals(Verdict.GREEN, s.overall) // not all inconclusive → mtu decides
|
|
||||||
}
|
|
||||||
|
|
||||||
@JTest
|
|
||||||
fun categoryMappingCoversFamilies() {
|
|
||||||
assertEquals(Category.CONNECTIVITY, TestType.category(TestType.TRACEROUTE_UDP4))
|
|
||||||
assertEquals(Category.IPV6, TestType.category(TestType.V6_BROKENNESS))
|
|
||||||
assertEquals(Category.LOCAL, TestType.category(TestType.PEER_MULTICAST))
|
|
||||||
assertEquals(Category.PERFORMANCE, TestType.category(TestType.PERF_BUFFERBLOAT))
|
|
||||||
assertEquals(Category.CONNECTIVITY, TestType.category(TestType.TIME_SERVER_OFFSET))
|
|
||||||
}
|
|
||||||
}
|
|
||||||
+205
@@ -0,0 +1,205 @@
|
|||||||
|
// SPDX-FileCopyrightText: 2026 Echolot contributors
|
||||||
|
// SPDX-License-Identifier: GPL-3.0-or-later
|
||||||
|
|
||||||
|
package app.echo_lot.measurement
|
||||||
|
|
||||||
|
/**
|
||||||
|
* The registry of finding codes (measurement-schema.md §9, open item 1).
|
||||||
|
*
|
||||||
|
* A finding code is the stable, machine-readable half of a result: the prose changes, the code is
|
||||||
|
* what a dashboard groups by and what someone greps a year of archived runs for. That only holds
|
||||||
|
* if a code means exactly one thing forever — which is not something ad-hoc string literals at
|
||||||
|
* fifteen call sites can promise.
|
||||||
|
*
|
||||||
|
* The failure this exists to prevent had already happened by the time it was written. Two
|
||||||
|
* independently-added emitters produced `connectivity.downstream_loss` and
|
||||||
|
* `connectivity.loss_downstream` for the same concept, and nothing anywhere objected. Anyone
|
||||||
|
* aggregating either one would have silently seen half their data.
|
||||||
|
*
|
||||||
|
* So codes are declared here as typed specs, each carrying its category and default severity, and
|
||||||
|
* emitters reference the spec rather than retyping the string. That makes a typo a compile error,
|
||||||
|
* and makes it impossible for two call sites to disagree about which category a finding belongs
|
||||||
|
* to — a disagreement that would otherwise split one fault across two verdict lights.
|
||||||
|
*/
|
||||||
|
data class FindingSpec(
|
||||||
|
val code: String,
|
||||||
|
val category: Category,
|
||||||
|
/** Severity when nothing about the specific run argues otherwise; emitters may escalate. */
|
||||||
|
val severity: Severity,
|
||||||
|
/** One line: what this finding asserts. Present tense, no hedging. */
|
||||||
|
val meaning: String,
|
||||||
|
/**
|
||||||
|
* What the finding rules *out*, where that is the useful half. "Loss upstream" is worth much
|
||||||
|
* more when it also says the return path is fine, because that halves where to look next.
|
||||||
|
*/
|
||||||
|
val rulesOut: String? = null,
|
||||||
|
)
|
||||||
|
|
||||||
|
object FindingRegistry {
|
||||||
|
|
||||||
|
// ---- connectivity ----------------------------------------------------------------
|
||||||
|
|
||||||
|
// Renamed from nat.* before anything shipped: neither of these is about NAT, and the
|
||||||
|
// prefix is what decides which category - and therefore which verdict light - a finding
|
||||||
|
// rolls up into. A nat.* code landing under connectivity would be a permanent puzzle.
|
||||||
|
val UDP_UNREACHABLE = FindingSpec(
|
||||||
|
"connectivity.udp_unreachable", Category.CONNECTIVITY, Severity.HIGH,
|
||||||
|
"No UDP echo replies came back from the server at all.",
|
||||||
|
)
|
||||||
|
|
||||||
|
val UDP_UNREACHABLE_UPSTREAM = FindingSpec(
|
||||||
|
"connectivity.udp_unreachable_upstream", Category.CONNECTIVITY, Severity.HIGH,
|
||||||
|
"The server received none of the probes, so traffic is dropped on the way out.",
|
||||||
|
rulesOut = "The return path: nothing arrived to be replied to.",
|
||||||
|
)
|
||||||
|
|
||||||
|
val UDP_LOSS = FindingSpec(
|
||||||
|
"connectivity.udp_loss", Category.CONNECTIVITY, Severity.MEDIUM,
|
||||||
|
"A large fraction of round-trip probes were lost, direction unknown.",
|
||||||
|
)
|
||||||
|
|
||||||
|
val LOSS_UPSTREAM = FindingSpec(
|
||||||
|
"connectivity.loss_upstream", Category.CONNECTIVITY, Severity.MEDIUM,
|
||||||
|
"Probes were lost on the way to the server.",
|
||||||
|
rulesOut = "The return path: replies came back for everything that arrived.",
|
||||||
|
)
|
||||||
|
|
||||||
|
/**
|
||||||
|
* The single code for "lost on the return path", whichever measurement found it.
|
||||||
|
*
|
||||||
|
* Two emitters had independently invented `connectivity.downstream_loss` and
|
||||||
|
* `connectivity.loss_downstream` for this, and nothing objected. Anyone aggregating either
|
||||||
|
* one would have silently seen half their data. Paired with [LOSS_UPSTREAM] so the two
|
||||||
|
* directions read as a set.
|
||||||
|
*/
|
||||||
|
val LOSS_DOWNSTREAM = FindingSpec(
|
||||||
|
"connectivity.loss_downstream", Category.CONNECTIVITY, Severity.MEDIUM,
|
||||||
|
"Packets were lost on the way back from the server.",
|
||||||
|
rulesOut = "The outbound path: the server received what it was answering.",
|
||||||
|
)
|
||||||
|
|
||||||
|
val DOWNSTREAM_BLOCKED = FindingSpec(
|
||||||
|
"connectivity.downstream_blocked", Category.CONNECTIVITY, Severity.HIGH,
|
||||||
|
"Server-initiated packets never arrive, although round trips work.",
|
||||||
|
rulesOut = "Basic reachability: the path forwards replies, just not unsolicited traffic.",
|
||||||
|
)
|
||||||
|
|
||||||
|
val DOWNSTREAM_REORDER = FindingSpec(
|
||||||
|
"connectivity.downstream_reorder", Category.CONNECTIVITY, Severity.LOW,
|
||||||
|
"Downstream packets arrive in a different order than they were sent.",
|
||||||
|
)
|
||||||
|
|
||||||
|
// MEDIUM, not HIGH: a captive portal is a condition to report, not necessarily a fault - on
|
||||||
|
// hotel or cafe wifi it is exactly what should be there, and logging in clears it. NO_INTERNET
|
||||||
|
// is the HIGH one, because nothing the user does locally fixes that. The registry first said
|
||||||
|
// HIGH; the probe emitting it had always said MEDIUM, and the probe was the considered value.
|
||||||
|
val CAPTIVE_PORTAL = FindingSpec(
|
||||||
|
"connectivity.captive_portal", Category.CONNECTIVITY, Severity.MEDIUM,
|
||||||
|
"A captive portal is intercepting connectivity checks.",
|
||||||
|
)
|
||||||
|
|
||||||
|
val NO_INTERNET = FindingSpec(
|
||||||
|
"connectivity.no_internet", Category.CONNECTIVITY, Severity.HIGH,
|
||||||
|
"Android's own connectivity checks fail on this network.",
|
||||||
|
)
|
||||||
|
|
||||||
|
// ---- mtu -------------------------------------------------------------------------
|
||||||
|
|
||||||
|
val MTU_REDUCED_DOWNSTREAM = FindingSpec(
|
||||||
|
"mtu.reduced_downstream", Category.MTU, Severity.LOW,
|
||||||
|
"The downstream path MTU is below the usual 1500 bytes.",
|
||||||
|
)
|
||||||
|
|
||||||
|
val MTU_DOWNSTREAM_BLACKHOLE = FindingSpec(
|
||||||
|
"mtu.downstream_blackhole", Category.MTU, Severity.MEDIUM,
|
||||||
|
"Datagrams above the path MTU are dropped downstream, fragmented or not.",
|
||||||
|
)
|
||||||
|
|
||||||
|
val FRAGMENTS_BLOCKED = FindingSpec(
|
||||||
|
"mtu.fragments_blocked", Category.MTU, Severity.MEDIUM,
|
||||||
|
"IP fragments do not reach this device even when sent in order.",
|
||||||
|
)
|
||||||
|
|
||||||
|
val FRAGMENT_REORDER_SENSITIVE = FindingSpec(
|
||||||
|
"mtu.fragment_reorder_sensitive", Category.MTU, Severity.LOW,
|
||||||
|
"Fragments are delivered in order but dropped when reordered or delayed.",
|
||||||
|
rulesOut = "Fragmentation itself: in-order fragments arrive fine.",
|
||||||
|
)
|
||||||
|
|
||||||
|
// ---- nat -------------------------------------------------------------------------
|
||||||
|
|
||||||
|
val NAT_UDP_REBINDING = FindingSpec(
|
||||||
|
"nat.udp_rebinding", Category.NAT, Severity.MEDIUM,
|
||||||
|
"A NAT remapped the UDP source port mid-flow.",
|
||||||
|
)
|
||||||
|
|
||||||
|
val NAT_SYMMETRIC = FindingSpec(
|
||||||
|
"nat.symmetric", Category.NAT, Severity.MEDIUM,
|
||||||
|
"The NAT assigns a different external port per destination.",
|
||||||
|
)
|
||||||
|
|
||||||
|
// ---- perf ------------------------------------------------------------------------
|
||||||
|
|
||||||
|
val THROUGHPUT_NO_DELIVERY = FindingSpec(
|
||||||
|
"perf.throughput_no_delivery", Category.PERFORMANCE, Severity.HIGH,
|
||||||
|
"No throughput traffic arrived, although the server sent it.",
|
||||||
|
)
|
||||||
|
|
||||||
|
val THROUGHPUT_BELOW_OFFERED = FindingSpec(
|
||||||
|
"perf.throughput_below_offered", Category.PERFORMANCE, Severity.LOW,
|
||||||
|
"Less throughput arrived than the server sent for the whole run.",
|
||||||
|
)
|
||||||
|
|
||||||
|
// ---- dns -------------------------------------------------------------------------
|
||||||
|
|
||||||
|
val DNS_ANSWER_REWRITTEN = FindingSpec(
|
||||||
|
"dns.answer_rewritten", Category.DNS, Severity.HIGH,
|
||||||
|
"A resolver returned an answer that differs from the authoritative record.",
|
||||||
|
)
|
||||||
|
|
||||||
|
val DNS_AUTHORITATIVE_UNREACHABLE = FindingSpec(
|
||||||
|
"dns.authoritative_unreachable", Category.DNS, Severity.MEDIUM,
|
||||||
|
"The canary zone's authoritative server could not be reached.",
|
||||||
|
)
|
||||||
|
|
||||||
|
// ---- v6 ----------------------------------------------------------------------------
|
||||||
|
//
|
||||||
|
// Prefix is `v6.`, matching the test-type registry (v6.brokenness, v6.happy_eyeballs, ...).
|
||||||
|
// These were `ipv6.*` while declaring Category.IPV6, but the prefix map only knows "v6", so
|
||||||
|
// they silently rolled up under connectivity: the third instance of a prefix disagreeing with
|
||||||
|
// its category and quietly moving a fault to a different verdict light.
|
||||||
|
|
||||||
|
val V6_BROKEN = FindingSpec(
|
||||||
|
"v6.broken", Category.IPV6, Severity.MEDIUM,
|
||||||
|
"IPv6 is configured on this network but does not work.",
|
||||||
|
rulesOut = "Absence of IPv6: it is provisioned, it simply fails.",
|
||||||
|
)
|
||||||
|
|
||||||
|
/**
|
||||||
|
* INFO deliberately, and it needs to stay that way.
|
||||||
|
*
|
||||||
|
* Most networks still do not offer IPv6, and that is not a fault. Reporting it as a warning
|
||||||
|
* lights a yellow verdict on a perfectly healthy network, which teaches people to ignore the
|
||||||
|
* light — the one thing a diagnostic must never do.
|
||||||
|
*/
|
||||||
|
val V6_NOT_OFFERED = FindingSpec(
|
||||||
|
"v6.not_offered", Category.IPV6, Severity.INFO,
|
||||||
|
"This network does not offer IPv6.",
|
||||||
|
)
|
||||||
|
|
||||||
|
/** Every registered finding, in declaration order. */
|
||||||
|
val all: List<FindingSpec> = listOf(
|
||||||
|
UDP_UNREACHABLE, UDP_UNREACHABLE_UPSTREAM, UDP_LOSS, LOSS_UPSTREAM, LOSS_DOWNSTREAM,
|
||||||
|
DOWNSTREAM_BLOCKED, DOWNSTREAM_REORDER, CAPTIVE_PORTAL, NO_INTERNET,
|
||||||
|
MTU_REDUCED_DOWNSTREAM, MTU_DOWNSTREAM_BLACKHOLE, FRAGMENTS_BLOCKED,
|
||||||
|
FRAGMENT_REORDER_SENSITIVE,
|
||||||
|
NAT_UDP_REBINDING, NAT_SYMMETRIC,
|
||||||
|
THROUGHPUT_NO_DELIVERY, THROUGHPUT_BELOW_OFFERED,
|
||||||
|
DNS_ANSWER_REWRITTEN, DNS_AUTHORITATIVE_UNREACHABLE,
|
||||||
|
V6_BROKEN, V6_NOT_OFFERED,
|
||||||
|
)
|
||||||
|
|
||||||
|
private val byCode: Map<String, FindingSpec> = all.associateBy { it.code }
|
||||||
|
|
||||||
|
fun byCode(code: String): FindingSpec? = byCode[code]
|
||||||
|
}
|
||||||
@@ -69,12 +69,16 @@ object TestType {
|
|||||||
const val TRACEROUTE_ICMP6 = "traceroute.icmp6"
|
const val TRACEROUTE_ICMP6 = "traceroute.icmp6"
|
||||||
// train
|
// train
|
||||||
const val TRAIN_UDP_UPDOWN = "train.udp_updown"
|
const val TRAIN_UDP_UPDOWN = "train.udp_updown"
|
||||||
|
/** Server-to-client train under a §3.4 grant: the direction a round trip cannot separate. */
|
||||||
|
const val TRAIN_UDP_DOWNSTREAM = "train.udp_downstream"
|
||||||
// mtu
|
// mtu
|
||||||
const val MTU_PMTUD_UP = "mtu.pmtud_up"
|
const val MTU_PMTUD_UP = "mtu.pmtud_up"
|
||||||
const val MTU_PMTUD_DOWN = "mtu.pmtud_down"
|
const val MTU_PMTUD_DOWN = "mtu.pmtud_down"
|
||||||
const val MTU_BLACKHOLE = "mtu.blackhole"
|
const val MTU_BLACKHOLE = "mtu.blackhole"
|
||||||
const val MTU_MSS_OBSERVED = "mtu.mss_observed"
|
const val MTU_MSS_OBSERVED = "mtu.mss_observed"
|
||||||
const val MTU_FRAG_DELIVERY = "mtu.frag_delivery"
|
const val MTU_FRAG_DELIVERY = "mtu.frag_delivery"
|
||||||
|
/** Whether fragments survive arriving out of order, not merely whether they survive. */
|
||||||
|
const val MTU_FRAG_ORDERING = "mtu.frag_ordering"
|
||||||
// nat
|
// nat
|
||||||
const val NAT_STUN_5780 = "nat.stun_5780"
|
const val NAT_STUN_5780 = "nat.stun_5780"
|
||||||
const val NAT_MAPPING_LIFETIME_UDP = "nat.mapping_lifetime_udp"
|
const val NAT_MAPPING_LIFETIME_UDP = "nat.mapping_lifetime_udp"
|
||||||
|
|||||||
+133
@@ -0,0 +1,133 @@
|
|||||||
|
// SPDX-FileCopyrightText: 2026 Echolot contributors
|
||||||
|
// SPDX-License-Identifier: GPL-3.0-or-later
|
||||||
|
|
||||||
|
package app.echo_lot.measurement
|
||||||
|
|
||||||
|
import java.io.File
|
||||||
|
import kotlin.test.Test
|
||||||
|
import kotlin.test.assertEquals
|
||||||
|
import kotlin.test.assertTrue
|
||||||
|
import kotlin.test.fail
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Keeps the finding registry honest.
|
||||||
|
*
|
||||||
|
* The interesting test is the last one: it reads `docs/findings-registry.md` and fails when the
|
||||||
|
* document and the code disagree. Documentation that drifts from its implementation is worse than
|
||||||
|
* none, because it still looks authoritative — and a finding registry is precisely the artifact
|
||||||
|
* other people build tooling against.
|
||||||
|
*/
|
||||||
|
class FindingRegistryTest {
|
||||||
|
|
||||||
|
@Test
|
||||||
|
fun codesAreUnique() {
|
||||||
|
val dupes = FindingRegistry.all.groupBy { it.code }.filterValues { it.size > 1 }.keys
|
||||||
|
assertTrue(dupes.isEmpty(), "duplicate finding codes: $dupes")
|
||||||
|
}
|
||||||
|
|
||||||
|
@Test
|
||||||
|
fun everyDeclaredSpecIsInTheAllList() {
|
||||||
|
// Reflection over the object's properties: a spec that is declared but left out of `all`
|
||||||
|
// is invisible to the doc check and to any consumer enumerating the registry.
|
||||||
|
val declared = FindingRegistry::class.java.declaredMethods
|
||||||
|
.filter { it.parameterCount == 0 && it.returnType == FindingSpec::class.java }
|
||||||
|
.mapNotNull { runCatching { it.invoke(FindingRegistry) as FindingSpec }.getOrNull() }
|
||||||
|
.map { it.code }
|
||||||
|
.toSet()
|
||||||
|
val listed = FindingRegistry.all.map { it.code }.toSet()
|
||||||
|
assertEquals(declared, listed, "declared specs and the `all` list disagree")
|
||||||
|
}
|
||||||
|
|
||||||
|
// The prefix decides the category, and the category decides which verdict light the finding
|
||||||
|
// rolls up into. A code whose prefix disagrees with its category silently moves a fault to a
|
||||||
|
// different light — the exact bug that got two codes renamed out of nat.*.
|
||||||
|
@Test
|
||||||
|
fun everyPrefixMatchesItsCategory() {
|
||||||
|
for (spec in FindingRegistry.all) {
|
||||||
|
val fromPrefix = TestType.category(spec.code)
|
||||||
|
assertEquals(
|
||||||
|
fromPrefix, spec.category,
|
||||||
|
"${spec.code} is declared as ${spec.category} but its prefix maps to $fromPrefix",
|
||||||
|
)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
@Test
|
||||||
|
fun codesFollowTheNamingConvention() {
|
||||||
|
val shape = Regex("^[a-z0-9]+\\.[a-z0-9_]+$")
|
||||||
|
for (spec in FindingRegistry.all) {
|
||||||
|
assertTrue(shape.matches(spec.code), "malformed code: ${spec.code}")
|
||||||
|
assertTrue(spec.meaning.isNotBlank(), "${spec.code} has no meaning")
|
||||||
|
assertTrue(
|
||||||
|
spec.meaning.trimEnd().endsWith("."),
|
||||||
|
"${spec.code}'s meaning should be a sentence: '${spec.meaning}'",
|
||||||
|
)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Two near-identical codes are how one fault ends up split across two dashboards. This is a
|
||||||
|
// blunt check — it will not catch every synonym — but it catches the shape that already
|
||||||
|
// happened: the same words in a different order.
|
||||||
|
@Test
|
||||||
|
fun noTwoCodesAreAnagramsOfEachOther() {
|
||||||
|
val normalised = FindingRegistry.all.associate { spec ->
|
||||||
|
spec.code to spec.code.substringAfter('.').split('_').sorted().joinToString("_")
|
||||||
|
}
|
||||||
|
val clashes = normalised.entries.groupBy { it.value }.filterValues { it.size > 1 }
|
||||||
|
if (clashes.isNotEmpty()) {
|
||||||
|
fail("codes differing only in word order: ${clashes.values.map { g -> g.map { it.key } }}")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
@Test
|
||||||
|
fun theDocumentAndTheRegistryAgree() {
|
||||||
|
val doc = findDoc() ?: run {
|
||||||
|
println("findings-registry.md not found from ${File(".").absolutePath} — skipping")
|
||||||
|
return
|
||||||
|
}
|
||||||
|
val text = doc.readText()
|
||||||
|
|
||||||
|
// Only table rows count as "documented". Prose may legitimately mention a code that no
|
||||||
|
// longer exists — the rules section explains why two were merged — and treating that as
|
||||||
|
// a registry entry would force the document to forget its own history.
|
||||||
|
val documented = text.lines()
|
||||||
|
.filter { it.trimStart().startsWith("|") }
|
||||||
|
.flatMap { row -> Regex("`([a-z0-9]+\\.[a-z0-9_]+)`").findAll(row).map { it.groupValues[1] } }
|
||||||
|
.toSet()
|
||||||
|
val registered = FindingRegistry.all.map { it.code }.toSet()
|
||||||
|
|
||||||
|
val missingFromDoc = registered - documented
|
||||||
|
val missingFromCode = documented - registered
|
||||||
|
assertTrue(
|
||||||
|
missingFromDoc.isEmpty(),
|
||||||
|
"these codes exist in FindingRegistry but not in docs/findings-registry.md: $missingFromDoc",
|
||||||
|
)
|
||||||
|
assertTrue(
|
||||||
|
missingFromCode.isEmpty(),
|
||||||
|
"docs/findings-registry.md documents codes that no longer exist: $missingFromCode",
|
||||||
|
)
|
||||||
|
|
||||||
|
// And the severities must match, or the document is describing a different system.
|
||||||
|
for (spec in FindingRegistry.all) {
|
||||||
|
val row = text.lines().firstOrNull {
|
||||||
|
it.trimStart().startsWith("|") && it.contains("`${spec.code}`")
|
||||||
|
} ?: continue
|
||||||
|
val severity = spec.severity.name.lowercase()
|
||||||
|
assertTrue(
|
||||||
|
row.contains("| $severity |"),
|
||||||
|
"${spec.code} is ${severity} in code but the doc row says otherwise: $row",
|
||||||
|
)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Walks up from the test's working directory to find the repo's docs/ folder. */
|
||||||
|
private fun findDoc(): File? {
|
||||||
|
var dir: File? = File(".").absoluteFile
|
||||||
|
repeat(6) {
|
||||||
|
val candidate = File(dir, "docs/findings-registry.md")
|
||||||
|
if (candidate.isFile) return candidate
|
||||||
|
dir = dir?.parentFile
|
||||||
|
}
|
||||||
|
return null
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -108,8 +108,14 @@ class Anonymizer(private val level: PrivacyLevel, private val salt: Salt) {
|
|||||||
is JsonObject -> walkObject(v, path)
|
is JsonObject -> walkObject(v, path)
|
||||||
is JsonArray -> JsonArray(v.map { walk(key, it, path) })
|
is JsonArray -> JsonArray(v.map { walk(key, it, path) })
|
||||||
is JsonPrimitive ->
|
is JsonPrimitive ->
|
||||||
if (v.isString) transform(Classification.typeOf(key, path), v.content).let(::JsonPrimitive)
|
if (v.isString) {
|
||||||
else v
|
// Name first (it is precise), then shape (it is exhaustive). A field nobody
|
||||||
|
// classified must not be a field that leaks.
|
||||||
|
val type = Classification.typeOf(key, path) ?: Classification.inferFromValue(v.content)
|
||||||
|
JsonPrimitive(transform(type, v.content))
|
||||||
|
} else {
|
||||||
|
v
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
private fun transform(type: LogicalType?, value: String): String = when (type) {
|
private fun transform(type: LogicalType?, value: String): String = when (type) {
|
||||||
@@ -147,6 +153,11 @@ class Anonymizer(private val level: PrivacyLevel, private val salt: Salt) {
|
|||||||
* Public addresses keep only their /16 so the network is still locatable at ISP granularity.
|
* Public addresses keep only their /16 so the network is still locatable at ISP granularity.
|
||||||
*/
|
*/
|
||||||
private fun ip4(value: String): String {
|
private fun ip4(value: String): String {
|
||||||
|
// A route destination carries a prefix length; pseudonymize the address and put it back,
|
||||||
|
// or "0.0.0.0/0" turns into nonsense and the routing table becomes unreadable.
|
||||||
|
value.substringAfter('/', "").takeIf { it.isNotEmpty() && value.contains('/') }?.let { len ->
|
||||||
|
return ip4(value.substringBefore('/')) + "/" + len
|
||||||
|
}
|
||||||
val o = value.split(".")
|
val o = value.split(".")
|
||||||
if (o.size != 4 || o.any { it.toIntOrNull() == null }) return value
|
if (o.size != 4 || o.any { it.toIntOrNull() == null }) return value
|
||||||
val n = o.map { it.toInt() }
|
val n = o.map { it.toInt() }
|
||||||
@@ -167,8 +178,36 @@ class Anonymizer(private val level: PrivacyLevel, private val salt: Salt) {
|
|||||||
* is a device fingerprint, especially with EUI-64.
|
* is a device fingerprint, especially with EUI-64.
|
||||||
*/
|
*/
|
||||||
private fun ip6(value: String): String {
|
private fun ip6(value: String): String {
|
||||||
|
// Dotted quads reach here through the family-agnostic field names (addr, gateway, dst);
|
||||||
|
// hand them to the IPv4 path rather than mangling them as if they were v6.
|
||||||
|
if (value.count { it == ':' } < 2) return ip4(value)
|
||||||
|
if (value.contains('/')) {
|
||||||
|
return ip6(value.substringBefore('/')) + "/" + value.substringAfter('/')
|
||||||
|
}
|
||||||
val v = value.lowercase(Locale.ROOT)
|
val v = value.lowercase(Locale.ROOT)
|
||||||
|
// The unspecified address and the default route are not identities; mangling them would
|
||||||
|
// make a routing table unreadable for no privacy gain.
|
||||||
if (v == "::1" || v == "::" || v.startsWith("fe80:") || v.startsWith("ff")) return v
|
if (v == "::1" || v == "::" || v.startsWith("fe80:") || v.startsWith("ff")) return v
|
||||||
|
|
||||||
|
// Unique local addresses (fc00::/7) need the *whole* prefix replaced, not the tail.
|
||||||
|
//
|
||||||
|
// They look like the v6 equivalent of RFC1918, and the first instinct is to keep them for
|
||||||
|
// the same reason: private, topological, says nothing about anyone. That reasoning does
|
||||||
|
// not carry over. An RFC1918 prefix is shared by millions of networks and identifies
|
||||||
|
// none of them; a ULA global ID is 40 *random* bits, unique to one network by
|
||||||
|
// construction (RFC 4193). It is a network fingerprint. Passing the leading groups
|
||||||
|
// through - which is what the general path does - leaked 32 of those 40 bits.
|
||||||
|
//
|
||||||
|
// The prefix is pseudonymized as a unit, so two addresses on the same ULA subnet still
|
||||||
|
// land on the same pseudonymous prefix. "These hosts are on one network" survives;
|
||||||
|
// "this is *that* network" does not.
|
||||||
|
if (v.startsWith("fc") || v.startsWith("fd")) {
|
||||||
|
val groups = v.substringBefore('%').split(":")
|
||||||
|
val prefix = pseudo("ula-prefix", groups.take(3).joinToString(":")) { it }
|
||||||
|
val host = pseudo("ula-host", v) { it }
|
||||||
|
return "fd${prefix.substring(0, 2)}:${prefix.substring(2, 6)}:${prefix.substring(6, 10)}" +
|
||||||
|
"::${host.substring(0, 4)}"
|
||||||
|
}
|
||||||
val groups = v.substringBefore('%').split(":")
|
val groups = v.substringBefore('%').split(":")
|
||||||
if (groups.size < 3) return v
|
if (groups.size < 3) return v
|
||||||
val h = pseudo("ip6", value) { it }
|
val h = pseudo("ip6", value) { it }
|
||||||
|
|||||||
@@ -32,6 +32,16 @@ object Classification {
|
|||||||
"link_local", "ra_source", "prefix",
|
"link_local", "ra_source", "prefix",
|
||||||
).forEach { put(it, LogicalType.IP6) }
|
).forEach { put(it, LogicalType.IP6) }
|
||||||
|
|
||||||
|
// Family-agnostic address fields — the names the models actually use (Address.addr,
|
||||||
|
// Route.gateway, Route.dst, DnsConfig.servers). Their absence here was a real leak: the
|
||||||
|
// device's own global IPv6 address went out verbatim at the level whose description
|
||||||
|
// promises addresses are pseudonymized. Typed IP6 because the transform detects the
|
||||||
|
// family from the value, falling through to the IPv4 path for a dotted quad.
|
||||||
|
listOf(
|
||||||
|
"addr", "address", "gateway", "dst", "src", "servers", "server", "resolver",
|
||||||
|
"next_hop", "via", "public_ip", "observed_ip",
|
||||||
|
).forEach { put(it, LogicalType.IP6) }
|
||||||
|
|
||||||
listOf("mac", "hw_addr", "gateway_mac", "router_mac", "sender_mac", "peer_mac")
|
listOf("mac", "hw_addr", "gateway_mac", "router_mac", "sender_mac", "peer_mac")
|
||||||
.forEach { put(it, LogicalType.MAC) }
|
.forEach { put(it, LogicalType.MAC) }
|
||||||
listOf("bssid", "ap_mac").forEach { put(it, LogicalType.BSSID) }
|
listOf("bssid", "ap_mac").forEach { put(it, LogicalType.BSSID) }
|
||||||
@@ -40,6 +50,8 @@ object Classification {
|
|||||||
listOf(
|
listOf(
|
||||||
"fqdn", "hostname", "host", "name", "reverse_dns", "ptr", "domain", "query_name",
|
"fqdn", "hostname", "host", "name", "reverse_dns", "ptr", "domain", "query_name",
|
||||||
"friendly_name", "server_name", "sni", "cname", "search_domain", "device_name",
|
"friendly_name", "server_name", "sni", "cname", "search_domain", "device_name",
|
||||||
|
// Plural and prefixed variants the models actually use.
|
||||||
|
"search_domains", "private_dns_hostname", "domains", "hostnames",
|
||||||
).forEach { put(it, LogicalType.FQDN) }
|
).forEach { put(it, LogicalType.FQDN) }
|
||||||
|
|
||||||
listOf("session_id", "credential", "token", "device_id", "android_id", "serial", "imsi", "iccid")
|
listOf("session_id", "credential", "token", "device_id", "android_id", "serial", "imsi", "iccid")
|
||||||
@@ -78,6 +90,49 @@ object Classification {
|
|||||||
return null
|
return null
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Last-resort classification from the *value*, when the field name is unrecognised.
|
||||||
|
*
|
||||||
|
* A name table can only protect fields somebody remembered to add, which is the wrong
|
||||||
|
* property for a privacy control: the dangerous field is the one nobody thought of. This
|
||||||
|
* exists because that failed once already — `addresses[].addr` holds the device's own global
|
||||||
|
* IPv6 address, the table had never heard of the name, and it went out verbatim.
|
||||||
|
*
|
||||||
|
* Only addresses and MACs are inferred, because only those have shapes that cannot be
|
||||||
|
* mistaken for something else. Hostnames deliberately are not: `train.udp_updown` is
|
||||||
|
* indistinguishable from a domain by shape, and mangling a test type would corrupt the
|
||||||
|
* document to protect nothing.
|
||||||
|
*/
|
||||||
|
fun inferFromValue(value: String): LogicalType? {
|
||||||
|
val v = value.trim()
|
||||||
|
if (v.isEmpty() || v.length > 64) return null
|
||||||
|
if (looksLikeMac(v)) return LogicalType.MAC
|
||||||
|
if (looksLikeIp6(v)) return LogicalType.IP6
|
||||||
|
if (looksLikeIp4(v)) return LogicalType.IP4
|
||||||
|
return null
|
||||||
|
}
|
||||||
|
|
||||||
|
private fun isHex(c: Char) = c in '0'..'9' || c in 'a'..'f' || c in 'A'..'F'
|
||||||
|
|
||||||
|
private fun looksLikeMac(v: String): Boolean {
|
||||||
|
val parts = v.split(':', '-')
|
||||||
|
return parts.size == 6 && parts.all { p -> p.length == 2 && p.all(::isHex) }
|
||||||
|
}
|
||||||
|
|
||||||
|
private fun looksLikeIp4(v: String): Boolean {
|
||||||
|
val parts = v.substringBefore('/').split('.')
|
||||||
|
return parts.size == 4 && parts.all { p ->
|
||||||
|
p.isNotEmpty() && p.length <= 3 && p.all(Char::isDigit) && p.toInt() <= 255
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
private fun looksLikeIp6(v: String): Boolean {
|
||||||
|
val core = v.substringBefore('/').substringBefore('%')
|
||||||
|
// Two colons minimum, so a time or a MAC fragment does not qualify, and nothing but the
|
||||||
|
// characters an address may contain.
|
||||||
|
return core.count { it == ':' } >= 2 && core.all { it == ':' || isHex(it) }
|
||||||
|
}
|
||||||
|
|
||||||
fun dropAtBalanced(path: List<String>): Boolean {
|
fun dropAtBalanced(path: List<String>): Boolean {
|
||||||
if (path.isNotEmpty() && path.last() in droppedKeys) return true
|
if (path.isNotEmpty() && path.last() in droppedKeys) return true
|
||||||
return droppedPaths.any { dropped -> dropped.all { path.contains(it) } }
|
return droppedPaths.any { dropped -> dropped.all { path.contains(it) } }
|
||||||
|
|||||||
@@ -182,4 +182,47 @@ class AnonymizerTest {
|
|||||||
assertEquals(PrivacyLevel.BALANCED, PrivacyLevel.max(PrivacyLevel.BALANCED, PrivacyLevel.FULL))
|
assertEquals(PrivacyLevel.BALANCED, PrivacyLevel.max(PrivacyLevel.BALANCED, PrivacyLevel.FULL))
|
||||||
assertEquals(PrivacyLevel.FULL, PrivacyLevel.fromWire("nonsense"))
|
assertEquals(PrivacyLevel.FULL, PrivacyLevel.fromWire("nonsense"))
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// A ULA looks like the v6 RFC1918 and is not. Its global ID is 40 random bits, unique to one
|
||||||
|
// network by construction (RFC 4193), so the prefix IS the identifier - unlike 192.168.x,
|
||||||
|
// which millions of networks share. Passing the leading groups through leaked most of it.
|
||||||
|
@Test
|
||||||
|
fun ulaPrefixesArePseudonymizedWhole() {
|
||||||
|
val doc = json.parseToJsonElement(
|
||||||
|
"""{"run":{"id":"r"},"networks":[{"link":{"dns":{"servers":["fda1:3fb1:ff92:6696::2662"]}}}]}"""
|
||||||
|
).jsonObject
|
||||||
|
val out = flat(anon(PrivacyLevel.BALANCED, doc))
|
||||||
|
assertFalse(out.contains("fda1"), "the ULA global ID survived: $out")
|
||||||
|
assertFalse(out.contains("3fb1"), "part of the ULA global ID survived: $out")
|
||||||
|
assertTrue(out.contains("fd"), "the result should still read as a ULA: $out")
|
||||||
|
}
|
||||||
|
|
||||||
|
// Pseudonymizing the prefix as a unit keeps the one fact that is diagnostically useful:
|
||||||
|
// whether two addresses sit on the same network.
|
||||||
|
@Test
|
||||||
|
fun addressesOnOneUlaSubnetStayRelated() {
|
||||||
|
val doc = json.parseToJsonElement(
|
||||||
|
"""{"run":{"id":"r"},"networks":[{"link":{"dns":{"servers":[
|
||||||
|
"fda1:3fb1:ff92:6696::1","fda1:3fb1:ff92:6696::2","fdff:9999:8888:7777::1"]}}}]}"""
|
||||||
|
).jsonObject
|
||||||
|
val servers = anon(PrivacyLevel.BALANCED, doc)["networks"]!!.jsonArray[0].jsonObject["link"]!!
|
||||||
|
.jsonObject["dns"]!!.jsonObject["servers"]!!.jsonArray.map { it.jsonPrimitive.content }
|
||||||
|
val prefixOf = { s: String -> s.substringBeforeLast("::") }
|
||||||
|
assertEquals(prefixOf(servers[0]), prefixOf(servers[1]),
|
||||||
|
"two addresses on one ULA subnet should share a pseudonymous prefix")
|
||||||
|
assertNotEquals(prefixOf(servers[0]), prefixOf(servers[2]),
|
||||||
|
"a different ULA network must not collide with the first")
|
||||||
|
}
|
||||||
|
|
||||||
|
// RFC1918 stays readable, and this is the contrast that justifies it: a shared, meaningless
|
||||||
|
// prefix is topology; a unique random one is identity.
|
||||||
|
@Test
|
||||||
|
fun rfc1918StaysReadableUnlikeUla() {
|
||||||
|
val doc = json.parseToJsonElement(
|
||||||
|
"""{"run":{"id":"r"},"networks":[{"link":{"dns":{"servers":["192.168.1.1","10.13.102.1"]}}}]}"""
|
||||||
|
).jsonObject
|
||||||
|
val out = flat(anon(PrivacyLevel.BALANCED, doc))
|
||||||
|
assertTrue(out.contains("192.168.1.1"), "RFC1918 should survive: $out")
|
||||||
|
assertTrue(out.contains("10.13.102.1"), "RFC1918 should survive: $out")
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -0,0 +1,113 @@
|
|||||||
|
// SPDX-FileCopyrightText: 2026 Echolot contributors
|
||||||
|
// SPDX-License-Identifier: GPL-3.0-or-later
|
||||||
|
|
||||||
|
package app.echo_lot.privacy
|
||||||
|
|
||||||
|
import kotlinx.serialization.json.Json
|
||||||
|
import kotlinx.serialization.json.jsonObject
|
||||||
|
import kotlin.test.Test
|
||||||
|
import kotlin.test.assertTrue
|
||||||
|
|
||||||
|
/**
|
||||||
|
* The blunt instrument: build a document with identifying values in every place one can actually
|
||||||
|
* occur, anonymize it, and assert none of them survive.
|
||||||
|
*
|
||||||
|
* [AnonymizerTest] checks that the fields the classification table knows about are handled
|
||||||
|
* correctly. This checks the other half — the fields it does *not* know about. A per-field test
|
||||||
|
* can only fail for a field someone remembered to write a case for, which is exactly the wrong
|
||||||
|
* property for a privacy check: the dangerous field is the one nobody thought of.
|
||||||
|
*
|
||||||
|
* Concretely, this is written the way it is because the schema's own field names disagree with
|
||||||
|
* the classifier's. `Address.addr` carries an IP and is documented as such in
|
||||||
|
* measurement-schema.md §8, but the classifier keys on names like `ip4` and `gateway_ip4` and had
|
||||||
|
* never heard of `addr`.
|
||||||
|
*/
|
||||||
|
class LeakTest {
|
||||||
|
|
||||||
|
private val json = Json { prettyPrint = false }
|
||||||
|
private val salt = Salt.perRun(ByteArray(32) { 3 })
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Every string here is something that identifies a person, a household or a device, placed
|
||||||
|
* where the real models actually put it (`core-measurement`'s Network/Link/Address/DnsConfig).
|
||||||
|
*/
|
||||||
|
private val secrets = listOf(
|
||||||
|
"Rambossek WLAN", // ssid
|
||||||
|
"78:9a:18:aa:bb:cc", // bssid
|
||||||
|
"aa:bb:cc:dd:ee:11", // gateway mac
|
||||||
|
"2001:1ad0:c4fe:6767::150", // global v6 address on the interface
|
||||||
|
"2a02:1748:dead:beef::1", // v6 default gateway
|
||||||
|
"203.0.113.77", // public v4
|
||||||
|
"nas.rambossek.lan", // private-dns hostname
|
||||||
|
"rambossek.lan", // search domain
|
||||||
|
"Anna's Chromecast", // neighbour name
|
||||||
|
"kitchen table", // free-text note
|
||||||
|
)
|
||||||
|
|
||||||
|
private fun document(): String = """
|
||||||
|
{
|
||||||
|
"schema": "echolot/measurement",
|
||||||
|
"run": {
|
||||||
|
"id": "run-1", "trigger": "manual", "notes": "${secrets[9]}",
|
||||||
|
"device": {"manufacturer": "OnePlus", "model": "CPH2747"}
|
||||||
|
},
|
||||||
|
"networks": [{
|
||||||
|
"id": "net-1", "transport": "wifi",
|
||||||
|
"link": {
|
||||||
|
"mtu": 1500,
|
||||||
|
"addresses": [
|
||||||
|
{"addr": "${secrets[3]}", "prefix_len": 64, "scope": "global"},
|
||||||
|
{"addr": "192.168.1.44", "prefix_len": 24, "scope": "global"}
|
||||||
|
],
|
||||||
|
"routes": [
|
||||||
|
{"dst": "::/0", "gateway": "${secrets[4]}", "iface": "wlan0"},
|
||||||
|
{"dst": "0.0.0.0/0", "gateway": "192.168.1.1", "iface": "wlan0"}
|
||||||
|
],
|
||||||
|
"dns": {
|
||||||
|
"servers": ["${secrets[5]}", "192.168.1.1"],
|
||||||
|
"private_dns_hostname": "${secrets[6]}",
|
||||||
|
"search_domains": ["${secrets[7]}"]
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"wifi": {"ssid": "${secrets[0]}", "bssid": "${secrets[1]}"},
|
||||||
|
"neighbors": [{"name": "${secrets[8]}", "mac": "${secrets[2]}"}]
|
||||||
|
}],
|
||||||
|
"tests": [{"id": "t1", "type": "train.udp_updown", "status": "ok",
|
||||||
|
"metrics": {"rtt_ms_avg": 12.4}}],
|
||||||
|
"findings": [],
|
||||||
|
"summary": {"verdict": "ok"}
|
||||||
|
}
|
||||||
|
""".trimIndent()
|
||||||
|
|
||||||
|
private fun anonymized(level: PrivacyLevel): String =
|
||||||
|
json.encodeToString(
|
||||||
|
kotlinx.serialization.json.JsonObject.serializer(),
|
||||||
|
Anonymizer(level, salt).anonymize(json.parseToJsonElement(document()).jsonObject),
|
||||||
|
)
|
||||||
|
|
||||||
|
@Test
|
||||||
|
fun nothingIdentifyingSurvivesBalanced() {
|
||||||
|
val out = anonymized(PrivacyLevel.BALANCED)
|
||||||
|
val leaked = secrets.filter { out.contains(it) }
|
||||||
|
assertTrue(
|
||||||
|
leaked.isEmpty(),
|
||||||
|
"these identifying values were uploaded verbatim at BALANCED: $leaked\n\n$out",
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
@Test
|
||||||
|
fun nothingIdentifyingSurvivesStrict() {
|
||||||
|
val out = anonymized(PrivacyLevel.STRICT)
|
||||||
|
val leaked = secrets.filter { out.contains(it) }
|
||||||
|
assertTrue(leaked.isEmpty(), "leaked at STRICT: $leaked\n\n$out")
|
||||||
|
}
|
||||||
|
|
||||||
|
// Private addresses are kept on purpose — they describe the topology and not the person — so
|
||||||
|
// this pins that the leak test above is not passing by accident of over-redaction.
|
||||||
|
@Test
|
||||||
|
fun privateAddressesAreStillReadable() {
|
||||||
|
val out = anonymized(PrivacyLevel.BALANCED)
|
||||||
|
assertTrue(out.contains("192.168.1.1"), "RFC1918 gateway should survive: $out")
|
||||||
|
assertTrue(out.contains("192.168.1.44"), "RFC1918 interface address should survive: $out")
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,202 @@
|
|||||||
|
// SPDX-FileCopyrightText: 2026 Echolot contributors
|
||||||
|
// SPDX-License-Identifier: GPL-3.0-or-later
|
||||||
|
|
||||||
|
package app.echo_lot.protocol
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Version compatibility, the client side of the same question the server asks about us.
|
||||||
|
*
|
||||||
|
* Versions are SemVer, but what is really being checked is whether the peer speaks a wire protocol
|
||||||
|
* and schema this build understands; the version is a proxy, and it only works because the
|
||||||
|
* breaking axis is bumped when the contract changes. Bounds therefore sit at breaking boundaries
|
||||||
|
* rather than at individual releases — a server patch release must never make the app refuse to
|
||||||
|
* talk to it.
|
||||||
|
*
|
||||||
|
* Mirrors `server/internal/compat`. Two implementations of one rule is a duplication worth
|
||||||
|
* accepting: each side must be able to state and enforce its own limits without asking the other,
|
||||||
|
* which is the entire point of a compatibility check.
|
||||||
|
*/
|
||||||
|
data class SemVer(
|
||||||
|
val major: Int,
|
||||||
|
val minor: Int,
|
||||||
|
val patch: Int,
|
||||||
|
val pre: String = "",
|
||||||
|
) : Comparable<SemVer> {
|
||||||
|
|
||||||
|
override fun compareTo(other: SemVer): Int {
|
||||||
|
(major - other.major).let { if (it != 0) return it.coerceIn(-1, 1) }
|
||||||
|
(minor - other.minor).let { if (it != 0) return it.coerceIn(-1, 1) }
|
||||||
|
(patch - other.patch).let { if (it != 0) return it.coerceIn(-1, 1) }
|
||||||
|
// A pre-release sorts below the same version without one (SemVer §11).
|
||||||
|
return when {
|
||||||
|
pre == other.pre -> 0
|
||||||
|
pre.isEmpty() -> 1
|
||||||
|
other.pre.isEmpty() -> -1
|
||||||
|
else -> pre.compareTo(other.pre).coerceIn(-1, 1)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
override fun toString(): String = "$major.$minor.$patch" + if (pre.isEmpty()) "" else "-$pre"
|
||||||
|
|
||||||
|
/**
|
||||||
|
* The first version that may break compatibility with this one. Below 1.0.0 the minor is the
|
||||||
|
* breaking axis (SemVer §4), so 0.4.2's next break is 0.5.0 — treating it as 1.0.0 would let
|
||||||
|
* this build accept a peer it cannot actually talk to.
|
||||||
|
*/
|
||||||
|
fun nextBreaking(): SemVer =
|
||||||
|
if (major == 0) SemVer(0, minor + 1, 0) else SemVer(major + 1, 0, 0)
|
||||||
|
|
||||||
|
companion object {
|
||||||
|
/** Accepts "1.2.3", "v1.2.3" and namespaced tags like "server-v1.2.3". Null if unusable. */
|
||||||
|
fun parse(raw: String?): SemVer? {
|
||||||
|
var s = raw?.trim().orEmpty()
|
||||||
|
if (s.isEmpty()) return null
|
||||||
|
// Strip a tag prefix ending in "v", guarded on the prefix having no digits so a
|
||||||
|
// pre-release identifier containing a "v" is left alone.
|
||||||
|
val v = s.lastIndexOf('v')
|
||||||
|
if (v >= 0 && v + 1 < s.length && s[v + 1].isDigit() && s.take(v).none { it.isDigit() }) {
|
||||||
|
s = s.substring(v + 1)
|
||||||
|
}
|
||||||
|
s = s.substringBefore('+')
|
||||||
|
val pre = s.substringAfter('-', "")
|
||||||
|
val core = s.substringBefore('-')
|
||||||
|
val parts = core.split(".")
|
||||||
|
if (parts.size != 3) return null
|
||||||
|
val nums = parts.map { it.toIntOrNull() ?: return null }
|
||||||
|
if (nums.any { it < 0 }) return null
|
||||||
|
return SemVer(nums[0], nums[1], nums[2], pre)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/** [min, max): minimum inclusive, maximum exclusive. A null max is unbounded. */
|
||||||
|
data class VersionRange(val min: SemVer, val max: SemVer? = null) {
|
||||||
|
operator fun contains(v: SemVer): Boolean = v >= min && (max == null || v < max)
|
||||||
|
override fun toString(): String = ">= $min" + (max?.let { ", < $it" } ?: "")
|
||||||
|
|
||||||
|
companion object {
|
||||||
|
fun of(min: String, max: String?): VersionRange {
|
||||||
|
val lo = requireNotNull(SemVer.parse(min)) { "bad minimum version: $min" }
|
||||||
|
val hi = max?.takeIf { it.isNotBlank() }?.let {
|
||||||
|
requireNotNull(SemVer.parse(it)) { "bad maximum version: $it" }
|
||||||
|
}
|
||||||
|
require(hi == null || hi > lo) { "maximum $max is not above minimum $min" }
|
||||||
|
return VersionRange(lo, hi)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* What this build of the app requires of a server, and what it tells servers about itself.
|
||||||
|
*
|
||||||
|
* Two separate questions, deliberately not conflated:
|
||||||
|
*
|
||||||
|
* - **Protocol version** — can these builds talk at all? This is the correctness axis, and its
|
||||||
|
* breaking boundary is enforced strictly.
|
||||||
|
* - **Release-version window** — should they, per policy? A coarse safety net over the peer's
|
||||||
|
* SemVer, with bounds at breaking boundaries so a patch release never strands anyone. Both
|
||||||
|
* sides publish their own, and the operator can tighten the server's.
|
||||||
|
*
|
||||||
|
* `MIN_SERVER` is 0.4.2 for a concrete reason, not caution: below it a multi-homed server sent
|
||||||
|
* granted traffic from an address the session never used, so every downstream packet was dropped
|
||||||
|
* in transit and reported as 100 % downstream loss. A confidently wrong measurement is worse than
|
||||||
|
* a refused one, so talking to those builds is not something to allow "just in case".
|
||||||
|
*/
|
||||||
|
object Compat {
|
||||||
|
/** Header the app sets on every control-plane request. */
|
||||||
|
const val APP_VERSION_HEADER = "X-Echolot-App-Version"
|
||||||
|
|
||||||
|
/** The wire contract (probe-protocol.md) this build implements. */
|
||||||
|
const val PROTOCOL_VERSION = "1.0.0"
|
||||||
|
|
||||||
|
const val MIN_SERVER = "0.4.2"
|
||||||
|
|
||||||
|
/** Exclusive. The next breaking series is refused until this app is taught about it. */
|
||||||
|
const val MAX_SERVER = "1.0.0"
|
||||||
|
|
||||||
|
val serverRange: VersionRange = VersionRange.of(MIN_SERVER, MAX_SERVER)
|
||||||
|
|
||||||
|
enum class Verdict { OK, PROTOCOL_MISMATCH, SERVER_TOO_OLD, SERVER_TOO_NEW, APP_REFUSED, UNKNOWN }
|
||||||
|
|
||||||
|
/**
|
||||||
|
* The result of checking a server, with a message written for the person holding the phone.
|
||||||
|
* [usable] is what callers branch on; [message] is what they show.
|
||||||
|
*/
|
||||||
|
data class Result(val verdict: Verdict, val message: String?) {
|
||||||
|
val usable: Boolean get() = verdict == Verdict.OK || verdict == Verdict.UNKNOWN
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Checks a profile both ways: is the server within our range, and are we within the server's.
|
||||||
|
*
|
||||||
|
* Asking both is the point of advertising the window in the profile. Discovering that the
|
||||||
|
* server will refuse us only when a measurement fails halfway through is a much worse
|
||||||
|
* experience than being told before the run starts.
|
||||||
|
*/
|
||||||
|
fun check(profile: Profile, appVersion: String): Result {
|
||||||
|
// The protocol version is the axis that decides whether these two builds *can* talk;
|
||||||
|
// the release-version window below is the operator's policy about whether they *may*.
|
||||||
|
// Checking the real thing first means a mismatch is reported as what it is.
|
||||||
|
val ours = SemVer.parse(PROTOCOL_VERSION)!!
|
||||||
|
val theirs = SemVer.parse(profile.compat.protocolVersion)
|
||||||
|
if (theirs != null && theirs >= ours.nextBreaking()) {
|
||||||
|
return Result(
|
||||||
|
Verdict.PROTOCOL_MISMATCH,
|
||||||
|
"This server speaks probe protocol ${profile.compat.protocolVersion}; this app " +
|
||||||
|
"speaks $PROTOCOL_VERSION and does not understand that revision. Update the app.",
|
||||||
|
)
|
||||||
|
}
|
||||||
|
if (theirs != null && ours >= theirs.nextBreaking()) {
|
||||||
|
return Result(
|
||||||
|
Verdict.PROTOCOL_MISMATCH,
|
||||||
|
"This server speaks probe protocol ${profile.compat.protocolVersion}, which this " +
|
||||||
|
"app ($PROTOCOL_VERSION) has moved past. Update the server.",
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
val server = SemVer.parse(profile.serverVersion)
|
||||||
|
?: return Result(
|
||||||
|
Verdict.UNKNOWN,
|
||||||
|
"Server did not report a usable version (\"${profile.serverVersion}\") — " +
|
||||||
|
"continuing without a compatibility check.",
|
||||||
|
)
|
||||||
|
|
||||||
|
if (server < serverRange.min) {
|
||||||
|
return Result(
|
||||||
|
Verdict.SERVER_TOO_OLD,
|
||||||
|
"This server runs $server; Echolot needs $serverRange. " +
|
||||||
|
"Measurements against older servers can be wrong rather than merely missing, " +
|
||||||
|
"so update the server.",
|
||||||
|
)
|
||||||
|
}
|
||||||
|
if (serverRange.max != null && server >= serverRange.max) {
|
||||||
|
return Result(
|
||||||
|
Verdict.SERVER_TOO_NEW,
|
||||||
|
"This server runs $server, which is newer than this app understands " +
|
||||||
|
"($serverRange). Update the app.",
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
// And the server's own view of us.
|
||||||
|
val app = SemVer.parse(appVersion)
|
||||||
|
val serverWantsMin = SemVer.parse(profile.compat.appMin)
|
||||||
|
val serverWantsMax = SemVer.parse(profile.compat.appMax)
|
||||||
|
if (app != null && serverWantsMin != null) {
|
||||||
|
if (app < serverWantsMin) {
|
||||||
|
return Result(
|
||||||
|
Verdict.APP_REFUSED,
|
||||||
|
"This server only accepts Echolot $serverWantsMin or newer; this app is " +
|
||||||
|
"$app. Update the app.",
|
||||||
|
)
|
||||||
|
}
|
||||||
|
if (serverWantsMax != null && app >= serverWantsMax) {
|
||||||
|
return Result(
|
||||||
|
Verdict.APP_REFUSED,
|
||||||
|
"This server refuses Echolot $serverWantsMax and newer; this app is $app. " +
|
||||||
|
"Use an older app, or a server that has caught up.",
|
||||||
|
)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return Result(Verdict.OK, null)
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -4,9 +4,21 @@
|
|||||||
package app.echo_lot.protocol
|
package app.echo_lot.protocol
|
||||||
|
|
||||||
import kotlinx.serialization.json.Json
|
import kotlinx.serialization.json.Json
|
||||||
|
import kotlinx.serialization.json.jsonObject
|
||||||
|
import kotlinx.serialization.json.jsonPrimitive
|
||||||
import java.net.URL
|
import java.net.URL
|
||||||
import javax.net.ssl.HttpsURLConnection
|
import javax.net.ssl.HttpsURLConnection
|
||||||
|
|
||||||
|
/** The server declined to store this run, per the operator's policy. Not a transport failure. */
|
||||||
|
class UploadRefused(message: String) : Exception(message)
|
||||||
|
|
||||||
|
/**
|
||||||
|
* The server refused this app's version. Distinct from a transport failure and from an auth
|
||||||
|
* failure: nothing about the request was wrong, the two builds simply do not go together, and the
|
||||||
|
* message says which versions do.
|
||||||
|
*/
|
||||||
|
class VersionRefused(message: String) : Exception(message)
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* The control-plane client (probe-protocol.md §2): enrollment, profile, sessions — over
|
* The control-plane client (probe-protocol.md §2): enrollment, profile, sessions — over
|
||||||
* SPKI-pinned HTTPS. Uses HttpsURLConnection (available since Android API 1, unlike
|
* SPKI-pinned HTTPS. Uses HttpsURLConnection (available since Android API 1, unlike
|
||||||
@@ -16,11 +28,14 @@ import javax.net.ssl.HttpsURLConnection
|
|||||||
*
|
*
|
||||||
* @param controlUrl e.g. "https://fmr-1.echo-lot.app:8443"
|
* @param controlUrl e.g. "https://fmr-1.echo-lot.app:8443"
|
||||||
* @param pins the `pin-sha256` value(s) from the enrollment QR (base64, no prefix)
|
* @param pins the `pin-sha256` value(s) from the enrollment QR (base64, no prefix)
|
||||||
|
* @param appVersion this build's SemVer, sent on every request so the server can refuse a build
|
||||||
|
* it cannot serve *before* a measurement half-runs (BuildConfig.VERSION_NAME).
|
||||||
*/
|
*/
|
||||||
/** The server declined to store this run, per the operator's policy. Not a transport failure. */
|
class ControlClient(
|
||||||
class UploadRefused(message: String) : Exception(message)
|
private val controlUrl: String,
|
||||||
|
pins: Set<String>,
|
||||||
class ControlClient(private val controlUrl: String, pins: Set<String>) {
|
private val appVersion: String = "",
|
||||||
|
) {
|
||||||
|
|
||||||
private val json = Json { ignoreUnknownKeys = true }
|
private val json = Json { ignoreUnknownKeys = true }
|
||||||
private val socketFactory = Pinning.sslContext(pins).socketFactory
|
private val socketFactory = Pinning.sslContext(pins).socketFactory
|
||||||
@@ -33,12 +48,42 @@ class ControlClient(private val controlUrl: String, pins: Set<String>) {
|
|||||||
conn.connectTimeout = 10_000
|
conn.connectTimeout = 10_000
|
||||||
conn.readTimeout = 10_000
|
conn.readTimeout = 10_000
|
||||||
credential?.let { conn.setRequestProperty("Authorization", "Bearer $it") }
|
credential?.let { conn.setRequestProperty("Authorization", "Bearer $it") }
|
||||||
|
if (appVersion.isNotBlank()) conn.setRequestProperty(Compat.APP_VERSION_HEADER, appVersion)
|
||||||
return conn
|
return conn
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* 426 is the server saying "your version, not your request". Raised as a distinct exception
|
||||||
|
* from every call site so callers never report it as a network error — the whole value of the
|
||||||
|
* check is that the failure is legible.
|
||||||
|
*/
|
||||||
|
private fun checkVersion(conn: HttpsURLConnection, body: String) {
|
||||||
|
if (conn.responseCode == 426) throw VersionRefused(extractError(body) ?: body.take(200))
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Pulls the "error" string out of a JSON body.
|
||||||
|
*
|
||||||
|
* Parsed rather than pattern-matched: an encoder may legitimately escape characters in the
|
||||||
|
* message (Go escapes ">" by default), and a regex hands the user "needs \u003e= 0.2.0".
|
||||||
|
* The parser knows how to undo every escape; a regex would have to be taught each one.
|
||||||
|
*/
|
||||||
|
private fun extractError(body: String): String? = runCatching {
|
||||||
|
json.parseToJsonElement(body).jsonObject["error"]?.jsonPrimitive?.content
|
||||||
|
}.getOrNull()
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Reads the response body, and turns a 426 into [VersionRefused] first.
|
||||||
|
*
|
||||||
|
* Every call goes through here, so the version check cannot be forgotten at a new call site —
|
||||||
|
* the alternative (a check per method) is exactly the kind of thing that gets missed once and
|
||||||
|
* then reports "upload failed: 426" to a user for a year.
|
||||||
|
*/
|
||||||
private fun body(conn: HttpsURLConnection): String {
|
private fun body(conn: HttpsURLConnection): String {
|
||||||
val stream = if (conn.responseCode in 200..299) conn.inputStream else conn.errorStream
|
val stream = if (conn.responseCode in 200..299) conn.inputStream else conn.errorStream
|
||||||
return stream?.bufferedReader()?.use { it.readText() } ?: ""
|
val text = stream?.bufferedReader()?.use { it.readText() } ?: ""
|
||||||
|
checkVersion(conn, text)
|
||||||
|
return text
|
||||||
}
|
}
|
||||||
|
|
||||||
private fun writeJson(conn: HttpsURLConnection, payload: String) {
|
private fun writeJson(conn: HttpsURLConnection, payload: String) {
|
||||||
@@ -66,21 +111,24 @@ class ControlClient(private val controlUrl: String, pins: Set<String>) {
|
|||||||
val conn = open("/v1/enroll", "POST", null)
|
val conn = open("/v1/enroll", "POST", null)
|
||||||
conn.setRequestProperty("Authorization", "Bearer $token")
|
conn.setRequestProperty("Authorization", "Bearer $token")
|
||||||
writeJson(conn, if (name != null) """{"name":${jstr(name)}}""" else "{}")
|
writeJson(conn, if (name != null) """{"name":${jstr(name)}}""" else "{}")
|
||||||
check(conn.responseCode == 201) { "enroll failed: ${conn.responseCode} ${body(conn)}" }
|
val text = body(conn) // reads and raises VersionRefused on 426
|
||||||
return json.decodeFromString(EnrollResponse.serializer(), body(conn))
|
check(conn.responseCode == 201) { "enroll failed: ${conn.responseCode} $text" }
|
||||||
|
return json.decodeFromString(EnrollResponse.serializer(), text)
|
||||||
}
|
}
|
||||||
|
|
||||||
fun profile(credential: String): Profile {
|
fun profile(credential: String): Profile {
|
||||||
val conn = open("/v1/profile", "GET", credential)
|
val conn = open("/v1/profile", "GET", credential)
|
||||||
check(conn.responseCode == 200) { "profile failed: ${conn.responseCode} ${body(conn)}" }
|
val text = body(conn)
|
||||||
return json.decodeFromString(Profile.serializer(), body(conn))
|
check(conn.responseCode == 200) { "profile failed: ${conn.responseCode} $text" }
|
||||||
|
return json.decodeFromString(Profile.serializer(), text)
|
||||||
}
|
}
|
||||||
|
|
||||||
fun createSession(credential: String, target: String): SessionResponse {
|
fun createSession(credential: String, target: String): SessionResponse {
|
||||||
val conn = open("/v1/sessions", "POST", credential)
|
val conn = open("/v1/sessions", "POST", credential)
|
||||||
writeJson(conn, """{"target":${jstr(target)}}""")
|
writeJson(conn, """{"target":${jstr(target)}}""")
|
||||||
check(conn.responseCode == 201) { "session failed: ${conn.responseCode} ${body(conn)}" }
|
val text = body(conn)
|
||||||
return json.decodeFromString(SessionResponse.serializer(), body(conn))
|
check(conn.responseCode == 201) { "session failed: ${conn.responseCode} $text" }
|
||||||
|
return json.decodeFromString(SessionResponse.serializer(), text)
|
||||||
}
|
}
|
||||||
|
|
||||||
/**
|
/**
|
||||||
@@ -111,7 +159,7 @@ class ControlClient(private val controlUrl: String, pins: Set<String>) {
|
|||||||
val body = body(conn)
|
val body = body(conn)
|
||||||
when (conn.responseCode) {
|
when (conn.responseCode) {
|
||||||
in 200..299 -> return body
|
in 200..299 -> return body
|
||||||
403 -> throw UploadRefused(body)
|
403 -> throw UploadRefused(extractError(body) ?: body.take(200))
|
||||||
413 -> throw UploadRefused("run is larger than this server accepts: $body")
|
413 -> throw UploadRefused("run is larger than this server accepts: $body")
|
||||||
else -> error("upload failed: ${conn.responseCode} $body")
|
else -> error("upload failed: ${conn.responseCode} $body")
|
||||||
}
|
}
|
||||||
@@ -120,14 +168,16 @@ class ControlClient(private val controlUrl: String, pins: Set<String>) {
|
|||||||
/** Lists this device's runs stored on the server. */
|
/** Lists this device's runs stored on the server. */
|
||||||
fun listRuns(credential: String): String {
|
fun listRuns(credential: String): String {
|
||||||
val conn = open("/v1/runs", "GET", credential)
|
val conn = open("/v1/runs", "GET", credential)
|
||||||
check(conn.responseCode == 200) { "list runs failed: ${conn.responseCode}" }
|
val text = body(conn)
|
||||||
return body(conn)
|
check(conn.responseCode == 200) { "list runs failed: ${conn.responseCode} $text" }
|
||||||
|
return text
|
||||||
}
|
}
|
||||||
|
|
||||||
fun getRun(credential: String, runId: String): String {
|
fun getRun(credential: String, runId: String): String {
|
||||||
val conn = open("/v1/runs/$runId", "GET", credential)
|
val conn = open("/v1/runs/$runId", "GET", credential)
|
||||||
check(conn.responseCode == 200) { "get run failed: ${conn.responseCode}" }
|
val text = body(conn)
|
||||||
return body(conn)
|
check(conn.responseCode == 200) { "get run failed: ${conn.responseCode} $text" }
|
||||||
|
return text
|
||||||
}
|
}
|
||||||
|
|
||||||
fun deleteRun(credential: String, runId: String) {
|
fun deleteRun(credential: String, runId: String) {
|
||||||
@@ -136,8 +186,9 @@ class ControlClient(private val controlUrl: String, pins: Set<String>) {
|
|||||||
|
|
||||||
fun observations(credential: String, sessionId: String): String {
|
fun observations(credential: String, sessionId: String): String {
|
||||||
val conn = open("/v1/sessions/$sessionId/observations", "GET", credential)
|
val conn = open("/v1/sessions/$sessionId/observations", "GET", credential)
|
||||||
check(conn.responseCode == 200) { "observations failed: ${conn.responseCode}" }
|
val text = body(conn)
|
||||||
return body(conn)
|
check(conn.responseCode == 200) { "observations failed: ${conn.responseCode} $text" }
|
||||||
|
return text
|
||||||
}
|
}
|
||||||
|
|
||||||
fun deleteSession(credential: String, sessionId: String) {
|
fun deleteSession(credential: String, sessionId: String) {
|
||||||
|
|||||||
@@ -0,0 +1,119 @@
|
|||||||
|
// SPDX-FileCopyrightText: 2026 Echolot contributors
|
||||||
|
// SPDX-License-Identifier: GPL-3.0-or-later
|
||||||
|
|
||||||
|
package app.echo_lot.protocol
|
||||||
|
|
||||||
|
import java.net.URLDecoder
|
||||||
|
import java.net.URLEncoder
|
||||||
|
|
||||||
|
/**
|
||||||
|
* The enrollment bootstrap of probe-protocol.md §2.1.
|
||||||
|
*
|
||||||
|
* ```
|
||||||
|
* echolot://enroll?v=1&u=<control-URL, urlencoded>&p=pin-sha256:<b64 SPKI hash>&t=<token>
|
||||||
|
* ```
|
||||||
|
*
|
||||||
|
* One string carries everything a device needs to start trusting a server: where it is, which key
|
||||||
|
* to pin, and a single-use token proving the operator meant to admit this device. That is the
|
||||||
|
* whole point — it is why enrollment can be a paste or a QR scan rather than three fields typed
|
||||||
|
* from a screenshot, which is what people actually do wrong.
|
||||||
|
*
|
||||||
|
* **The link is a secret.** It contains a bearer token; anyone who sees it before the device does
|
||||||
|
* can enroll instead. Tokens are single-use and short-lived precisely so a leaked link is a
|
||||||
|
* bounded problem, but it should be treated like a password while it is live.
|
||||||
|
*/
|
||||||
|
data class EnrollmentLink(
|
||||||
|
/** e.g. "https://fmr-1.echo-lot.app:8443" */
|
||||||
|
val controlUrl: String,
|
||||||
|
/** Base64 SPKI hash, without the "pin-sha256:" prefix — the form [ControlClient] wants. */
|
||||||
|
val pin: String,
|
||||||
|
val token: String,
|
||||||
|
) {
|
||||||
|
/** Rebuilds the URI. Round-trips with [parse]; used for tests and for sharing a link on. */
|
||||||
|
fun toUri(): String = buildString {
|
||||||
|
append("echolot://enroll?v=1")
|
||||||
|
append("&u=").append(enc(controlUrl))
|
||||||
|
append("&p=").append(enc(PIN_PREFIX + pin))
|
||||||
|
append("&t=").append(enc(token))
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Redeems the token and returns a usable server configuration.
|
||||||
|
*
|
||||||
|
* The pin is applied to the very request that redeems the token, so a link pointing at an
|
||||||
|
* impostor fails at the TLS handshake rather than after handing it a token. That ordering is
|
||||||
|
* the reason the pin travels in the link at all.
|
||||||
|
*/
|
||||||
|
fun redeem(deviceName: String? = null, appVersion: String = ""): Enrolled {
|
||||||
|
val client = ControlClient(controlUrl, setOf(pin), appVersion)
|
||||||
|
val response = client.enroll(token, deviceName)
|
||||||
|
val profile = client.profile(response.credential)
|
||||||
|
return Enrolled(
|
||||||
|
controlUrl = controlUrl,
|
||||||
|
pin = pin,
|
||||||
|
credential = response.credential,
|
||||||
|
deviceId = response.deviceId,
|
||||||
|
profile = profile,
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
companion object {
|
||||||
|
const val SCHEME = "echolot"
|
||||||
|
const val HOST = "enroll"
|
||||||
|
private const val PIN_PREFIX = "pin-sha256:"
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Parses a bootstrap link. Returns null for anything that is not one — a malformed link
|
||||||
|
* must not be half-applied, because a half-configured server is a confusing failure much
|
||||||
|
* later rather than an obvious one now.
|
||||||
|
*/
|
||||||
|
fun parse(raw: String?): EnrollmentLink? {
|
||||||
|
val s = raw?.trim() ?: return null
|
||||||
|
val scheme = s.substringBefore("://", "")
|
||||||
|
if (!scheme.equals(SCHEME, ignoreCase = true)) return null
|
||||||
|
val rest = s.substringAfter("://")
|
||||||
|
val host = rest.substringBefore('?').trim('/')
|
||||||
|
if (!host.equals(HOST, ignoreCase = true)) return null
|
||||||
|
|
||||||
|
val params = HashMap<String, String>()
|
||||||
|
for (pair in rest.substringAfter('?', "").split('&')) {
|
||||||
|
if (pair.isEmpty()) continue
|
||||||
|
val k = pair.substringBefore('=')
|
||||||
|
val v = pair.substringAfter('=', "")
|
||||||
|
params[k] = dec(v)
|
||||||
|
}
|
||||||
|
|
||||||
|
// v is the link format, not the protocol. Unknown versions are refused rather than
|
||||||
|
// guessed at: the fields could mean anything.
|
||||||
|
val version = params["v"] ?: "1"
|
||||||
|
if (version != "1") return null
|
||||||
|
|
||||||
|
val url = params["u"]?.trim().orEmpty()
|
||||||
|
val pinRaw = params["p"]?.trim().orEmpty()
|
||||||
|
val token = params["t"]?.trim().orEmpty()
|
||||||
|
if (url.isEmpty() || pinRaw.isEmpty() || token.isEmpty()) return null
|
||||||
|
if (!url.startsWith("https://", ignoreCase = true)) return null
|
||||||
|
|
||||||
|
// A "+" in a query string decodes to a space, so a link whose base64 pin was pasted
|
||||||
|
// in unencoded arrives with spaces where "+" belonged — and a pin that is wrong by
|
||||||
|
// one character does not fail loudly, it just never matches, which surfaces much
|
||||||
|
// later as an inexplicable TLS error. Base64 has no spaces, so putting them back is
|
||||||
|
// unambiguous and cannot damage a correctly-encoded pin.
|
||||||
|
val pin = pinRaw.removePrefix(PIN_PREFIX).replace(' ', '+')
|
||||||
|
if (pin.isEmpty()) return null
|
||||||
|
return EnrollmentLink(controlUrl = url.trimEnd('/'), pin = pin, token = token)
|
||||||
|
}
|
||||||
|
|
||||||
|
private fun enc(s: String) = URLEncoder.encode(s, "UTF-8")
|
||||||
|
private fun dec(s: String) = runCatching { URLDecoder.decode(s, "UTF-8") }.getOrDefault(s)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/** A server this device is now enrolled with, ready to be stored in settings. */
|
||||||
|
data class Enrolled(
|
||||||
|
val controlUrl: String,
|
||||||
|
val pin: String,
|
||||||
|
val credential: String,
|
||||||
|
val deviceId: String,
|
||||||
|
val profile: Profile,
|
||||||
|
)
|
||||||
@@ -13,8 +13,16 @@ import kotlinx.serialization.json.JsonElement
|
|||||||
@Serializable
|
@Serializable
|
||||||
data class EnrollResponse(
|
data class EnrollResponse(
|
||||||
@SerialName("device_id") val deviceId: String,
|
@SerialName("device_id") val deviceId: String,
|
||||||
val credential: String,
|
/** The spec's name (§2.1). */
|
||||||
)
|
@SerialName("device_credential") val deviceCredential: String? = null,
|
||||||
|
/** What the first server implementation shipped. Read for older servers; do not emit. */
|
||||||
|
@SerialName("credential") val legacyCredential: String? = null,
|
||||||
|
) {
|
||||||
|
/** Whichever field the server used. */
|
||||||
|
val credential: String
|
||||||
|
get() = deviceCredential ?: legacyCredential
|
||||||
|
?: error("enroll response carried no credential")
|
||||||
|
}
|
||||||
|
|
||||||
@Serializable
|
@Serializable
|
||||||
data class Target(
|
data class Target(
|
||||||
@@ -56,6 +64,16 @@ data class UploadPolicy(
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/** The server's declaration of what it speaks and which app versions it will serve. */
|
||||||
|
@Serializable
|
||||||
|
data class CompatInfo(
|
||||||
|
@SerialName("protocol_version") val protocolVersion: String = "",
|
||||||
|
@SerialName("schema_version") val schemaVersion: String = "",
|
||||||
|
@SerialName("app_min") val appMin: String = "",
|
||||||
|
/** Exclusive; empty means the server sets no upper bound. */
|
||||||
|
@SerialName("app_max") val appMax: String = "",
|
||||||
|
)
|
||||||
|
|
||||||
@Serializable
|
@Serializable
|
||||||
data class Profile(
|
data class Profile(
|
||||||
@SerialName("profile_version") val profileVersion: Int = 0,
|
@SerialName("profile_version") val profileVersion: Int = 0,
|
||||||
@@ -67,6 +85,7 @@ data class Profile(
|
|||||||
@SerialName("server_selftest") val serverSelftest: SelfTest? = null,
|
@SerialName("server_selftest") val serverSelftest: SelfTest? = null,
|
||||||
val pins: List<String> = emptyList(),
|
val pins: List<String> = emptyList(),
|
||||||
val uploads: UploadPolicy = UploadPolicy(),
|
val uploads: UploadPolicy = UploadPolicy(),
|
||||||
|
val compat: CompatInfo = CompatInfo(),
|
||||||
) {
|
) {
|
||||||
fun supports(capability: String) = capability in capabilities
|
fun supports(capability: String) = capability in capabilities
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -12,6 +12,11 @@ import java.util.Base64
|
|||||||
* A data-plane session (probe-protocol.md §3): derives the session key, then sends signed ELT1
|
* A data-plane session (probe-protocol.md §3): derives the session key, then sends signed ELT1
|
||||||
* packets to the server's UDP endpoint and reads back verified responses. One session ↔ one
|
* packets to the server's UDP endpoint and reads back verified responses. One session ↔ one
|
||||||
* server target. Blocking; the caller owns threading.
|
* server target. Blocking; the caller owns threading.
|
||||||
|
*
|
||||||
|
* One instance per server session, for its whole lifetime. Sequence numbers start at zero here
|
||||||
|
* while the server's anti-replay window (§3.2) keeps counting, so a second instance sharing a
|
||||||
|
* session id has all its packets discarded as replays — and, because the server then never
|
||||||
|
* records the new source, any granted send still targets the socket that was closed.
|
||||||
*/
|
*/
|
||||||
class ProbeSession(
|
class ProbeSession(
|
||||||
private val credential: String,
|
private val credential: String,
|
||||||
@@ -39,13 +44,26 @@ class ProbeSession(
|
|||||||
*/
|
*/
|
||||||
fun echo(paddingBytes: Int = 40): EchoResult? {
|
fun echo(paddingBytes: Int = 40): EchoResult? {
|
||||||
val t0 = System.nanoTime()
|
val t0 = System.nanoTime()
|
||||||
val pkt = Wire.build(Wire.TYPE_ECHO_REQ, prefix, ++seq, nowNs(), key, ByteArray(paddingBytes))
|
val wireSeq = ++seq
|
||||||
|
val pkt = Wire.build(Wire.TYPE_ECHO_REQ, prefix, wireSeq, nowNs(), key, ByteArray(paddingBytes))
|
||||||
socket.send(DatagramPacket(pkt, pkt.size, server))
|
socket.send(DatagramPacket(pkt, pkt.size, server))
|
||||||
|
// A lost probe still has a sequence number, and that number is what lets the server's
|
||||||
|
// observations say whether it was lost going out or coming back — so report it either way.
|
||||||
|
lastSeq = wireSeq
|
||||||
val resp = receive(Wire.TYPE_ECHO_RESP) ?: return null
|
val resp = receive(Wire.TYPE_ECHO_RESP) ?: return null
|
||||||
val rttMs = (System.nanoTime() - t0) / 1_000_000.0
|
val rttMs = (System.nanoTime() - t0) / 1_000_000.0
|
||||||
return EchoResult(rttMs, Observation.parse(resp.payload))
|
return EchoResult(rttMs, Observation.parse(resp.payload), wireSeq)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* The wire sequence number of the most recent [echo], including one that was lost.
|
||||||
|
*
|
||||||
|
* Exposed because the caller cannot derive it: the counter is shared with every other packet
|
||||||
|
* type on this session, so "the nth echo" is not "sequence n".
|
||||||
|
*/
|
||||||
|
var lastSeq: Int = 0
|
||||||
|
private set
|
||||||
|
|
||||||
/** One MTU probe of [totalSize] bytes (DF is set by the OS on the socket where supported).
|
/** One MTU probe of [totalSize] bytes (DF is set by the OS on the socket where supported).
|
||||||
* Returns the size the server acknowledged receiving, or null if the probe was lost. */
|
* Returns the size the server acknowledged receiving, or null if the probe was lost. */
|
||||||
fun mtuProbe(totalSize: Int): Int? {
|
fun mtuProbe(totalSize: Int): Int? {
|
||||||
@@ -91,6 +109,49 @@ class ProbeSession(
|
|||||||
return out
|
return out
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Sends paced upstream traffic for [durationMs] and reports what was put on the wire.
|
||||||
|
*
|
||||||
|
* Paced rather than flat out, for the same reason the server paces: an unpaced burst measures
|
||||||
|
* the local NIC and the first queue it meets, then collapses into loss that reads as a network
|
||||||
|
* fault. The schedule is absolute rather than sleep-per-packet, which accumulates the
|
||||||
|
* scheduler's error and drifts the achieved rate below target over a multi-second run.
|
||||||
|
*
|
||||||
|
* Nothing comes back — the server counts and stays silent — so the result here is only the
|
||||||
|
* send side. The measurement is the gap between this and the server's tally.
|
||||||
|
*/
|
||||||
|
fun sendThroughput(durationMs: Long, kbps: Int, sizeBytes: Int = 1200): Sent {
|
||||||
|
val size = sizeBytes.coerceIn(Wire.HEADER_SIZE + 16, 1472)
|
||||||
|
val payload = ByteArray(size - Wire.HEADER_SIZE)
|
||||||
|
val perPacketNs = (size.toLong() * 8 * 1_000_000 / kbps.coerceAtLeast(1)).coerceAtLeast(1_000)
|
||||||
|
|
||||||
|
val start = System.nanoTime()
|
||||||
|
val deadline = start + durationMs * 1_000_000
|
||||||
|
var next = start
|
||||||
|
var packets = 0
|
||||||
|
var bytes = 0L
|
||||||
|
while (System.nanoTime() < deadline) {
|
||||||
|
val pkt = Wire.build(Wire.TYPE_THROUGHPUT_UP, prefix, ++seq, nowNs(), key, payload)
|
||||||
|
try {
|
||||||
|
socket.send(DatagramPacket(pkt, pkt.size, server))
|
||||||
|
} catch (e: java.io.IOException) {
|
||||||
|
// A local send failure is our condition, not the path's. Stop and report what
|
||||||
|
// actually left, rather than counting the remainder as loss on the network.
|
||||||
|
break
|
||||||
|
}
|
||||||
|
packets++
|
||||||
|
bytes += pkt.size
|
||||||
|
next += perPacketNs
|
||||||
|
val sleepNs = next - System.nanoTime()
|
||||||
|
if (sleepNs > 0) Thread.sleep(sleepNs / 1_000_000, (sleepNs % 1_000_000).toInt())
|
||||||
|
}
|
||||||
|
val elapsedMs = (System.nanoTime() - start) / 1_000_000
|
||||||
|
return Sent(packets, bytes, elapsedMs, if (elapsedMs > 0) (bytes * 8 / elapsedMs).toInt() else 0)
|
||||||
|
}
|
||||||
|
|
||||||
|
/** What one upstream run put on the wire locally. */
|
||||||
|
data class Sent(val packets: Int, val bytes: Long, val durationMs: Long, val kbps: Int)
|
||||||
|
|
||||||
/** One packet received from the server, with the wire size actually delivered. */
|
/** One packet received from the server, with the wire size actually delivered. */
|
||||||
data class Received(val type: Int, val seq: Int, val sizeBytes: Int, val tRxNs: Long)
|
data class Received(val type: Int, val seq: Int, val sizeBytes: Int, val tRxNs: Long)
|
||||||
|
|
||||||
@@ -107,5 +168,5 @@ class ProbeSession(
|
|||||||
|
|
||||||
override fun close() = socket.close()
|
override fun close() = socket.close()
|
||||||
|
|
||||||
data class EchoResult(val rttMs: Double, val observation: Observation?)
|
data class EchoResult(val rttMs: Double, val observation: Observation?, val seq: Int = 0)
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -32,6 +32,22 @@ object Wire {
|
|||||||
const val TYPE_DOWNTRAIN_DATA: Int = 0x06
|
const val TYPE_DOWNTRAIN_DATA: Int = 0x06
|
||||||
const val TYPE_BIG_SEND: Int = 0x0C
|
const val TYPE_BIG_SEND: Int = 0x0C
|
||||||
|
|
||||||
|
/**
|
||||||
|
* A datagram the server deliberately fragmented. Its arrival IS the measurement: it can only
|
||||||
|
* be delivered if every fragment survived the path and the local stack reassembled them.
|
||||||
|
*/
|
||||||
|
const val TYPE_FRAG_DATA: Int = 0x0D
|
||||||
|
|
||||||
|
/** One packet of a sustained-rate downstream run. */
|
||||||
|
const val TYPE_THROUGHPUT_DATA: Int = 0x0E
|
||||||
|
|
||||||
|
/**
|
||||||
|
* One packet of a client-driven upstream run. The server counts it and does not answer:
|
||||||
|
* a reply would double the traffic and drag the return path into a measurement that is
|
||||||
|
* specifically about the outbound one.
|
||||||
|
*/
|
||||||
|
const val TYPE_THROUGHPUT_UP: Int = 0x0F
|
||||||
|
|
||||||
/** The 8-byte on-the-wire prefix = first 16 hex chars of the session id, decoded. */
|
/** The 8-byte on-the-wire prefix = first 16 hex chars of the session id, decoded. */
|
||||||
fun wirePrefix(sessionId: String): ByteArray {
|
fun wirePrefix(sessionId: String): ByteArray {
|
||||||
require(sessionId.length >= 16) { "session id too short" }
|
require(sessionId.length >= 16) { "session id too short" }
|
||||||
|
|||||||
@@ -0,0 +1,162 @@
|
|||||||
|
// SPDX-FileCopyrightText: 2026 Echolot contributors
|
||||||
|
// SPDX-License-Identifier: GPL-3.0-or-later
|
||||||
|
|
||||||
|
package app.echo_lot.protocol
|
||||||
|
|
||||||
|
import kotlin.test.Test
|
||||||
|
import kotlin.test.assertEquals
|
||||||
|
import kotlin.test.assertNotNull
|
||||||
|
import kotlin.test.assertNull
|
||||||
|
import kotlin.test.assertTrue
|
||||||
|
|
||||||
|
/**
|
||||||
|
* The client half of the compatibility rule. Deliberately mirrors `compat_test.go`: the two
|
||||||
|
* implementations must agree on where the boundaries are, or one side refuses a peer the other
|
||||||
|
* accepts and the disagreement surfaces as an inexplicable failure in the field.
|
||||||
|
*/
|
||||||
|
class CompatTest {
|
||||||
|
|
||||||
|
@Test
|
||||||
|
fun parsesTheFormsThatActuallyReachUs() {
|
||||||
|
assertEquals(SemVer(1, 2, 3), SemVer.parse("1.2.3"))
|
||||||
|
assertEquals(SemVer(1, 2, 3), SemVer.parse("v1.2.3"))
|
||||||
|
assertEquals(SemVer(0, 4, 2), SemVer.parse("server-v0.4.2"))
|
||||||
|
assertEquals(SemVer(0, 2, 0), SemVer.parse(" 0.2.0 "))
|
||||||
|
assertEquals(SemVer(1, 0, 0, "rc1"), SemVer.parse("1.0.0-rc1"))
|
||||||
|
assertEquals(SemVer(1, 0, 0), SemVer.parse("1.0.0+build.7"))
|
||||||
|
assertEquals(SemVer(1, 0, 0, "rc1"), SemVer.parse("1.0.0-rc1+meta"))
|
||||||
|
// A pre-release identifier containing a "v" is not a tag prefix.
|
||||||
|
assertEquals(SemVer(1, 2, 3, "rcv1"), SemVer.parse("1.2.3-rcv1"))
|
||||||
|
|
||||||
|
for (bad in listOf("", "dev", "1.2", "1.2.3.4", "x.y.z", "-1.0.0", "1.2.beta", null)) {
|
||||||
|
assertNull(SemVer.parse(bad), "should not parse: $bad")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
@Test
|
||||||
|
fun ordersPreReleasesBelowTheirRelease() {
|
||||||
|
fun lt(a: String, b: String) {
|
||||||
|
val x = assertNotNull(SemVer.parse(a))
|
||||||
|
val y = assertNotNull(SemVer.parse(b))
|
||||||
|
assertTrue(x < y, "$a should sort below $b")
|
||||||
|
assertTrue(y > x)
|
||||||
|
}
|
||||||
|
lt("0.9.9", "1.0.0")
|
||||||
|
lt("1.0.0", "1.0.1")
|
||||||
|
lt("1.0.0", "1.1.0")
|
||||||
|
lt("1.0.0-rc1", "1.0.0")
|
||||||
|
lt("1.0.0-rc1", "1.0.0-rc2")
|
||||||
|
assertEquals(0, SemVer.parse("1.2.3")!!.compareTo(SemVer.parse("v1.2.3")!!))
|
||||||
|
}
|
||||||
|
|
||||||
|
// Below 1.0.0 the minor is the breaking axis. Must match Go's NextBreaking exactly.
|
||||||
|
@Test
|
||||||
|
fun nextBreakingUsesTheMinorBelowOne() {
|
||||||
|
assertEquals("0.5.0", SemVer.parse("0.4.2")!!.nextBreaking().toString())
|
||||||
|
assertEquals("0.1.0", SemVer.parse("0.0.9")!!.nextBreaking().toString())
|
||||||
|
assertEquals("2.0.0", SemVer.parse("1.2.3")!!.nextBreaking().toString())
|
||||||
|
}
|
||||||
|
|
||||||
|
@Test
|
||||||
|
fun rangeIsMinInclusiveMaxExclusive() {
|
||||||
|
val r = VersionRange.of("0.2.0", "1.0.0")
|
||||||
|
for (s in listOf("0.2.0", "0.2.1", "0.9.9", "1.0.0-rc1")) {
|
||||||
|
assertTrue(SemVer.parse(s)!! in r, "$s should be inside $r")
|
||||||
|
}
|
||||||
|
for (s in listOf("0.1.9", "1.0.0", "1.0.1", "2.0.0")) {
|
||||||
|
assertTrue(SemVer.parse(s)!! !in r, "$s should be outside $r")
|
||||||
|
}
|
||||||
|
assertTrue(SemVer.parse("99.0.0")!! in VersionRange.of("0.2.0", null), "empty max is unbounded")
|
||||||
|
}
|
||||||
|
|
||||||
|
private fun profile(serverVersion: String, appMin: String = "0.2.0", appMax: String = "1.0.0") =
|
||||||
|
Profile(
|
||||||
|
serverVersion = serverVersion,
|
||||||
|
compat = CompatInfo(protocolVersion = "1.0.0", appMin = appMin, appMax = appMax),
|
||||||
|
)
|
||||||
|
|
||||||
|
@Test
|
||||||
|
fun acceptsAServerInsideTheWindow() {
|
||||||
|
val r = Compat.check(profile("0.4.2"), appVersion = "0.2.0")
|
||||||
|
assertEquals(Compat.Verdict.OK, r.verdict)
|
||||||
|
assertTrue(r.usable)
|
||||||
|
assertNull(r.message)
|
||||||
|
}
|
||||||
|
|
||||||
|
// 0.4.2 is the minimum for a concrete reason: older multi-homed servers mis-address granted
|
||||||
|
// sends and the client reports 100% downstream loss that never happened.
|
||||||
|
@Test
|
||||||
|
fun refusesAServerBelowTheMinimum() {
|
||||||
|
val r = Compat.check(profile("0.4.1"), appVersion = "0.2.0")
|
||||||
|
assertEquals(Compat.Verdict.SERVER_TOO_OLD, r.verdict)
|
||||||
|
assertTrue(!r.usable)
|
||||||
|
assertTrue(r.message!!.contains("0.4.1") && r.message!!.contains("0.4.2"),
|
||||||
|
"the message must name both versions: ${r.message}")
|
||||||
|
}
|
||||||
|
|
||||||
|
@Test
|
||||||
|
fun refusesAServerFromANewerBreakingSeries() {
|
||||||
|
val r = Compat.check(profile("1.0.0"), appVersion = "0.2.0")
|
||||||
|
assertEquals(Compat.Verdict.SERVER_TOO_NEW, r.verdict)
|
||||||
|
assertTrue(r.message!!.contains("Update the app"), "should tell the user what to do")
|
||||||
|
}
|
||||||
|
|
||||||
|
// Learning that the server will refuse us only when a measurement fails halfway is a much
|
||||||
|
// worse experience than being told before the run starts — so the check goes both ways.
|
||||||
|
@Test
|
||||||
|
fun detectsThatTheServerWouldRefuseThisApp() {
|
||||||
|
val old = Compat.check(profile("0.4.2", appMin = "0.5.0"), appVersion = "0.2.0")
|
||||||
|
assertEquals(Compat.Verdict.APP_REFUSED, old.verdict)
|
||||||
|
assertTrue(old.message!!.contains("0.5.0"))
|
||||||
|
|
||||||
|
val tooNew = Compat.check(profile("0.4.2", appMax = "0.3.0"), appVersion = "0.4.0")
|
||||||
|
assertEquals(Compat.Verdict.APP_REFUSED, tooNew.verdict)
|
||||||
|
}
|
||||||
|
|
||||||
|
// A development build reports something unparseable. Locking a developer out of their own
|
||||||
|
// server would be a poor trade for a check meant to make failures clearer.
|
||||||
|
@Test
|
||||||
|
fun unknownVersionsAreUsableWithAnExplanation() {
|
||||||
|
val r = Compat.check(profile("dev"), appVersion = "0.2.0")
|
||||||
|
assertEquals(Compat.Verdict.UNKNOWN, r.verdict)
|
||||||
|
assertTrue(r.usable, "an unidentifiable server must not be treated as incompatible")
|
||||||
|
assertNotNull(r.message)
|
||||||
|
}
|
||||||
|
|
||||||
|
@Test
|
||||||
|
fun aServerThatDeclaresNoWindowIsNotTreatedAsRefusingUs() {
|
||||||
|
// An older server predating the compat block sends nothing; absence must not read as
|
||||||
|
// a restriction.
|
||||||
|
val r = Compat.check(Profile(serverVersion = "0.4.2"), appVersion = "0.2.0")
|
||||||
|
assertEquals(Compat.Verdict.OK, r.verdict)
|
||||||
|
}
|
||||||
|
|
||||||
|
// The protocol version decides whether the builds *can* talk; the release window is only
|
||||||
|
// policy about whether they *may*. A protocol break must be reported as a protocol break,
|
||||||
|
// even when both release versions sit comfortably inside their windows.
|
||||||
|
@Test
|
||||||
|
fun aProtocolBreakIsReportedAsOne() {
|
||||||
|
val newer = Profile(
|
||||||
|
serverVersion = "0.9.0",
|
||||||
|
compat = CompatInfo(protocolVersion = "2.0.0", appMin = "0.2.0", appMax = "1.0.0"),
|
||||||
|
)
|
||||||
|
val r = Compat.check(newer, appVersion = "0.2.0")
|
||||||
|
assertEquals(Compat.Verdict.PROTOCOL_MISMATCH, r.verdict)
|
||||||
|
assertTrue(r.message!!.contains("2.0.0"))
|
||||||
|
|
||||||
|
// Same protocol series, different patch: fine. A protocol bugfix must not split a fleet.
|
||||||
|
val samePatch = Profile(
|
||||||
|
serverVersion = "0.9.0",
|
||||||
|
compat = CompatInfo(protocolVersion = "1.0.4", appMin = "0.2.0", appMax = "1.0.0"),
|
||||||
|
)
|
||||||
|
assertEquals(Compat.Verdict.OK, Compat.check(samePatch, appVersion = "0.2.0").verdict)
|
||||||
|
}
|
||||||
|
|
||||||
|
@Test
|
||||||
|
fun thisBuildsOwnBoundsAreWellFormed() {
|
||||||
|
val r = Compat.serverRange
|
||||||
|
assertEquals(SemVer.parse(Compat.MIN_SERVER), r.min)
|
||||||
|
assertEquals(SemVer.parse(Compat.MAX_SERVER), r.max)
|
||||||
|
assertTrue(r.min < r.max!!, "the built-in window must be non-empty")
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,101 @@
|
|||||||
|
// SPDX-FileCopyrightText: 2026 Echolot contributors
|
||||||
|
// SPDX-License-Identifier: GPL-3.0-or-later
|
||||||
|
|
||||||
|
package app.echo_lot.protocol
|
||||||
|
|
||||||
|
import kotlin.test.Test
|
||||||
|
import kotlin.test.assertEquals
|
||||||
|
import kotlin.test.assertNotNull
|
||||||
|
import kotlin.test.assertNull
|
||||||
|
|
||||||
|
class EnrollmentTest {
|
||||||
|
|
||||||
|
private val pin = "zRV9qkiLnRexAeh4RrSfJzbPWO+U/2Oj2/NVM/KfXlg="
|
||||||
|
private val url = "https://fmr-1.echo-lot.app:8443"
|
||||||
|
private val token = "abc123-token_value"
|
||||||
|
|
||||||
|
@Test
|
||||||
|
fun parsesTheSpecFormat() {
|
||||||
|
val link = assertNotNull(
|
||||||
|
EnrollmentLink.parse(
|
||||||
|
"echolot://enroll?v=1&u=https%3A%2F%2Ffmr-1.echo-lot.app%3A8443" +
|
||||||
|
"&p=pin-sha256%3AzRV9qkiLnRexAeh4RrSfJzbPWO%2BU%2F2Oj2%2FNVM%2FKfXlg%3D" +
|
||||||
|
"&t=abc123-token_value"
|
||||||
|
)
|
||||||
|
)
|
||||||
|
assertEquals(url, link.controlUrl)
|
||||||
|
assertEquals(pin, link.pin, "the pin-sha256: prefix should be stripped for ControlClient")
|
||||||
|
assertEquals(token, link.token)
|
||||||
|
}
|
||||||
|
|
||||||
|
// The pin is base64: it contains +, / and = , every one of which means something else in a
|
||||||
|
// query string. Getting the decoding wrong yields a pin that silently never matches.
|
||||||
|
@Test
|
||||||
|
fun survivesBase64PunctuationThroughARoundTrip() {
|
||||||
|
val original = EnrollmentLink(url, pin, token)
|
||||||
|
val reparsed = assertNotNull(EnrollmentLink.parse(original.toUri()))
|
||||||
|
assertEquals(original, reparsed)
|
||||||
|
}
|
||||||
|
|
||||||
|
@Test
|
||||||
|
fun acceptsAnUnprefixedPin() {
|
||||||
|
val link = assertNotNull(EnrollmentLink.parse("echolot://enroll?v=1&u=$url&p=$pin&t=$token"))
|
||||||
|
assertEquals(pin, link.pin)
|
||||||
|
}
|
||||||
|
|
||||||
|
// A hand-assembled link often has its base64 pin pasted in raw. "+" then decodes to a space
|
||||||
|
// and the pin is wrong by one character — which does not fail loudly, it just never matches.
|
||||||
|
// Base64 contains no spaces, so restoring them is unambiguous.
|
||||||
|
@Test
|
||||||
|
fun repairsAPinWhosePlusSignsWereNotEncoded() {
|
||||||
|
val mangled = pin.replace("+", " ")
|
||||||
|
val link = assertNotNull(EnrollmentLink.parse("echolot://enroll?v=1&u=$url&p=$mangled&t=$token"))
|
||||||
|
assertEquals(pin, link.pin)
|
||||||
|
}
|
||||||
|
|
||||||
|
@Test
|
||||||
|
fun toleratesSurroundingWhitespaceAndCaseFromAPaste() {
|
||||||
|
val link = assertNotNull(
|
||||||
|
EnrollmentLink.parse(" ECHOLOT://ENROLL?v=1&u=$url&p=$pin&t=$token\n")
|
||||||
|
)
|
||||||
|
assertEquals(url, link.controlUrl)
|
||||||
|
}
|
||||||
|
|
||||||
|
// A half-applied link is a confusing failure much later; a rejected one is an obvious failure
|
||||||
|
// now. So anything missing or unrecognised parses to null rather than to a partial config.
|
||||||
|
@Test
|
||||||
|
fun rejectsAnythingItCannotFullyUnderstand() {
|
||||||
|
val bad = listOf(
|
||||||
|
null,
|
||||||
|
"",
|
||||||
|
"not a uri",
|
||||||
|
"https://fmr-1.echo-lot.app:8443", // a plain URL is not a bootstrap link
|
||||||
|
"echolot://run?v=1&u=$url&p=$pin&t=$token", // wrong action
|
||||||
|
"echolot://enroll?v=2&u=$url&p=$pin&t=$token", // unknown link version
|
||||||
|
"echolot://enroll?v=1&p=$pin&t=$token", // no url
|
||||||
|
"echolot://enroll?v=1&u=$url&t=$token", // no pin
|
||||||
|
"echolot://enroll?v=1&u=$url&p=$pin", // no token
|
||||||
|
"echolot://enroll?v=1&u=$url&p=pin-sha256:&t=$token", // empty pin
|
||||||
|
)
|
||||||
|
for (s in bad) assertNull(EnrollmentLink.parse(s), "should not parse: $s")
|
||||||
|
}
|
||||||
|
|
||||||
|
// The pin is the entire basis of trust, and it only protects the connection if the connection
|
||||||
|
// is TLS. A cleartext control URL would hand the token to anyone on the path.
|
||||||
|
@Test
|
||||||
|
fun refusesACleartextControlUrl() {
|
||||||
|
assertNull(EnrollmentLink.parse("echolot://enroll?v=1&u=http://fmr-1.echo-lot.app:8443&p=$pin&t=$token"))
|
||||||
|
}
|
||||||
|
|
||||||
|
@Test
|
||||||
|
fun aMissingVersionIsTreatedAsTheOnlyVersionThatExists() {
|
||||||
|
val link = assertNotNull(EnrollmentLink.parse("echolot://enroll?u=$url&p=$pin&t=$token"))
|
||||||
|
assertEquals(token, link.token)
|
||||||
|
}
|
||||||
|
|
||||||
|
@Test
|
||||||
|
fun trailingSlashesOnTheControlUrlAreNormalised() {
|
||||||
|
val link = assertNotNull(EnrollmentLink.parse("echolot://enroll?v=1&u=$url/&p=$pin&t=$token"))
|
||||||
|
assertEquals(url, link.controlUrl, "a trailing slash would double up when paths are appended")
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,42 @@
|
|||||||
|
#!/usr/bin/env bash
|
||||||
|
# SPDX-FileCopyrightText: 2026 Echolot contributors
|
||||||
|
# SPDX-License-Identifier: GPL-3.0-or-later
|
||||||
|
#
|
||||||
|
# Mints an enrollment link on the probe server and prints it — as text, as a QR code if
|
||||||
|
# `qrencode` is around, and as an adb command if a device is attached.
|
||||||
|
#
|
||||||
|
# The admin listener is localhost-only by design, so this goes over SSH. The link carries a
|
||||||
|
# single-use bearer token: treat it like a password until it is redeemed.
|
||||||
|
#
|
||||||
|
# Usage: echolot-app/scripts/enroll-link.sh [note]
|
||||||
|
set -euo pipefail
|
||||||
|
|
||||||
|
SSH_HOST="${ECHOLOT_SSH:-claude-echolot}"
|
||||||
|
NOTE="${1:-manual}"
|
||||||
|
|
||||||
|
MINTED=$(ssh -o BatchMode=yes "$SSH_HOST" \
|
||||||
|
"curl -s -X POST 'http://127.0.0.1:8444/admin/enroll-tokens?note=$NOTE'")
|
||||||
|
|
||||||
|
URI=$(printf '%s' "$MINTED" | python -c 'import json,sys;print(json.load(sys.stdin).get("enroll_uri",""))')
|
||||||
|
if [ -z "$URI" ]; then
|
||||||
|
echo "server returned no enroll_uri (needs server-v0.5.4+):" >&2
|
||||||
|
echo "$MINTED" >&2
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
|
||||||
|
echo "$URI"
|
||||||
|
echo
|
||||||
|
|
||||||
|
# A QR is the point of the format: scanning beats pasting a 200-character string onto a phone.
|
||||||
|
if command -v qrencode >/dev/null 2>&1; then
|
||||||
|
qrencode -t ANSIUTF8 "$URI"
|
||||||
|
else
|
||||||
|
echo "(install qrencode to get a scannable QR here)"
|
||||||
|
fi
|
||||||
|
|
||||||
|
# With a device attached, the deep link can be delivered straight to the app — no typing at all.
|
||||||
|
if command -v adb >/dev/null 2>&1 && [ -n "$(adb devices | sed -n '2p')" ]; then
|
||||||
|
echo
|
||||||
|
echo "attached device — deliver it directly with:"
|
||||||
|
echo " adb shell am start -a android.intent.action.VIEW -d '$URI'"
|
||||||
|
fi
|
||||||
@@ -19,13 +19,23 @@ UDP_PORT="${ECHOLOT_UDP_PORT:-8442}"
|
|||||||
CTL_URL="https://${CTL_HOST}:${CTL_PORT}"
|
CTL_URL="https://${CTL_HOST}:${CTL_PORT}"
|
||||||
|
|
||||||
echo "· minting enrollment token on ${SSH_HOST} ..."
|
echo "· minting enrollment token on ${SSH_HOST} ..."
|
||||||
TOKEN=$(ssh -o BatchMode=yes "$SSH_HOST" \
|
MINTED=$(ssh -o BatchMode=yes "$SSH_HOST" \
|
||||||
'curl -s -X POST http://127.0.0.1:8444/admin/enroll-tokens' \
|
'curl -s -X POST http://127.0.0.1:8444/admin/enroll-tokens')
|
||||||
| python -c 'import json,sys;print(json.load(sys.stdin)["token"])')
|
TOKEN=$(printf '%s' "$MINTED" | python -c 'import json,sys;print(json.load(sys.stdin)["token"])')
|
||||||
|
# The server also returns the whole §2.1 bootstrap link. LiveEnrollmentTest redeems that link,
|
||||||
|
# which is what proves the Go side and the Kotlin side agree on its encoding — a disagreement
|
||||||
|
# there yields a pin wrong by one character, which fails much later and looks like anything but.
|
||||||
|
ENROLL_URI=$(printf '%s' "$MINTED" \
|
||||||
|
| python -c 'import json,sys;print(json.load(sys.stdin).get("enroll_uri",""))')
|
||||||
|
|
||||||
echo "· enrolling over ${CTL_URL} ..."
|
echo "· enrolling over ${CTL_URL} ..."
|
||||||
CRED=$(curl -sk -X POST "${CTL_URL}/v1/enroll" -H "Authorization: Bearer ${TOKEN}" \
|
# A second token, because the one above is single-use and may be spent by LiveEnrollmentTest.
|
||||||
| python -c 'import json,sys;print(json.load(sys.stdin)["credential"])')
|
TOKEN2=$(ssh -o BatchMode=yes "$SSH_HOST" \
|
||||||
|
'curl -s -X POST http://127.0.0.1:8444/admin/enroll-tokens' \
|
||||||
|
| python -c 'import json,sys;print(json.load(sys.stdin)["token"])')
|
||||||
|
CRED=$(curl -sk -X POST "${CTL_URL}/v1/enroll" -H "Authorization: Bearer ${TOKEN2}" \
|
||||||
|
-H "X-Echolot-App-Version: 0.2.0" \
|
||||||
|
| python -c 'import json,sys;d=json.load(sys.stdin);print(d.get("device_credential") or d["credential"])')
|
||||||
|
|
||||||
echo "· computing SPKI pin from served cert ..."
|
echo "· computing SPKI pin from served cert ..."
|
||||||
PIN=$(echo | openssl s_client -connect "${CTL_HOST}:${CTL_PORT}" 2>/dev/null \
|
PIN=$(echo | openssl s_client -connect "${CTL_HOST}:${CTL_PORT}" 2>/dev/null \
|
||||||
@@ -43,5 +53,6 @@ ECHOLOT_LIVE_PIN="$PIN" \
|
|||||||
ECHOLOT_LIVE_CRED="$CRED" \
|
ECHOLOT_LIVE_CRED="$CRED" \
|
||||||
ECHOLOT_LIVE_UDP="${CTL_HOST}:${UDP_PORT}" \
|
ECHOLOT_LIVE_UDP="${CTL_HOST}:${UDP_PORT}" \
|
||||||
ECHOLOT_LIVE_TARGET="${ECHOLOT_LIVE_TARGET:-fmr}" \
|
ECHOLOT_LIVE_TARGET="${ECHOLOT_LIVE_TARGET:-fmr}" \
|
||||||
|
ECHOLOT_ENROLL_URI="$ENROLL_URI" \
|
||||||
./gradlew "$TASK" --tests "$FILTER" --info --rerun-tasks --console=plain \
|
./gradlew "$TASK" --tests "$FILTER" --info --rerun-tasks --console=plain \
|
||||||
2>&1 | grep -E "profile:|capabilities:|session:|echo |primed|mtu probe|downtrain|big_send|largest|observations bytes|Live[A-Za-z]*Test|BUILD|FAIL|PASS|^e:" || true
|
2>&1 | grep -E "profile:|capabilities:|session:|echo |primed|mtu probe|downtrain|big_send|largest|observations bytes|link:|parsed:|enrolled:|refused|Live[A-Za-z]*Test|BUILD|FAIL|PASS|^e:" || true
|
||||||
|
|||||||
@@ -31,11 +31,13 @@ import (
|
|||||||
"os/signal"
|
"os/signal"
|
||||||
"path/filepath"
|
"path/filepath"
|
||||||
"strconv"
|
"strconv"
|
||||||
|
"strings"
|
||||||
"sync/atomic"
|
"sync/atomic"
|
||||||
"syscall"
|
"syscall"
|
||||||
"time"
|
"time"
|
||||||
|
|
||||||
"echo-lot.app/server/internal/canarydns"
|
"echo-lot.app/server/internal/canarydns"
|
||||||
|
"echo-lot.app/server/internal/compat"
|
||||||
"echo-lot.app/server/internal/config"
|
"echo-lot.app/server/internal/config"
|
||||||
"echo-lot.app/server/internal/control"
|
"echo-lot.app/server/internal/control"
|
||||||
"echo-lot.app/server/internal/dataplane"
|
"echo-lot.app/server/internal/dataplane"
|
||||||
@@ -109,7 +111,15 @@ func serve(cfg *config.Config) error {
|
|||||||
TLSConfig: &tls.Config{Certificates: []tls.Certificate{cert}, MinVersion: tls.VersionTLS12},
|
TLSConfig: &tls.Config{Certificates: []tls.Certificate{cert}, MinVersion: tls.VersionTLS12},
|
||||||
}
|
}
|
||||||
|
|
||||||
caps := []string{"udp-probe", "delayed-echo", "connect-back", "http-echo", "downtrain", "big-send"}
|
caps := []string{"udp-probe", "delayed-echo", "connect-back", "http-echo", "downtrain", "big-send", "throughput"}
|
||||||
|
// Crafted fragments need a raw socket. Advertised only when one can actually be opened —
|
||||||
|
// a capability we cannot deliver turns a missing feature into a failed measurement.
|
||||||
|
rawFrag := dataplane.RawFragSupported()
|
||||||
|
if rawFrag {
|
||||||
|
caps = append(caps, "frag-send")
|
||||||
|
} else {
|
||||||
|
slog.Info("frag-send unavailable: no raw socket (needs CAP_NET_RAW)")
|
||||||
|
}
|
||||||
if len(config.Addrs(cfg.TCPListen)) > 0 {
|
if len(config.Addrs(cfg.TCPListen)) > 0 {
|
||||||
caps = append(caps, "tcp-echo", "tls-echo")
|
caps = append(caps, "tcp-echo", "tls-echo")
|
||||||
}
|
}
|
||||||
@@ -131,6 +141,15 @@ func serve(cfg *config.Config) error {
|
|||||||
"retention_days", cfg.UploadRetentionDays, "max_runs_per_device", cfg.UploadMaxRuns)
|
"retention_days", cfg.UploadRetentionDays, "max_runs_per_device", cfg.UploadMaxRuns)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// A malformed window is fatal rather than ignored: an operator who set a restriction must
|
||||||
|
// not end up running without one because of a typo.
|
||||||
|
appRange, err := compat.ParseRange(cfg.MinAppVersion, cfg.MaxAppVersion)
|
||||||
|
if err != nil {
|
||||||
|
return fmt.Errorf("app version window: %w", err)
|
||||||
|
}
|
||||||
|
slog.Info("client compatibility", "accepts_app", appRange.String(),
|
||||||
|
"protocol", control.ProtocolVersion, "schema", control.SchemaVersion)
|
||||||
|
|
||||||
ctl := &control.Server{
|
ctl := &control.Server{
|
||||||
Store: st, Sessions: sessions, Name: cfg.Name,
|
Store: st, Sessions: sessions, Name: cfg.Name,
|
||||||
UDPPort: mustPort(firstAddr(cfg.UDPListen)), TCPPort: mustPort(firstAddr(cfg.TCPListen)),
|
UDPPort: mustPort(firstAddr(cfg.UDPListen)), TCPPort: mustPort(firstAddr(cfg.TCPListen)),
|
||||||
@@ -140,7 +159,15 @@ func serve(cfg *config.Config) error {
|
|||||||
BigSend: dp.BigSend,
|
BigSend: dp.BigSend,
|
||||||
TCPRecent: func(ip string) any { return tcpSrv.RecentFor(ip) },
|
TCPRecent: func(ip string) any { return tcpSrv.RecentFor(ip) },
|
||||||
Runs: runStore,
|
Runs: runStore,
|
||||||
|
AppRange: appRange,
|
||||||
|
PublicControlURL: publicControlURL(cfg),
|
||||||
}
|
}
|
||||||
|
// Left nil when there is no raw socket, so the handler answers "not implemented" with a
|
||||||
|
// reason rather than failing somewhere deeper.
|
||||||
|
if rawFrag {
|
||||||
|
ctl.FragSend = dp.FragSend
|
||||||
|
}
|
||||||
|
ctl.DownThroughput = dp.DownThroughput
|
||||||
|
|
||||||
ctx, stop := signal.NotifyContext(context.Background(), syscall.SIGINT, syscall.SIGTERM)
|
ctx, stop := signal.NotifyContext(context.Background(), syscall.SIGINT, syscall.SIGTERM)
|
||||||
defer stop()
|
defer stop()
|
||||||
@@ -221,7 +248,17 @@ func serve(cfg *config.Config) error {
|
|||||||
http.Error(w, err.Error(), 500)
|
http.Error(w, err.Error(), 500)
|
||||||
return
|
return
|
||||||
}
|
}
|
||||||
fmt.Fprintf(w, `{"token":%q,"expires_in_s":86400}`+"\n", tok)
|
// The whole bootstrap, not just the token: this is what gets pasted or turned into a
|
||||||
|
// QR code, and assembling it here is what keeps an operator from transcribing a pin by
|
||||||
|
// hand — a pin wrong by one character fails as an inscrutable TLS error days later.
|
||||||
|
w.Header().Set("Content-Type", "application/json")
|
||||||
|
enc := json.NewEncoder(w)
|
||||||
|
enc.SetEscapeHTML(false) // the link is full of / and =; escaping them helps nobody
|
||||||
|
_ = enc.Encode(map[string]any{
|
||||||
|
"token": tok,
|
||||||
|
"expires_in_s": 86400,
|
||||||
|
"enroll_uri": ctl.EnrollmentLink(tok),
|
||||||
|
})
|
||||||
})
|
})
|
||||||
adminSrv := &http.Server{Addr: cfg.AdminListen, Handler: admin, ReadHeaderTimeout: 10 * time.Second}
|
adminSrv := &http.Server{Addr: cfg.AdminListen, Handler: admin, ReadHeaderTimeout: 10 * time.Second}
|
||||||
go func() { errCh <- fmt.Errorf("admin: %w", adminSrv.ListenAndServe()) }()
|
go func() { errCh <- fmt.Errorf("admin: %w", adminSrv.ListenAndServe()) }()
|
||||||
@@ -435,3 +472,24 @@ func loadOrCreateCert(cfg *config.Config) (tls.Certificate, error) {
|
|||||||
slog.Info("generated self-signed certificate", "cert", certPath)
|
slog.Info("generated self-signed certificate", "cert", certPath)
|
||||||
return tls.X509KeyPair(certPem, keyPem)
|
return tls.X509KeyPair(certPem, keyPem)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// publicControlURL is where clients should reach this server's control plane.
|
||||||
|
//
|
||||||
|
// Configured wins; otherwise the first control listen address is used, which is correct for the
|
||||||
|
// plain case (bind an address, hand out that address). A wildcard bind has no single right answer,
|
||||||
|
// so it is left to the operator rather than guessed — a link pointing at 0.0.0.0 is worse than a
|
||||||
|
// link the operator was told to configure.
|
||||||
|
func publicControlURL(cfg *config.Config) string {
|
||||||
|
if cfg.PublicControlURL != "" {
|
||||||
|
return strings.TrimRight(cfg.PublicControlURL, "/")
|
||||||
|
}
|
||||||
|
addr := firstAddr(cfg.ControlListen)
|
||||||
|
if addr == "" {
|
||||||
|
return ""
|
||||||
|
}
|
||||||
|
if strings.HasPrefix(addr, ":") || strings.HasPrefix(addr, "0.0.0.0:") || strings.HasPrefix(addr, "[::]:") {
|
||||||
|
slog.Warn("control plane is bound to a wildcard address; set ECHOLOT_PUBLIC_URL "+
|
||||||
|
"so enrollment links point somewhere reachable", "listen", addr)
|
||||||
|
}
|
||||||
|
return "https://" + addr
|
||||||
|
}
|
||||||
|
|||||||
@@ -0,0 +1,200 @@
|
|||||||
|
// SPDX-FileCopyrightText: 2026 Echolot contributors
|
||||||
|
// SPDX-License-Identifier: GPL-3.0-or-later
|
||||||
|
|
||||||
|
// Package compat decides whether two Echolot builds should talk to each other.
|
||||||
|
//
|
||||||
|
// Versions are SemVer. What is actually being checked, though, is not "is this release recent"
|
||||||
|
// but "does this peer speak a wire protocol and schema I understand" — the version is a proxy for
|
||||||
|
// that, and the proxy only holds because we bump the breaking axis when the contract changes.
|
||||||
|
// So the bounds here are set at *breaking boundaries*, not at every release: a patch bump must
|
||||||
|
// never strand a fleet, and the range must not need editing to ship a bugfix.
|
||||||
|
//
|
||||||
|
// The one rule that shapes the rest: refusing a peer must still tell it why. A client that cannot
|
||||||
|
// reach the profile endpoint cannot learn what version it should be, so it has nothing to show its
|
||||||
|
// user but a network error. GET /v1/profile is therefore always reachable, whatever the range says.
|
||||||
|
package compat
|
||||||
|
|
||||||
|
import (
|
||||||
|
"fmt"
|
||||||
|
"strconv"
|
||||||
|
"strings"
|
||||||
|
)
|
||||||
|
|
||||||
|
// Version is a parsed SemVer. Build metadata is discarded (it is explicitly not part of
|
||||||
|
// precedence); pre-release is kept and compared, because "1.0.0-rc1" must sort below "1.0.0".
|
||||||
|
type Version struct {
|
||||||
|
Major, Minor, Patch int
|
||||||
|
Pre string
|
||||||
|
}
|
||||||
|
|
||||||
|
// Parse accepts "1.2.3", "v1.2.3" and our namespaced release tags ("server-v1.2.3"), because
|
||||||
|
// those are the forms that actually reach this code: a tag, a --version output, and an HTTP
|
||||||
|
// header written by three different pieces of software.
|
||||||
|
func Parse(s string) (Version, bool) {
|
||||||
|
s = strings.TrimSpace(s)
|
||||||
|
// Strip a tag prefix ending in "v" ("v1.2.3", "server-v1.2.3"). Guarded on the prefix having
|
||||||
|
// no digits so a pre-release identifier that happens to contain a "v" is left alone.
|
||||||
|
if i := strings.LastIndexByte(s, 'v'); i >= 0 && i+1 < len(s) &&
|
||||||
|
s[i+1] >= '0' && s[i+1] <= '9' && !strings.ContainsAny(s[:i], "0123456789") {
|
||||||
|
s = s[i+1:]
|
||||||
|
}
|
||||||
|
if plus := strings.IndexByte(s, '+'); plus >= 0 {
|
||||||
|
s = s[:plus]
|
||||||
|
}
|
||||||
|
var pre string
|
||||||
|
if dash := strings.IndexByte(s, '-'); dash >= 0 {
|
||||||
|
pre, s = s[dash+1:], s[:dash]
|
||||||
|
}
|
||||||
|
parts := strings.Split(s, ".")
|
||||||
|
if len(parts) != 3 {
|
||||||
|
return Version{}, false
|
||||||
|
}
|
||||||
|
out := Version{Pre: pre}
|
||||||
|
for i, p := range parts {
|
||||||
|
n, err := strconv.Atoi(p)
|
||||||
|
if err != nil || n < 0 {
|
||||||
|
return Version{}, false
|
||||||
|
}
|
||||||
|
switch i {
|
||||||
|
case 0:
|
||||||
|
out.Major = n
|
||||||
|
case 1:
|
||||||
|
out.Minor = n
|
||||||
|
case 2:
|
||||||
|
out.Patch = n
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return out, true
|
||||||
|
}
|
||||||
|
|
||||||
|
func (v Version) String() string {
|
||||||
|
s := fmt.Sprintf("%d.%d.%d", v.Major, v.Minor, v.Patch)
|
||||||
|
if v.Pre != "" {
|
||||||
|
s += "-" + v.Pre
|
||||||
|
}
|
||||||
|
return s
|
||||||
|
}
|
||||||
|
|
||||||
|
// Compare returns -1, 0 or 1. A pre-release sorts below the same version without one (SemVer §11);
|
||||||
|
// two pre-releases compare lexically, which is close enough for the identifiers we use.
|
||||||
|
func (v Version) Compare(o Version) int {
|
||||||
|
for _, d := range []int{v.Major - o.Major, v.Minor - o.Minor, v.Patch - o.Patch} {
|
||||||
|
if d != 0 {
|
||||||
|
return sign(d)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
switch {
|
||||||
|
case v.Pre == o.Pre:
|
||||||
|
return 0
|
||||||
|
case v.Pre == "":
|
||||||
|
return 1
|
||||||
|
case o.Pre == "":
|
||||||
|
return -1
|
||||||
|
case v.Pre < o.Pre:
|
||||||
|
return -1
|
||||||
|
}
|
||||||
|
return 1
|
||||||
|
}
|
||||||
|
|
||||||
|
func (v Version) Less(o Version) bool { return v.Compare(o) < 0 }
|
||||||
|
|
||||||
|
// NextBreaking is the first version that may break compatibility with v.
|
||||||
|
//
|
||||||
|
// Below 1.0.0 the minor is the breaking axis (SemVer §4: anything may change in 0.y), so 0.4.2's
|
||||||
|
// next break is 0.5.0, not 1.0.0. Getting this wrong in the permissive direction would let a
|
||||||
|
// 0.5 server accept a 0.4 app that cannot speak to it.
|
||||||
|
func (v Version) NextBreaking() Version {
|
||||||
|
if v.Major == 0 {
|
||||||
|
return Version{Major: 0, Minor: v.Minor + 1}
|
||||||
|
}
|
||||||
|
return Version{Major: v.Major + 1}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Range is [Min, Max): minimum inclusive, maximum exclusive. An unset Max means unbounded.
|
||||||
|
//
|
||||||
|
// Exclusive on the upper end because the useful bound is always "the version that broke it",
|
||||||
|
// and writing that literally ("< 1.0.0") is unambiguous in a way that "<= 0.999.999" is not.
|
||||||
|
type Range struct {
|
||||||
|
Min Version
|
||||||
|
Max Version
|
||||||
|
HasMax bool
|
||||||
|
}
|
||||||
|
|
||||||
|
func (r Range) Contains(v Version) bool {
|
||||||
|
if v.Less(r.Min) {
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
if r.HasMax && !v.Less(r.Max) {
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
return true
|
||||||
|
}
|
||||||
|
|
||||||
|
func (r Range) String() string {
|
||||||
|
if !r.HasMax {
|
||||||
|
return ">= " + r.Min.String()
|
||||||
|
}
|
||||||
|
return ">= " + r.Min.String() + ", < " + r.Max.String()
|
||||||
|
}
|
||||||
|
|
||||||
|
// ParseRange builds a range from two strings; an empty max means unbounded. A malformed bound is
|
||||||
|
// an error rather than a silently ignored one: a typo in an operator's config must not quietly
|
||||||
|
// turn a restriction off.
|
||||||
|
func ParseRange(min, max string) (Range, error) {
|
||||||
|
lo, ok := Parse(min)
|
||||||
|
if !ok {
|
||||||
|
return Range{}, fmt.Errorf("bad minimum version %q", min)
|
||||||
|
}
|
||||||
|
if strings.TrimSpace(max) == "" {
|
||||||
|
return Range{Min: lo}, nil
|
||||||
|
}
|
||||||
|
hi, ok := Parse(max)
|
||||||
|
if !ok {
|
||||||
|
return Range{}, fmt.Errorf("bad maximum version %q", max)
|
||||||
|
}
|
||||||
|
if hi.Less(lo) || hi.Compare(lo) == 0 {
|
||||||
|
return Range{}, fmt.Errorf("maximum %s is not above minimum %s", max, min)
|
||||||
|
}
|
||||||
|
return Range{Min: lo, Max: hi, HasMax: true}, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// Verdict is the outcome of a compatibility check.
|
||||||
|
type Verdict int
|
||||||
|
|
||||||
|
const (
|
||||||
|
OK Verdict = iota
|
||||||
|
TooOld
|
||||||
|
TooNew
|
||||||
|
// Unknown means the peer did not say, or said something unparseable — a development build,
|
||||||
|
// or a client too old to send its version at all.
|
||||||
|
Unknown
|
||||||
|
)
|
||||||
|
|
||||||
|
// Check reports whether peer falls in r, and explains the answer in words meant for a person.
|
||||||
|
// The message is deliberately actionable: it names both versions and what to do about it, since
|
||||||
|
// it is the only thing the user on the other end will see.
|
||||||
|
func Check(peer string, r Range, peerName string) (Verdict, string) {
|
||||||
|
v, ok := Parse(peer)
|
||||||
|
if !ok {
|
||||||
|
return Unknown, fmt.Sprintf("%s did not report a usable version (%q); proceeding without a compatibility check", peerName, peer)
|
||||||
|
}
|
||||||
|
switch {
|
||||||
|
case v.Less(r.Min):
|
||||||
|
return TooOld, fmt.Sprintf("%s %s is older than this build supports (needs %s). Update the %s.",
|
||||||
|
peerName, v, r, peerName)
|
||||||
|
case r.HasMax && !v.Less(r.Max):
|
||||||
|
return TooNew, fmt.Sprintf("%s %s is newer than this build supports (accepts %s). Update this side, or use a version of the %s within that range.",
|
||||||
|
peerName, v, r, peerName)
|
||||||
|
}
|
||||||
|
return OK, ""
|
||||||
|
}
|
||||||
|
|
||||||
|
func sign(d int) int {
|
||||||
|
if d < 0 {
|
||||||
|
return -1
|
||||||
|
}
|
||||||
|
if d > 0 {
|
||||||
|
return 1
|
||||||
|
}
|
||||||
|
return 0
|
||||||
|
}
|
||||||
@@ -0,0 +1,156 @@
|
|||||||
|
// SPDX-FileCopyrightText: 2026 Echolot contributors
|
||||||
|
// SPDX-License-Identifier: GPL-3.0-or-later
|
||||||
|
|
||||||
|
package compat
|
||||||
|
|
||||||
|
import "testing"
|
||||||
|
|
||||||
|
func TestParseAcceptsTheFormsThatActuallyReachUs(t *testing.T) {
|
||||||
|
cases := map[string]Version{
|
||||||
|
"1.2.3": {Major: 1, Minor: 2, Patch: 3},
|
||||||
|
"v1.2.3": {Major: 1, Minor: 2, Patch: 3},
|
||||||
|
"server-v0.4.2": {Minor: 4, Patch: 2},
|
||||||
|
" 0.2.0 ": {Minor: 2},
|
||||||
|
"1.0.0-rc1": {Major: 1, Pre: "rc1"},
|
||||||
|
"1.0.0+build.7": {Major: 1},
|
||||||
|
"1.0.0-rc1+meta": {Major: 1, Pre: "rc1"},
|
||||||
|
}
|
||||||
|
for in, want := range cases {
|
||||||
|
got, ok := Parse(in)
|
||||||
|
if !ok || got != want {
|
||||||
|
t.Errorf("Parse(%q) = %v,%v; want %v", in, got, ok, want)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
// A pre-release identifier containing a "v" must not be mistaken for a tag prefix.
|
||||||
|
if got, ok := Parse("1.2.3-rcv1"); !ok || got.Pre != "rcv1" || got.Major != 1 {
|
||||||
|
t.Errorf("Parse(1.2.3-rcv1) = %v,%v", got, ok)
|
||||||
|
}
|
||||||
|
for _, bad := range []string{"", "dev", "1.2", "1.2.3.4", "x.y.z", "-1.0.0", "1.2.beta"} {
|
||||||
|
if _, ok := Parse(bad); ok {
|
||||||
|
t.Errorf("Parse(%q) should have failed", bad)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestCompareOrdersPreReleasesBelowTheirRelease(t *testing.T) {
|
||||||
|
lt := func(a, b string) {
|
||||||
|
t.Helper()
|
||||||
|
x, _ := Parse(a)
|
||||||
|
y, _ := Parse(b)
|
||||||
|
if x.Compare(y) != -1 || y.Compare(x) != 1 {
|
||||||
|
t.Errorf("expected %s < %s", a, b)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
lt("0.9.9", "1.0.0")
|
||||||
|
lt("1.0.0", "1.0.1")
|
||||||
|
lt("1.0.0", "1.1.0")
|
||||||
|
lt("1.0.0-rc1", "1.0.0")
|
||||||
|
lt("1.0.0-rc1", "1.0.0-rc2")
|
||||||
|
a, _ := Parse("1.2.3")
|
||||||
|
b, _ := Parse("v1.2.3")
|
||||||
|
if a.Compare(b) != 0 {
|
||||||
|
t.Error("the same version written two ways must compare equal")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Below 1.0.0 the minor is the breaking axis. Treating 1.0.0 as the next break for a 0.x build
|
||||||
|
// would let a 0.5 server accept a 0.4 client it cannot actually talk to.
|
||||||
|
func TestNextBreakingUsesTheMinorBelowOne(t *testing.T) {
|
||||||
|
cases := map[string]string{
|
||||||
|
"0.4.2": "0.5.0",
|
||||||
|
"0.0.9": "0.1.0",
|
||||||
|
"1.2.3": "2.0.0",
|
||||||
|
"2.0.0": "3.0.0",
|
||||||
|
}
|
||||||
|
for in, want := range cases {
|
||||||
|
v, _ := Parse(in)
|
||||||
|
if got := v.NextBreaking().String(); got != want {
|
||||||
|
t.Errorf("NextBreaking(%s) = %s, want %s", in, got, want)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestRangeIsMinInclusiveMaxExclusive(t *testing.T) {
|
||||||
|
r, err := ParseRange("0.2.0", "1.0.0")
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
in := []string{"0.2.0", "0.2.1", "0.9.9", "1.0.0-rc1"}
|
||||||
|
out := []string{"0.1.9", "1.0.0", "1.0.1", "2.0.0"}
|
||||||
|
for _, s := range in {
|
||||||
|
v, _ := Parse(s)
|
||||||
|
if !r.Contains(v) {
|
||||||
|
t.Errorf("%s should be inside %s", s, r)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
for _, s := range out {
|
||||||
|
v, _ := Parse(s)
|
||||||
|
if r.Contains(v) {
|
||||||
|
t.Errorf("%s should be outside %s", s, r)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestUnboundedRangeHasNoCeiling(t *testing.T) {
|
||||||
|
r, err := ParseRange("0.2.0", "")
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
v, _ := Parse("99.0.0")
|
||||||
|
if !r.Contains(v) {
|
||||||
|
t.Error("an empty maximum must mean unbounded")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// A typo in an operator's config must not silently disable the restriction it was meant to set.
|
||||||
|
func TestMalformedBoundsAreErrorsNotSilentPermissiveness(t *testing.T) {
|
||||||
|
for _, c := range [][2]string{
|
||||||
|
{"nonsense", "1.0.0"},
|
||||||
|
{"0.2.0", "nonsense"},
|
||||||
|
{"1.0.0", "0.9.0"}, // max below min
|
||||||
|
{"1.0.0", "1.0.0"}, // empty window: nothing could ever satisfy it
|
||||||
|
} {
|
||||||
|
if _, err := ParseRange(c[0], c[1]); err == nil {
|
||||||
|
t.Errorf("ParseRange(%q, %q) should have failed", c[0], c[1])
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestCheckExplainsItself(t *testing.T) {
|
||||||
|
r, _ := ParseRange("0.2.0", "1.0.0")
|
||||||
|
|
||||||
|
if v, msg := Check("0.5.0", r, "app"); v != OK || msg != "" {
|
||||||
|
t.Errorf("in-range check should pass silently: %v %q", v, msg)
|
||||||
|
}
|
||||||
|
v, msg := Check("0.1.0", r, "app")
|
||||||
|
if v != TooOld {
|
||||||
|
t.Fatalf("want TooOld, got %v", v)
|
||||||
|
}
|
||||||
|
for _, want := range []string{"0.1.0", "0.2.0", "Update"} {
|
||||||
|
if !contains(msg, want) {
|
||||||
|
t.Errorf("the refusal must name %q so the user can act on it: %q", want, msg)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if v, _ := Check("1.4.0", r, "app"); v != TooNew {
|
||||||
|
t.Errorf("want TooNew, got %v", v)
|
||||||
|
}
|
||||||
|
|
||||||
|
// A development build reports "dev". Locking developers out of their own server would be a
|
||||||
|
// poor trade for a check that exists to prevent confusing failures.
|
||||||
|
if v, msg := Check("dev", r, "server"); v != Unknown || msg == "" {
|
||||||
|
t.Errorf("unparseable version should be Unknown with an explanation, got %v %q", v, msg)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func contains(h, n string) bool {
|
||||||
|
return len(h) >= len(n) && (h == n || len(n) == 0 || indexOf(h, n) >= 0)
|
||||||
|
}
|
||||||
|
|
||||||
|
func indexOf(h, n string) int {
|
||||||
|
for i := 0; i+len(n) <= len(h); i++ {
|
||||||
|
if h[i:i+len(n)] == n {
|
||||||
|
return i
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return -1
|
||||||
|
}
|
||||||
@@ -58,6 +58,15 @@ type Config struct {
|
|||||||
UploadMaxRuns int // ECHOLOT_UPLOAD_MAX_RUNS / --upload-max-runs (per device)
|
UploadMaxRuns int // ECHOLOT_UPLOAD_MAX_RUNS / --upload-max-runs (per device)
|
||||||
UploadMinAnon string // ECHOLOT_UPLOAD_MIN_ANONYMIZATION / --upload-min-anonymization
|
UploadMinAnon string // ECHOLOT_UPLOAD_MIN_ANONYMIZATION / --upload-min-anonymization
|
||||||
|
|
||||||
|
// Client compatibility window. Bounds are SemVer; an empty maximum means unbounded. The
|
||||||
|
// defaults sit at breaking boundaries, so shipping a patch never requires changing them.
|
||||||
|
MinAppVersion string // ECHOLOT_MIN_APP_VERSION / --min-app-version
|
||||||
|
MaxAppVersion string // ECHOLOT_MAX_APP_VERSION / --max-app-version (exclusive)
|
||||||
|
|
||||||
|
// Where clients reach the control plane, for enrollment links. Empty = derive from the
|
||||||
|
// first control listen address.
|
||||||
|
PublicControlURL string // ECHOLOT_PUBLIC_URL / --public-url
|
||||||
|
|
||||||
// Mode
|
// Mode
|
||||||
Docker bool // --docker (or autodetected; env ECHOLOT_DOCKER=1 forces)
|
Docker bool // --docker (or autodetected; env ECHOLOT_DOCKER=1 forces)
|
||||||
}
|
}
|
||||||
@@ -106,6 +115,9 @@ func Load(args []string) (*Config, *Actions, error) {
|
|||||||
fs.IntVar(&c.UploadRetentionDays, "upload-retention-days", envInt("UPLOAD_RETENTION_DAYS", 90), "delete uploaded runs older than this; 0 disables")
|
fs.IntVar(&c.UploadRetentionDays, "upload-retention-days", envInt("UPLOAD_RETENTION_DAYS", 90), "delete uploaded runs older than this; 0 disables")
|
||||||
fs.IntVar(&c.UploadMaxRuns, "upload-max-runs", envInt("UPLOAD_MAX_RUNS", 200), "keep at most this many runs per device; 0 disables")
|
fs.IntVar(&c.UploadMaxRuns, "upload-max-runs", envInt("UPLOAD_MAX_RUNS", 200), "keep at most this many runs per device; 0 disables")
|
||||||
fs.StringVar(&c.UploadMinAnon, "upload-min-anonymization", envOr("UPLOAD_MIN_ANONYMIZATION", "full"), "least anonymization accepted: full|balanced|strict")
|
fs.StringVar(&c.UploadMinAnon, "upload-min-anonymization", envOr("UPLOAD_MIN_ANONYMIZATION", "full"), "least anonymization accepted: full|balanced|strict")
|
||||||
|
fs.StringVar(&c.PublicControlURL, "public-url", envOr("PUBLIC_URL", ""), "public control-plane URL for enrollment links, e.g. https://probe.example.net:8443")
|
||||||
|
fs.StringVar(&c.MinAppVersion, "min-app-version", envOr("MIN_APP_VERSION", "0.2.0"), "oldest app version this server will serve (SemVer, inclusive)")
|
||||||
|
fs.StringVar(&c.MaxAppVersion, "max-app-version", envOr("MAX_APP_VERSION", "1.0.0"), "first app version this server will refuse (SemVer, exclusive); empty = unbounded")
|
||||||
fs.BoolVar(&c.Docker, "docker", envOr("DOCKER", "") == "1", "force container mode (config from env, no systemd/self-update)")
|
fs.BoolVar(&c.Docker, "docker", envOr("DOCKER", "") == "1", "force container mode (config from env, no systemd/self-update)")
|
||||||
|
|
||||||
fs.BoolVar(&a.InstallSystemd, "install-systemd", false, "install a systemd unit for this binary and exit")
|
fs.BoolVar(&a.InstallSystemd, "install-systemd", false, "install a systemd unit for this binary and exit")
|
||||||
|
|||||||
@@ -0,0 +1,113 @@
|
|||||||
|
// SPDX-FileCopyrightText: 2026 Echolot contributors
|
||||||
|
// SPDX-License-Identifier: GPL-3.0-or-later
|
||||||
|
|
||||||
|
package control
|
||||||
|
|
||||||
|
import (
|
||||||
|
"encoding/json"
|
||||||
|
"net/http"
|
||||||
|
"net/http/httptest"
|
||||||
|
"strings"
|
||||||
|
"testing"
|
||||||
|
|
||||||
|
"echo-lot.app/server/internal/compat"
|
||||||
|
"echo-lot.app/server/internal/store"
|
||||||
|
)
|
||||||
|
|
||||||
|
func rangeOrDie(t *testing.T, min, max string) compat.Range {
|
||||||
|
t.Helper()
|
||||||
|
r, err := compat.ParseRange(min, max)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
return r
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestGateRefusesOutOfRangeApps(t *testing.T) {
|
||||||
|
s := &Server{AppRange: rangeOrDie(t, "0.2.0", "1.0.0")}
|
||||||
|
reached := false
|
||||||
|
h := s.requireCompatibleApp(func(w http.ResponseWriter, _ *http.Request) {
|
||||||
|
reached = true
|
||||||
|
w.WriteHeader(http.StatusOK)
|
||||||
|
})
|
||||||
|
|
||||||
|
for _, tc := range []struct {
|
||||||
|
version string
|
||||||
|
wantCode int
|
||||||
|
wantThru bool
|
||||||
|
}{
|
||||||
|
{"0.2.0", http.StatusOK, true}, // exactly the minimum is in range
|
||||||
|
{"0.9.9", http.StatusOK, true},
|
||||||
|
{"0.1.9", http.StatusUpgradeRequired, false}, // too old
|
||||||
|
{"1.0.0", http.StatusUpgradeRequired, false}, // maximum is exclusive
|
||||||
|
{"2.0.0", http.StatusUpgradeRequired, false}, // too new
|
||||||
|
{"", http.StatusOK, true}, // unknown: allowed, see below
|
||||||
|
{"dev", http.StatusOK, true}, // development build
|
||||||
|
} {
|
||||||
|
reached = false
|
||||||
|
req := httptest.NewRequest("GET", "/v1/sessions", nil)
|
||||||
|
if tc.version != "" {
|
||||||
|
req.Header.Set(AppVersionHeader, tc.version)
|
||||||
|
}
|
||||||
|
rec := httptest.NewRecorder()
|
||||||
|
h(rec, req)
|
||||||
|
if rec.Code != tc.wantCode || reached != tc.wantThru {
|
||||||
|
t.Errorf("app %q: got code=%d reached=%v, want code=%d reached=%v",
|
||||||
|
tc.version, rec.Code, reached, tc.wantCode, tc.wantThru)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// A refusal that does not say what version to install is only marginally better than a timeout.
|
||||||
|
func TestRefusalNamesTheAcceptedWindow(t *testing.T) {
|
||||||
|
s := &Server{AppRange: rangeOrDie(t, "0.2.0", "1.0.0")}
|
||||||
|
rec := httptest.NewRecorder()
|
||||||
|
req := httptest.NewRequest("POST", "/v1/runs", nil)
|
||||||
|
req.Header.Set(AppVersionHeader, "0.1.0")
|
||||||
|
s.requireCompatibleApp(func(http.ResponseWriter, *http.Request) {})(rec, req)
|
||||||
|
|
||||||
|
var body map[string]any
|
||||||
|
if err := json.Unmarshal(rec.Body.Bytes(), &body); err != nil {
|
||||||
|
t.Fatalf("refusal body is not JSON: %v", err)
|
||||||
|
}
|
||||||
|
for _, key := range []string{"error", "app_version", "accepts_app", "protocol_version"} {
|
||||||
|
if body[key] == nil || body[key] == "" {
|
||||||
|
t.Errorf("refusal omits %q, so the client cannot explain itself: %v", key, body)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if !strings.Contains(body["accepts_app"].(string), "0.2.0") {
|
||||||
|
t.Errorf("accepts_app should state the minimum: %v", body["accepts_app"])
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// The profile is how a refused client learns which version it needs. Gating it would leave the
|
||||||
|
// user with a network error instead of an answer, which defeats the whole check.
|
||||||
|
func TestProfileIsReachableRegardlessOfVersion(t *testing.T) {
|
||||||
|
st, err := store.Open(t.TempDir())
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
s := &Server{
|
||||||
|
AppRange: rangeOrDie(t, "9.0.0", ""), // nothing current could satisfy this
|
||||||
|
Store: st,
|
||||||
|
}
|
||||||
|
req := httptest.NewRequest("GET", "/v1/profile", nil)
|
||||||
|
req.Header.Set(AppVersionHeader, "0.1.0")
|
||||||
|
rec := httptest.NewRecorder()
|
||||||
|
s.Handler().ServeHTTP(rec, req)
|
||||||
|
|
||||||
|
// The request carries no credential, so the handler answers 401 — the point is that it is
|
||||||
|
// the *handler* answering, not the version gate turning it into 426.
|
||||||
|
if rec.Code == http.StatusUpgradeRequired {
|
||||||
|
t.Fatal("the profile endpoint must never be gated on app version")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestZeroValueRangeFallsBackToTheBuiltInDefault(t *testing.T) {
|
||||||
|
s := &Server{} // nothing configured
|
||||||
|
got := s.appRange()
|
||||||
|
want := DefaultAppRange()
|
||||||
|
if got.Min != want.Min || got.HasMax != want.HasMax || got.Max != want.Max {
|
||||||
|
t.Fatalf("appRange() = %s, want the built-in default %s", got, want)
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -20,10 +20,12 @@ import (
|
|||||||
"net"
|
"net"
|
||||||
"net/http"
|
"net/http"
|
||||||
"net/netip"
|
"net/netip"
|
||||||
|
"net/url"
|
||||||
"strconv"
|
"strconv"
|
||||||
"strings"
|
"strings"
|
||||||
"time"
|
"time"
|
||||||
|
|
||||||
|
"echo-lot.app/server/internal/compat"
|
||||||
"echo-lot.app/server/internal/dataplane"
|
"echo-lot.app/server/internal/dataplane"
|
||||||
"echo-lot.app/server/internal/runs"
|
"echo-lot.app/server/internal/runs"
|
||||||
"echo-lot.app/server/internal/session"
|
"echo-lot.app/server/internal/session"
|
||||||
@@ -57,6 +59,11 @@ type Server struct {
|
|||||||
BigSend func(sess *session.Session, g *session.Grant, sizes []int, df bool) ([]dataplane.BigSendResult, error)
|
BigSend func(sess *session.Session, g *session.Grant, sizes []int, df bool) ([]dataplane.BigSendResult, error)
|
||||||
// Runs stores uploaded measurement documents (may be nil: uploads unsupported).
|
// Runs stores uploaded measurement documents (may be nil: uploads unsupported).
|
||||||
Runs *runs.Store
|
Runs *runs.Store
|
||||||
|
// FragSend emits one datagram as hand-built IP fragments in a chosen order (may be nil:
|
||||||
|
// needs a raw socket, so it is unavailable to an unprivileged server).
|
||||||
|
FragSend func(sess *session.Session, g *session.Grant, sizeBytes int, mode dataplane.FragMode, fragSize int) (dataplane.FragResult, error)
|
||||||
|
// DownThroughput sends paced traffic toward the client for a bounded time (may be nil).
|
||||||
|
DownThroughput func(sess *session.Session, g *session.Grant, durationMs, kbps, sizeBytes int) (dataplane.ThroughputResult, error)
|
||||||
// EgressMTU reports the server's own measured egress path MTU (0 = unknown). With DF set
|
// EgressMTU reports the server's own measured egress path MTU (0 = unknown). With DF set
|
||||||
// we cannot emit a datagram larger than this, so requested sizes above it are refused up
|
// we cannot emit a datagram larger than this, so requested sizes above it are refused up
|
||||||
// front and reported as such — the client must not read that as a downstream path limit.
|
// front and reported as such — the client must not read that as a downstream path limit.
|
||||||
@@ -70,22 +77,91 @@ type Server struct {
|
|||||||
// server's own egress isn't full-MTU, client MTU results measure the
|
// server's own egress isn't full-MTU, client MTU results measure the
|
||||||
// server, not the client.
|
// server, not the client.
|
||||||
ProvenGood func() (mtuOK, sysctlOK bool)
|
ProvenGood func() (mtuOK, sysctlOK bool)
|
||||||
|
|
||||||
|
// PublicControlURL is where clients reach this server, for the enrollment link (§2.1).
|
||||||
|
// Empty means "derive from the address we are listening on", which is right for a plain
|
||||||
|
// deployment and wrong behind a proxy or a name — hence the override.
|
||||||
|
PublicControlURL string
|
||||||
|
// AppRange is the app-version window this server will serve. Zero value means the built-in
|
||||||
|
// default (see DefaultAppRange).
|
||||||
|
AppRange compat.Range
|
||||||
|
}
|
||||||
|
|
||||||
|
// AppVersionHeader is how a client states its version. A client too old to send it is treated as
|
||||||
|
// unknown rather than refused: the check exists to turn confusing failures into clear ones, and
|
||||||
|
// refusing something we cannot identify achieves the opposite.
|
||||||
|
const AppVersionHeader = "X-Echolot-App-Version"
|
||||||
|
|
||||||
|
// ProtocolVersion is the wire contract (probe-protocol.md) this build implements. It is what the
|
||||||
|
// version window is really about; the release version is only a proxy for it.
|
||||||
|
const ProtocolVersion = "1.0.0"
|
||||||
|
|
||||||
|
// SchemaVersion is the measurement-document format this server can store.
|
||||||
|
const SchemaVersion = "1.0.0"
|
||||||
|
|
||||||
|
// DefaultAppRange: everything from the first app that speaks this protocol up to — but not
|
||||||
|
// including — the next breaking series. Bounds sit at breaking boundaries so shipping a patch
|
||||||
|
// never requires touching this.
|
||||||
|
func DefaultAppRange() compat.Range {
|
||||||
|
r, err := compat.ParseRange("0.2.0", "1.0.0")
|
||||||
|
if err != nil {
|
||||||
|
panic("built-in app range is malformed: " + err.Error())
|
||||||
|
}
|
||||||
|
return r
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s *Server) appRange() compat.Range {
|
||||||
|
if s.AppRange.Min == (compat.Version{}) && !s.AppRange.HasMax {
|
||||||
|
return DefaultAppRange()
|
||||||
|
}
|
||||||
|
return s.AppRange
|
||||||
|
}
|
||||||
|
|
||||||
|
// requireCompatibleApp wraps a handler with the version window.
|
||||||
|
//
|
||||||
|
// Deliberately NOT applied to GET /v1/profile: that is where a client learns which version it
|
||||||
|
// should be. Gating it would leave a refused client with nothing to show its user but a timeout,
|
||||||
|
// which is precisely the confusion this check exists to remove.
|
||||||
|
func (s *Server) requireCompatibleApp(next http.HandlerFunc) http.HandlerFunc {
|
||||||
|
return func(w http.ResponseWriter, r *http.Request) {
|
||||||
|
verdict, msg := compat.Check(r.Header.Get(AppVersionHeader), s.appRange(), "app")
|
||||||
|
switch verdict {
|
||||||
|
case compat.TooOld, compat.TooNew:
|
||||||
|
slog.Info("refused incompatible app", "app_version", r.Header.Get(AppVersionHeader),
|
||||||
|
"accepts", s.appRange().String(), "path", r.URL.Path)
|
||||||
|
// 426 says exactly this and nothing else; the body carries the window so the app can
|
||||||
|
// show the user the number to reach, not just that it failed.
|
||||||
|
writeJSON(w, http.StatusUpgradeRequired, map[string]any{
|
||||||
|
"error": msg,
|
||||||
|
"app_version": r.Header.Get(AppVersionHeader),
|
||||||
|
"accepts_app": s.appRange().String(),
|
||||||
|
"server_version": Version,
|
||||||
|
"protocol_version": ProtocolVersion,
|
||||||
|
})
|
||||||
|
return
|
||||||
|
}
|
||||||
|
next(w, r)
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
func (s *Server) Handler() http.Handler {
|
func (s *Server) Handler() http.Handler {
|
||||||
mux := http.NewServeMux()
|
mux := http.NewServeMux()
|
||||||
mux.HandleFunc("POST /v1/enroll", s.enroll)
|
// Always reachable, whatever the version window says: this is how a client discovers the
|
||||||
|
// window it has to satisfy.
|
||||||
mux.HandleFunc("GET /v1/profile", s.profile)
|
mux.HandleFunc("GET /v1/profile", s.profile)
|
||||||
mux.HandleFunc("POST /v1/sessions", s.newSession)
|
|
||||||
mux.HandleFunc("DELETE /v1/sessions/{id}", s.deleteSession)
|
gate := s.requireCompatibleApp
|
||||||
mux.HandleFunc("GET /v1/sessions/{id}/observations", s.observations)
|
mux.HandleFunc("POST /v1/enroll", gate(s.enroll))
|
||||||
mux.HandleFunc("POST /v1/sessions/{id}/actions", s.actions)
|
mux.HandleFunc("POST /v1/sessions", gate(s.newSession))
|
||||||
mux.HandleFunc("POST /v1/echo", s.httpEcho)
|
mux.HandleFunc("DELETE /v1/sessions/{id}", gate(s.deleteSession))
|
||||||
mux.HandleFunc("GET /v1/tls-reference", s.tlsReference)
|
mux.HandleFunc("GET /v1/sessions/{id}/observations", gate(s.observations))
|
||||||
mux.HandleFunc("POST /v1/runs", s.uploadRun)
|
mux.HandleFunc("POST /v1/sessions/{id}/actions", gate(s.actions))
|
||||||
mux.HandleFunc("GET /v1/runs", s.listRuns)
|
mux.HandleFunc("POST /v1/echo", gate(s.httpEcho))
|
||||||
mux.HandleFunc("GET /v1/runs/{id}", s.getRun)
|
mux.HandleFunc("GET /v1/tls-reference", gate(s.tlsReference))
|
||||||
mux.HandleFunc("DELETE /v1/runs/{id}", s.deleteRun)
|
mux.HandleFunc("POST /v1/runs", gate(s.uploadRun))
|
||||||
|
mux.HandleFunc("GET /v1/runs", gate(s.listRuns))
|
||||||
|
mux.HandleFunc("GET /v1/runs/{id}", gate(s.getRun))
|
||||||
|
mux.HandleFunc("DELETE /v1/runs/{id}", gate(s.deleteRun))
|
||||||
// TODO(spec §5): frag_send, throughput (both build on the same grant machinery)
|
// TODO(spec §5): frag_send, throughput (both build on the same grant machinery)
|
||||||
return mux
|
return mux
|
||||||
}
|
}
|
||||||
@@ -147,6 +223,10 @@ func (s *Server) observations(w http.ResponseWriter, r *http.Request) {
|
|||||||
"udp": map[string]any{"packets_seen": packetsSeen, "packets": udp},
|
"udp": map[string]any{"packets_seen": packetsSeen, "packets": udp},
|
||||||
"tcp": tcp,
|
"tcp": tcp,
|
||||||
"connect_back": cb,
|
"connect_back": cb,
|
||||||
|
// The sender's own count, which is what makes the receiver's count mean something.
|
||||||
|
"throughput": sess.ThroughputReports(),
|
||||||
|
// The receiver's count for upstream runs — same idea, other direction.
|
||||||
|
"throughput_up": upstreamJSON(sess),
|
||||||
"dns_canary": dnsCanary,
|
"dns_canary": dnsCanary,
|
||||||
// TODO(spec §6): http echo records
|
// TODO(spec §6): http echo records
|
||||||
})
|
})
|
||||||
@@ -170,6 +250,12 @@ func (s *Server) actions(w http.ResponseWriter, r *http.Request) {
|
|||||||
IntervalUs int `json:"interval_us"`
|
IntervalUs int `json:"interval_us"`
|
||||||
SizesBytes []int `json:"sizes_bytes"`
|
SizesBytes []int `json:"sizes_bytes"`
|
||||||
DF *bool `json:"df"`
|
DF *bool `json:"df"`
|
||||||
|
Mode string `json:"mode"`
|
||||||
|
FragBytes int `json:"frag_bytes"`
|
||||||
|
Direction string `json:"direction"`
|
||||||
|
DurationS int `json:"duration_s"`
|
||||||
|
Kbps int `json:"kbps"`
|
||||||
|
Streams int `json:"streams"`
|
||||||
}
|
}
|
||||||
if err := json.NewDecoder(r.Body).Decode(&req); err != nil {
|
if err := json.NewDecoder(r.Body).Decode(&req); err != nil {
|
||||||
writeJSON(w, http.StatusBadRequest, map[string]string{"error": "bad body"})
|
writeJSON(w, http.StatusBadRequest, map[string]string{"error": "bad body"})
|
||||||
@@ -302,6 +388,105 @@ func (s *Server) actions(w http.ResponseWriter, r *http.Request) {
|
|||||||
"grant": map[string]any{"max_bytes": g.MaxBytes, "max_kbps": g.MaxKbps},
|
"grant": map[string]any{"max_bytes": g.MaxBytes, "max_kbps": g.MaxKbps},
|
||||||
})
|
})
|
||||||
|
|
||||||
|
case "frag_send":
|
||||||
|
if s.FragSend == nil {
|
||||||
|
writeJSON(w, http.StatusNotImplemented, map[string]string{
|
||||||
|
"error": "frag_send needs a raw socket, which this server does not have",
|
||||||
|
})
|
||||||
|
return
|
||||||
|
}
|
||||||
|
size := clamp(req.SizeBytes, 1600, 8000) // must exceed the path MTU or nothing fragments
|
||||||
|
mode := dataplane.FragMode(req.Mode)
|
||||||
|
switch mode {
|
||||||
|
case dataplane.FragInOrder, dataplane.FragReversed, dataplane.FragFirstLast:
|
||||||
|
default:
|
||||||
|
mode = dataplane.FragInOrder
|
||||||
|
}
|
||||||
|
fragBytes := clamp(req.FragBytes, 8, 1400)
|
||||||
|
g := sess.NewGrant(actionID, int64(size), 0, session.DefaultGrantLimits)
|
||||||
|
if g == nil {
|
||||||
|
writeJSON(w, http.StatusConflict, noDataPlaneYet)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
// Synchronous: the whole burst is a few kB and at most a few hundred milliseconds, and
|
||||||
|
// the caller wants to know it was actually emitted before it starts listening. An
|
||||||
|
// asynchronous send would make "nothing arrived" ambiguous between a path drop and a
|
||||||
|
// send that never happened — the one distinction this test exists to make.
|
||||||
|
result, err := s.FragSend(sess, g, size, mode, fragBytes)
|
||||||
|
slog.Info("frag_send finished", "action", actionID, "mode", mode,
|
||||||
|
"size", size, "fragments", result.Fragments, "err", err)
|
||||||
|
if err != nil {
|
||||||
|
writeJSON(w, http.StatusConflict, map[string]any{
|
||||||
|
"error": err.Error(), "action_id": actionID, "result": result,
|
||||||
|
})
|
||||||
|
return
|
||||||
|
}
|
||||||
|
writeJSON(w, http.StatusAccepted, map[string]any{
|
||||||
|
"action_id": actionID, "mode": string(mode), "size_bytes": size,
|
||||||
|
"frag_bytes": fragBytes, "fragments": result.Fragments,
|
||||||
|
"grant": map[string]any{"max_bytes": g.MaxBytes, "max_kbps": g.MaxKbps},
|
||||||
|
})
|
||||||
|
|
||||||
|
case "throughput":
|
||||||
|
if s.DownThroughput == nil {
|
||||||
|
writeJSON(w, http.StatusNotImplemented, map[string]string{"error": "throughput not wired"})
|
||||||
|
return
|
||||||
|
}
|
||||||
|
// Only the downstream direction needs the server to send. Upstream is the client
|
||||||
|
// sending and the server counting, which needs no action at all — so asking for it here
|
||||||
|
// is a client bug worth naming rather than silently doing the other thing.
|
||||||
|
if req.Direction == "up" {
|
||||||
|
// Upstream needs nothing sent from here — the client generates the traffic and the
|
||||||
|
// server counts it. The only thing an action can usefully do is zero the counter so
|
||||||
|
// the run measures itself rather than inheriting an earlier one.
|
||||||
|
sess.ResetUpstream()
|
||||||
|
writeJSON(w, http.StatusAccepted, map[string]any{
|
||||||
|
"action_id": actionID, "direction": "up", "reset": true,
|
||||||
|
"note": "send TYPE_THROUGHPUT_UP packets, then read observations.throughput_up",
|
||||||
|
})
|
||||||
|
return
|
||||||
|
}
|
||||||
|
if req.Direction != "" && req.Direction != "down" {
|
||||||
|
writeJSON(w, http.StatusBadRequest, map[string]string{
|
||||||
|
"error": "direction must be up or down",
|
||||||
|
})
|
||||||
|
return
|
||||||
|
}
|
||||||
|
// Planned once, here, so the response promises exactly what the run will do. A request
|
||||||
|
// that would outlast the server's byte cap comes back with a shorter duration rather
|
||||||
|
// than being truncated halfway.
|
||||||
|
durationMs, kbps := dataplane.ThroughputPlan(
|
||||||
|
clamp(req.DurationS, 1, 30)*1000, clamp(req.Kbps, 100, 200_000))
|
||||||
|
size := clamp(req.SizeBytes, dataMinPacket, 1472)
|
||||||
|
if req.SizeBytes == 0 {
|
||||||
|
size = 1200
|
||||||
|
}
|
||||||
|
g := sess.NewGrant(actionID, 0, kbps, dataplane.ThroughputLimits(durationMs, kbps))
|
||||||
|
if g == nil {
|
||||||
|
writeJSON(w, http.StatusConflict, noDataPlaneYet)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
// Answered before the run so the client can start listening, then reported through the
|
||||||
|
// observations API. Doing it the other way round would have the client miss the first
|
||||||
|
// second of a ten-second test.
|
||||||
|
writeJSON(w, http.StatusAccepted, map[string]any{
|
||||||
|
"action_id": actionID, "direction": "down",
|
||||||
|
"duration_s": durationMs / 1000, "duration_ms": durationMs,
|
||||||
|
"requested_duration_s": clamp(req.DurationS, 1, 30),
|
||||||
|
"kbps": kbps, "size_bytes": size,
|
||||||
|
"grant": map[string]any{"max_bytes": g.MaxBytes, "max_kbps": g.MaxKbps},
|
||||||
|
})
|
||||||
|
if f, ok := w.(http.Flusher); ok {
|
||||||
|
f.Flush()
|
||||||
|
}
|
||||||
|
go func() {
|
||||||
|
result, err := s.DownThroughput(sess, g, durationMs, kbps, size)
|
||||||
|
slog.Info("throughput finished", "action", actionID, "packets", result.Packets,
|
||||||
|
"bytes", result.Bytes, "kbps", result.Kbps, "limited_by", result.LimitedBy, "err", err)
|
||||||
|
sess.RecordThroughput(actionID, result.Packets, result.Bytes, result.DurationMs,
|
||||||
|
result.Kbps, result.LimitedBy)
|
||||||
|
}()
|
||||||
|
|
||||||
default:
|
default:
|
||||||
writeJSON(w, http.StatusBadRequest, map[string]string{"error": "unknown or unimplemented action"})
|
writeJSON(w, http.StatusBadRequest, map[string]string{"error": "unknown or unimplemented action"})
|
||||||
}
|
}
|
||||||
@@ -364,7 +549,12 @@ func bearer(r *http.Request) string {
|
|||||||
func writeJSON(w http.ResponseWriter, code int, v any) {
|
func writeJSON(w http.ResponseWriter, code int, v any) {
|
||||||
w.Header().Set("Content-Type", "application/json")
|
w.Header().Set("Content-Type", "application/json")
|
||||||
w.WriteHeader(code)
|
w.WriteHeader(code)
|
||||||
_ = json.NewEncoder(w).Encode(v)
|
enc := json.NewEncoder(w)
|
||||||
|
// Go escapes <, > and & by default, for JSON embedded in HTML. This is an API, and the
|
||||||
|
// escaping is actively harmful here: a refusal message reading "needs >= 0.2.0" is what
|
||||||
|
// the user ends up seeing. Nothing we emit is ever interpolated into a page.
|
||||||
|
enc.SetEscapeHTML(false)
|
||||||
|
_ = enc.Encode(v)
|
||||||
}
|
}
|
||||||
|
|
||||||
// enroll redeems a single-use enrollment token for a device credential (§2.1).
|
// enroll redeems a single-use enrollment token for a device credential (§2.1).
|
||||||
@@ -386,7 +576,11 @@ func (s *Server) enroll(w http.ResponseWriter, r *http.Request) {
|
|||||||
slog.Info("device enrolled", "device", dev.ID, "name", dev.Name)
|
slog.Info("device enrolled", "device", dev.ID, "name", dev.Name)
|
||||||
writeJSON(w, http.StatusCreated, map[string]string{
|
writeJSON(w, http.StatusCreated, map[string]string{
|
||||||
"device_id": dev.ID,
|
"device_id": dev.ID,
|
||||||
"credential": dev.Credential, // returned exactly once
|
// The spec (§2.1) names this device_credential; the first implementation shipped
|
||||||
|
// "credential". Both are sent while deployed 0.5.x clients still read the old name;
|
||||||
|
// the client prefers the spec's. Drop "credential" once nothing reads it.
|
||||||
|
"device_credential": dev.Credential, // returned exactly once
|
||||||
|
"credential": dev.Credential, // deprecated alias, see above
|
||||||
})
|
})
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -424,6 +618,14 @@ func (s *Server) profile(w http.ResponseWriter, r *http.Request) {
|
|||||||
// The app needs the upload rules before it offers the switch: whether uploads are
|
// The app needs the upload rules before it offers the switch: whether uploads are
|
||||||
// accepted at all, and how much identifying detail it must strip first.
|
// accepted at all, and how much identifying detail it must strip first.
|
||||||
"uploads": s.uploadPolicy(),
|
"uploads": s.uploadPolicy(),
|
||||||
|
// What this build speaks, and which app versions it will serve. A client checks the
|
||||||
|
// server side of the same question against its own bounds.
|
||||||
|
"compat": map[string]any{
|
||||||
|
"protocol_version": ProtocolVersion,
|
||||||
|
"schema_version": SchemaVersion,
|
||||||
|
"app_min": s.appRange().Min.String(),
|
||||||
|
"app_max": maxOrEmpty(s.appRange()),
|
||||||
|
},
|
||||||
})
|
})
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -574,3 +776,35 @@ func (s *Server) deleteRun(w http.ResponseWriter, r *http.Request) {
|
|||||||
}
|
}
|
||||||
w.WriteHeader(http.StatusNoContent)
|
w.WriteHeader(http.StatusNoContent)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// maxOrEmpty renders an unbounded ceiling as "" rather than as a sentinel version, so a client
|
||||||
|
// reading the profile cannot mistake a placeholder for a real bound.
|
||||||
|
func maxOrEmpty(r compat.Range) string {
|
||||||
|
if !r.HasMax {
|
||||||
|
return ""
|
||||||
|
}
|
||||||
|
return r.Max.String()
|
||||||
|
}
|
||||||
|
|
||||||
|
// EnrollmentLink builds the §2.1 bootstrap string for a freshly minted token.
|
||||||
|
//
|
||||||
|
// The server assembles it rather than the operator, because it is the only party that knows all
|
||||||
|
// three parts at once — its own URL, its own SPKI pin, and the token. An operator copying a pin
|
||||||
|
// by hand is the step that goes wrong, and a pin wrong by one character does not fail loudly.
|
||||||
|
func (s *Server) EnrollmentLink(token string) string {
|
||||||
|
u := s.PublicControlURL
|
||||||
|
return "echolot://enroll?v=1" +
|
||||||
|
"&u=" + url.QueryEscape(strings.TrimRight(u, "/")) +
|
||||||
|
"&p=" + url.QueryEscape("pin-sha256:"+s.PinB64) +
|
||||||
|
"&t=" + url.QueryEscape(token)
|
||||||
|
}
|
||||||
|
|
||||||
|
// upstreamJSON renders the upstream tally with the derived figures already computed, so every
|
||||||
|
// consumer does not have to repeat (and risk fumbling) the same arithmetic.
|
||||||
|
func upstreamJSON(sess *session.Session) map[string]any {
|
||||||
|
u := sess.Upstream()
|
||||||
|
return map[string]any{
|
||||||
|
"packets": u.Packets, "bytes": u.Bytes,
|
||||||
|
"span_ms": u.SpanMs(), "kbps": u.Kbps(),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|||||||
@@ -0,0 +1,263 @@
|
|||||||
|
// SPDX-FileCopyrightText: 2026 Echolot contributors
|
||||||
|
// SPDX-License-Identifier: GPL-3.0-or-later
|
||||||
|
|
||||||
|
//go:build linux
|
||||||
|
|
||||||
|
package dataplane
|
||||||
|
|
||||||
|
import (
|
||||||
|
"encoding/binary"
|
||||||
|
"fmt"
|
||||||
|
"net/netip"
|
||||||
|
"sync/atomic"
|
||||||
|
"syscall"
|
||||||
|
"time"
|
||||||
|
|
||||||
|
"echo-lot.app/server/internal/session"
|
||||||
|
)
|
||||||
|
|
||||||
|
// Crafted IPv4 fragmentation (spec §5 frag_send).
|
||||||
|
//
|
||||||
|
// Letting the kernel fragment an oversized datagram — which is what big_send with df=false does —
|
||||||
|
// answers one question: do fragments get through at all. It cannot answer the more interesting
|
||||||
|
// one, because the kernel always emits fragments in order, first one first.
|
||||||
|
//
|
||||||
|
// The classic middlebox fault is precisely about that ordering. Only the *first* fragment carries
|
||||||
|
// the UDP header, and therefore the ports; a stateful firewall or NAT that has not seen it has no
|
||||||
|
// flow to match later fragments against. Plenty of implementations drop them. Others hold them
|
||||||
|
// briefly and reassemble; others leak. The difference is invisible to any test that sends
|
||||||
|
// fragments in order, and it shows up in the real world as "large DNS answers fail on this
|
||||||
|
// network" or "the VPN works until the MTU drops".
|
||||||
|
//
|
||||||
|
// So this builds the fragments by hand and controls their order and timing. That needs a raw
|
||||||
|
// socket (CAP_NET_RAW); when we do not have one the capability is not advertised, rather than
|
||||||
|
// advertised and failing later.
|
||||||
|
|
||||||
|
// FragMode is how a fragmented datagram is put on the wire.
|
||||||
|
type FragMode string
|
||||||
|
|
||||||
|
const (
|
||||||
|
// FragInOrder is the baseline: first fragment first, as the kernel would. A path that fails
|
||||||
|
// this fails everything, and it tells the others apart from a path that drops all fragments.
|
||||||
|
FragInOrder FragMode = "in_order"
|
||||||
|
// FragReversed sends the last fragment first. This is the one that finds stateful devices
|
||||||
|
// which need the first fragment to build state.
|
||||||
|
FragReversed FragMode = "reversed"
|
||||||
|
// FragFirstLast holds the first fragment back until the others have arrived, which tests
|
||||||
|
// whether the path buffers non-first fragments at all and for how long.
|
||||||
|
FragFirstLast FragMode = "first_last"
|
||||||
|
)
|
||||||
|
|
||||||
|
var fragIPID atomic.Uint32
|
||||||
|
|
||||||
|
// RawFragSupported reports whether crafted fragments can actually be sent here.
|
||||||
|
//
|
||||||
|
// Checked by opening the socket rather than by inspecting capabilities: the question is "will
|
||||||
|
// this work", and a permission model has more ways to say no than a capability bit has to say yes
|
||||||
|
// (user namespaces, seccomp, LSM). Advertising a capability we cannot deliver would turn a
|
||||||
|
// missing feature into a failed measurement.
|
||||||
|
func RawFragSupported() bool {
|
||||||
|
fd, err := syscall.Socket(syscall.AF_INET, syscall.SOCK_RAW, syscall.IPPROTO_RAW)
|
||||||
|
if err != nil {
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
_ = syscall.Close(fd)
|
||||||
|
return true
|
||||||
|
}
|
||||||
|
|
||||||
|
// FragResult is what happened to one crafted fragment burst.
|
||||||
|
type FragResult struct {
|
||||||
|
Mode FragMode `json:"mode"`
|
||||||
|
SizeBytes int `json:"size_bytes"`
|
||||||
|
Fragments int `json:"fragments"`
|
||||||
|
Sent bool `json:"sent"`
|
||||||
|
Err string `json:"err,omitempty"`
|
||||||
|
}
|
||||||
|
|
||||||
|
// FragSend emits one ELT1 packet of sizeBytes as hand-built IPv4 fragments, in the given order.
|
||||||
|
//
|
||||||
|
// The datagram is assembled whole and then cut up, so what the client reassembles — if it
|
||||||
|
// reassembles — is a normal, HMAC-valid packet indistinguishable from any other. That matters:
|
||||||
|
// the client must not be able to tell a crafted fragment burst from a kernel one, or it would be
|
||||||
|
// measuring our sender rather than the path.
|
||||||
|
func (s *Server) FragSend(
|
||||||
|
sess *session.Session, g *session.Grant, sizeBytes int, mode FragMode, fragSize int,
|
||||||
|
) (FragResult, error) {
|
||||||
|
res := FragResult{Mode: mode, SizeBytes: sizeBytes}
|
||||||
|
|
||||||
|
target := sess.DataSource()
|
||||||
|
if !target.IsValid() {
|
||||||
|
return res, fmt.Errorf("no observed data-plane source")
|
||||||
|
}
|
||||||
|
if !target.Addr().Unmap().Is4() {
|
||||||
|
// IPv6 has no in-network fragmentation: only the source may fragment, via an extension
|
||||||
|
// header. Worth building, but it is a different mechanism and belongs in its own code
|
||||||
|
// path rather than pretending this one covers it.
|
||||||
|
return res, fmt.Errorf("crafted fragmentation is IPv4-only for now")
|
||||||
|
}
|
||||||
|
conn := s.connFor(target, sess.DataLocal())
|
||||||
|
if conn == nil {
|
||||||
|
return res, fmt.Errorf("no data-plane socket matches target family")
|
||||||
|
}
|
||||||
|
local := sess.DataLocal()
|
||||||
|
if !local.IsValid() {
|
||||||
|
return res, fmt.Errorf("session has no recorded local address")
|
||||||
|
}
|
||||||
|
|
||||||
|
if sizeBytes < HeaderSize+8 {
|
||||||
|
sizeBytes = HeaderSize + 8
|
||||||
|
}
|
||||||
|
if sizeBytes > 8000 {
|
||||||
|
sizeBytes = 8000
|
||||||
|
}
|
||||||
|
if !g.Allow(sizeBytes) {
|
||||||
|
return res, fmt.Errorf("grant exhausted")
|
||||||
|
}
|
||||||
|
|
||||||
|
// The ELT1 packet, signed exactly as any other, then wrapped in UDP.
|
||||||
|
payload := make([]byte, sizeBytes-HeaderSize)
|
||||||
|
binary.BigEndian.PutUint32(payload[0:4], uint32(sizeBytes))
|
||||||
|
copy(payload[4:], mode)
|
||||||
|
elt := s.buildPacket(sess, TypeFragData, 0, payload)
|
||||||
|
|
||||||
|
udp := buildUDP(local, target, elt)
|
||||||
|
|
||||||
|
// Fragment offsets are in 8-byte units, so every fragment except the last must be a multiple
|
||||||
|
// of 8. A payload that is not is not an error — it is a fragment that no host will reassemble.
|
||||||
|
if fragSize <= 0 {
|
||||||
|
fragSize = 576
|
||||||
|
}
|
||||||
|
fragSize = (fragSize / 8) * 8
|
||||||
|
if fragSize < 8 {
|
||||||
|
fragSize = 8
|
||||||
|
}
|
||||||
|
|
||||||
|
fragments := splitIPv4(local.Addr(), target.Addr(), udp, fragSize, uint16(fragIPID.Add(1)))
|
||||||
|
res.Fragments = len(fragments)
|
||||||
|
|
||||||
|
fd, err := syscall.Socket(syscall.AF_INET, syscall.SOCK_RAW, syscall.IPPROTO_RAW)
|
||||||
|
if err != nil {
|
||||||
|
res.Err = err.Error()
|
||||||
|
return res, err
|
||||||
|
}
|
||||||
|
defer syscall.Close(fd)
|
||||||
|
if err := syscall.SetsockoptInt(fd, syscall.IPPROTO_IP, syscall.IP_HDRINCL, 1); err != nil {
|
||||||
|
res.Err = err.Error()
|
||||||
|
return res, err
|
||||||
|
}
|
||||||
|
|
||||||
|
dst := syscall.SockaddrInet4{}
|
||||||
|
copy(dst.Addr[:], target.Addr().Unmap().AsSlice())
|
||||||
|
|
||||||
|
send := func(pkt []byte) error { return syscall.Sendto(fd, pkt, 0, &dst) }
|
||||||
|
|
||||||
|
switch mode {
|
||||||
|
case FragReversed:
|
||||||
|
for i := len(fragments) - 1; i >= 0; i-- {
|
||||||
|
if err := send(fragments[i]); err != nil {
|
||||||
|
res.Err = err.Error()
|
||||||
|
return res, err
|
||||||
|
}
|
||||||
|
time.Sleep(time.Millisecond)
|
||||||
|
}
|
||||||
|
case FragFirstLast:
|
||||||
|
for i := 1; i < len(fragments); i++ {
|
||||||
|
if err := send(fragments[i]); err != nil {
|
||||||
|
res.Err = err.Error()
|
||||||
|
return res, err
|
||||||
|
}
|
||||||
|
time.Sleep(time.Millisecond)
|
||||||
|
}
|
||||||
|
// Long enough to be a real test of whether anything holds fragments, short enough to stay
|
||||||
|
// inside the usual 30-second reassembly timeout by a wide margin.
|
||||||
|
time.Sleep(250 * time.Millisecond)
|
||||||
|
if err := send(fragments[0]); err != nil {
|
||||||
|
res.Err = err.Error()
|
||||||
|
return res, err
|
||||||
|
}
|
||||||
|
default:
|
||||||
|
for _, f := range fragments {
|
||||||
|
if err := send(f); err != nil {
|
||||||
|
res.Err = err.Error()
|
||||||
|
return res, err
|
||||||
|
}
|
||||||
|
time.Sleep(time.Millisecond)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
res.Sent = true
|
||||||
|
return res, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// buildUDP wraps a payload in a UDP header with a computed checksum.
|
||||||
|
//
|
||||||
|
// The checksum is optional in IPv4 and it would be less code to send zero, but a zero-checksum
|
||||||
|
// datagram is dropped by some middleboxes — and that drop would be recorded as a fragmentation
|
||||||
|
// failure, which is exactly the wrong conclusion.
|
||||||
|
func buildUDP(src, dst netip.AddrPort, payload []byte) []byte {
|
||||||
|
out := make([]byte, 8+len(payload))
|
||||||
|
binary.BigEndian.PutUint16(out[0:2], src.Port())
|
||||||
|
binary.BigEndian.PutUint16(out[2:4], dst.Port())
|
||||||
|
binary.BigEndian.PutUint16(out[4:6], uint16(8+len(payload)))
|
||||||
|
copy(out[8:], payload)
|
||||||
|
|
||||||
|
// Pseudo-header + UDP header + data, per RFC 768.
|
||||||
|
var sum uint32
|
||||||
|
s4, d4 := src.Addr().Unmap().As4(), dst.Addr().Unmap().As4()
|
||||||
|
for _, b := range [][]byte{s4[:], d4[:]} {
|
||||||
|
sum += uint32(binary.BigEndian.Uint16(b[0:2]))
|
||||||
|
sum += uint32(binary.BigEndian.Uint16(b[2:4]))
|
||||||
|
}
|
||||||
|
sum += uint32(syscall.IPPROTO_UDP)
|
||||||
|
sum += uint32(len(out))
|
||||||
|
for i := 0; i+1 < len(out); i += 2 {
|
||||||
|
sum += uint32(binary.BigEndian.Uint16(out[i : i+2]))
|
||||||
|
}
|
||||||
|
if len(out)%2 == 1 {
|
||||||
|
sum += uint32(out[len(out)-1]) << 8
|
||||||
|
}
|
||||||
|
for sum>>16 != 0 {
|
||||||
|
sum = (sum & 0xFFFF) + (sum >> 16)
|
||||||
|
}
|
||||||
|
ck := ^uint16(sum)
|
||||||
|
if ck == 0 {
|
||||||
|
ck = 0xFFFF // 0 means "no checksum" in IPv4; the all-ones form is the same value
|
||||||
|
}
|
||||||
|
binary.BigEndian.PutUint16(out[6:8], ck)
|
||||||
|
return out
|
||||||
|
}
|
||||||
|
|
||||||
|
// splitIPv4 cuts a UDP datagram into IPv4 fragments of at most fragSize payload bytes each.
|
||||||
|
//
|
||||||
|
// Every fragment carries the same IP ID — that is what marks them as one datagram — and every one
|
||||||
|
// but the last sets MF. The kernel fills in the header checksum and total length for us under
|
||||||
|
// IP_HDRINCL (raw(7)); the ID it only fills when zero, which is why it is set explicitly here.
|
||||||
|
func splitIPv4(src, dst netip.Addr, udp []byte, fragSize int, id uint16) [][]byte {
|
||||||
|
s4, d4 := src.Unmap().As4(), dst.Unmap().As4()
|
||||||
|
var out [][]byte
|
||||||
|
for off := 0; off < len(udp); off += fragSize {
|
||||||
|
end := off + fragSize
|
||||||
|
if end > len(udp) {
|
||||||
|
end = len(udp)
|
||||||
|
}
|
||||||
|
chunk := udp[off:end]
|
||||||
|
more := end < len(udp)
|
||||||
|
|
||||||
|
hdr := make([]byte, 20, 20+len(chunk))
|
||||||
|
hdr[0] = 0x45 // IPv4, 5 words of header
|
||||||
|
hdr[1] = 0 // DSCP/ECN
|
||||||
|
binary.BigEndian.PutUint16(hdr[2:4], uint16(20+len(chunk)))
|
||||||
|
binary.BigEndian.PutUint16(hdr[4:6], id)
|
||||||
|
flagsOff := uint16(off / 8)
|
||||||
|
if more {
|
||||||
|
flagsOff |= 0x2000 // MF
|
||||||
|
}
|
||||||
|
binary.BigEndian.PutUint16(hdr[6:8], flagsOff)
|
||||||
|
hdr[8] = 64 // TTL
|
||||||
|
hdr[9] = syscall.IPPROTO_UDP
|
||||||
|
// hdr[10:12] checksum left zero: the kernel computes it under IP_HDRINCL.
|
||||||
|
copy(hdr[12:16], s4[:])
|
||||||
|
copy(hdr[16:20], d4[:])
|
||||||
|
out = append(out, append(hdr, chunk...))
|
||||||
|
}
|
||||||
|
return out
|
||||||
|
}
|
||||||
@@ -0,0 +1,160 @@
|
|||||||
|
// SPDX-FileCopyrightText: 2026 Echolot contributors
|
||||||
|
// SPDX-License-Identifier: GPL-3.0-or-later
|
||||||
|
|
||||||
|
//go:build linux
|
||||||
|
|
||||||
|
package dataplane
|
||||||
|
|
||||||
|
import (
|
||||||
|
"encoding/binary"
|
||||||
|
"net/netip"
|
||||||
|
"testing"
|
||||||
|
)
|
||||||
|
|
||||||
|
// Fragment headers are the kind of thing that is either exactly right or silently useless: a
|
||||||
|
// wrong offset unit, a missing MF bit or a bad checksum produces packets that leave the machine
|
||||||
|
// and are dropped by the receiver's IP stack without a word. Nothing downstream would notice —
|
||||||
|
// the client would simply record "fragments do not get through", which is a wrong answer rather
|
||||||
|
// than a missing one. Hence these check the bytes.
|
||||||
|
|
||||||
|
func testAddrs() (netip.AddrPort, netip.AddrPort) {
|
||||||
|
return netip.MustParseAddrPort("192.0.2.1:8442"), netip.MustParseAddrPort("198.51.100.9:41000")
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestSplitCoversThePayloadExactlyOnce(t *testing.T) {
|
||||||
|
src, dst := testAddrs()
|
||||||
|
udp := buildUDP(src, dst, make([]byte, 2000))
|
||||||
|
|
||||||
|
frags := splitIPv4(src.Addr(), dst.Addr(), udp, 576, 0x1234)
|
||||||
|
if len(frags) < 3 {
|
||||||
|
t.Fatalf("expected several fragments for %d bytes, got %d", len(udp), len(frags))
|
||||||
|
}
|
||||||
|
|
||||||
|
// Reassemble the way a receiver would: place each fragment's payload at its offset.
|
||||||
|
rebuilt := make([]byte, len(udp))
|
||||||
|
covered := make([]bool, len(udp))
|
||||||
|
for _, f := range frags {
|
||||||
|
flagsOff := binary.BigEndian.Uint16(f[6:8])
|
||||||
|
off := int(flagsOff&0x1FFF) * 8
|
||||||
|
body := f[20:]
|
||||||
|
if off+len(body) > len(udp) {
|
||||||
|
t.Fatalf("fragment at offset %d overruns the datagram", off)
|
||||||
|
}
|
||||||
|
for i, b := range body {
|
||||||
|
if covered[off+i] {
|
||||||
|
t.Fatalf("byte %d delivered twice", off+i)
|
||||||
|
}
|
||||||
|
covered[off+i] = true
|
||||||
|
rebuilt[off+i] = b
|
||||||
|
}
|
||||||
|
}
|
||||||
|
for i, c := range covered {
|
||||||
|
if !c {
|
||||||
|
t.Fatalf("byte %d was never sent", i)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
for i := range udp {
|
||||||
|
if rebuilt[i] != udp[i] {
|
||||||
|
t.Fatalf("reassembled byte %d differs", i)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestFragmentHeadersAreWellFormed(t *testing.T) {
|
||||||
|
src, dst := testAddrs()
|
||||||
|
udp := buildUDP(src, dst, make([]byte, 3000))
|
||||||
|
frags := splitIPv4(src.Addr(), dst.Addr(), udp, 800, 0xBEEF)
|
||||||
|
|
||||||
|
for i, f := range frags {
|
||||||
|
if got := f[0]; got != 0x45 {
|
||||||
|
t.Errorf("fragment %d: version/IHL = %#x, want 0x45", i, got)
|
||||||
|
}
|
||||||
|
if got := f[9]; got != 17 {
|
||||||
|
t.Errorf("fragment %d: protocol = %d, want 17 (UDP)", i, got)
|
||||||
|
}
|
||||||
|
if got := binary.BigEndian.Uint16(f[4:6]); got != 0xBEEF {
|
||||||
|
t.Errorf("fragment %d: IP ID = %#x — all fragments of one datagram must share it", i, got)
|
||||||
|
}
|
||||||
|
if got := binary.BigEndian.Uint16(f[2:4]); int(got) != len(f) {
|
||||||
|
t.Errorf("fragment %d: total length = %d, actual %d", i, got, len(f))
|
||||||
|
}
|
||||||
|
flagsOff := binary.BigEndian.Uint16(f[6:8])
|
||||||
|
mf := flagsOff&0x2000 != 0
|
||||||
|
wantMF := i < len(frags)-1
|
||||||
|
if mf != wantMF {
|
||||||
|
t.Errorf("fragment %d: MF = %v, want %v", i, mf, wantMF)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Offsets are counted in 8-byte units, so every fragment but the last must be a multiple of 8.
|
||||||
|
// A 100-byte "fragment size" that silently becomes 100 bytes on the wire produces a datagram no
|
||||||
|
// host will ever reassemble.
|
||||||
|
func TestNonFinalFragmentsAreEightByteMultiples(t *testing.T) {
|
||||||
|
src, dst := testAddrs()
|
||||||
|
udp := buildUDP(src, dst, make([]byte, 2500))
|
||||||
|
for _, size := range []int{8, 100, 576, 999, 1400} {
|
||||||
|
frags := splitIPv4(src.Addr(), dst.Addr(), udp, (size/8)*8, 1)
|
||||||
|
for i, f := range frags[:len(frags)-1] {
|
||||||
|
if body := len(f) - 20; body%8 != 0 {
|
||||||
|
t.Errorf("size %d: non-final fragment %d carries %d bytes, not a multiple of 8",
|
||||||
|
size, i, body)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// The UDP checksum is optional in IPv4, and sending zero would be less code — but a
|
||||||
|
// zero-checksum datagram is dropped by some middleboxes, and that drop would be recorded as a
|
||||||
|
// fragmentation failure. So it must be present and correct.
|
||||||
|
func TestUDPChecksumVerifies(t *testing.T) {
|
||||||
|
src, dst := testAddrs()
|
||||||
|
for _, n := range []int{0, 1, 7, 8, 100, 1001} { // odd lengths exercise the tail-byte path
|
||||||
|
udp := buildUDP(src, dst, make([]byte, n))
|
||||||
|
if got := binary.BigEndian.Uint16(udp[6:8]); got == 0 {
|
||||||
|
t.Fatalf("payload %d: checksum is zero, which means 'not computed'", n)
|
||||||
|
}
|
||||||
|
if sum := verifyUDPChecksum(src.Addr(), dst.Addr(), udp); sum != 0xFFFF {
|
||||||
|
t.Errorf("payload %d: checksum does not verify (one's complement sum %#x)", n, sum)
|
||||||
|
}
|
||||||
|
if got := binary.BigEndian.Uint16(udp[4:6]); int(got) != len(udp) {
|
||||||
|
t.Errorf("payload %d: UDP length field %d, actual %d", n, got, len(udp))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestUDPPortsComeFromTheSessionAddresses(t *testing.T) {
|
||||||
|
src, dst := testAddrs()
|
||||||
|
udp := buildUDP(src, dst, []byte("x"))
|
||||||
|
if got := binary.BigEndian.Uint16(udp[0:2]); got != src.Port() {
|
||||||
|
t.Errorf("source port = %d, want %d", got, src.Port())
|
||||||
|
}
|
||||||
|
// The destination port must be the client's observed source port, or the datagram arrives
|
||||||
|
// at the machine and is discarded before any socket sees it.
|
||||||
|
if got := binary.BigEndian.Uint16(udp[2:4]); got != dst.Port() {
|
||||||
|
t.Errorf("destination port = %d, want %d", got, dst.Port())
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Recomputes the one's complement sum over the pseudo-header and datagram; a correct checksum
|
||||||
|
// makes the total 0xFFFF.
|
||||||
|
func verifyUDPChecksum(src, dst netip.Addr, udp []byte) uint16 {
|
||||||
|
var sum uint32
|
||||||
|
s4, d4 := src.Unmap().As4(), dst.Unmap().As4()
|
||||||
|
for _, b := range [][]byte{s4[:], d4[:]} {
|
||||||
|
sum += uint32(binary.BigEndian.Uint16(b[0:2]))
|
||||||
|
sum += uint32(binary.BigEndian.Uint16(b[2:4]))
|
||||||
|
}
|
||||||
|
sum += 17
|
||||||
|
sum += uint32(len(udp))
|
||||||
|
for i := 0; i+1 < len(udp); i += 2 {
|
||||||
|
sum += uint32(binary.BigEndian.Uint16(udp[i : i+2]))
|
||||||
|
}
|
||||||
|
if len(udp)%2 == 1 {
|
||||||
|
sum += uint32(udp[len(udp)-1]) << 8
|
||||||
|
}
|
||||||
|
for sum>>16 != 0 {
|
||||||
|
sum = (sum & 0xFFFF) + (sum >> 16)
|
||||||
|
}
|
||||||
|
return uint16(sum)
|
||||||
|
}
|
||||||
@@ -0,0 +1,41 @@
|
|||||||
|
// SPDX-FileCopyrightText: 2026 Echolot contributors
|
||||||
|
// SPDX-License-Identifier: GPL-3.0-or-later
|
||||||
|
|
||||||
|
//go:build !linux
|
||||||
|
|
||||||
|
package dataplane
|
||||||
|
|
||||||
|
import (
|
||||||
|
"fmt"
|
||||||
|
|
||||||
|
"echo-lot.app/server/internal/session"
|
||||||
|
)
|
||||||
|
|
||||||
|
// Crafting IP fragments needs a raw socket and Linux's IP_HDRINCL semantics. Off Linux the
|
||||||
|
// capability is simply not advertised, so a client never asks for it — better than answering
|
||||||
|
// with a measurement we cannot actually make.
|
||||||
|
|
||||||
|
type FragMode string
|
||||||
|
|
||||||
|
const (
|
||||||
|
FragInOrder FragMode = "in_order"
|
||||||
|
FragReversed FragMode = "reversed"
|
||||||
|
FragFirstLast FragMode = "first_last"
|
||||||
|
)
|
||||||
|
|
||||||
|
type FragResult struct {
|
||||||
|
Mode FragMode `json:"mode"`
|
||||||
|
SizeBytes int `json:"size_bytes"`
|
||||||
|
Fragments int `json:"fragments"`
|
||||||
|
Sent bool `json:"sent"`
|
||||||
|
Err string `json:"err,omitempty"`
|
||||||
|
}
|
||||||
|
|
||||||
|
func RawFragSupported() bool { return false }
|
||||||
|
|
||||||
|
func (s *Server) FragSend(
|
||||||
|
sess *session.Session, g *session.Grant, sizeBytes int, mode FragMode, fragSize int,
|
||||||
|
) (FragResult, error) {
|
||||||
|
return FragResult{Mode: mode, SizeBytes: sizeBytes},
|
||||||
|
fmt.Errorf("crafted fragmentation is only implemented on Linux")
|
||||||
|
}
|
||||||
@@ -0,0 +1,187 @@
|
|||||||
|
// SPDX-FileCopyrightText: 2026 Echolot contributors
|
||||||
|
// SPDX-License-Identifier: GPL-3.0-or-later
|
||||||
|
|
||||||
|
package dataplane
|
||||||
|
|
||||||
|
import (
|
||||||
|
"encoding/binary"
|
||||||
|
"fmt"
|
||||||
|
"time"
|
||||||
|
|
||||||
|
"echo-lot.app/server/internal/session"
|
||||||
|
)
|
||||||
|
|
||||||
|
// Sustained-rate sending (spec §5 throughput).
|
||||||
|
//
|
||||||
|
// This is the most expensive thing the server will do on a client's say-so, so it is also the
|
||||||
|
// action where the §3.4 anti-amplification rules matter most. Three bounds apply, and all three
|
||||||
|
// are enforced here rather than trusted to the caller:
|
||||||
|
//
|
||||||
|
// - the destination is the session's *observed* data-plane source, verified by an HMAC-signed
|
||||||
|
// ECHO that arrived from that address, so this cannot be aimed at a third party;
|
||||||
|
// - the grant carries a byte budget and an average-rate ceiling, and the send stops the moment
|
||||||
|
// either is reached;
|
||||||
|
// - the duration is hard-capped, so a client that vanishes mid-test costs a bounded amount of
|
||||||
|
// traffic rather than an open-ended one.
|
||||||
|
//
|
||||||
|
// The measurement this produces is honest only if the client is told which limit it hit. A run
|
||||||
|
// that saturates the grant ceiling has measured *us*, not the network, and reporting that as
|
||||||
|
// throughput would be worse than not measuring at all — see ThroughputResult.LimitedBy.
|
||||||
|
|
||||||
|
// ThroughputResult is what the server actually managed to send.
|
||||||
|
type ThroughputResult struct {
|
||||||
|
Packets int `json:"packets"`
|
||||||
|
Bytes int64 `json:"bytes"`
|
||||||
|
DurationMs int64 `json:"duration_ms"`
|
||||||
|
Kbps int `json:"kbps"`
|
||||||
|
// LimitedBy says what stopped it: "duration" (ran the full time, so the rate is the path's
|
||||||
|
// or ours to give), "budget" (hit the grant's byte ceiling), or "rate" (the pacing ceiling
|
||||||
|
// held it back). Only "duration" makes the number a property of the network.
|
||||||
|
LimitedBy string `json:"limited_by"`
|
||||||
|
}
|
||||||
|
|
||||||
|
// ThroughputLimits derives a grant sized for one throughput run.
|
||||||
|
//
|
||||||
|
// The default 8 MiB action budget is deliberately far too small for this — ten seconds at
|
||||||
|
// 50 Mbps is 62 MB — so throughput gets its own budget computed from what it asked for, still
|
||||||
|
// clamped to a ceiling. Sizing the budget to the request (rather than raising the global default)
|
||||||
|
// keeps every *other* action bounded at 8 MiB.
|
||||||
|
func ThroughputLimits(durationMs, kbps int) session.GrantLimits {
|
||||||
|
durationMs, kbps = ThroughputPlan(durationMs, kbps)
|
||||||
|
// bytes = kbps * 1000 / 8 * seconds, with a little headroom so the byte budget is not what
|
||||||
|
// stops a run that was meant to be stopped by the clock.
|
||||||
|
budget := int64(kbps) * 1000 / 8 * int64(durationMs) / 1000
|
||||||
|
budget = budget * 11 / 10
|
||||||
|
if budget > maxThroughputBytes {
|
||||||
|
budget = maxThroughputBytes
|
||||||
|
}
|
||||||
|
return session.GrantLimits{
|
||||||
|
MaxBytes: budget,
|
||||||
|
// A little above the pacing target on purpose: the pacer should be what controls the
|
||||||
|
// rate, and the grant should be the safety net. If they are equal, ordinary scheduling
|
||||||
|
// jitter trips the grant and the run is cut short for no real reason.
|
||||||
|
MaxKbps: kbps * 12 / 10,
|
||||||
|
MaxHold: time.Duration(durationMs)*time.Millisecond + 5*time.Second,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// ThroughputPlan reduces a request to what this server will actually run, and is the single
|
||||||
|
// place that decides it.
|
||||||
|
//
|
||||||
|
// When the byte cap binds before the clock does, the *duration* is shortened rather than the run
|
||||||
|
// being cut off partway. Truncating mid-run is not wrong exactly — the rate is still computed
|
||||||
|
// over the elapsed time and limited_by says "budget" — but it means promising a client thirty
|
||||||
|
// seconds and giving it twenty-one. Saying "twenty-one seconds" up front is the same information
|
||||||
|
// without the surprise, and it keeps "the clock ended the run" as the normal case, which is the
|
||||||
|
// only case where the number is a clean property of the network.
|
||||||
|
func ThroughputPlan(durationMs, kbps int) (effectiveMs, effectiveKbps int) {
|
||||||
|
if durationMs <= 0 {
|
||||||
|
durationMs = 10_000
|
||||||
|
}
|
||||||
|
if durationMs > maxThroughputMs {
|
||||||
|
durationMs = maxThroughputMs
|
||||||
|
}
|
||||||
|
if kbps <= 0 || kbps > maxThroughputKbps {
|
||||||
|
kbps = maxThroughputKbps
|
||||||
|
}
|
||||||
|
bytesPerMs := int64(kbps) * 1000 / 8 / 1000
|
||||||
|
if bytesPerMs > 0 {
|
||||||
|
if maxMs := maxThroughputBytes / bytesPerMs; int64(durationMs) > maxMs {
|
||||||
|
durationMs = int(maxMs)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return durationMs, kbps
|
||||||
|
}
|
||||||
|
|
||||||
|
const (
|
||||||
|
maxThroughputMs = 30_000
|
||||||
|
maxThroughputKbps = 200_000
|
||||||
|
maxThroughputBytes = 256 << 20
|
||||||
|
)
|
||||||
|
|
||||||
|
// DownThroughput sends paced traffic toward the client for up to durationMs.
|
||||||
|
//
|
||||||
|
// Pacing is deliberate rather than "send as fast as possible": an unpaced burst measures the
|
||||||
|
// server's NIC and the first queue it meets, then collapses into loss that looks like a network
|
||||||
|
// fault. Spacing packets at the target rate makes loss mean what a reader will assume it means.
|
||||||
|
func (s *Server) DownThroughput(
|
||||||
|
sess *session.Session, g *session.Grant, durationMs, kbps, sizeBytes int,
|
||||||
|
) (ThroughputResult, error) {
|
||||||
|
res := ThroughputResult{}
|
||||||
|
|
||||||
|
target := sess.DataSource()
|
||||||
|
if !target.IsValid() {
|
||||||
|
return res, fmt.Errorf("no observed data-plane source")
|
||||||
|
}
|
||||||
|
conn := s.connFor(target, sess.DataLocal())
|
||||||
|
if conn == nil {
|
||||||
|
return res, fmt.Errorf("no data-plane socket matches target family")
|
||||||
|
}
|
||||||
|
|
||||||
|
// Same plan the grant was sized from, so the two cannot disagree.
|
||||||
|
durationMs, kbps = ThroughputPlan(durationMs, kbps)
|
||||||
|
if sizeBytes < HeaderSize+16 {
|
||||||
|
sizeBytes = 1200 // a size that survives every common path unfragmented
|
||||||
|
}
|
||||||
|
if sizeBytes > 1472 {
|
||||||
|
sizeBytes = 1472
|
||||||
|
}
|
||||||
|
|
||||||
|
// Nanoseconds between packets to hit the target rate.
|
||||||
|
perPacketNs := int64(sizeBytes) * 8 * 1_000_000 / int64(kbps)
|
||||||
|
if perPacketNs < 1_000 {
|
||||||
|
perPacketNs = 1_000
|
||||||
|
}
|
||||||
|
|
||||||
|
payload := make([]byte, sizeBytes-HeaderSize)
|
||||||
|
deadline := time.Now().Add(time.Duration(durationMs) * time.Millisecond)
|
||||||
|
start := time.Now()
|
||||||
|
next := start
|
||||||
|
|
||||||
|
var seq uint32
|
||||||
|
for time.Now().Before(deadline) {
|
||||||
|
ok, why := g.TryAllow(sizeBytes)
|
||||||
|
if !ok {
|
||||||
|
if why == session.RefusalRate {
|
||||||
|
// Transient: the bucket is momentarily empty. Wait for the next slot and carry
|
||||||
|
// on. Ending the run here would report a rate measured over a fraction of a
|
||||||
|
// second, which is worse than reporting no rate at all.
|
||||||
|
res.LimitedBy = "rate"
|
||||||
|
time.Sleep(time.Duration(perPacketNs))
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
// Terminal: the budget is spent, or the grant expired.
|
||||||
|
res.LimitedBy = why
|
||||||
|
break
|
||||||
|
}
|
||||||
|
// Reaching here means the run is progressing normally; the clock will end it.
|
||||||
|
res.LimitedBy = "duration"
|
||||||
|
binary.BigEndian.PutUint32(payload[0:4], seq)
|
||||||
|
binary.BigEndian.PutUint64(payload[4:12], uint64(time.Since(s.start).Nanoseconds()))
|
||||||
|
if err := s.sendErr(conn, target, sess, TypeThroughputData, seq, payload); err != nil {
|
||||||
|
// A send error mid-run is a local condition (buffer full, route gone). Stop and
|
||||||
|
// report what got out rather than pretending the rest was lost on the path.
|
||||||
|
res.LimitedBy = "send_error"
|
||||||
|
break
|
||||||
|
}
|
||||||
|
res.Packets++
|
||||||
|
res.Bytes += int64(sizeBytes)
|
||||||
|
seq++
|
||||||
|
|
||||||
|
// Absolute schedule, not sleep-per-packet: sleeping a fixed interval accumulates the
|
||||||
|
// scheduler's error and drifts the achieved rate below the target over a 10-second run.
|
||||||
|
next = next.Add(time.Duration(perPacketNs))
|
||||||
|
if d := time.Until(next); d > 0 {
|
||||||
|
time.Sleep(d)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
elapsed := time.Since(start)
|
||||||
|
res.DurationMs = elapsed.Milliseconds()
|
||||||
|
// bits per millisecond is kilobits per second, so no scaling constant is needed - and none
|
||||||
|
// can be got wrong. Guarded because a run that ends inside a millisecond has no rate.
|
||||||
|
if res.DurationMs > 0 {
|
||||||
|
res.Kbps = int(res.Bytes * 8 / res.DurationMs)
|
||||||
|
}
|
||||||
|
return res, nil
|
||||||
|
}
|
||||||
@@ -0,0 +1,99 @@
|
|||||||
|
// SPDX-FileCopyrightText: 2026 Echolot contributors
|
||||||
|
// SPDX-License-Identifier: GPL-3.0-or-later
|
||||||
|
|
||||||
|
package dataplane
|
||||||
|
|
||||||
|
import (
|
||||||
|
"testing"
|
||||||
|
"time"
|
||||||
|
)
|
||||||
|
|
||||||
|
// The grant has to be big enough that the *clock* ends a throughput run, not the byte budget. Get
|
||||||
|
// this wrong and the test still "works": it stops early, reports a rate computed over a truncated
|
||||||
|
// window, and nothing anywhere says the number is meaningless. So the sizing is pinned.
|
||||||
|
func TestThroughputBudgetOutlastsTheRequestedRun(t *testing.T) {
|
||||||
|
cases := []struct{ durationMs, kbps int }{
|
||||||
|
{1_000, 1_000},
|
||||||
|
{10_000, 50_000},
|
||||||
|
{10_000, 200_000},
|
||||||
|
{30_000, 100_000},
|
||||||
|
}
|
||||||
|
for _, c := range cases {
|
||||||
|
// Against the *planned* duration, which is what will actually be run: a request the
|
||||||
|
// server shortens is answered with the shorter number, not truncated halfway.
|
||||||
|
planMs, planKbps := ThroughputPlan(c.durationMs, c.kbps)
|
||||||
|
lim := ThroughputLimits(c.durationMs, c.kbps)
|
||||||
|
needed := int64(planKbps) * 1000 / 8 * int64(planMs) / 1000
|
||||||
|
if lim.MaxBytes < needed {
|
||||||
|
t.Errorf("%d ms at %d kbps (planned %d ms) needs %d bytes, budget is %d - the run "+
|
||||||
|
"would stop early and report a rate over a truncated window",
|
||||||
|
c.durationMs, c.kbps, planMs, needed, lim.MaxBytes)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// The pacer should control the rate and the grant should be the safety net. If the grant's
|
||||||
|
// ceiling equals the pacing target, ordinary scheduling jitter trips it and cuts the run short
|
||||||
|
// for no real reason.
|
||||||
|
func TestGrantRateCeilingSitsAboveThePacingTarget(t *testing.T) {
|
||||||
|
lim := ThroughputLimits(10_000, 50_000)
|
||||||
|
if lim.MaxKbps <= 50_000 {
|
||||||
|
t.Fatalf("grant ceiling %d kbps is not above the 50000 kbps pacing target", lim.MaxKbps)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// A client asking for more than the server will do must get the server's number, not its own.
|
||||||
|
func TestThroughputRequestsAreClamped(t *testing.T) {
|
||||||
|
lim := ThroughputLimits(10*60*1000, 10_000_000) // ten minutes at 10 Gbps
|
||||||
|
if lim.MaxBytes > maxThroughputBytes {
|
||||||
|
t.Errorf("byte budget %d exceeds the hard cap %d", lim.MaxBytes, maxThroughputBytes)
|
||||||
|
}
|
||||||
|
if lim.MaxKbps > maxThroughputKbps*12/10 {
|
||||||
|
t.Errorf("rate ceiling %d exceeds the hard cap", lim.MaxKbps)
|
||||||
|
}
|
||||||
|
// The hold has to outlast the planned run, or the grant expires mid-send and the run is
|
||||||
|
// reported as rate-limited when it was really time-limited.
|
||||||
|
planMs, _ := ThroughputPlan(10*60*1000, 10_000_000)
|
||||||
|
if lim.MaxHold < time.Duration(planMs)*time.Millisecond {
|
||||||
|
t.Errorf("hold %v is shorter than the planned run of %d ms", lim.MaxHold, planMs)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// When the byte cap binds before the clock does, the server shortens the run and says so, rather
|
||||||
|
// than accepting thirty seconds and delivering twenty-one. Same information, no surprise - and it
|
||||||
|
// keeps "the clock ended the run" as the normal case, which is the only case where the resulting
|
||||||
|
// rate is a clean property of the network.
|
||||||
|
func TestAnOversizedRequestComesBackShorterRatherThanTruncated(t *testing.T) {
|
||||||
|
const kbps = 200_000
|
||||||
|
askedMs := 30_000
|
||||||
|
planMs, planKbps := ThroughputPlan(askedMs, kbps)
|
||||||
|
|
||||||
|
if planKbps != kbps {
|
||||||
|
t.Errorf("rate was reduced to %d; the duration should absorb the cap, not the rate", planKbps)
|
||||||
|
}
|
||||||
|
if planMs >= askedMs {
|
||||||
|
t.Fatalf("plan kept the full %d ms at %d kbps, which exceeds the %d byte cap",
|
||||||
|
askedMs, kbps, maxThroughputBytes)
|
||||||
|
}
|
||||||
|
// And what it does promise must fit.
|
||||||
|
if got := int64(planKbps) * 1000 / 8 * int64(planMs) / 1000; got > maxThroughputBytes {
|
||||||
|
t.Errorf("planned run needs %d bytes, over the %d cap", got, maxThroughputBytes)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// A short, ordinary request must come back untouched - the clamping only exists for the extremes.
|
||||||
|
func TestAnOrdinaryRequestIsNotRewritten(t *testing.T) {
|
||||||
|
planMs, planKbps := ThroughputPlan(10_000, 50_000)
|
||||||
|
if planMs != 10_000 || planKbps != 50_000 {
|
||||||
|
t.Errorf("10 s at 50 Mbps was rewritten to %d ms at %d kbps", planMs, planKbps)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Every action other than throughput stays on the small default budget. Throughput needs a big
|
||||||
|
// one; raising the global default to suit it would quietly unbound everything else.
|
||||||
|
func TestOnlyThroughputGetsTheLargeBudget(t *testing.T) {
|
||||||
|
big := ThroughputLimits(10_000, 50_000)
|
||||||
|
if big.MaxBytes <= 8<<20 {
|
||||||
|
t.Fatalf("throughput budget %d is no larger than the default action budget", big.MaxBytes)
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -35,6 +35,14 @@ const (
|
|||||||
// Server->client under an asymmetric grant (spec §3.4/§5).
|
// Server->client under an asymmetric grant (spec §3.4/§5).
|
||||||
TypeDownTrainData = 0x06
|
TypeDownTrainData = 0x06
|
||||||
TypeBigSend = 0x0C
|
TypeBigSend = 0x0C
|
||||||
|
// TypeFragData is delivered only after IP reassembly, so its arrival IS the measurement.
|
||||||
|
TypeFragData = 0x0D
|
||||||
|
// TypeThroughputData is one packet of a sustained-rate downstream run.
|
||||||
|
TypeThroughputData = 0x0E
|
||||||
|
// TypeThroughputUp is one packet of a client-driven upstream run. The server counts it and
|
||||||
|
// deliberately does not answer: a reply would double the traffic and measure the return
|
||||||
|
// path at the same time, which is the one thing this test is trying not to do.
|
||||||
|
TypeThroughputUp = 0x0F
|
||||||
)
|
)
|
||||||
|
|
||||||
type Server struct {
|
type Server struct {
|
||||||
@@ -148,6 +156,14 @@ func (s *Server) handle(conn *net.UDPConn, raddr netip.AddrPort, pkt []byte, tRx
|
|||||||
if la, ok := conn.LocalAddr().(*net.UDPAddr); ok {
|
if la, ok := conn.LocalAddr().(*net.UDPAddr); ok {
|
||||||
sess.NoteDataLocal(la.AddrPort())
|
sess.NoteDataLocal(la.AddrPort())
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// Upstream throughput short-circuits before the observation log. Recording one struct per
|
||||||
|
// packet here would mean tens of thousands of allocations for a single run; the counter is
|
||||||
|
// all anyone needs, since the client holds the send-side record.
|
||||||
|
if typ == TypeThroughputUp {
|
||||||
|
sess.CountUpstream(len(pkt), tRxNs)
|
||||||
|
return
|
||||||
|
}
|
||||||
sess.RecordUDP(session.UDPObservation{
|
sess.RecordUDP(session.UDPObservation{
|
||||||
Seq: seq, TRxNs: tRxNs, TTxNs: time.Since(s.start).Nanoseconds(),
|
Seq: seq, TRxNs: tRxNs, TTxNs: time.Since(s.start).Nanoseconds(),
|
||||||
Src: raddr.String(), Size: len(pkt), Type: typ,
|
Src: raddr.String(), Size: len(pkt), Type: typ,
|
||||||
@@ -177,6 +193,7 @@ func (s *Server) mtuAck(conn *net.UDPConn, raddr netip.AddrPort, sess *session.S
|
|||||||
}
|
}
|
||||||
|
|
||||||
// Observation block (spec §3.3), fixed 40 bytes appended to the RESP header:
|
// Observation block (spec §3.3), fixed 40 bytes appended to the RESP header:
|
||||||
|
//
|
||||||
// 0 8 t_rx_ns (server clock, process epoch)
|
// 0 8 t_rx_ns (server clock, process epoch)
|
||||||
// 8 8 t_tx_ns
|
// 8 8 t_tx_ns
|
||||||
// 16 16 observed source IP (v4-mapped when v4)
|
// 16 16 observed source IP (v4-mapped when v4)
|
||||||
@@ -231,6 +248,17 @@ func (s *Server) send(conn *net.UDPConn, raddr netip.AddrPort, sess *session.Ses
|
|||||||
// EMSGSIZE means our own egress MTU refused the datagram, which is a different fact from the
|
// EMSGSIZE means our own egress MTU refused the datagram, which is a different fact from the
|
||||||
// client not receiving it.
|
// client not receiving it.
|
||||||
func (s *Server) sendErr(conn *net.UDPConn, raddr netip.AddrPort, sess *session.Session, typ byte, seq uint32, payload []byte) error {
|
func (s *Server) sendErr(conn *net.UDPConn, raddr netip.AddrPort, sess *session.Session, typ byte, seq uint32, payload []byte) error {
|
||||||
|
pkt := s.buildPacket(sess, typ, seq, payload)
|
||||||
|
_, err := conn.WriteToUDPAddrPort(pkt, raddr)
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
|
||||||
|
// buildPacket assembles and signs an ELT1 packet without sending it.
|
||||||
|
//
|
||||||
|
// Split out for the crafted-fragment path, which needs the bytes so it can cut them up itself.
|
||||||
|
// What arrives after reassembly must be indistinguishable from an ordinary packet, or the client
|
||||||
|
// would be measuring our sender rather than the path — so it goes through exactly this function.
|
||||||
|
func (s *Server) buildPacket(sess *session.Session, typ byte, seq uint32, payload []byte) []byte {
|
||||||
pkt := make([]byte, HeaderSize+len(payload))
|
pkt := make([]byte, HeaderSize+len(payload))
|
||||||
copy(pkt[0:4], Magic)
|
copy(pkt[0:4], Magic)
|
||||||
pkt[4] = typ
|
pkt[4] = typ
|
||||||
@@ -246,8 +274,7 @@ func (s *Server) sendErr(conn *net.UDPConn, raddr netip.AddrPort, sess *session.
|
|||||||
mac.Write(pkt[0:28])
|
mac.Write(pkt[0:28])
|
||||||
mac.Write(payload)
|
mac.Write(payload)
|
||||||
copy(pkt[28:32], mac.Sum(nil)[:4])
|
copy(pkt[28:32], mac.Sum(nil)[:4])
|
||||||
_, err := conn.WriteToUDPAddrPort(pkt, raddr)
|
return pkt
|
||||||
return err
|
|
||||||
}
|
}
|
||||||
|
|
||||||
func hexByte(hi, lo byte) byte {
|
func hexByte(hi, lo byte) byte {
|
||||||
|
|||||||
@@ -74,22 +74,64 @@ func (s *Session) NewGrant(actionID string, wantBytes int64, wantKbps int, lim G
|
|||||||
// enforces the byte ceiling, the expiry, and the average rate (by refusing early sends rather
|
// enforces the byte ceiling, the expiry, and the average rate (by refusing early sends rather
|
||||||
// than sleeping, so callers stay in control of pacing).
|
// than sleeping, so callers stay in control of pacing).
|
||||||
func (g *Grant) Allow(n int) bool {
|
func (g *Grant) Allow(n int) bool {
|
||||||
|
ok, _ := g.TryAllow(n)
|
||||||
|
return ok
|
||||||
|
}
|
||||||
|
|
||||||
|
// Refusal reasons from TryAllow. The distinction is not cosmetic: "too fast just now" is
|
||||||
|
// transient and a caller should pace and carry on, while "budget" and "expired" are terminal and
|
||||||
|
// a caller that keeps trying is only wasting its own run.
|
||||||
|
const (
|
||||||
|
RefusalNone = ""
|
||||||
|
RefusalBudget = "budget"
|
||||||
|
RefusalExpired = "expired"
|
||||||
|
RefusalRate = "rate"
|
||||||
|
)
|
||||||
|
|
||||||
|
// TryAllow reports whether n more bytes may be sent now, consuming the budget when they may, and
|
||||||
|
// says why not when they may not.
|
||||||
|
//
|
||||||
|
// The rate limit is a token bucket: allowance = burst + rate x elapsed. An earlier version
|
||||||
|
// exempted the first 50 ms from the check entirely, meaning to be lenient at startup. The effect
|
||||||
|
// was the opposite - a sender could dump an unbounded burst into that window, and the moment the
|
||||||
|
// check switched on it compared those bytes against 50 ms worth of allowance and refused
|
||||||
|
// everything until real time caught up. A sustained send died about fifty milliseconds in, having
|
||||||
|
// looked perfectly fine in every short test. A bucket has no such cliff: it is smooth from t=0.
|
||||||
|
func (g *Grant) TryAllow(n int) (bool, string) {
|
||||||
g.mu.Lock()
|
g.mu.Lock()
|
||||||
defer g.mu.Unlock()
|
defer g.mu.Unlock()
|
||||||
if time.Now().After(g.ExpiresAt) {
|
if time.Now().After(g.ExpiresAt) {
|
||||||
return false
|
return false, RefusalExpired
|
||||||
}
|
}
|
||||||
if g.sentBytes+int64(n) > g.MaxBytes {
|
if g.sentBytes+int64(n) > g.MaxBytes {
|
||||||
return false
|
return false, RefusalBudget
|
||||||
}
|
}
|
||||||
// Average-rate check: bytes allowed so far = kbps/8 * elapsed_seconds.
|
// kbps -> bytes/s is kbps*1000/8 = kbps*125.
|
||||||
|
bytesPerSec := float64(g.MaxKbps) * 125
|
||||||
elapsed := time.Since(g.started).Seconds()
|
elapsed := time.Since(g.started).Seconds()
|
||||||
allowed := float64(g.MaxKbps) * 125 * elapsed // kbps -> bytes/s is kbps*1000/8 = kbps*125
|
allowed := burstBytes(bytesPerSec) + bytesPerSec*elapsed
|
||||||
if elapsed > 0.05 && float64(g.sentBytes+int64(n)) > allowed {
|
if float64(g.sentBytes+int64(n)) > allowed {
|
||||||
return false
|
return false, RefusalRate
|
||||||
}
|
}
|
||||||
g.sentBytes += int64(n)
|
g.sentBytes += int64(n)
|
||||||
return true
|
return true, RefusalNone
|
||||||
|
}
|
||||||
|
|
||||||
|
// burstBytes is the bucket's depth: 100 ms of the allowed rate, floored at a single ordinary
|
||||||
|
// datagram.
|
||||||
|
//
|
||||||
|
// The floor exists only so that one packet is never refused outright by a very slow grant — it is
|
||||||
|
// deliberately one datagram and not more. A generous floor would undo the rate ceiling at low
|
||||||
|
// rates: at 8 kbps a 64 KB burst is sixty-four seconds' worth, which is exactly the instant dump
|
||||||
|
// the ceiling is there to prevent. One datagram is 1.5 seconds' worth at that rate and nothing at
|
||||||
|
// any realistic one.
|
||||||
|
func burstBytes(bytesPerSec float64) float64 {
|
||||||
|
const oneDatagram = 1500
|
||||||
|
b := bytesPerSec * 0.1
|
||||||
|
if b < oneDatagram {
|
||||||
|
b = oneDatagram
|
||||||
|
}
|
||||||
|
return b
|
||||||
}
|
}
|
||||||
|
|
||||||
// Sent returns how many bytes this grant has consumed.
|
// Sent returns how many bytes this grant has consumed.
|
||||||
|
|||||||
@@ -92,3 +92,67 @@ func TestGrantEnforcesRate(t *testing.T) {
|
|||||||
t.Fatalf("rate limit let %d bytes through in ~60ms at 8kbps", sent)
|
t.Fatalf("rate limit let %d bytes through in ~60ms at 8kbps", sent)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// The bug this pins: the rate check used to exempt the first 50 ms entirely, so a sender could
|
||||||
|
// dump an unbounded burst into that window and then be refused for as long as it took real time
|
||||||
|
// to catch up. Every short test passed; a sustained send died about fifty milliseconds in. A
|
||||||
|
// token bucket has no such cliff, and the property that matters is that a sender pacing *at* the
|
||||||
|
// allowed rate is never refused for long.
|
||||||
|
func TestSustainedSendAtTheAllowedRateIsNotCutOff(t *testing.T) {
|
||||||
|
s := sessionWithSource(t)
|
||||||
|
const kbps = 8000 // 1 MB/s
|
||||||
|
const packet = 1200 // bytes
|
||||||
|
g := s.NewGrant("a1", 8<<20, kbps, DefaultGrantLimits)
|
||||||
|
|
||||||
|
// Pace at the allowed rate for a short run and count how much got through. A correct
|
||||||
|
// limiter passes essentially all of it; the old one stopped almost immediately.
|
||||||
|
perPacket := time.Duration(float64(packet) / (float64(kbps) * 125) * float64(time.Second))
|
||||||
|
deadline := time.Now().Add(300 * time.Millisecond)
|
||||||
|
sent, refusals := 0, 0
|
||||||
|
for time.Now().Before(deadline) {
|
||||||
|
if ok, why := g.TryAllow(packet); ok {
|
||||||
|
sent += packet
|
||||||
|
} else if why == RefusalRate {
|
||||||
|
refusals++
|
||||||
|
} else {
|
||||||
|
t.Fatalf("unexpected terminal refusal %q after %d bytes", why, sent)
|
||||||
|
}
|
||||||
|
time.Sleep(perPacket)
|
||||||
|
}
|
||||||
|
|
||||||
|
// 300 ms at 1 MB/s is ~300 KB. Allow generous slack for scheduler granularity, but a run
|
||||||
|
// that delivered only a few packets means the limiter cut it off.
|
||||||
|
if sent < 100_000 {
|
||||||
|
t.Fatalf("a sender pacing at the allowed rate got only %d bytes through in 300ms "+
|
||||||
|
"(%d rate refusals) — the limiter is cutting off sustained sends", sent, refusals)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// The other half: a rate refusal must be distinguishable from a spent budget, because one is
|
||||||
|
// transient and one is terminal, and a caller that cannot tell them apart either gives up early
|
||||||
|
// or spins forever.
|
||||||
|
func TestRefusalReasonsAreDistinguishable(t *testing.T) {
|
||||||
|
s := sessionWithSource(t)
|
||||||
|
|
||||||
|
// Budget: tiny ceiling, plenty of rate.
|
||||||
|
g := s.NewGrant("a1", 1000, 100_000, DefaultGrantLimits)
|
||||||
|
for i := 0; i < 20; i++ {
|
||||||
|
g.TryAllow(100)
|
||||||
|
}
|
||||||
|
if ok, why := g.TryAllow(100); ok || why != RefusalBudget {
|
||||||
|
t.Errorf("spent budget reported as ok=%v why=%q, want %q", ok, why, RefusalBudget)
|
||||||
|
}
|
||||||
|
|
||||||
|
// Rate: huge ceiling, minimal rate, so only the bucket can refuse.
|
||||||
|
g2 := s.NewGrant("a2", 1<<20, 8, DefaultGrantLimits)
|
||||||
|
sawRate := false
|
||||||
|
for i := 0; i < 100; i++ {
|
||||||
|
if ok, why := g2.TryAllow(1000); !ok && why == RefusalRate {
|
||||||
|
sawRate = true
|
||||||
|
break
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if !sawRate {
|
||||||
|
t.Error("a sender far above the rate ceiling never got a rate refusal")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|||||||
@@ -40,6 +40,8 @@ type Session struct {
|
|||||||
packetsSeen uint64
|
packetsSeen uint64
|
||||||
udpObs []UDPObservation // ring, newest last, cap obsCap
|
udpObs []UDPObservation // ring, newest last, cap obsCap
|
||||||
connectBack []ConnectBackResult
|
connectBack []ConnectBackResult
|
||||||
|
throughput []ThroughputReport
|
||||||
|
upstream UpstreamCounter
|
||||||
}
|
}
|
||||||
|
|
||||||
const obsCap = 4096
|
const obsCap = 4096
|
||||||
@@ -54,6 +56,48 @@ type UDPObservation struct {
|
|||||||
Type uint8 `json:"type"`
|
Type uint8 `json:"type"`
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// ThroughputReport is the server's own account of a sustained send: what it managed to put on
|
||||||
|
// the wire, and what stopped it. The client needs this to interpret its own count — the gap
|
||||||
|
// between the two IS the loss, and without the sender's number a receiver can only guess.
|
||||||
|
type ThroughputReport struct {
|
||||||
|
ActionID string `json:"action_id"`
|
||||||
|
Packets int `json:"packets"`
|
||||||
|
Bytes int64 `json:"bytes"`
|
||||||
|
DurationMs int64 `json:"duration_ms"`
|
||||||
|
Kbps int `json:"kbps"`
|
||||||
|
LimitedBy string `json:"limited_by"`
|
||||||
|
}
|
||||||
|
|
||||||
|
// UpstreamCounter is the server's tally of a client-driven throughput run.
|
||||||
|
//
|
||||||
|
// Deliberately a counter and not a list. A five-second upstream run at 20 Mbps is around ten
|
||||||
|
// thousand packets; one observation struct each would turn a measurement into an allocation
|
||||||
|
// storm on a shared server, and nothing downstream needs the per-packet detail - the client
|
||||||
|
// already has its own send record. The gap between the two counts IS the loss.
|
||||||
|
type UpstreamCounter struct {
|
||||||
|
Packets int `json:"packets"`
|
||||||
|
Bytes int64 `json:"bytes"`
|
||||||
|
FirstRxNs int64 `json:"first_rx_ns"`
|
||||||
|
LastRxNs int64 `json:"last_rx_ns"`
|
||||||
|
}
|
||||||
|
|
||||||
|
// SpanMs is the time between the first and last packet, which is the interval the rate should be
|
||||||
|
// computed over - not the client's requested duration, which includes ramp-up and the tail.
|
||||||
|
func (u UpstreamCounter) SpanMs() int64 {
|
||||||
|
if u.Packets < 2 || u.LastRxNs <= u.FirstRxNs {
|
||||||
|
return 0
|
||||||
|
}
|
||||||
|
return (u.LastRxNs - u.FirstRxNs) / 1_000_000
|
||||||
|
}
|
||||||
|
|
||||||
|
// Kbps is bits per millisecond, which is kilobits per second - no scaling constant to get wrong.
|
||||||
|
func (u UpstreamCounter) Kbps() int {
|
||||||
|
if ms := u.SpanMs(); ms > 0 {
|
||||||
|
return int(u.Bytes * 8 / ms)
|
||||||
|
}
|
||||||
|
return 0
|
||||||
|
}
|
||||||
|
|
||||||
// ConnectBackResult records one connect-back action outcome.
|
// ConnectBackResult records one connect-back action outcome.
|
||||||
type ConnectBackResult struct {
|
type ConnectBackResult struct {
|
||||||
ActionID string `json:"action_id"`
|
ActionID string `json:"action_id"`
|
||||||
@@ -87,6 +131,57 @@ func (s *Session) Observations() (packetsSeen uint64, udp []UDPObservation, cb [
|
|||||||
append([]ConnectBackResult(nil), s.connectBack...)
|
append([]ConnectBackResult(nil), s.connectBack...)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// CountUpstream tallies one client-sent throughput packet.
|
||||||
|
//
|
||||||
|
// Called on the hot path for every packet of an upstream run, so it does exactly two additions
|
||||||
|
// and two comparisons under the lock and allocates nothing.
|
||||||
|
func (s *Session) CountUpstream(sizeBytes int, tRxNs int64) {
|
||||||
|
s.mu.Lock()
|
||||||
|
defer s.mu.Unlock()
|
||||||
|
if s.upstream.Packets == 0 {
|
||||||
|
s.upstream.FirstRxNs = tRxNs
|
||||||
|
}
|
||||||
|
s.upstream.Packets++
|
||||||
|
s.upstream.Bytes += int64(sizeBytes)
|
||||||
|
s.upstream.LastRxNs = tRxNs
|
||||||
|
}
|
||||||
|
|
||||||
|
// Upstream returns the tally so far.
|
||||||
|
func (s *Session) Upstream() UpstreamCounter {
|
||||||
|
s.mu.Lock()
|
||||||
|
defer s.mu.Unlock()
|
||||||
|
return s.upstream
|
||||||
|
}
|
||||||
|
|
||||||
|
// ResetUpstream clears the tally, so a second run in one session measures itself rather than
|
||||||
|
// inheriting the first one's packets.
|
||||||
|
func (s *Session) ResetUpstream() {
|
||||||
|
s.mu.Lock()
|
||||||
|
defer s.mu.Unlock()
|
||||||
|
s.upstream = UpstreamCounter{}
|
||||||
|
}
|
||||||
|
|
||||||
|
// RecordThroughput stores the server's account of one sustained send.
|
||||||
|
//
|
||||||
|
// Kept as a per-action summary rather than per-packet records: a ten-second run at 50 Mbps is
|
||||||
|
// half a million packets, and holding one struct each would turn a measurement into a memory
|
||||||
|
// exhaustion. The client has the per-packet view; the server only needs to say how many it sent.
|
||||||
|
func (s *Session) RecordThroughput(actionID string, packets int, bytes, durationMs int64, kbps int, limitedBy string) {
|
||||||
|
s.mu.Lock()
|
||||||
|
defer s.mu.Unlock()
|
||||||
|
s.throughput = append(s.throughput, ThroughputReport{
|
||||||
|
ActionID: actionID, Packets: packets, Bytes: bytes,
|
||||||
|
DurationMs: durationMs, Kbps: kbps, LimitedBy: limitedBy,
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
// ThroughputReports returns the server's account of every sustained send in this session.
|
||||||
|
func (s *Session) ThroughputReports() []ThroughputReport {
|
||||||
|
s.mu.Lock()
|
||||||
|
defer s.mu.Unlock()
|
||||||
|
return append([]ThroughputReport(nil), s.throughput...)
|
||||||
|
}
|
||||||
|
|
||||||
// DataSource returns the last verified data-plane source (invalid when the
|
// DataSource returns the last verified data-plane source (invalid when the
|
||||||
// session has not sent data-plane traffic yet).
|
// session has not sent data-plane traffic yet).
|
||||||
func (s *Session) DataSource() netip.AddrPort {
|
func (s *Session) DataSource() netip.AddrPort {
|
||||||
|
|||||||
@@ -0,0 +1,74 @@
|
|||||||
|
# Echolot website (`web/`)
|
||||||
|
|
||||||
|
Minimal single-page site for [echo-lot.app](https://echo-lot.app), served from Cloudflare
|
||||||
|
Workers. Static files in `public/` are served straight from the edge; the tiny Worker in
|
||||||
|
`src/index.js` only runs for paths that aren't files:
|
||||||
|
|
||||||
|
| Path | Behavior |
|
||||||
|
| ------------- | ------------------------------------------------------------------------ |
|
||||||
|
| `/apk` | 302 → newest `.apk` asset of the latest Gitea release (QR-code friendly) |
|
||||||
|
| `/apk.sha256` | 302 → the matching `.sha256` asset |
|
||||||
|
| `/api/latest` | JSON `{version, published_at, apk, sha256}` — the homepage's version readout |
|
||||||
|
| `/fdroid`, `/source` | 302 → the URLs configured in `wrangler.jsonc` vars |
|
||||||
|
|
||||||
|
The latest release is resolved from the Gitea API **at request time** (edge-cached 5 min), so
|
||||||
|
publishing a release — `git tag v0.2.0 && git push origin v0.2.0`, which triggers
|
||||||
|
`.gitea/workflows/release.yml` — is the only release step. The site never needs a redeploy for
|
||||||
|
a new version, and empty/unreachable values fall back to the homepage instead of 404ing.
|
||||||
|
|
||||||
|
Light/dark follows the OS (`prefers-color-scheme`), no toggle, no JS required for it. Colors
|
||||||
|
come from the branding palette (teal = instrument, single amber point = finding).
|
||||||
|
|
||||||
|
`public/assets/` (favicon, wordmark, social preview) are **copies** of `../assets/branding/` —
|
||||||
|
that directory is the source of truth; re-copy after any branding change.
|
||||||
|
|
||||||
|
## Deploy
|
||||||
|
|
||||||
|
Everything is driven by [wrangler](https://developers.cloudflare.com/workers/wrangler/), config
|
||||||
|
in `wrangler.jsonc`. No build step, no node_modules to commit.
|
||||||
|
|
||||||
|
### One-time setup
|
||||||
|
|
||||||
|
1. In the Cloudflare dashboard, add **echo-lot.app** as a zone (and point the domain's
|
||||||
|
nameservers at Cloudflare). The `routes` in `wrangler.jsonc` use `custom_domain: true`, so
|
||||||
|
wrangler creates the DNS records for `echo-lot.app` and `www` automatically on first deploy —
|
||||||
|
the zone just has to exist in the same account.
|
||||||
|
2. Auth, either flavor:
|
||||||
|
- **Interactive:** `npx wrangler login` (opens the browser once, stores an OAuth token).
|
||||||
|
- **API token (also what CI uses):** dashboard → My Profile → API Tokens → create from the
|
||||||
|
**"Edit Cloudflare Workers"** template. Then:
|
||||||
|
|
||||||
|
```
|
||||||
|
$env:CLOUDFLARE_API_TOKEN = "..." # PowerShell; export ... on POSIX
|
||||||
|
$env:CLOUDFLARE_ACCOUNT_ID = "..." # dashboard → Workers & Pages, right sidebar
|
||||||
|
```
|
||||||
|
|
||||||
|
### Deploy
|
||||||
|
|
||||||
|
```
|
||||||
|
cd web
|
||||||
|
npx wrangler@4 deploy
|
||||||
|
```
|
||||||
|
|
||||||
|
That's it — uploads `src/index.js` + the `public/` assets, wires the custom domains. Useful
|
||||||
|
extras: `npx wrangler dev` (local preview at localhost:8787), `npx wrangler tail` (live logs),
|
||||||
|
`npx wrangler versions list`.
|
||||||
|
|
||||||
|
### CI deploy (Gitea Actions)
|
||||||
|
|
||||||
|
`.gitea/workflows/deploy-site.yml` runs `wrangler deploy` on every push to `main`/`master` that
|
||||||
|
touches `web/`. It stays inert until you add two repo secrets (Settings → Actions → Secrets):
|
||||||
|
`CLOUDFLARE_API_TOKEN` and `CLOUDFLARE_ACCOUNT_ID` (same values as above).
|
||||||
|
|
||||||
|
Cloudflare's raw REST API (`PUT /accounts/:id/workers/scripts/...`) exists, but the assets
|
||||||
|
upload needs a manifest/session dance that wrangler already implements — use wrangler even in
|
||||||
|
automation.
|
||||||
|
|
||||||
|
## Config knobs (`wrangler.jsonc` → `vars`)
|
||||||
|
|
||||||
|
- `GITEA_REPO_API` — Gitea repo API base; releases must be publicly readable.
|
||||||
|
- `DOWNLOAD_URL` — manual `/apk` fallback while Gitea is unreachable.
|
||||||
|
- `FDROID_URL` — set when the F-Droid listing exists; until then `/fdroid` loops home.
|
||||||
|
- `SOURCE_URL` — public source mirror for the footer + `/source`.
|
||||||
|
|
||||||
|
Vars are plain (non-secret) config; change + `wrangler deploy` to apply.
|
||||||
Binary file not shown.
|
After Width: | Height: | Size: 23 KiB |
@@ -0,0 +1,24 @@
|
|||||||
|
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 96 96">
|
||||||
|
<defs>
|
||||||
|
<linearGradient id="tile" x1="0" y1="0" x2="0" y2="1">
|
||||||
|
<stop offset="0" stop-color="#0E2433"/>
|
||||||
|
<stop offset="1" stop-color="#071522"/>
|
||||||
|
</linearGradient>
|
||||||
|
</defs>
|
||||||
|
<rect width="96" height="96" rx="21" fill="url(#tile)"/>
|
||||||
|
<!-- the network, at rest -->
|
||||||
|
<g fill="#1E4A5C">
|
||||||
|
<circle cx="24" cy="24" r="2.6"/><circle cx="48" cy="24" r="2.6"/><circle cx="72" cy="24" r="2.6"/>
|
||||||
|
<circle cx="24" cy="48" r="2.6"/> <circle cx="72" cy="48" r="2.6"/>
|
||||||
|
<circle cx="24" cy="72" r="2.6"/><circle cx="48" cy="72" r="2.6"/><circle cx="72" cy="72" r="2.6"/>
|
||||||
|
</g>
|
||||||
|
<!-- one node, under examination -->
|
||||||
|
<circle cx="48" cy="48" r="8" fill="none" stroke="#FFB454" stroke-opacity="0.3" stroke-width="2"/>
|
||||||
|
<circle cx="48" cy="48" r="4.5" fill="#FFB454"/>
|
||||||
|
<g fill="none" stroke="#35E0C4" stroke-width="3.5" stroke-linecap="round" stroke-linejoin="round">
|
||||||
|
<path d="M 33 41 V 33 H 41"/>
|
||||||
|
<path d="M 55 33 H 63 V 41"/>
|
||||||
|
<path d="M 63 55 V 63 H 55"/>
|
||||||
|
<path d="M 41 63 H 33 V 55"/>
|
||||||
|
</g>
|
||||||
|
</svg>
|
||||||
|
After Width: | Height: | Size: 1.1 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 26 KiB |
@@ -0,0 +1,17 @@
|
|||||||
|
<svg xmlns="http://www.w3.org/2000/svg" viewBox="-10 -92 433 104">
|
||||||
|
<!-- "echolot" — hand-drawn monoline letterforms (no font dependency). For dark grounds. -->
|
||||||
|
<g fill="none" stroke="#E8F4F2" stroke-width="11" stroke-linecap="round">
|
||||||
|
<path d="M 0 -24 H 48"/>
|
||||||
|
<path d="M 48 -24 A 24 24 0 1 0 40.97 -7.03"/>
|
||||||
|
<path d="M 110.97 -40.97 A 24 24 0 1 0 110.97 -7.03"/>
|
||||||
|
<path d="M 140 -76 V 0"/>
|
||||||
|
<path d="M 140 -24 A 24 24 0 0 1 188 -24 L 188 0"/>
|
||||||
|
<circle cx="234" cy="-24" r="24"/>
|
||||||
|
<path d="M 280 -76 V 0"/>
|
||||||
|
<circle cx="326" cy="-24" r="24" stroke="#35E0C4"/>
|
||||||
|
<path d="M 372 -48 H 402"/>
|
||||||
|
<path d="M 387 -68 V 0"/>
|
||||||
|
</g>
|
||||||
|
<!-- the finding -->
|
||||||
|
<circle cx="326" cy="-24" r="6.5" fill="#FFB454"/>
|
||||||
|
</svg>
|
||||||
|
After Width: | Height: | Size: 742 B |
@@ -0,0 +1,17 @@
|
|||||||
|
<svg xmlns="http://www.w3.org/2000/svg" viewBox="-10 -92 433 104">
|
||||||
|
<!-- "echolot" — hand-drawn monoline letterforms (no font dependency). For light grounds. -->
|
||||||
|
<g fill="none" stroke="#1B3540" stroke-width="11" stroke-linecap="round">
|
||||||
|
<path d="M 0 -24 H 48"/>
|
||||||
|
<path d="M 48 -24 A 24 24 0 1 0 40.97 -7.03"/>
|
||||||
|
<path d="M 110.97 -40.97 A 24 24 0 1 0 110.97 -7.03"/>
|
||||||
|
<path d="M 140 -76 V 0"/>
|
||||||
|
<path d="M 140 -24 A 24 24 0 0 1 188 -24 L 188 0"/>
|
||||||
|
<circle cx="234" cy="-24" r="24"/>
|
||||||
|
<path d="M 280 -76 V 0"/>
|
||||||
|
<circle cx="326" cy="-24" r="24" stroke="#0E9384"/>
|
||||||
|
<path d="M 372 -48 H 402"/>
|
||||||
|
<path d="M 387 -68 V 0"/>
|
||||||
|
</g>
|
||||||
|
<!-- the finding -->
|
||||||
|
<circle cx="326" cy="-24" r="6.5" fill="#E08A1E"/>
|
||||||
|
</svg>
|
||||||
|
After Width: | Height: | Size: 743 B |
+156
-87
@@ -5,103 +5,124 @@
|
|||||||
<head>
|
<head>
|
||||||
<meta charset="utf-8">
|
<meta charset="utf-8">
|
||||||
<meta name="viewport" content="width=device-width, initial-scale=1">
|
<meta name="viewport" content="width=device-width, initial-scale=1">
|
||||||
<title>Echolot — depth soundings for your local network</title>
|
<title>Echolot — measure, don't guess</title>
|
||||||
<meta name="description" content="Free Android app for detecting and debugging local network issues: rogue DHCP, broken IPv6 RAs, MTU black holes, multicast loss, lying DNS. No root required.">
|
<meta name="description" content="Free Android app for detecting and debugging local network issues: rogue DHCP, broken IPv6 RAs, MTU black holes, multicast loss, lying DNS. No root required.">
|
||||||
<meta property="og:title" content="Echolot">
|
<meta property="og:title" content="Echolot">
|
||||||
<meta property="og:description" content="Depth soundings for your local network. F/OSS Android network diagnostics — no root required.">
|
<meta property="og:description" content="Measure, don't guess. F/OSS Android network diagnostics — no root required.">
|
||||||
<meta property="og:url" content="https://echo-lot.app/">
|
<meta property="og:url" content="https://echo-lot.app/">
|
||||||
<link rel="icon" href="data:image/svg+xml,%3Csvg xmlns='http://www.w3.org/2000/svg' viewBox='0 0 32 32'%3E%3Crect width='32' height='32' fill='%23071A29'/%3E%3Cg fill='none' stroke='%23FFB454' stroke-width='2'%3E%3Ccircle cx='16' cy='16' r='3' fill='%23FFB454' stroke='none'/%3E%3Cpath d='M16 6a10 10 0 0 1 10 10'/%3E%3Cpath d='M16 1a15 15 0 0 1 15 15' opacity='.5'/%3E%3C/g%3E%3C/svg%3E">
|
<meta property="og:image" content="https://echo-lot.app/assets/social-preview.png">
|
||||||
|
<meta name="theme-color" media="(prefers-color-scheme: dark)" content="#071522">
|
||||||
|
<meta name="theme-color" media="(prefers-color-scheme: light)" content="#F2F6F7">
|
||||||
|
<link rel="icon" href="/assets/icon.svg" type="image/svg+xml">
|
||||||
|
<link rel="icon" href="/assets/icon.png" type="image/png" sizes="512x512">
|
||||||
|
<link rel="apple-touch-icon" href="/assets/icon.png">
|
||||||
<style>
|
<style>
|
||||||
|
/* Branding: teal is always the instrument (links, brackets, controls);
|
||||||
|
the single amber point is the finding (the focused node, the version
|
||||||
|
readout). Dark = the instrument's own display; light = the same tokens
|
||||||
|
on paper. Palette from assets/branding/. */
|
||||||
:root {
|
:root {
|
||||||
--depth-0: #0B2437; /* surface */
|
color-scheme: dark;
|
||||||
--depth-1: #092031; /* photic */
|
--bg-0: #0C2130; /* top of page */
|
||||||
--depth-2: #071A29; /* mid */
|
--bg-1: #071522; /* abyss — page floor, panel ground */
|
||||||
--depth-3: #051320; /* floor */
|
--tile: #0E2433; /* raised surfaces */
|
||||||
--foam: #DCE9F1; /* primary text */
|
--foam: #E8F4F2; /* primary text */
|
||||||
--slate: #8AA5B8; /* secondary text */
|
--slate: #7DA2AC; /* secondary text */
|
||||||
--grid: #16374E; /* hairlines, chart grid */
|
--caption:#5E8B96; /* mono captions, from the banner */
|
||||||
--ping: #FFB454; /* the one accent: sonar amber */
|
--grid: #10303F; /* hairlines */
|
||||||
--ok: #7BC98F; /* verdict green, chips only */
|
--rest: #1E4A5C; /* the network, at rest */
|
||||||
|
--teal: #35E0C4; /* instrument */
|
||||||
|
--on-teal:#04212B; /* text on teal */
|
||||||
|
--amber: #FFB454; /* the finding */
|
||||||
--mono: "Cascadia Code", "SF Mono", Consolas, "Liberation Mono", Menlo, monospace;
|
--mono: "Cascadia Code", "SF Mono", Consolas, "Liberation Mono", Menlo, monospace;
|
||||||
--sans: "Segoe UI", system-ui, -apple-system, "Helvetica Neue", Arial, sans-serif;
|
--sans: "Segoe UI", system-ui, -apple-system, "Helvetica Neue", Arial, sans-serif;
|
||||||
}
|
}
|
||||||
|
@media (prefers-color-scheme: light) {
|
||||||
|
:root {
|
||||||
|
color-scheme: light;
|
||||||
|
--bg-0: #F2F6F7;
|
||||||
|
--bg-1: #E4ECEF;
|
||||||
|
--tile: #EBF1F3;
|
||||||
|
--foam: #1B3540; /* ink, from wordmark-on-light */
|
||||||
|
--slate: #47656F;
|
||||||
|
--caption:#5E8B96;
|
||||||
|
--grid: #C4D3D8;
|
||||||
|
--rest: #9FB8C1;
|
||||||
|
--teal: #0E9384; /* instrument, printable contrast */
|
||||||
|
--on-teal:#F5FBFA;
|
||||||
|
--amber: #C77413; /* finding ink */
|
||||||
|
}
|
||||||
|
}
|
||||||
* { box-sizing: border-box; margin: 0; }
|
* { box-sizing: border-box; margin: 0; }
|
||||||
html { scroll-behavior: smooth; }
|
html { scroll-behavior: smooth; }
|
||||||
body {
|
body {
|
||||||
font-family: var(--sans);
|
font-family: var(--sans);
|
||||||
color: var(--foam);
|
color: var(--foam);
|
||||||
background: linear-gradient(var(--depth-0), var(--depth-1) 30%, var(--depth-2) 65%, var(--depth-3));
|
background: linear-gradient(var(--bg-0), var(--bg-1) 70%);
|
||||||
|
min-height: 100vh;
|
||||||
line-height: 1.6;
|
line-height: 1.6;
|
||||||
-webkit-font-smoothing: antialiased;
|
-webkit-font-smoothing: antialiased;
|
||||||
}
|
}
|
||||||
a { color: var(--ping); text-decoration-thickness: 1px; text-underline-offset: 3px; }
|
a { color: var(--teal); text-decoration-thickness: 1px; text-underline-offset: 3px; }
|
||||||
a:hover { text-decoration-thickness: 2px; }
|
a:hover { text-decoration-thickness: 2px; }
|
||||||
:focus-visible { outline: 2px solid var(--ping); outline-offset: 3px; border-radius: 2px; }
|
:focus-visible { outline: 2px solid var(--teal); outline-offset: 3px; border-radius: 2px; }
|
||||||
|
|
||||||
.col { max-width: 46rem; margin: 0 auto; padding: 0 1.25rem; }
|
.col { max-width: 46rem; margin: 0 auto; padding: 0 1.25rem; }
|
||||||
|
|
||||||
/* Depth ruler: fixed left margin scale, desktop only. Marks are set per-section
|
/* Graduated rule: the tick motif from the banner, vertical. Fixed left
|
||||||
by scroll position purely decoratively — it is a ruler, not navigation. */
|
margin, desktop only, purely decorative. */
|
||||||
.ruler {
|
.rule {
|
||||||
position: fixed; top: 0; bottom: 0; left: 0; width: 3.5rem;
|
position: fixed; top: 0; bottom: 0; left: 0; width: 3.5rem;
|
||||||
border-right: 1px solid var(--grid);
|
border-right: 1px solid var(--grid);
|
||||||
font-family: var(--mono); font-size: .65rem; color: var(--slate);
|
|
||||||
display: none;
|
display: none;
|
||||||
}
|
}
|
||||||
@media (min-width: 72rem) { .ruler { display: block; } }
|
@media (min-width: 72rem) { .rule { display: block; } }
|
||||||
.ruler span {
|
.rule::after {
|
||||||
position: absolute; right: .5rem; transform: translateY(-50%);
|
content: ""; position: absolute; right: 0; top: 0; bottom: 0; width: .4rem;
|
||||||
}
|
background: repeating-linear-gradient(to bottom, var(--rest) 0 1.5px, transparent 1.5px 60px);
|
||||||
.ruler span::after {
|
|
||||||
content: ""; position: absolute; right: -.55rem; top: 50%;
|
|
||||||
width: .35rem; height: 1px; background: var(--slate);
|
|
||||||
}
|
}
|
||||||
|
|
||||||
header.hero { padding: 4.5rem 0 3rem; }
|
header.hero { padding: 4.5rem 0 3rem; }
|
||||||
.wordmark {
|
.wordmark { display: block; height: 30px; width: auto; }
|
||||||
font-family: var(--mono); font-size: .8rem; letter-spacing: .35em;
|
|
||||||
text-transform: uppercase; color: var(--slate);
|
|
||||||
}
|
|
||||||
.wordmark b { color: var(--ping); font-weight: 600; }
|
|
||||||
h1 {
|
h1 {
|
||||||
font-size: clamp(1.9rem, 5vw, 3rem);
|
font-size: clamp(1.9rem, 5vw, 3rem);
|
||||||
font-weight: 650; letter-spacing: -.02em; line-height: 1.15;
|
font-weight: 650; letter-spacing: -.02em; line-height: 1.15;
|
||||||
margin: 1rem 0 .75rem; max-width: 30ch;
|
margin: 1.75rem 0 .75rem; max-width: 30ch;
|
||||||
}
|
}
|
||||||
.hero p.lede { color: var(--slate); max-width: 52ch; font-size: 1.05rem; }
|
.hero p.lede { color: var(--slate); max-width: 52ch; font-size: 1.05rem; }
|
||||||
.hero p.lede strong { color: var(--foam); font-weight: 600; }
|
.hero p.lede strong { color: var(--foam); font-weight: 600; }
|
||||||
|
|
||||||
/* Echogram: the signature. A chart-recorder trace of ping RTTs; the sweep
|
/* Focus panel: the signature, straight from the mark. The network at rest,
|
||||||
line is the sounder, the profile is the "seabed" the echoes draw. */
|
one node under examination — teal brackets are the instrument, the amber
|
||||||
figure.echogram {
|
point is the finding. */
|
||||||
|
figure.focus {
|
||||||
margin: 2.5rem 0 0; border: 1px solid var(--grid); border-radius: 4px;
|
margin: 2.5rem 0 0; border: 1px solid var(--grid); border-radius: 4px;
|
||||||
background:
|
background: var(--tile);
|
||||||
repeating-linear-gradient(to right, transparent 0 39px, var(--grid) 39px 40px),
|
|
||||||
repeating-linear-gradient(to bottom, transparent 0 31px, var(--grid) 31px 32px),
|
|
||||||
var(--depth-3);
|
|
||||||
position: relative; overflow: hidden;
|
position: relative; overflow: hidden;
|
||||||
}
|
}
|
||||||
.echogram svg { display: block; width: 100%; height: auto; }
|
.focus svg { display: block; width: 100%; height: auto; }
|
||||||
.echogram figcaption {
|
.focus .rest-node { fill: var(--rest); }
|
||||||
|
.focus .bracket { fill: none; stroke: var(--teal); stroke-width: 3; stroke-linecap: round; stroke-linejoin: round; }
|
||||||
|
.focus .finding { fill: var(--amber); }
|
||||||
|
.focus .halo { fill: none; stroke: var(--amber); stroke-width: 2; opacity: .3; }
|
||||||
|
.focus .readout { font-family: var(--mono); font-size: 11px; fill: var(--slate); }
|
||||||
|
.focus .readout .flag { fill: var(--amber); }
|
||||||
|
.focus .lead { stroke: var(--grid); stroke-width: 1; }
|
||||||
|
.focus figcaption {
|
||||||
position: absolute; top: .5rem; left: .75rem;
|
position: absolute; top: .5rem; left: .75rem;
|
||||||
font-family: var(--mono); font-size: .65rem; color: var(--slate);
|
font-family: var(--mono); font-size: .65rem; color: var(--caption);
|
||||||
}
|
}
|
||||||
.sweep {
|
@keyframes examine { 0%, 100% { opacity: .3; } 50% { opacity: .1; } }
|
||||||
position: absolute; top: 0; bottom: 0; width: 1px;
|
.focus .halo { animation: examine 3.2s ease-in-out infinite; }
|
||||||
background: var(--ping); opacity: .8;
|
|
||||||
box-shadow: 0 0 8px var(--ping);
|
|
||||||
animation: sweep 7s linear infinite;
|
|
||||||
}
|
|
||||||
@keyframes sweep { from { left: 0; } to { left: 100%; } }
|
|
||||||
@media (prefers-reduced-motion: reduce) {
|
@media (prefers-reduced-motion: reduce) {
|
||||||
.sweep { animation: none; left: 62%; }
|
.focus .halo { animation: none; }
|
||||||
html { scroll-behavior: auto; }
|
html { scroll-behavior: auto; }
|
||||||
}
|
}
|
||||||
|
|
||||||
section { padding: 3.5rem 0 0; }
|
section { padding: 3.5rem 0 0; }
|
||||||
.eyebrow {
|
.eyebrow {
|
||||||
font-family: var(--mono); font-size: .7rem; letter-spacing: .25em;
|
font-family: var(--mono); font-size: .7rem; letter-spacing: .25em;
|
||||||
text-transform: uppercase; color: var(--ping);
|
text-transform: uppercase; color: var(--teal);
|
||||||
}
|
}
|
||||||
h2 { font-size: 1.35rem; font-weight: 650; margin: .5rem 0 1rem; letter-spacing: -.01em; }
|
h2 { font-size: 1.35rem; font-weight: 650; margin: .5rem 0 1rem; letter-spacing: -.01em; }
|
||||||
section > .col > p { color: var(--slate); max-width: 58ch; }
|
section > .col > p { color: var(--slate); max-width: 58ch; }
|
||||||
@@ -129,21 +150,26 @@
|
|||||||
border: 1px solid var(--grid); border-radius: 3px; padding: .35rem .6rem;
|
border: 1px solid var(--grid); border-radius: 3px; padding: .35rem .6rem;
|
||||||
color: var(--slate);
|
color: var(--slate);
|
||||||
}
|
}
|
||||||
.tier b { color: var(--ok); font-weight: 600; }
|
.tier b { color: var(--teal); font-weight: 600; }
|
||||||
|
|
||||||
/* Install */
|
/* Install */
|
||||||
.buttons { display: flex; gap: .75rem; flex-wrap: wrap; margin: 1.5rem 0 1rem; }
|
.release {
|
||||||
|
font-family: var(--mono); font-size: .8rem; color: var(--slate);
|
||||||
|
margin-top: 1.25rem;
|
||||||
|
}
|
||||||
|
.release b { color: var(--amber); font-weight: 600; }
|
||||||
|
.buttons { display: flex; gap: .75rem; flex-wrap: wrap; margin: 1rem 0 1rem; }
|
||||||
.btn {
|
.btn {
|
||||||
display: inline-block; padding: .7rem 1.3rem; border-radius: 4px;
|
display: inline-block; padding: .7rem 1.3rem; border-radius: 4px;
|
||||||
font-weight: 600; text-decoration: none; font-size: .95rem;
|
font-weight: 600; text-decoration: none; font-size: .95rem;
|
||||||
}
|
}
|
||||||
.btn.primary { background: var(--ping); color: var(--depth-3); }
|
.btn.primary { background: var(--teal); color: var(--on-teal); }
|
||||||
.btn.primary:hover { filter: brightness(1.08); }
|
.btn.primary:hover { filter: brightness(1.08); }
|
||||||
.btn.ghost { border: 1px solid var(--grid); color: var(--foam); }
|
.btn.ghost { border: 1px solid var(--grid); color: var(--foam); }
|
||||||
.btn.ghost:hover { border-color: var(--slate); }
|
.btn.ghost:hover { border-color: var(--slate); }
|
||||||
.note {
|
.note {
|
||||||
font-size: .85rem; color: var(--slate);
|
font-size: .85rem; color: var(--slate);
|
||||||
border-left: 2px solid var(--ping); padding-left: .9rem; max-width: 52ch;
|
border-left: 2px solid var(--teal); padding-left: .9rem; max-width: 52ch;
|
||||||
}
|
}
|
||||||
.checksum { font-family: var(--mono); font-size: .75rem; color: var(--slate); margin-top: 1rem; }
|
.checksum { font-family: var(--mono); font-size: .75rem; color: var(--slate); margin-top: 1rem; }
|
||||||
|
|
||||||
@@ -157,45 +183,51 @@
|
|||||||
</head>
|
</head>
|
||||||
<body>
|
<body>
|
||||||
|
|
||||||
<div class="ruler" aria-hidden="true">
|
<div class="rule" aria-hidden="true"></div>
|
||||||
<span style="top:6%">0 m</span>
|
|
||||||
<span style="top:28%">─ 20</span>
|
|
||||||
<span style="top:50%">─ 40</span>
|
|
||||||
<span style="top:72%">─ 60</span>
|
|
||||||
<span style="top:94%">─ 80</span>
|
|
||||||
</div>
|
|
||||||
|
|
||||||
<header class="hero">
|
<header class="hero">
|
||||||
<div class="col">
|
<div class="col">
|
||||||
<p class="wordmark"><b>●</b> echo·lot <span aria-hidden="true">/ˈɛçolo:t/ — echo sounder</span></p>
|
<picture>
|
||||||
<h1>Depth soundings for your local network.</h1>
|
<source srcset="/assets/wordmark-on-dark.svg" media="(prefers-color-scheme: dark)">
|
||||||
<p class="lede">An echo sounder maps the seabed by timing returns. <strong>Echolot</strong> does the
|
<img class="wordmark" src="/assets/wordmark-on-light.svg" alt="echolot" width="125" height="30">
|
||||||
same to your network: free Android diagnostics for the layer where things actually break —
|
</picture>
|
||||||
|
<h1>Measure, don't guess.</h1>
|
||||||
|
<p class="lede">Free Android diagnostics for the layer where networks actually break.
|
||||||
|
<strong>Echolot</strong> takes the failure you can feel and pins it to a fact you can show —
|
||||||
<strong>no root required</strong>. Built for people who know what a neighbor table is.</p>
|
<strong>no root required</strong>. Built for people who know what a neighbor table is.</p>
|
||||||
|
|
||||||
<figure class="echogram">
|
<figure class="focus">
|
||||||
<figcaption>trace · icmp.ping4 · rtt ms ↓ / t →</figcaption>
|
<figcaption>focus · dhcp.rogue_detect · tier:app</figcaption>
|
||||||
<svg viewBox="0 0 720 190" role="img" aria-label="Chart-recorder style trace of ping round-trip times, drawn like a sonar seabed profile">
|
<svg viewBox="0 0 720 240" role="img" aria-label="A grid of network nodes at rest; one node is framed by viewfinder brackets, highlighted as a finding: two DHCP servers answered the same DISCOVER">
|
||||||
<!-- echo returns: the profile -->
|
<!-- the network, at rest -->
|
||||||
<polyline fill="none" stroke="#FFB454" stroke-width="1.5" opacity=".9"
|
<g class="rest-node">
|
||||||
points="0,138 40,136 80,139 120,135 160,137 200,141 240,138 260,120 280,96 300,88 320,94 340,118 360,134 400,136 440,133 480,158 500,171 520,168 540,150 560,139 600,137 640,140 680,136 720,138"/>
|
<circle cx="120" cy="60" r="3"/><circle cx="240" cy="60" r="3"/><circle cx="360" cy="60" r="3"/><circle cx="480" cy="60" r="3"/><circle cx="600" cy="60" r="3"/>
|
||||||
<!-- second, fainter return (multipath) -->
|
<circle cx="120" cy="120" r="3"/><circle cx="480" cy="120" r="3"/><circle cx="600" cy="120" r="3"/>
|
||||||
<polyline fill="none" stroke="#FFB454" stroke-width="1" opacity=".25"
|
<circle cx="120" cy="180" r="3"/><circle cx="240" cy="180" r="3"/><circle cx="360" cy="180" r="3"/><circle cx="480" cy="180" r="3"/><circle cx="600" cy="180" r="3"/>
|
||||||
points="0,148 40,146 80,149 120,145 160,147 200,151 240,148 260,132 280,110 300,101 320,107 340,129 360,144 400,146 440,143 480,168 500,180 520,177 540,160 560,149 600,147 640,150 680,146 720,148"/>
|
</g>
|
||||||
<!-- dropped probes -->
|
<!-- one node, under examination -->
|
||||||
<g fill="#8AA5B8" font-family="monospace" font-size="9">
|
<circle class="halo" cx="240" cy="120" r="11"/>
|
||||||
<text x="497" y="30">×</text><text x="507" y="30">×</text>
|
<circle class="finding" cx="240" cy="120" r="5"/>
|
||||||
<text x="288" y="30">▲ spike: wifi→cell handover</text>
|
<g class="bracket">
|
||||||
|
<path d="M 222 111 V 102 H 231"/>
|
||||||
|
<path d="M 249 102 H 258 V 111"/>
|
||||||
|
<path d="M 258 129 V 138 H 249"/>
|
||||||
|
<path d="M 231 138 H 222 V 129"/>
|
||||||
|
</g>
|
||||||
|
<!-- the readout -->
|
||||||
|
<line class="lead" x1="262" y1="120" x2="296" y2="120"/>
|
||||||
|
<g class="readout">
|
||||||
|
<text x="304" y="112">DISCOVER → 2 OFFERs</text>
|
||||||
|
<text x="304" y="130">192.168.1.1 gw · <tspan class="flag">192.168.1.223 — who is this?</tspan></text>
|
||||||
</g>
|
</g>
|
||||||
</svg>
|
</svg>
|
||||||
<div class="sweep" aria-hidden="true"></div>
|
|
||||||
</figure>
|
</figure>
|
||||||
</div>
|
</div>
|
||||||
</header>
|
</header>
|
||||||
|
|
||||||
<section id="what">
|
<section id="what">
|
||||||
<div class="col">
|
<div class="col">
|
||||||
<p class="eyebrow">What it sounds out</p>
|
<p class="eyebrow">What it measures</p>
|
||||||
<h2>Signal bars lie. Timings don't.</h2>
|
<h2>Signal bars lie. Timings don't.</h2>
|
||||||
<p>Most wifi apps show you signal strength and call it a diagnosis. The failures that ruin
|
<p>Most wifi apps show you signal strength and call it a diagnosis. The failures that ruin
|
||||||
home and office networks live deeper: a second DHCP server nobody admits to, IPv6 router
|
home and office networks live deeper: a second DHCP server nobody admits to, IPv6 router
|
||||||
@@ -234,15 +266,16 @@
|
|||||||
<div class="col">
|
<div class="col">
|
||||||
<p class="eyebrow">Install</p>
|
<p class="eyebrow">Install</p>
|
||||||
<h2>Get Echolot</h2>
|
<h2>Get Echolot</h2>
|
||||||
|
<p class="release" id="release" hidden></p>
|
||||||
<div class="buttons">
|
<div class="buttons">
|
||||||
<a class="btn primary" href="/apk">Download APK</a>
|
<a class="btn primary" id="dl-btn" href="/apk">Download APK</a>
|
||||||
<a class="btn ghost" href="/fdroid">F-Droid</a>
|
<a class="btn ghost" href="/fdroid">F-Droid</a>
|
||||||
</div>
|
</div>
|
||||||
<p class="note"><strong>Pre-release.</strong> The capability prober is running on real
|
<p class="note" id="prerelease-note"><strong>Pre-release.</strong> The capability prober is
|
||||||
hardware; the production app is under construction. These links go live with the first
|
running on real hardware; the production app is under construction. These links go live with
|
||||||
release — until then they loop back here. No mailing list, no tracker: check back, or watch
|
the first release — until then they loop back here. No mailing list, no tracker: check back,
|
||||||
the <a href="/source">repository</a>.</p>
|
or watch the <a href="/source">repository</a>.</p>
|
||||||
<p class="checksum">releases will ship with sha256sums + a signing key you can pin</p>
|
<p class="checksum" id="checksum">releases will ship with sha256sums + a signing key you can pin</p>
|
||||||
</div>
|
</div>
|
||||||
</section>
|
</section>
|
||||||
|
|
||||||
@@ -253,5 +286,41 @@
|
|||||||
</div>
|
</div>
|
||||||
</footer>
|
</footer>
|
||||||
|
|
||||||
|
<script>
|
||||||
|
// Release readout: asks this site's own Worker (/api/latest, which proxies the
|
||||||
|
// Gitea "latest release" API, edge-cached). Progressive enhancement — with no
|
||||||
|
// JS, no network, or no release yet, the static pre-release copy above stands.
|
||||||
|
(async () => {
|
||||||
|
let rel;
|
||||||
|
try {
|
||||||
|
const res = await fetch("/api/latest");
|
||||||
|
if (!res.ok) return;
|
||||||
|
rel = await res.json();
|
||||||
|
} catch { return; }
|
||||||
|
if (!rel || !rel.available) return;
|
||||||
|
|
||||||
|
const line = document.getElementById("release");
|
||||||
|
const ver = document.createElement("b");
|
||||||
|
ver.textContent = rel.version;
|
||||||
|
line.append("» latest ", ver);
|
||||||
|
const date = (rel.published_at || "").slice(0, 10);
|
||||||
|
if (date) line.append(" · " + date);
|
||||||
|
if (rel.apk && rel.apk.size) {
|
||||||
|
line.append(" · " + (rel.apk.size / 1048576).toFixed(1) + " MiB");
|
||||||
|
}
|
||||||
|
line.hidden = false;
|
||||||
|
|
||||||
|
document.getElementById("prerelease-note").hidden = true;
|
||||||
|
if (rel.sha256) {
|
||||||
|
const c = document.getElementById("checksum");
|
||||||
|
c.textContent = "";
|
||||||
|
const a = document.createElement("a");
|
||||||
|
a.href = "/apk.sha256";
|
||||||
|
a.textContent = "sha256";
|
||||||
|
c.append(a, " · verify before you sideload");
|
||||||
|
}
|
||||||
|
})();
|
||||||
|
</script>
|
||||||
|
|
||||||
</body>
|
</body>
|
||||||
</html>
|
</html>
|
||||||
|
|||||||
+47
-14
@@ -3,28 +3,32 @@
|
|||||||
|
|
||||||
// Everything under public/ is served straight from the edge without invoking
|
// Everything under public/ is served straight from the edge without invoking
|
||||||
// this Worker. The Worker exists for the short stable URLs (/apk, /fdroid,
|
// this Worker. The Worker exists for the short stable URLs (/apk, /fdroid,
|
||||||
// /source) — short enough for a QR code — and to resolve "/apk" to the newest
|
// /source) — short enough for a QR code — and for /api/latest, which the
|
||||||
// release asset at request time, so tagging a release in Gitea is the only
|
// homepage uses to show the current version. Both resolve the newest release
|
||||||
// publish step. No site redeploy, no URL to update.
|
// from the Gitea API at request time, so tagging a release in Gitea is the
|
||||||
|
// only publish step. No site redeploy, no URL to update.
|
||||||
|
|
||||||
const STATIC_ROUTES = {
|
const STATIC_ROUTES = {
|
||||||
"/fdroid": "FDROID_URL",
|
"/fdroid": "FDROID_URL",
|
||||||
"/source": "SOURCE_URL",
|
"/source": "SOURCE_URL",
|
||||||
};
|
};
|
||||||
|
|
||||||
// Resolve the newest APK from the Gitea "latest release" API. Cached at the
|
// Fetch the Gitea "latest release" object. Cached at the edge for 5 minutes so
|
||||||
// edge for 5 minutes so a release becomes visible quickly, while Gitea sees
|
// a new release becomes visible quickly, while Gitea sees at most one API hit
|
||||||
// at most one API hit per POP per 5 min regardless of download traffic.
|
// per POP per 5 min regardless of traffic. Returns null on any failure —
|
||||||
async function latestApkUrl(env) {
|
// callers degrade to fallbacks rather than surfacing errors.
|
||||||
|
async function latestRelease(env) {
|
||||||
if (!env.GITEA_REPO_API) return null;
|
if (!env.GITEA_REPO_API) return null;
|
||||||
const res = await fetch(`${env.GITEA_REPO_API}/releases/latest`, {
|
const res = await fetch(`${env.GITEA_REPO_API}/releases/latest`, {
|
||||||
headers: { Accept: "application/json", "User-Agent": "echolot-site" },
|
headers: { Accept: "application/json", "User-Agent": "echolot-site" },
|
||||||
cf: { cacheTtl: 300, cacheEverything: true },
|
cf: { cacheTtl: 300, cacheEverything: true },
|
||||||
});
|
});
|
||||||
if (!res.ok) return null;
|
if (!res.ok) return null;
|
||||||
const rel = await res.json();
|
return res.json();
|
||||||
const apk = rel.assets?.find((a) => a.name?.endsWith(".apk"));
|
}
|
||||||
return apk?.browser_download_url ?? null;
|
|
||||||
|
function asset(rel, suffix) {
|
||||||
|
return rel?.assets?.find((a) => a.name?.endsWith(suffix)) ?? null;
|
||||||
}
|
}
|
||||||
|
|
||||||
function redirect(location) {
|
function redirect(location) {
|
||||||
@@ -38,21 +42,50 @@ function redirect(location) {
|
|||||||
});
|
});
|
||||||
}
|
}
|
||||||
|
|
||||||
|
function json(body, maxAge) {
|
||||||
|
return new Response(JSON.stringify(body), {
|
||||||
|
headers: {
|
||||||
|
"Content-Type": "application/json",
|
||||||
|
"Cache-Control": `public, max-age=${maxAge}`,
|
||||||
|
},
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
export default {
|
export default {
|
||||||
async fetch(request, env) {
|
async fetch(request, env) {
|
||||||
const { pathname } = new URL(request.url);
|
const { pathname } = new URL(request.url);
|
||||||
const path = pathname.replace(/\/$/, "");
|
const path = pathname.replace(/\/$/, "");
|
||||||
const fallback = new URL("/#install", request.url).toString();
|
const fallback = new URL("/#install", request.url).toString();
|
||||||
|
|
||||||
if (path === "/apk" || path === "/download") {
|
if (path === "/apk" || path === "/download" || path === "/apk.sha256") {
|
||||||
// Order: live Gitea release → manual override → install section.
|
const suffix = path === "/apk.sha256" ? ".sha256" : ".apk";
|
||||||
let target = null;
|
let target = null;
|
||||||
try {
|
try {
|
||||||
target = await latestApkUrl(env);
|
target = asset(await latestRelease(env), suffix)?.browser_download_url;
|
||||||
} catch {
|
} catch {
|
||||||
// Gitea unreachable — fall through rather than 500 on a download link.
|
// Gitea unreachable — fall through rather than 500 on a download link.
|
||||||
}
|
}
|
||||||
return redirect(target || env.DOWNLOAD_URL || fallback);
|
const override = suffix === ".apk" ? env.DOWNLOAD_URL : null;
|
||||||
|
return redirect(target || override || fallback);
|
||||||
|
}
|
||||||
|
|
||||||
|
if (path === "/api/latest") {
|
||||||
|
let rel = null;
|
||||||
|
try {
|
||||||
|
rel = await latestRelease(env);
|
||||||
|
} catch {}
|
||||||
|
const apk = asset(rel, ".apk");
|
||||||
|
if (!rel || !apk) return json({ available: false }, 60);
|
||||||
|
return json(
|
||||||
|
{
|
||||||
|
available: true,
|
||||||
|
version: rel.tag_name,
|
||||||
|
published_at: rel.published_at,
|
||||||
|
apk: { name: apk.name, size: apk.size, url: apk.browser_download_url },
|
||||||
|
sha256: Boolean(asset(rel, ".sha256")),
|
||||||
|
},
|
||||||
|
300,
|
||||||
|
);
|
||||||
}
|
}
|
||||||
|
|
||||||
const varName = STATIC_ROUTES[path];
|
const varName = STATIC_ROUTES[path];
|
||||||
|
|||||||
+5
-5
@@ -22,14 +22,14 @@
|
|||||||
// its route fall back to the homepage's install section, so nothing 404s
|
// its route fall back to the homepage's install section, so nothing 404s
|
||||||
// before the first release exists.
|
// before the first release exists.
|
||||||
"vars": {
|
"vars": {
|
||||||
// Gitea repo API base, e.g. "https://git.example.net/api/v1/repos/mram/echolot".
|
// Gitea repo API base. The repo (or at least its releases) must be
|
||||||
// The repo (or at least its releases) must be publicly readable.
|
// publicly readable for /apk and /api/latest to resolve.
|
||||||
"GITEA_REPO_API": "",
|
"GITEA_REPO_API": "https://git.rambossek.at/api/v1/repos/EchoLot/echolot",
|
||||||
// Manual override / fallback while GITEA_REPO_API is unset or unreachable.
|
// Manual override / fallback while GITEA_REPO_API is unreachable.
|
||||||
"DOWNLOAD_URL": "",
|
"DOWNLOAD_URL": "",
|
||||||
// F-Droid listing, once it exists: https://f-droid.org/packages/app.echo_lot.app/
|
// F-Droid listing, once it exists: https://f-droid.org/packages/app.echo_lot.app/
|
||||||
"FDROID_URL": "",
|
"FDROID_URL": "",
|
||||||
// Public source URL, the footer + /source target.
|
// Public source URL, the footer + /source target.
|
||||||
"SOURCE_URL": ""
|
"SOURCE_URL": "https://git.rambossek.at/EchoLot/echolot"
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
Reference in New Issue
Block a user