Compare commits

..
Author SHA1 Message Date
mrambossekandClaude Fable 5 a7dccf7da2 frag_send: crafted IP fragments, so ordering can be tested and not just delivery
server-release / image (push) Successful in 15s
server-test / test (push) Successful in 32s
server-release / release (push) Successful in 32s
Letting the kernel fragment an oversized datagram answers one question — do
fragments get through. It cannot answer the more interesting one, because the
kernel always emits them in order, first one first.

The classic middlebox fault is exactly about that ordering. Only the first
fragment carries the UDP header, and therefore the ports; a stateful firewall
or NAT that has not seen it has no flow to match the rest against, and many
drop them. That is invisible to any in-order test and shows up in the field as
"large DNS answers fail on this network" or "the tunnel breaks when the MTU
drops" — it works until the network reorders, then fails intermittently, which
is the hardest kind of fault to chase.

So the server now builds the fragments itself (raw socket, IP_HDRINCL) and
controls their order: in_order as a baseline, reversed, and first-fragment-last.
The datagram is assembled and signed whole before being cut up, so what the
client reassembles is indistinguishable from an ordinary packet — otherwise it
would be measuring our sender rather than the path.

Two details that would silently produce wrong answers:
  - The UDP checksum is computed rather than left zero. A zero-checksum datagram
    is dropped by some middleboxes, and that drop would be recorded as a
    fragmentation failure, which is the wrong conclusion entirely.
  - Fragment offsets are in 8-byte units, so non-final fragments are rounded to
    a multiple of 8. A 100-byte fragment is not an error, it is a datagram no
    host will ever reassemble.

frag-send is advertised only when a raw socket can actually be opened — checked
by opening one, since a permission model has more ways to say no than a
capability bit has to say yes.

Fragment header arithmetic is unit-tested (reassembly coverage, MF flags, shared
IP ID, 8-byte offsets, checksum verification), cross-compiled and run on Linux
since the code is build-tagged.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-01 13:45:09 +02:00
mrambossekandClaude Fable 5 4ffa6e4ae2 engine: split packet loss by direction using the server's observations
"3 % loss" sends an engineer looking in both directions at once. The server
records every packet it received per sequence number, so the two cases are
distinguishable: sent-but-never-seen is upstream loss, seen-but-no-reply is
downstream. The findings say which, and say what is not implicated.

Downstream loss is measured against what reached the server, not against what
was sent — the other denominator counts every upstream loss twice and
overstates the return path.

Per-direction jitter comes out of the same records without needing synchronised
clocks: (server_rx - client_tx) carries a constant unknown offset, and
differencing successive samples cancels it, so RFC 3393 variation is honestly
attributable to a direction even though absolute latency is not.

Correlation is by wire sequence number, not loop index — the counter is shared
with every packet type on the session. ProbeSession exposes it even for a lost
probe, since that is precisely the packet whose direction is in question.

Live against fmr: 0.08 ms upstream jitter vs 0.85 ms downstream, an asymmetry a
round-trip test cannot see.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-01 13:09:59 +02:00
mrambossekandClaude Fable 5 3e7e3b8d33 scripts: one command to mint an enrollment link, QR included
Scanning beats pasting a 200-character string onto a phone, and with a device
attached the deep link can be delivered by adb with no typing at all.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-01 12:12:28 +02:00
mrambossekandClaude Fable 5 199807a8c9 docs: enrollment link encoding rules in the spec, session log in build-status
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-01 12:11:37 +02:00
mrambossekandClaude Fable 5 5291bdd045 enrollment: actually emit enroll_uri from the admin endpoint
server-release / image (push) Successful in 15s
server-test / test (push) Successful in 30s
server-release / release (push) Successful in 31s
The previous commit's edit to the admin handler silently did not apply, so the
endpoint still returned just the token. Caught by deploying and looking at the
response rather than by trusting the build to have picked it up.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-01 12:08:11 +02:00
mrambossekandClaude Fable 5 fe3658e009 chore: ignore the Kotlin compiler's .kotlin scratch directory
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-01 12:06:47 +02:00
mrambossekandClaude Fable 5 ad85f3bfcd enrollment: the server mints the §2.1 bootstrap link, the app consumes it
server-release / image (push) Successful in 15s
server-test / test (push) Successful in 29s
server-release / release (push) Successful in 31s
POST /admin/enroll-tokens now returns the whole link, not just the token:

  echolot://enroll?v=1&u=<control URL>&p=pin-sha256:<b64>&t=<token>

The server is the only party that knows all three parts at once, and the part
an operator gets wrong by hand is the base64 pin — which does not fail loudly,
it just never matches, surfacing days later as an inscrutable TLS error. The
app takes the link from a paste or from an echolot:// deep link (QR scan), and
writes URL, pin and credential together or not at all.

One trap the tests pin: an unencoded "+" in a query string decodes to a space,
so a hand-assembled link arrives with a pin wrong by one character. Base64 has
no spaces, so they are restored — unambiguous, and it cannot damage a correctly
encoded pin.

Also fixes a spec divergence: §2.1 names the field device_credential and the
first implementation shipped "credential". Both are sent now and the client
prefers the spec's; the alias goes once nothing reads it.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-01 12:06:22 +02:00
mrambossekandClaude Fable 5 8166611af1 docs: record the compatibility-window work in build-status
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-01 11:43:02 +02:00
mrambossekandClaude Fable 5 33a6acb0bf compat: stop mangling refusal messages with HTML escapes
server-release / image (push) Successful in 14s
server-test / test (push) Successful in 29s
server-release / release (push) Successful in 31s
The server's 426 body reached the user as "needs \u003e= 0.2.0, \u003c 1.0.0":
Go escapes <, > and & by default for JSON destined for a page, which this is
not. Disabled at the encoder. The client now parses the error field rather than
pattern-matching it, so it survives whatever a future encoder decides to escape.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-01 11:41:00 +02:00
mrambossekandClaude Fable 5 9d6572bc33 compat: fix the too-new message's grammar, add a live gate test
server-release / image (push) Successful in 14s
server-test / test (push) Successful in 29s
server-release / release (push) Successful in 30s
The generated refusal read "point at a app within range". Also adds
LiveCompatTest, which checks the half a unit test cannot reach: that two
independently-built artifacts agree on the window, that the profile stays
readable for a version the server refuses, and that both bounds are enforced.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-01 11:38:45 +02:00
34 changed files with 1804 additions and 37 deletions
+3
View File
@@ -43,3 +43,6 @@ web/.wrangler/
# Eclipse/JDT output from the VSCodium Java extension — not a build artifact we own # Eclipse/JDT output from the VSCodium Java extension — not a build artifact we own
echolot-app/*/bin/ echolot-app/*/bin/
# Kotlin compiler scratch/error logs
echolot-app/.kotlin/
+8
View File
@@ -162,3 +162,11 @@ First build downloads AGP/Compose/Shizuku from Google Maven + Maven Central.
are SUPPORTED on both known devices via `Os.recvmsg` + `StructMsghdr` reflection. are SUPPORTED on both known devices via `Os.recvmsg` + `StructMsghdr` reflection.
3. Fold the confirmed capabilities + Shizuku dump-format samples back into the production 3. Fold the confirmed capabilities + Shizuku dump-format samples back into the production
`core-probe` / `core-shizuku` modules. `core-probe` / `core-shizuku` modules.
## Enrolling a device with a server
`echolot-app/scripts/enroll-link.sh [note]` mints a §2.1 bootstrap link on fmr over SSH and prints
it (plus a QR if `qrencode` is installed, plus the `adb shell am start -a …VIEW -d '<uri>'` command
when a device is attached). The link carries a single-use token — treat it as a secret until spent.
Never hand-assemble one: the base64 pin needs percent-encoding, and a pin wrong by one character
fails as an inscrutable TLS error rather than as a bad pin.
+107
View File
@@ -621,3 +621,110 @@ finding code and the metrics survive, then deleted it.
- Accounts/OIDC on the server, which is what `uploads=account` is waiting for. - Accounts/OIDC on the server, which is what `uploads=account` is waiting for.
- Nothing in this entry has been exercised on a phone yet — all of it was verified from the PC - Nothing in this entry has been exercised on a phone yet — all of it was verified from the PC
against the live server. On-device verification is the next step. against the live server. On-device verification is the next step.
### SemVer compatibility windows between app and server (server-v0.5.0 … v0.5.2, 2026-08-01)
Both artifacts are SemVer, and each now declares — and enforces — which peer versions it will talk
to. Spec: `docs/probe-protocol.md` §8.
**Two axes, deliberately not conflated.** Release versions are a *proxy* for what actually has to
match, so the real thing is checked first:
- `protocol_version` — **can** these builds talk. Advertised in the profile; a peer in a different
breaking series is refused whatever its release version says. Below 1.0.0 the **minor** is the
breaking axis (SemVer §4).
- release-version window — **may** they, per policy. `[min, max)`, min inclusive, max exclusive,
because the useful bound is always "the version that broke it".
Bounds sit at breaking boundaries, not at releases, so shipping a patch never requires editing a
range. The app requires server `>= 0.4.2` for a stated reason, not caution: earlier multi-homed
servers mis-addressed granted sends and the client measured 100 % downstream loss that never
happened. Operators override the server side with `ECHOLOT_MIN_APP_VERSION` /
`ECHOLOT_MAX_APP_VERSION`; a malformed bound is fatal at startup rather than ignored, so a typo
cannot silently disable a restriction.
Three rules that shaped the implementation:
1. **`GET /v1/profile` is never gated.** It is where a refused client learns which version it needs;
gating it leaves the user with a network error instead of an answer.
2. **An unparseable or absent version is `unknown`, and is allowed.** Dev builds report `dev`, and a
client too old to send the header cannot be identified anyway.
3. **Refusal is 426 with a body naming both versions and the window**, surfaced client-side as a
distinct `VersionRefused` rather than folded into "network error".
The app's `versionCode` is now derived from its SemVer (`major*1e6 + minor*1e4 + patch*10`) instead
of being a second number to remember.
Verified live against fmr (`LiveCompatTest`): profile advertises the window and stays readable for a
refused version; 0.1.0 and 99.0.0 are both refused with actionable messages; 0.2.0 and a missing
header are both served.
One user-visible bug caught in the process: Go's JSON encoder HTML-escapes `<`, `>` and `&` by
default, so the refusal reached the client as `needs \u003e= 0.2.0`. Disabled at the encoder (this
is an API, not a page), and the client now *parses* the error field instead of pattern-matching it,
so it survives whatever a future encoder decides to escape.
### Enrollment: the server mints the bootstrap link (server-v0.5.3 … v0.5.4, 2026-08-01)
Until now a device was configured by hand-typing a control URL, a base64 SPKI pin and a
credential. That is the step that goes wrong, and it goes wrong quietly: a pin off by one
character does not fail loudly, it just never matches, and surfaces days later as an inscrutable
TLS error.
`POST /admin/enroll-tokens` now returns the whole §2.1 bootstrap link alongside the token, because
the server is the only party holding all three parts at once. The app takes it from a paste or an
`echolot://enroll` deep link (so a QR scan configures a server in one action) and writes URL, pin
and credential **together or not at all** — a half-applied server fails later, somewhere else,
with an error pointing at the wrong thing.
The control URL comes from `ECHOLOT_PUBLIC_URL` (set on fmr to `https://fmr-1.echo-lot.app:8443`),
falling back to the first control listen address; a wildcard bind warns rather than emitting a
link to `0.0.0.0`.
**The encoding trap, which is the whole reason this is tested across both languages.** The pin is
base64, so it contains `+`, `/` and `=` — each of which means something else in a query string. An
unencoded `+` decodes to a space, leaving the pin wrong by exactly one character. Base64 has no
spaces, so the parser restores them; that cannot damage a correctly-encoded pin and it rescues
every hand-assembled link. `LiveEnrollmentTest` redeems a link the *server* produced, which is the
only way to catch a disagreement between the Go assembler and the Kotlin parser — a unit test on
either side alone cannot see it. It also asserts the token is refused the second time.
Also fixed a spec divergence found while reading §2.1: the spec names the field
`device_credential`, the first implementation shipped `credential`. The server now sends both and
the client prefers the spec's; the alias goes once nothing reads it.
Two process notes from this round:
- An edit to the admin handler silently failed to apply and the endpoint kept returning just the
token. Caught by deploying and *looking at the response*, not by trusting a green build.
- The live suite is now six tests (`LiveServerTest`, `LiveMeasurement`, `LiveGranted`,
`LiveUpload`, `LiveCompat`, `LiveEnrollment`), all green against fmr from the PC with no device.
### Directional loss: which way is the packet loss? (2026-08-01)
A round trip can only report that *something* was lost somewhere, which is the least useful form
of the answer — "3 % loss" sends an engineer looking in both directions at once. The server
already records every packet it received per sequence number (§6), so the two cases are actually
distinguishable, and `train.udp_updown` now reports them separately:
- sent, never seen by the server → **upstream** loss
- seen by the server, reply never arrived → **downstream** loss
Findings name the direction and say what is *not* implicated, which is half the value:
`connectivity.loss_upstream` ("the return path is not implicated: replies came back for everything
that arrived"), `connectivity.loss_downstream`, `nat.udp_unreachable_upstream`.
Two things the implementation gets deliberately right:
- **Downstream loss is measured against what reached the server**, not against what was sent.
Using "sent" as the denominator counts every upstream loss a second time and overstates the
return path. Pinned by a test with loss in both directions at once.
- **Per-direction jitter without synchronised clocks.** Absolute one-way delay would need clock
sync and we deliberately have none (the two-clock rule). But `server_rx client_tx` carries a
constant unknown offset, and differencing successive samples cancels it — so RFC 3393 one-way
delay variation *is* honestly attributable to a direction even though latency is not. A test
pins that a 10-second clock offset changes nothing.
Correlation is by **wire sequence number**, which is not the loop index: the counter is shared
with every other packet type on the session, so "the nth echo" is not "sequence n". `ProbeSession`
now exposes `lastSeq`, including for a probe that was lost — a lost packet still has a sequence
number, and that number is exactly what tells you which way it was lost.
Live against fmr: 20/20 both ways, and jitter of **0.08 ms upstream vs 0.85 ms downstream** — a
tenfold asymmetry that a round-trip measurement cannot see at all.
10 unit tests on the arithmetic (a wrong denominator here does not crash, it produces a plausible
number pointing at the wrong half of the network) plus the live correlation check.
+26 -3
View File
@@ -24,11 +24,34 @@ echolot://enroll?v=1&u=<control-URL, urlencoded>&p=pin-sha256:<b64 SPKI hash>&t=
``` ```
POST /v1/enroll Authorization: Bearer <enrollment-token> POST /v1/enroll Authorization: Bearer <enrollment-token>
→ 200 { "device_credential": "<random 256-bit, b64url>", → 201 { "device_credential": "<random 256-bit, b64url>",
"device_id": "uuid", "device_id": "uuid" }
"profile": { ... §2.2 ... } }
``` ```
The **server assembles the bootstrap link**, because it is the only party holding all three parts
at once, and the part an operator gets wrong by hand is the base64 pin — which does not fail
loudly, it just never matches, and surfaces later as an inscrutable TLS error:
```
POST /admin/enroll-tokens
→ { "token": "…", "expires_in_s": 86400,
"enroll_uri": "echolot://enroll?v=1&u=…&p=…&t=…" }
```
The control URL in the link comes from `ECHOLOT_PUBLIC_URL`, falling back to the first control
listen address. A wildcard bind has no single right answer, so it warns rather than guessing.
Encoding notes that matter in practice:
- `u`, `p` and `t` are **percent-encoded**. The pin is base64, so it contains `+`, `/` and `=`,
every one of which means something else in a query string.
- A `+` that was *not* encoded decodes to a space. Base64 contains no spaces, so a parser SHOULD
restore them — the alternative is a pin wrong by one character and a failure that points nowhere
near the cause.
- The control URL MUST be `https://`. The pin only protects a TLS connection; a cleartext URL
would hand the token to anyone on the path.
- **The link is a secret** while it is live: it carries a bearer token, so anyone who sees it
before the device does can enroll instead.
Enrollment tokens are single-use with expiry, created in the admin UI, scoped `enroll`. The device credential is a long-lived bearer secret, scoped `run-tests`; it is also the HKDF input for session keys. Revocation = deleting the device in the admin UI. Enrollment tokens are single-use with expiry, created in the admin UI, scoped `enroll`. The device credential is a long-lived bearer secret, scoped `run-tests`; it is also the HKDF input for session keys. Revocation = deleting the device in the admin UI.
### 2.2 Profile ### 2.2 Profile
@@ -27,6 +27,18 @@
<action android:name="android.intent.action.MAIN" /> <action android:name="android.intent.action.MAIN" />
<category android:name="android.intent.category.LAUNCHER" /> <category android:name="android.intent.category.LAUNCHER" />
</intent-filter> </intent-filter>
<!--
Enrollment bootstrap (probe-protocol.md §2.1): echolot://enroll?v=1&u=…&p=…&t=…
Scanning a QR or tapping a link the operator sent configures the server in one
action, instead of transcribing a URL, a base64 pin and a token by hand — the pin
in particular fails silently when it is wrong by one character.
-->
<intent-filter android:autoVerify="false">
<action android:name="android.intent.action.VIEW" />
<category android:name="android.intent.category.DEFAULT" />
<category android:name="android.intent.category.BROWSABLE" />
<data android:scheme="echolot" android:host="enroll" />
</intent-filter>
</activity> </activity>
<provider <provider
@@ -59,6 +59,17 @@ class MainActivity : ComponentActivity() {
// starts a run immediately and uploads the report, so an unattended // starts a run immediately and uploads the report, so an unattended
// measurement needs no UI tapping and no adb round-trip to collect. // measurement needs no UI tapping and no adb round-trip to collect.
val autorun = intent?.getBooleanExtra("autorun", false) == true val autorun = intent?.getBooleanExtra("autorun", false) == true
// An echolot://enroll link (QR scan, or a link the operator sent) opens the
// app straight into settings with the enrollment already done, so the user
// sees the result rather than a form they still have to fill in.
val enrollUri = intent?.takeIf { it.action == Intent.ACTION_VIEW }?.dataString
androidx.compose.runtime.LaunchedEffect(enrollUri) {
if (enrollUri != null) {
vm.enroll(enrollUri)
screen = Screen.SETTINGS
}
}
androidx.compose.runtime.LaunchedEffect(autorun) { androidx.compose.runtime.LaunchedEffect(autorun) {
if (autorun) vm.run(devUpload = true) if (autorun) vm.run(devUpload = true)
} }
@@ -89,6 +100,7 @@ class MainActivity : ComponentActivity() {
} }
}, },
onCheckServer = vm::checkServer, onCheckServer = vm::checkServer,
onEnroll = vm::enroll,
serverStatus = vm.state.archiveStatus, serverStatus = vm.state.archiveStatus,
onBack = { screen = Screen.RUN }, onBack = { screen = Screen.RUN },
) )
@@ -108,6 +108,35 @@ class RunStore(context: Context, private val settings: Settings) {
} }
} }
/**
* Redeems an enrollment link and stores the resulting server configuration (§2.1).
*
* Everything is written at once or not at all: a half-applied server — say a URL and pin with
* no credential — fails later, somewhere else, with an error that points at the wrong thing.
* Blocking; callers run it off the main thread.
*/
fun enroll(link: String, deviceName: String?): String {
val parsed = app.echo_lot.protocol.EnrollmentLink.parse(link)
?: return "That does not look like an Echolot enrollment link. It should start with " +
"echolot://enroll and carry a URL, a pin and a token."
return try {
val enrolled = parsed.redeem(deviceName, BuildConfig.APP_SEMVER)
val compat = Compat.check(enrolled.profile, BuildConfig.APP_SEMVER)
settings.serverUrl = enrolled.controlUrl
settings.serverPin = enrolled.pin
settings.serverCredential = enrolled.credential
val head = "Enrolled with ${enrolled.profile.name} " +
"(server ${enrolled.profile.serverVersion})."
if (compat.message != null) head + " " + compat.message else head
} catch (e: VersionRefused) {
"That server will not serve this app: ${e.message}"
} catch (t: Throwable) {
// The commonest causes are a spent token and a wrong pin, and they look nothing alike
// in the message — so pass it through rather than flattening it to "enrollment failed".
"Enrollment failed: ${t.message ?: t.javaClass.simpleName}"
}
}
/** /**
* Uploads one archived run to the configured server, redacting first. * Uploads one archived run to the configured server, redacting first.
* *
@@ -196,6 +196,14 @@ class RunViewModel(app: Application) : AndroidViewModel(app) {
fun archivedBytes(): Long = store.totalBytes() fun archivedBytes(): Long = store.totalBytes()
/** Redeems an enrollment link, from a paste or from an echolot:// deep link. */
fun enroll(link: String, deviceName: String? = android.os.Build.MODEL) {
viewModelScope.launch {
state = state.copy(archiveStatus = "enrolling …")
state = state.copy(archiveStatus = withContext(Dispatchers.IO) { store.enroll(link, deviceName) })
}
}
/** Settings-screen action: report what the configured server is and whether we can use it. */ /** Settings-screen action: report what the configured server is and whether we can use it. */
fun checkServer() { fun checkServer() {
viewModelScope.launch { viewModelScope.launch {
@@ -47,6 +47,7 @@ fun SettingsScreen(
onDeleteAll: () -> Unit, onDeleteAll: () -> Unit,
onPreviewUpload: () -> Unit, onPreviewUpload: () -> Unit,
onCheckServer: () -> Unit, onCheckServer: () -> Unit,
onEnroll: (String) -> Unit,
serverStatus: String?, serverStatus: String?,
onBack: () -> Unit, onBack: () -> Unit,
) { ) {
@@ -59,6 +60,7 @@ fun SettingsScreen(
var autoUpload by remember { mutableStateOf(settings.autoUpload) } var autoUpload by remember { mutableStateOf(settings.autoUpload) }
var privacy by remember { mutableStateOf(settings.privacyLevel) } var privacy by remember { mutableStateOf(settings.privacyLevel) }
var stableSalt by remember { mutableStateOf(settings.stableSalt) } var stableSalt by remember { mutableStateOf(settings.stableSalt) }
var enrollLink by remember { mutableStateOf("") }
var serverUrl by remember { mutableStateOf(settings.serverUrl) } var serverUrl by remember { mutableStateOf(settings.serverUrl) }
var serverPin by remember { mutableStateOf(settings.serverPin) } var serverPin by remember { mutableStateOf(settings.serverPin) }
var serverCred by remember { mutableStateOf(settings.serverCredential) } var serverCred by remember { mutableStateOf(settings.serverCredential) }
@@ -151,6 +153,32 @@ fun SettingsScreen(
checked = autoUpload, checked = autoUpload,
) { autoUpload = it; settings.autoUpload = it } ) { autoUpload = it; settings.autoUpload = it }
// Enrollment first, because it is the path that works: one link carries the
// URL, the pin and a single-use token. The three fields below exist for when
// someone has to reconstruct a configuration by hand, not as the normal route.
Text(
"Paste an enrollment link from your server operator, or scan its QR code. " +
"It fills in all three fields below. The link contains a one-time token — " +
"treat it like a password until it is used.",
style = MaterialTheme.typography.bodySmall,
)
OutlinedTextField(
value = enrollLink, onValueChange = { enrollLink = it },
label = { Text("echolot://enroll?…") }, singleLine = true,
textStyle = MaterialTheme.typography.bodySmall.copy(fontFamily = FontFamily.Monospace),
modifier = Modifier.fillMaxWidth(),
)
Button(
onClick = {
onEnroll(enrollLink)
enrollLink = "" // spent either way; leaving it around invites a retry
serverUrl = settings.serverUrl
serverPin = settings.serverPin
serverCred = settings.serverCredential
},
enabled = enrollLink.isNotBlank(),
) { Text("Enroll") }
OutlinedTextField( OutlinedTextField(
value = serverUrl, onValueChange = { serverUrl = it; settings.serverUrl = it }, value = serverUrl, onValueChange = { serverUrl = it; settings.serverUrl = it },
label = { Text("Server URL") }, singleLine = true, modifier = Modifier.fillMaxWidth(), label = { Text("Server URL") }, singleLine = true, modifier = Modifier.fillMaxWidth(),
+1 -1
View File
@@ -25,6 +25,6 @@ java { sourceCompatibility = JavaVersion.VERSION_17; targetCompatibility = JavaV
tasks.test { tasks.test {
useJUnitPlatform() useJUnitPlatform()
listOf("ECHOLOT_LIVE_URL","ECHOLOT_LIVE_PIN","ECHOLOT_LIVE_CRED","ECHOLOT_LIVE_UDP","ECHOLOT_LIVE_TARGET") listOf("ECHOLOT_LIVE_URL","ECHOLOT_LIVE_PIN","ECHOLOT_LIVE_CRED","ECHOLOT_LIVE_UDP","ECHOLOT_LIVE_TARGET","ECHOLOT_ENROLL_URI")
.forEach { k -> System.getenv(k)?.let { environment(k, it) } } .forEach { k -> System.getenv(k)?.let { environment(k, it) } }
} }
@@ -0,0 +1,107 @@
// SPDX-FileCopyrightText: 2026 Echolot contributors
// SPDX-License-Identifier: GPL-3.0-or-later
package app.echo_lot.engine
import kotlinx.serialization.SerialName
import kotlinx.serialization.Serializable
/**
* Splits a round-trip train into its two directions using what the server witnessed.
*
* A round trip can only report that *something* was lost somewhere. That is the least useful form
* of the answer: "3 % loss" sends an engineer looking in both directions at once. The server
* records every packet it received, per sequence number (probe-protocol.md §6), so the two cases
* are actually distinguishable:
*
* - sent, never seen by the server → **upstream** loss
* - seen by the server, reply never arrived → **downstream** loss
*
* The same records give one-way delay *variation* per direction. Absolute one-way delay would
* need synchronised clocks and we deliberately have none (measurement-schema.md's two-clock rule),
* but the variation does not: (server_rx client_tx) contains an unknown constant clock offset,
* and differencing successive samples cancels it. So jitter is honestly attributable to a
* direction even though latency is not.
*/
object Directional {
/** One probe as the client saw it. [tRxNs] null means no reply came back. */
data class Sample(val seq: Int, val tTxNs: Long, val tRxNs: Long?)
/** One probe as the server saw it: its own receive and transmit stamps, on its own clock. */
data class ServerSighting(val seq: Int, val tRxNs: Long, val tTxNs: Long)
fun analyse(sent: List<Sample>, seen: List<ServerSighting>): DirectionalMetrics {
val byServerSeq = seen.associateBy { it.seq }
// Only sequences we actually sent count. A server record for a sequence we have no note
// of is not evidence about this train — it is a bug or a stray, and silently folding it
// in would produce loss percentages above 100 or below zero.
val relevant = sent.filter { byServerSeq.containsKey(it.seq) }
val nSent = sent.size
val nSeen = relevant.size
val nReplied = sent.count { it.tRxNs != null }
// A reply can only exist if the request arrived, so downstream loss is measured against
// what the server saw, not against what we sent — otherwise upstream loss is counted twice.
val lostUp = nSent - nSeen
val lostDown = (nSeen - nReplied).coerceAtLeast(0)
val upDeltas = relevant.sortedBy { it.seq }
.map { byServerSeq.getValue(it.seq).tRxNs - it.tTxNs }
val downDeltas = sent.filter { it.tRxNs != null && byServerSeq.containsKey(it.seq) }
.sortedBy { it.seq }
.map { it.tRxNs!! - byServerSeq.getValue(it.seq).tTxNs }
return DirectionalMetrics(
sent = nSent,
seenByServer = nSeen,
repliesReceived = nReplied,
lostUpstream = lostUp,
lostDownstream = lostDown,
lossUpstreamPct = pct(lostUp, nSent),
// Denominator is what reached the server: of the packets that got there, how many
// replies came back.
lossDownstreamPct = pct(lostDown, nSeen),
jitterUpstreamMs = jitterMs(upDeltas),
jitterDownstreamMs = jitterMs(downDeltas),
/** True when the server saw nothing at all, which is a different fault from loss. */
noneReachedServer = nSent > 0 && nSeen == 0,
)
}
/**
* Mean absolute difference between consecutive one-way samples (RFC 3393 IPDV, averaged).
*
* Differencing is what makes this legitimate without synchronised clocks: each sample carries
* the same unknown offset between the two clocks, and the difference cancels it. Fewer than
* two samples yields null rather than zero — "no jitter" and "not enough data to say" are
* different claims and only one of them is true here.
*/
private fun jitterMs(oneWayNs: List<Long>): Double? {
if (oneWayNs.size < 2) return null
val deltas = oneWayNs.zipWithNext { a, b -> kotlin.math.abs(b - a) }
return round2(deltas.average() / 1_000_000.0)
}
private fun pct(part: Int, whole: Int): Double =
if (whole <= 0) 0.0 else round2(part * 100.0 / whole)
private fun round2(v: Double) = Math.round(v * 100.0) / 100.0
}
/** Directional metrics for train.udp_updown; recomputable from the columnar evidence. */
@Serializable
data class DirectionalMetrics(
val sent: Int,
@SerialName("seen_by_server") val seenByServer: Int,
@SerialName("replies_received") val repliesReceived: Int,
@SerialName("lost_upstream") val lostUpstream: Int,
@SerialName("lost_downstream") val lostDownstream: Int,
@SerialName("loss_upstream_pct") val lossUpstreamPct: Double,
@SerialName("loss_downstream_pct") val lossDownstreamPct: Double,
/** One-way delay variation (RFC 3393), per direction. Null when there were too few samples. */
@SerialName("jitter_upstream_ms") val jitterUpstreamMs: Double? = null,
@SerialName("jitter_downstream_ms") val jitterDownstreamMs: Double? = null,
@SerialName("none_reached_server") val noneReachedServer: Boolean = false,
)
@@ -37,6 +37,120 @@ class DownstreamMeasurement(private val ids: IdSource) {
/** How long to wait for a granted burst after the server accepts the action. */ /** How long to wait for a granted burst after the server accepts the action. */
private val collectWindowMs = 4_000L private val collectWindowMs = 4_000L
/**
* Shorter, but long enough to cover the first_last mode's deliberate 250 ms hold plus a
* reassembly. A fragment burst is one datagram: it is here quickly or not at all.
*/
private val fragWindowMs = 1_500L
/**
* Asks the server to send one deliberately-fragmented datagram per ordering, and reports
* which orderings survive the path.
*
* Kernel fragmentation always emits fragments in order, first one first, so an oversized
* datagram can only answer "do fragments get through at all". The interesting fault is about
* ordering: only the *first* fragment carries the UDP ports, so a stateful firewall that has
* not seen it has nothing to match the rest against, and many drop them. That failure is
* invisible to every in-order test and shows up in the field as "large DNS answers fail here"
* or "the tunnel breaks when the MTU drops".
*/
fun fragmentOrdering(
credential: String,
sessionId: String,
control: ControlClient,
probe: ProbeSession,
sessionRef: String,
sizeBytes: Int = 2000,
fragBytes: Int = 576,
): Pair<Test, List<Finding>> {
val testId = ids.uuid()
val started = ids.monoNs()
val delivered = LinkedHashMap<String, Boolean>()
val fragmentCounts = LinkedHashMap<String, Int>()
var unsupported = false
for (mode in FRAG_MODES) {
val reply = runCatching {
control.action(
credential, sessionId,
"""{"action":"frag_send","size_bytes":$sizeBytes,"mode":"$mode","frag_bytes":$fragBytes}""",
)
}
if (reply.isFailure) {
// A server without a raw socket says so; that is a missing capability, not a
// property of the network, and must not be recorded as a failed delivery.
unsupported = true
break
}
parseInt(reply.getOrNull(), "fragments")?.let { fragmentCounts[mode] = it }
// The burst is already on the wire when the action returns (it is sent
// synchronously), so anything that survived is either here or lost.
val got = probe.collectGranted(fragWindowMs).any { it.type == Wire.TYPE_FRAG_DATA }
delivered[mode] = got
}
if (unsupported) {
return Test(
id = testId, type = TestType.MTU_FRAG_ORDERING, sessionRef = sessionRef, tier = Tier.APP,
startedMonoNs = started, endedMonoNs = ids.monoNs(),
status = TestStatus.UNSUPPORTED,
error = TestError("no_raw_socket", "this server cannot craft fragments"),
) to emptyList()
}
val metrics = json.encodeToJsonElement(
FragOrderingMetrics(
sizeBytes = sizeBytes,
fragBytes = fragBytes,
fragmentsPerBurst = fragmentCounts,
deliveredByMode = delivered,
inOrderDelivered = delivered[FRAG_IN_ORDER] == true,
reorderedDelivered = delivered[FRAG_REVERSED] == true,
delayedFirstDelivered = delivered[FRAG_FIRST_LAST] == true,
),
) as JsonObject
val findings = ArrayList<Finding>()
val inOrder = delivered[FRAG_IN_ORDER] == true
val reversed = delivered[FRAG_REVERSED] == true
val firstLast = delivered[FRAG_FIRST_LAST] == true
if (!inOrder) {
findings.add(
finding(
"mtu.fragments_blocked", Category.MTU, Severity.MEDIUM, testId,
"IP fragments do not reach this device",
"A fragmented datagram sent in the normal order never arrived. Anything that " +
"relies on fragmentation — large DNS answers over UDP, some VPN traffic — " +
"will fail here rather than slow down.",
),
)
} else if (!reversed || !firstLast) {
// The precise and useful finding: fragments work, but only if they arrive tidily.
val which = buildList {
if (!reversed) add("out of order")
if (!firstLast) add("with the first fragment delayed")
}.joinToString(" or ")
findings.add(
finding(
"mtu.fragment_reorder_sensitive", Category.MTU, Severity.LOW, testId,
"Fragments are dropped when they arrive $which",
"In-order fragments are delivered, but the same datagram sent $which is not. " +
"Something on the path only reassembles when the first fragment (the one " +
"carrying the UDP ports) arrives first — typical of a stateful firewall " +
"or NAT. It works until the network reorders, then fails intermittently, " +
"which is the hardest kind of fault to chase.",
),
)
}
return Test(
id = testId, type = TestType.MTU_FRAG_ORDERING, sessionRef = sessionRef, tier = Tier.APP,
startedMonoNs = started, endedMonoNs = ids.monoNs(),
status = if (inOrder) TestStatus.OK else TestStatus.PARTIAL,
metrics = metrics,
) to findings
}
/** /**
* Runs all three against an already-primed session. * Runs all three against an already-primed session.
* *
@@ -66,6 +180,16 @@ class DownstreamMeasurement(private val ids: IdSource) {
tests.add(df.test); tests.add(frag.test); tests.add(train.test) tests.add(df.test); tests.add(frag.test); tests.add(train.test)
// Fragment ordering only makes sense once we know fragments arrive at all; when they do
// not, the ordering variants would all report "not delivered" and read as three faults
// instead of one.
if (frag.largestDelivered != null) {
val (fragTest, fragFindings) =
fragmentOrdering(credential, sessionId, control, probe, sessionRef)
tests.add(fragTest)
findings.addAll(fragFindings)
}
// A downstream MTU below the classic 1500-byte Ethernet payload is worth saying out loud: // A downstream MTU below the classic 1500-byte Ethernet payload is worth saying out loud:
// it is the usual cause of "small requests work, large responses hang". // it is the usual cause of "small requests work, large responses hang".
val pathMtu = df.largestDelivered val pathMtu = df.largestDelivered
@@ -310,6 +434,11 @@ class DownstreamMeasurement(private val ids: IdSource) {
/** IPv4 (20) + UDP (8). The v6 case is 48; reported per-family once v6 sessions land. */ /** IPv4 (20) + UDP (8). The v6 case is 48; reported per-family once v6 sessions land. */
const val IP_UDP_OVERHEAD4 = 28 const val IP_UDP_OVERHEAD4 = 28
const val FRAG_IN_ORDER = "in_order"
const val FRAG_REVERSED = "reversed"
const val FRAG_FIRST_LAST = "first_last"
val FRAG_MODES = listOf(FRAG_IN_ORDER, FRAG_REVERSED, FRAG_FIRST_LAST)
/** Straddles the usual suspects: 1500 Ethernet, 1492 PPPoE, 1400-ish tunnels. */ /** Straddles the usual suspects: 1500 Ethernet, 1492 PPPoE, 1400-ish tunnels. */
val DEFAULT_SIZES = listOf(600, 1200, 1372, 1400, 1450, 1472, 1500, 2000, 4000) val DEFAULT_SIZES = listOf(600, 1200, 1372, 1400, 1450, 1472, 1500, 2000, 4000)
@@ -330,6 +459,18 @@ data class BigSendMetrics(
@SerialName("path_mtu_bytes") val pathMtuBytes: Int? = null, @SerialName("path_mtu_bytes") val pathMtuBytes: Int? = null,
) )
/** Metrics for mtu.frag_ordering. */
@Serializable
data class FragOrderingMetrics(
@SerialName("size_bytes") val sizeBytes: Int,
@SerialName("frag_bytes") val fragBytes: Int,
@SerialName("fragments_per_burst") val fragmentsPerBurst: Map<String, Int>,
@SerialName("delivered_by_mode") val deliveredByMode: Map<String, Boolean>,
@SerialName("in_order_delivered") val inOrderDelivered: Boolean,
@SerialName("reordered_delivered") val reorderedDelivered: Boolean,
@SerialName("delayed_first_delivered") val delayedFirstDelivered: Boolean,
)
/** Metrics for train.udp_downstream. */ /** Metrics for train.udp_downstream. */
@Serializable @Serializable
data class DownTrainMetrics( data class DownTrainMetrics(
@@ -10,8 +10,13 @@ import app.echo_lot.measurement.*
import app.echo_lot.protocol.ControlClient import app.echo_lot.protocol.ControlClient
import app.echo_lot.protocol.ProbeSession import app.echo_lot.protocol.ProbeSession
import kotlinx.serialization.json.Json import kotlinx.serialization.json.Json
import kotlinx.serialization.json.JsonArray
import kotlinx.serialization.json.JsonObject import kotlinx.serialization.json.JsonObject
import kotlinx.serialization.json.encodeToJsonElement import kotlinx.serialization.json.encodeToJsonElement
import kotlinx.serialization.json.intOrNull
import kotlinx.serialization.json.jsonObject
import kotlinx.serialization.json.jsonPrimitive
import kotlinx.serialization.json.longOrNull
/** /**
* Runs the server-facing measurements against one target and assembles a [MeasurementDocument]: * Runs the server-facing measurements against one target and assembles a [MeasurementDocument]:
@@ -71,7 +76,7 @@ class ServerMeasurement(
// re-primed source is never recorded and every granted send goes to the old, closed port. // re-primed source is never recorded and every granted send goes to the old, closed port.
// Session identity lives on the server; the socket must live as long as it does. // Session identity lives on the server; the socket must live as long as it does.
ProbeSession(cfg.credential, session, cfg.udpHost, cfg.udpPort).use { ps -> ProbeSession(cfg.credential, session, cfg.udpHost, cfg.udpPort).use { ps ->
val (test, findings) = echoTrain(cfg, ps, startMono) val (test, findings) = echoTrain(cfg, ps, startMono, control, session.sessionId)
tests.add(test) tests.add(test)
allFindings.addAll(findings) allFindings.addAll(findings)
@@ -105,6 +110,7 @@ class ServerMeasurement(
private fun echoTrain( private fun echoTrain(
cfg: Config, ps: ProbeSession, startMono: Long, cfg: Config, ps: ProbeSession, startMono: Long,
control: ControlClient? = null, sessionId: String? = null,
): Pair<Test, List<Finding>> { ): Pair<Test, List<Finding>> {
val testId = ids.uuid() val testId = ids.uuid()
val seqs = ArrayList<Int>() val seqs = ArrayList<Int>()
@@ -114,9 +120,15 @@ class ServerMeasurement(
val rtts = ArrayList<Double>() val rtts = ArrayList<Double>()
val observedPorts = LinkedHashSet<Int>() val observedPorts = LinkedHashSet<Int>()
// Wire sequence numbers, kept so the server's observations can be correlated packet by
// packet. They are not 0..n-1: the counter is shared with every other packet type on the
// session, so "the nth echo" is not "sequence n".
val wireSeqs = ArrayList<Int>()
for (i in 0 until cfg.echoCount) { for (i in 0 until cfg.echoCount) {
val txMono = ids.monoNs() - startMono val txMono = ids.monoNs() - startMono
val r = ps.echo(cfg.echoPaddingBytes) val r = ps.echo(cfg.echoPaddingBytes)
wireSeqs.add(ps.lastSeq)
seqs.add(i) seqs.add(i)
tTx.add(txMono) tTx.add(txMono)
sizes.add(Wire_HEADER + cfg.echoPaddingBytes) sizes.add(Wire_HEADER + cfg.echoPaddingBytes)
@@ -129,6 +141,20 @@ class ServerMeasurement(
} }
} }
// Ask the server what it actually received. This is what turns "3 % loss somewhere" into
// "3 % loss upstream" - the least useful form of the answer into a usable one.
val directional: DirectionalMetrics? =
if (control != null && sessionId != null) {
runCatching {
val samples = wireSeqs.indices.map {
Directional.Sample(wireSeqs[it], tTx[it] ?: 0L, tRx[it])
}
Directional.analyse(samples, serverSightings(control, cfg, sessionId))
}.getOrNull() // an older server without the endpoint simply yields no split
} else {
null
}
val sent = cfg.echoCount val sent = cfg.echoCount
val received = rtts.size val received = rtts.size
val lossPct = if (sent == 0) 0.0 else (sent - received) * 100.0 / sent val lossPct = if (sent == 0) 0.0 else (sent - received) * 100.0 / sent
@@ -138,6 +164,9 @@ class ServerMeasurement(
epochMonoNs = startMono, seq = seqs, tTxNs = tTx, tRxNs = tRx, sizeBytes = sizes, epochMonoNs = startMono, seq = seqs, tTxNs = tTx, tRxNs = tRx, sizeBytes = sizes,
).toEvidence() ).toEvidence()
val directionalJson = directional?.let {
json.encodeToJsonElement(DirectionalMetrics.serializer(), it) as JsonObject
}
val metrics: JsonObject = json.encodeToJsonElement( val metrics: JsonObject = json.encodeToJsonElement(
EchoMetrics( EchoMetrics(
sent = sent, received = received, lossPct = round1(lossPct), sent = sent, received = received, lossPct = round1(lossPct),
@@ -147,7 +176,7 @@ class ServerMeasurement(
observedPorts = observedPorts.toList(), observedPorts = observedPorts.toList(),
natRebindingDetected = natRebinding, natRebindingDetected = natRebinding,
) )
) as JsonObject ).let { base -> JsonObject((base as JsonObject) + (directionalJson ?: JsonObject(emptyMap()))) }
val status = when { val status = when {
received == 0 -> TestStatus.FAILED received == 0 -> TestStatus.FAILED
@@ -170,6 +199,35 @@ class ServerMeasurement(
"High UDP loss to the server (${round1(lossPct)}%)", "High UDP loss to the server (${round1(lossPct)}%)",
"A large fraction of ECHO probes were lost, indicating an unreliable UDP path.")) "A large fraction of ECHO probes were lost, indicating an unreliable UDP path."))
} }
// Naming the direction is the entire value of the split, so the findings do.
directional?.let { d ->
when {
d.noneReachedServer && received == 0 -> findings.add(
finding("nat.udp_unreachable_upstream", Category.CONNECTIVITY, Severity.HIGH, testId,
"Nothing reached the server",
"The server received none of the ${d.sent} probes, so the traffic is being " +
"dropped on the way out, not on the way back. A firewall or NAT on " +
"this side of the path is the place to look."),
)
d.lossUpstreamPct >= 2.0 -> findings.add(
finding("connectivity.loss_upstream", Category.CONNECTIVITY, Severity.MEDIUM, testId,
"${d.lossUpstreamPct} % of probes were lost on the way to the server",
"${d.lostUpstream} of ${d.sent} probes never reached the server. The " +
"return path is not implicated: replies came back for everything that " +
"arrived."),
)
}
if (d.lossDownstreamPct >= 2.0) {
findings.add(
finding("connectivity.loss_downstream", Category.CONNECTIVITY, Severity.MEDIUM, testId,
"${d.lossDownstreamPct} % of replies were lost on the way back",
"The server received ${d.seenByServer} probes and answered them, but " +
"${d.lostDownstream} of those replies never arrived. The outbound path " +
"is fine; the fault is on the return leg."),
)
}
}
if (natRebinding) { if (natRebinding) {
findings.add(finding("nat.udp_rebinding", Category.NAT, Severity.MEDIUM, testId, findings.add(finding("nat.udp_rebinding", Category.NAT, Severity.MEDIUM, testId,
"NAT remapped the UDP source port mid-flow", "NAT remapped the UDP source port mid-flow",
@@ -178,6 +236,29 @@ class ServerMeasurement(
return test to findings return test to findings
} }
/**
* The server's per-packet record of this session's echoes (spec section 6). Filtered to
* ECHO_REQ, because the observation list also holds MTU probes and anything else we sent -
* counting those as train packets would invent loss that is not there.
*/
private fun serverSightings(
control: ControlClient, cfg: Config, sessionId: String,
): List<Directional.ServerSighting> {
val body = control.observations(cfg.credential, sessionId)
val packets = Json.parseToJsonElement(body).jsonObject["udp"]
?.jsonObject?.get("packets") as? JsonArray ?: return emptyList()
return packets.mapNotNull { el ->
val o = el as? JsonObject ?: return@mapNotNull null
val type = o["type"]?.jsonPrimitive?.intOrNull ?: return@mapNotNull null
if (type != ECHO_REQ_TYPE) return@mapNotNull null
Directional.ServerSighting(
seq = o["seq"]?.jsonPrimitive?.intOrNull ?: return@mapNotNull null,
tRxNs = o["t_rx_ns"]?.jsonPrimitive?.longOrNull ?: return@mapNotNull null,
tTxNs = o["t_tx_ns"]?.jsonPrimitive?.longOrNull ?: return@mapNotNull null,
)
}
}
private fun finding(code: String, cat: Category, sev: Severity, testId: String, title: String, desc: String) = private fun finding(code: String, cat: Category, sev: Severity, testId: String, title: String, desc: String) =
Finding( Finding(
id = ids.uuid(), code = code, category = cat, severity = sev, confidence = Confidence.HIGH, id = ids.uuid(), code = code, category = cat, severity = sev, confidence = Confidence.HIGH,
@@ -186,6 +267,7 @@ class ServerMeasurement(
private companion object { private companion object {
const val Wire_HEADER = 32 const val Wire_HEADER = 32
const val ECHO_REQ_TYPE = 0x01
fun round1(v: Double) = Math.round(v * 10.0) / 10.0 fun round1(v: Double) = Math.round(v * 10.0) / 10.0
} }
} }
@@ -0,0 +1,155 @@
// SPDX-FileCopyrightText: 2026 Echolot contributors
// SPDX-License-Identifier: GPL-3.0-or-later
package app.echo_lot.engine
import app.echo_lot.engine.Directional.Sample
import app.echo_lot.engine.Directional.ServerSighting
import kotlin.test.Test
import kotlin.test.assertEquals
import kotlin.test.assertFalse
import kotlin.test.assertNotNull
import kotlin.test.assertNull
import kotlin.test.assertTrue
/**
* The arithmetic that turns "3 % loss somewhere" into "3 % loss upstream". Getting a denominator
* wrong here does not crash anything — it produces a plausible number pointing at the wrong half
* of the network, which is worse than no number at all. Hence a test per claim.
*/
class DirectionalTest {
/** A clean train: every packet sent, seen and answered. Server clock offset by a constant. */
private fun clean(n: Int, offsetNs: Long = 5_000_000_000L): Pair<List<Sample>, List<ServerSighting>> {
val sent = (1..n).map { Sample(it, tTxNs = it * 10_000_000L, tRxNs = it * 10_000_000L + 4_000_000L) }
val seen = (1..n).map {
ServerSighting(it, tRxNs = offsetNs + it * 10_000_000L + 2_000_000L,
tTxNs = offsetNs + it * 10_000_000L + 2_100_000L)
}
return sent to seen
}
@Test
fun aCleanTrainReportsNoLossInEitherDirection() {
val (sent, seen) = clean(10)
val m = Directional.analyse(sent, seen)
assertEquals(10, m.sent)
assertEquals(10, m.seenByServer)
assertEquals(10, m.repliesReceived)
assertEquals(0.0, m.lossUpstreamPct)
assertEquals(0.0, m.lossDownstreamPct)
assertFalse(m.noneReachedServer)
}
// The whole point: a packet the server never saw was lost on the way there.
@Test
fun packetsTheServerNeverSawAreUpstreamLoss() {
val (sent, seen) = clean(10)
val m = Directional.analyse(sent, seen.filter { it.seq !in setOf(3, 7) })
assertEquals(2, m.lostUpstream)
assertEquals(0, m.lostDownstream)
assertEquals(20.0, m.lossUpstreamPct)
assertEquals(0.0, m.lossDownstreamPct, "a packet that never arrived cannot be lost coming back")
}
@Test
fun repliesThatNeverArrivedAreDownstreamLoss() {
val (sent, seen) = clean(10)
val withHoles = sent.map { if (it.seq in setOf(2, 5)) it.copy(tRxNs = null) else it }
val m = Directional.analyse(withHoles, seen)
assertEquals(0, m.lostUpstream)
assertEquals(2, m.lostDownstream)
assertEquals(20.0, m.lossDownstreamPct)
}
// Downstream loss is measured against what actually reached the server. Using "sent" as the
// denominator would count every upstream loss a second time and overstate the return path.
@Test
fun downstreamLossIsRelativeToWhatReachedTheServer() {
val (sent, seen) = clean(10)
// 5 lost on the way there; of the 5 that arrived, 1 reply is lost coming back.
val seenPartial = seen.filter { it.seq > 5 }
val withHole = sent.map {
when {
it.seq <= 5 -> it.copy(tRxNs = null) // never got there, so never came back
it.seq == 6 -> it.copy(tRxNs = null) // arrived, reply lost
else -> it
}
}
val m = Directional.analyse(withHole, seenPartial)
assertEquals(5, m.lostUpstream)
assertEquals(50.0, m.lossUpstreamPct)
assertEquals(1, m.lostDownstream)
assertEquals(20.0, m.lossDownstreamPct, "1 of the 5 that arrived, not 1 of 10")
}
@Test
fun aServerThatSawNothingIsCalledOutSeparately() {
val (sent, _) = clean(6)
val m = Directional.analyse(sent.map { it.copy(tRxNs = null) }, emptyList())
assertTrue(m.noneReachedServer)
assertEquals(100.0, m.lossUpstreamPct)
assertEquals(0.0, m.lossDownstreamPct, "with nothing arriving there is no return path to blame")
}
// Jitter is legitimate without synchronised clocks because the offset cancels when successive
// one-way samples are differenced. This pins that: a huge constant offset must not show up.
@Test
fun jitterIsUnaffectedByTheClockOffsetBetweenTheTwoMachines() {
val (sent, near) = clean(10, offsetNs = 0)
val (_, far) = clean(10, offsetNs = 9_999_999_999L)
val a = Directional.analyse(sent, near)
val b = Directional.analyse(sent, far)
assertEquals(a.jitterUpstreamMs, b.jitterUpstreamMs,
"a constant clock offset must cancel when consecutive samples are differenced")
assertEquals(0.0, assertNotNull(a.jitterUpstreamMs), "an evenly spaced train has no jitter")
}
@Test
fun jitterReflectsUnevenArrival() {
val sent = listOf(
Sample(1, 0, 10_000_000),
Sample(2, 10_000_000, 20_000_000),
Sample(3, 20_000_000, 30_000_000),
)
// Server receive times drift: +2ms, +7ms, +3ms relative to send.
val seen = listOf(
ServerSighting(1, 2_000_000, 2_100_000),
ServerSighting(2, 17_000_000, 17_100_000),
ServerSighting(3, 23_000_000, 23_100_000),
)
val m = Directional.analyse(sent, seen)
// one-way samples: 2ms, 7ms, 3ms → |7-2| and |3-7| → mean 4.5ms
assertEquals(4.5, assertNotNull(m.jitterUpstreamMs))
}
// "No jitter" and "not enough data to say" are different claims, and only one is true here.
@Test
fun tooFewSamplesReportsNoJitterRatherThanZero() {
val m = Directional.analyse(
listOf(Sample(1, 0, 10_000_000)),
listOf(ServerSighting(1, 2_000_000, 2_100_000)),
)
assertNull(m.jitterUpstreamMs)
assertNull(m.jitterDownstreamMs)
}
// A server record for a sequence we never sent is not evidence about this train; folding it
// in would yield loss percentages outside 0100.
@Test
fun strayServerRecordsAreIgnored() {
val (sent, seen) = clean(5)
val m = Directional.analyse(sent, seen + ServerSighting(99, 1, 2) + ServerSighting(100, 3, 4))
assertEquals(5, m.seenByServer)
assertEquals(0.0, m.lossUpstreamPct)
assertTrue(m.lossDownstreamPct in 0.0..100.0)
}
@Test
fun anEmptyTrainDoesNotDivideByZero() {
val m = Directional.analyse(emptyList(), emptyList())
assertEquals(0.0, m.lossUpstreamPct)
assertEquals(0.0, m.lossDownstreamPct)
assertFalse(m.noneReachedServer, "nothing sent is not the same as nothing arriving")
}
}
@@ -0,0 +1,71 @@
// SPDX-FileCopyrightText: 2026 Echolot contributors
// SPDX-License-Identifier: GPL-3.0-or-later
package app.echo_lot.engine
import app.echo_lot.protocol.Compat
import app.echo_lot.protocol.ControlClient
import app.echo_lot.protocol.VersionRefused
import kotlin.test.Test
import kotlin.test.assertEquals
import kotlin.test.assertNotNull
import kotlin.test.assertTrue
import kotlin.test.fail
/**
* Checks the version gate against a LIVE server — the half that unit tests cannot reach, because
* the whole point is that two independently-built artifacts agree. Self-skips without
* ECHOLOT_LIVE_*.
*/
class LiveCompatTest {
private val url = System.getenv("ECHOLOT_LIVE_URL")
private val pin = System.getenv("ECHOLOT_LIVE_PIN")
private val cred = System.getenv("ECHOLOT_LIVE_CRED")
private fun clientAs(version: String) = ControlClient(url!!, setOf(pin!!), version)
@Test
fun theServerAdvertisesAndEnforcesItsWindow() {
if (url == null || pin == null || cred == null) {
println("LiveCompatTest skipped (no ECHOLOT_LIVE_* env)"); return
}
// The profile must state the window — without it the app cannot pre-empt a refusal.
val profile = clientAs("0.2.0").profile(cred)
println("server ${profile.serverVersion} protocol=${profile.compat.protocolVersion} " +
"accepts app [${profile.compat.appMin}, ${profile.compat.appMax})")
assertTrue(profile.compat.protocolVersion.isNotBlank(), "profile omits protocol_version")
assertTrue(profile.compat.appMin.isNotBlank(), "profile omits app_min")
// This build must be inside it, or every other live test here is meaningless.
val verdict = Compat.check(profile, "0.2.0")
assertEquals(Compat.Verdict.OK, verdict.verdict, verdict.message ?: "")
// The profile stays reachable for a version the server would otherwise refuse: that is
// how a refused client discovers what it needs.
val ancient = clientAs("0.1.0")
val stillReadable = ancient.profile(cred)
assertEquals(profile.serverVersion, stillReadable.serverVersion,
"the profile endpoint must never be gated on app version")
// And a gated endpoint refuses it, with a message naming the window.
try {
ancient.createSession(cred, System.getenv("ECHOLOT_LIVE_TARGET") ?: "fmr")
fail("server accepted a session from an out-of-window app")
} catch (e: VersionRefused) {
val msg = assertNotNull(e.message)
println("refused as expected: $msg")
assertTrue(msg.contains("0.1.0"), "refusal should name the offending version: $msg")
assertTrue(msg.contains(profile.compat.appMin), "refusal should name the window: $msg")
}
// Too new is refused the same way — the window is a range, not a floor.
try {
clientAs("99.0.0").createSession(cred, System.getenv("ECHOLOT_LIVE_TARGET") ?: "fmr")
fail("server accepted a session from an app above its window")
} catch (e: VersionRefused) {
println("too-new refused as expected: ${e.message}")
}
}
}
@@ -0,0 +1,63 @@
// SPDX-FileCopyrightText: 2026 Echolot contributors
// SPDX-License-Identifier: GPL-3.0-or-later
package app.echo_lot.engine
import app.echo_lot.protocol.EnrollmentLink
import kotlin.test.Test
import kotlin.test.assertEquals
import kotlin.test.assertNotNull
import kotlin.test.assertTrue
import kotlin.test.fail
/**
* Enrolls against a LIVE server using the link the server itself minted (probe-protocol.md §2.1).
*
* This is the test that matters for enrollment, because the failure mode it guards against is a
* *disagreement* between two programs: the Go side assembles the link, the Kotlin side takes it
* apart, and if they differ by one percent-encoding the pin is wrong by one character — which
* does not fail loudly, it fails as an inscrutable TLS error days later. A unit test on either
* side alone cannot see that.
*
* Needs ECHOLOT_ENROLL_URI (minted over SSH by scripts/test-fmr.sh); self-skips without it.
*/
class LiveEnrollmentTest {
private val enrollUri = System.getenv("ECHOLOT_ENROLL_URI")
@Test
fun enrollsFromTheServersOwnLink() {
if (enrollUri.isNullOrBlank()) {
println("LiveEnrollmentTest skipped (no ECHOLOT_ENROLL_URI)"); return
}
println("link: ${enrollUri.take(60)}")
val link = assertNotNull(
EnrollmentLink.parse(enrollUri),
"the client could not parse a link the server produced — the two sides disagree",
)
println("parsed: url=${link.controlUrl} pin=${link.pin.take(12)}… token=${link.token.take(8)}")
// Redeeming applies the pin to the very request that spends the token, so a wrong pin
// fails here at the handshake rather than after the token is gone.
val enrolled = link.redeem(deviceName = "live-test", appVersion = "0.2.0")
assertTrue(enrolled.credential.isNotBlank(), "no credential came back")
assertTrue(enrolled.deviceId.isNotBlank(), "no device id came back")
println("enrolled: device=${enrolled.deviceId} server=${enrolled.profile.name} " +
"${enrolled.profile.serverVersion}")
// The credential must actually work, and the pin from the link must be the one that
// verifies the server — that is the whole claim the link is making.
assertEquals(link.controlUrl, enrolled.controlUrl)
assertTrue(enrolled.profile.capabilities.contains("udp-probe"),
"profile fetched with the new credential looks wrong: ${enrolled.profile.capabilities}")
// Single-use: a token that still works after redemption is a token an attacker can reuse.
try {
link.redeem(deviceName = "should-not-happen", appVersion = "0.2.0")
fail("the enrollment token was accepted twice — it must be single-use")
} catch (t: Throwable) {
println("second redemption correctly refused: ${t.message?.take(120)}")
}
}
}
@@ -7,6 +7,7 @@ import app.echo_lot.measurement.*
import kotlinx.serialization.json.Json import kotlinx.serialization.json.Json
import kotlin.test.Test import kotlin.test.Test
import kotlin.test.assertEquals import kotlin.test.assertEquals
import kotlin.test.assertNotNull
import kotlin.test.assertTrue import kotlin.test.assertTrue
/** /**
@@ -59,6 +60,18 @@ class LiveMeasurementTest {
println("metrics: $metrics") println("metrics: $metrics")
assertTrue(metrics.toString().contains("rtt_ms_avg")) assertTrue(metrics.toString().contains("rtt_ms_avg"))
// The directional split is the point of asking the server what it saw: without it a
// lossy path is reported as "loss" with no direction, which sends an engineer looking
// in both at once. Correlation is by wire sequence number, so a mismatch here means the
// two sides disagree about which packet is which.
val m = metrics.toString()
assertTrue(m.contains("seen_by_server"), "no directional split in the metrics: $m")
val seen = Regex(""""seen_by_server":(\d+)""").find(m)?.groupValues?.get(1)?.toInt()
assertNotNull(seen, "seen_by_server missing")
assertEquals(20, seen, "the server should have seen every probe on a healthy path")
assertTrue(m.contains("jitter_upstream_ms"), "no per-direction jitter: $m")
println("directional: $m")
assertTrue(doc.summary != null) assertTrue(doc.summary != null)
// A healthy local->fmr path should be green (no loss, no rebinding) or yellow. // A healthy local->fmr path should be green (no loss, no rebinding) or yellow.
println("summary: ${doc.summary}") println("summary: ${doc.summary}")
@@ -77,6 +77,8 @@ object TestType {
const val MTU_BLACKHOLE = "mtu.blackhole" const val MTU_BLACKHOLE = "mtu.blackhole"
const val MTU_MSS_OBSERVED = "mtu.mss_observed" const val MTU_MSS_OBSERVED = "mtu.mss_observed"
const val MTU_FRAG_DELIVERY = "mtu.frag_delivery" const val MTU_FRAG_DELIVERY = "mtu.frag_delivery"
/** Whether fragments survive arriving out of order, not merely whether they survive. */
const val MTU_FRAG_ORDERING = "mtu.frag_ordering"
// nat // nat
const val NAT_STUN_5780 = "nat.stun_5780" const val NAT_STUN_5780 = "nat.stun_5780"
const val NAT_MAPPING_LIFETIME_UDP = "nat.mapping_lifetime_udp" const val NAT_MAPPING_LIFETIME_UDP = "nat.mapping_lifetime_udp"
@@ -4,6 +4,8 @@
package app.echo_lot.protocol package app.echo_lot.protocol
import kotlinx.serialization.json.Json import kotlinx.serialization.json.Json
import kotlinx.serialization.json.jsonObject
import kotlinx.serialization.json.jsonPrimitive
import java.net.URL import java.net.URL
import javax.net.ssl.HttpsURLConnection import javax.net.ssl.HttpsURLConnection
@@ -59,13 +61,16 @@ class ControlClient(
if (conn.responseCode == 426) throw VersionRefused(extractError(body) ?: body.take(200)) if (conn.responseCode == 426) throw VersionRefused(extractError(body) ?: body.take(200))
} }
/** Pulls the "error" string out of a JSON body without pulling in a parser for one field. */ /**
private fun extractError(body: String): String? = * Pulls the "error" string out of a JSON body.
Regex(""""error"\s*:\s*"((?:[^"\\]|\\.)*)"""").find(body) *
?.groupValues?.get(1) * Parsed rather than pattern-matched: an encoder may legitimately escape characters in the
?.replace("\\\"", "\"") * message (Go escapes ">" by default), and a regex hands the user "needs \u003e= 0.2.0".
?.replace("\\n", "\n") * The parser knows how to undo every escape; a regex would have to be taught each one.
?.replace("\\\\", "\\") */
private fun extractError(body: String): String? = runCatching {
json.parseToJsonElement(body).jsonObject["error"]?.jsonPrimitive?.content
}.getOrNull()
/** /**
* Reads the response body, and turns a 426 into [VersionRefused] first. * Reads the response body, and turns a 426 into [VersionRefused] first.
@@ -0,0 +1,119 @@
// SPDX-FileCopyrightText: 2026 Echolot contributors
// SPDX-License-Identifier: GPL-3.0-or-later
package app.echo_lot.protocol
import java.net.URLDecoder
import java.net.URLEncoder
/**
* The enrollment bootstrap of probe-protocol.md §2.1.
*
* ```
* echolot://enroll?v=1&u=<control-URL, urlencoded>&p=pin-sha256:<b64 SPKI hash>&t=<token>
* ```
*
* One string carries everything a device needs to start trusting a server: where it is, which key
* to pin, and a single-use token proving the operator meant to admit this device. That is the
* whole point — it is why enrollment can be a paste or a QR scan rather than three fields typed
* from a screenshot, which is what people actually do wrong.
*
* **The link is a secret.** It contains a bearer token; anyone who sees it before the device does
* can enroll instead. Tokens are single-use and short-lived precisely so a leaked link is a
* bounded problem, but it should be treated like a password while it is live.
*/
data class EnrollmentLink(
/** e.g. "https://fmr-1.echo-lot.app:8443" */
val controlUrl: String,
/** Base64 SPKI hash, without the "pin-sha256:" prefix — the form [ControlClient] wants. */
val pin: String,
val token: String,
) {
/** Rebuilds the URI. Round-trips with [parse]; used for tests and for sharing a link on. */
fun toUri(): String = buildString {
append("echolot://enroll?v=1")
append("&u=").append(enc(controlUrl))
append("&p=").append(enc(PIN_PREFIX + pin))
append("&t=").append(enc(token))
}
/**
* Redeems the token and returns a usable server configuration.
*
* The pin is applied to the very request that redeems the token, so a link pointing at an
* impostor fails at the TLS handshake rather than after handing it a token. That ordering is
* the reason the pin travels in the link at all.
*/
fun redeem(deviceName: String? = null, appVersion: String = ""): Enrolled {
val client = ControlClient(controlUrl, setOf(pin), appVersion)
val response = client.enroll(token, deviceName)
val profile = client.profile(response.credential)
return Enrolled(
controlUrl = controlUrl,
pin = pin,
credential = response.credential,
deviceId = response.deviceId,
profile = profile,
)
}
companion object {
const val SCHEME = "echolot"
const val HOST = "enroll"
private const val PIN_PREFIX = "pin-sha256:"
/**
* Parses a bootstrap link. Returns null for anything that is not one — a malformed link
* must not be half-applied, because a half-configured server is a confusing failure much
* later rather than an obvious one now.
*/
fun parse(raw: String?): EnrollmentLink? {
val s = raw?.trim() ?: return null
val scheme = s.substringBefore("://", "")
if (!scheme.equals(SCHEME, ignoreCase = true)) return null
val rest = s.substringAfter("://")
val host = rest.substringBefore('?').trim('/')
if (!host.equals(HOST, ignoreCase = true)) return null
val params = HashMap<String, String>()
for (pair in rest.substringAfter('?', "").split('&')) {
if (pair.isEmpty()) continue
val k = pair.substringBefore('=')
val v = pair.substringAfter('=', "")
params[k] = dec(v)
}
// v is the link format, not the protocol. Unknown versions are refused rather than
// guessed at: the fields could mean anything.
val version = params["v"] ?: "1"
if (version != "1") return null
val url = params["u"]?.trim().orEmpty()
val pinRaw = params["p"]?.trim().orEmpty()
val token = params["t"]?.trim().orEmpty()
if (url.isEmpty() || pinRaw.isEmpty() || token.isEmpty()) return null
if (!url.startsWith("https://", ignoreCase = true)) return null
// A "+" in a query string decodes to a space, so a link whose base64 pin was pasted
// in unencoded arrives with spaces where "+" belonged — and a pin that is wrong by
// one character does not fail loudly, it just never matches, which surfaces much
// later as an inexplicable TLS error. Base64 has no spaces, so putting them back is
// unambiguous and cannot damage a correctly-encoded pin.
val pin = pinRaw.removePrefix(PIN_PREFIX).replace(' ', '+')
if (pin.isEmpty()) return null
return EnrollmentLink(controlUrl = url.trimEnd('/'), pin = pin, token = token)
}
private fun enc(s: String) = URLEncoder.encode(s, "UTF-8")
private fun dec(s: String) = runCatching { URLDecoder.decode(s, "UTF-8") }.getOrDefault(s)
}
}
/** A server this device is now enrolled with, ready to be stored in settings. */
data class Enrolled(
val controlUrl: String,
val pin: String,
val credential: String,
val deviceId: String,
val profile: Profile,
)
@@ -13,8 +13,16 @@ import kotlinx.serialization.json.JsonElement
@Serializable @Serializable
data class EnrollResponse( data class EnrollResponse(
@SerialName("device_id") val deviceId: String, @SerialName("device_id") val deviceId: String,
val credential: String, /** The spec's name (§2.1). */
) @SerialName("device_credential") val deviceCredential: String? = null,
/** What the first server implementation shipped. Read for older servers; do not emit. */
@SerialName("credential") val legacyCredential: String? = null,
) {
/** Whichever field the server used. */
val credential: String
get() = deviceCredential ?: legacyCredential
?: error("enroll response carried no credential")
}
@Serializable @Serializable
data class Target( data class Target(
@@ -44,13 +44,26 @@ class ProbeSession(
*/ */
fun echo(paddingBytes: Int = 40): EchoResult? { fun echo(paddingBytes: Int = 40): EchoResult? {
val t0 = System.nanoTime() val t0 = System.nanoTime()
val pkt = Wire.build(Wire.TYPE_ECHO_REQ, prefix, ++seq, nowNs(), key, ByteArray(paddingBytes)) val wireSeq = ++seq
val pkt = Wire.build(Wire.TYPE_ECHO_REQ, prefix, wireSeq, nowNs(), key, ByteArray(paddingBytes))
socket.send(DatagramPacket(pkt, pkt.size, server)) socket.send(DatagramPacket(pkt, pkt.size, server))
// A lost probe still has a sequence number, and that number is what lets the server's
// observations say whether it was lost going out or coming back — so report it either way.
lastSeq = wireSeq
val resp = receive(Wire.TYPE_ECHO_RESP) ?: return null val resp = receive(Wire.TYPE_ECHO_RESP) ?: return null
val rttMs = (System.nanoTime() - t0) / 1_000_000.0 val rttMs = (System.nanoTime() - t0) / 1_000_000.0
return EchoResult(rttMs, Observation.parse(resp.payload)) return EchoResult(rttMs, Observation.parse(resp.payload), wireSeq)
} }
/**
* The wire sequence number of the most recent [echo], including one that was lost.
*
* Exposed because the caller cannot derive it: the counter is shared with every other packet
* type on this session, so "the nth echo" is not "sequence n".
*/
var lastSeq: Int = 0
private set
/** One MTU probe of [totalSize] bytes (DF is set by the OS on the socket where supported). /** One MTU probe of [totalSize] bytes (DF is set by the OS on the socket where supported).
* Returns the size the server acknowledged receiving, or null if the probe was lost. */ * Returns the size the server acknowledged receiving, or null if the probe was lost. */
fun mtuProbe(totalSize: Int): Int? { fun mtuProbe(totalSize: Int): Int? {
@@ -112,5 +125,5 @@ class ProbeSession(
override fun close() = socket.close() override fun close() = socket.close()
data class EchoResult(val rttMs: Double, val observation: Observation?) data class EchoResult(val rttMs: Double, val observation: Observation?, val seq: Int = 0)
} }
@@ -32,6 +32,12 @@ object Wire {
const val TYPE_DOWNTRAIN_DATA: Int = 0x06 const val TYPE_DOWNTRAIN_DATA: Int = 0x06
const val TYPE_BIG_SEND: Int = 0x0C const val TYPE_BIG_SEND: Int = 0x0C
/**
* A datagram the server deliberately fragmented. Its arrival IS the measurement: it can only
* be delivered if every fragment survived the path and the local stack reassembled them.
*/
const val TYPE_FRAG_DATA: Int = 0x0D
/** The 8-byte on-the-wire prefix = first 16 hex chars of the session id, decoded. */ /** The 8-byte on-the-wire prefix = first 16 hex chars of the session id, decoded. */
fun wirePrefix(sessionId: String): ByteArray { fun wirePrefix(sessionId: String): ByteArray {
require(sessionId.length >= 16) { "session id too short" } require(sessionId.length >= 16) { "session id too short" }
@@ -0,0 +1,101 @@
// SPDX-FileCopyrightText: 2026 Echolot contributors
// SPDX-License-Identifier: GPL-3.0-or-later
package app.echo_lot.protocol
import kotlin.test.Test
import kotlin.test.assertEquals
import kotlin.test.assertNotNull
import kotlin.test.assertNull
class EnrollmentTest {
private val pin = "zRV9qkiLnRexAeh4RrSfJzbPWO+U/2Oj2/NVM/KfXlg="
private val url = "https://fmr-1.echo-lot.app:8443"
private val token = "abc123-token_value"
@Test
fun parsesTheSpecFormat() {
val link = assertNotNull(
EnrollmentLink.parse(
"echolot://enroll?v=1&u=https%3A%2F%2Ffmr-1.echo-lot.app%3A8443" +
"&p=pin-sha256%3AzRV9qkiLnRexAeh4RrSfJzbPWO%2BU%2F2Oj2%2FNVM%2FKfXlg%3D" +
"&t=abc123-token_value"
)
)
assertEquals(url, link.controlUrl)
assertEquals(pin, link.pin, "the pin-sha256: prefix should be stripped for ControlClient")
assertEquals(token, link.token)
}
// The pin is base64: it contains +, / and = , every one of which means something else in a
// query string. Getting the decoding wrong yields a pin that silently never matches.
@Test
fun survivesBase64PunctuationThroughARoundTrip() {
val original = EnrollmentLink(url, pin, token)
val reparsed = assertNotNull(EnrollmentLink.parse(original.toUri()))
assertEquals(original, reparsed)
}
@Test
fun acceptsAnUnprefixedPin() {
val link = assertNotNull(EnrollmentLink.parse("echolot://enroll?v=1&u=$url&p=$pin&t=$token"))
assertEquals(pin, link.pin)
}
// A hand-assembled link often has its base64 pin pasted in raw. "+" then decodes to a space
// and the pin is wrong by one character — which does not fail loudly, it just never matches.
// Base64 contains no spaces, so restoring them is unambiguous.
@Test
fun repairsAPinWhosePlusSignsWereNotEncoded() {
val mangled = pin.replace("+", " ")
val link = assertNotNull(EnrollmentLink.parse("echolot://enroll?v=1&u=$url&p=$mangled&t=$token"))
assertEquals(pin, link.pin)
}
@Test
fun toleratesSurroundingWhitespaceAndCaseFromAPaste() {
val link = assertNotNull(
EnrollmentLink.parse(" ECHOLOT://ENROLL?v=1&u=$url&p=$pin&t=$token\n")
)
assertEquals(url, link.controlUrl)
}
// A half-applied link is a confusing failure much later; a rejected one is an obvious failure
// now. So anything missing or unrecognised parses to null rather than to a partial config.
@Test
fun rejectsAnythingItCannotFullyUnderstand() {
val bad = listOf(
null,
"",
"not a uri",
"https://fmr-1.echo-lot.app:8443", // a plain URL is not a bootstrap link
"echolot://run?v=1&u=$url&p=$pin&t=$token", // wrong action
"echolot://enroll?v=2&u=$url&p=$pin&t=$token", // unknown link version
"echolot://enroll?v=1&p=$pin&t=$token", // no url
"echolot://enroll?v=1&u=$url&t=$token", // no pin
"echolot://enroll?v=1&u=$url&p=$pin", // no token
"echolot://enroll?v=1&u=$url&p=pin-sha256:&t=$token", // empty pin
)
for (s in bad) assertNull(EnrollmentLink.parse(s), "should not parse: $s")
}
// The pin is the entire basis of trust, and it only protects the connection if the connection
// is TLS. A cleartext control URL would hand the token to anyone on the path.
@Test
fun refusesACleartextControlUrl() {
assertNull(EnrollmentLink.parse("echolot://enroll?v=1&u=http://fmr-1.echo-lot.app:8443&p=$pin&t=$token"))
}
@Test
fun aMissingVersionIsTreatedAsTheOnlyVersionThatExists() {
val link = assertNotNull(EnrollmentLink.parse("echolot://enroll?u=$url&p=$pin&t=$token"))
assertEquals(token, link.token)
}
@Test
fun trailingSlashesOnTheControlUrlAreNormalised() {
val link = assertNotNull(EnrollmentLink.parse("echolot://enroll?v=1&u=$url/&p=$pin&t=$token"))
assertEquals(url, link.controlUrl, "a trailing slash would double up when paths are appended")
}
}
+42
View File
@@ -0,0 +1,42 @@
#!/usr/bin/env bash
# SPDX-FileCopyrightText: 2026 Echolot contributors
# SPDX-License-Identifier: GPL-3.0-or-later
#
# Mints an enrollment link on the probe server and prints it — as text, as a QR code if
# `qrencode` is around, and as an adb command if a device is attached.
#
# The admin listener is localhost-only by design, so this goes over SSH. The link carries a
# single-use bearer token: treat it like a password until it is redeemed.
#
# Usage: echolot-app/scripts/enroll-link.sh [note]
set -euo pipefail
SSH_HOST="${ECHOLOT_SSH:-claude-echolot}"
NOTE="${1:-manual}"
MINTED=$(ssh -o BatchMode=yes "$SSH_HOST" \
"curl -s -X POST 'http://127.0.0.1:8444/admin/enroll-tokens?note=$NOTE'")
URI=$(printf '%s' "$MINTED" | python -c 'import json,sys;print(json.load(sys.stdin).get("enroll_uri",""))')
if [ -z "$URI" ]; then
echo "server returned no enroll_uri (needs server-v0.5.4+):" >&2
echo "$MINTED" >&2
exit 1
fi
echo "$URI"
echo
# A QR is the point of the format: scanning beats pasting a 200-character string onto a phone.
if command -v qrencode >/dev/null 2>&1; then
qrencode -t ANSIUTF8 "$URI"
else
echo "(install qrencode to get a scannable QR here)"
fi
# With a device attached, the deep link can be delivered straight to the app — no typing at all.
if command -v adb >/dev/null 2>&1 && [ -n "$(adb devices | sed -n '2p')" ]; then
echo
echo "attached device — deliver it directly with:"
echo " adb shell am start -a android.intent.action.VIEW -d '$URI'"
fi
+17 -6
View File
@@ -19,13 +19,23 @@ UDP_PORT="${ECHOLOT_UDP_PORT:-8442}"
CTL_URL="https://${CTL_HOST}:${CTL_PORT}" CTL_URL="https://${CTL_HOST}:${CTL_PORT}"
echo "· minting enrollment token on ${SSH_HOST} ..." echo "· minting enrollment token on ${SSH_HOST} ..."
TOKEN=$(ssh -o BatchMode=yes "$SSH_HOST" \ MINTED=$(ssh -o BatchMode=yes "$SSH_HOST" \
'curl -s -X POST http://127.0.0.1:8444/admin/enroll-tokens' \ 'curl -s -X POST http://127.0.0.1:8444/admin/enroll-tokens')
| python -c 'import json,sys;print(json.load(sys.stdin)["token"])') TOKEN=$(printf '%s' "$MINTED" | python -c 'import json,sys;print(json.load(sys.stdin)["token"])')
# The server also returns the whole §2.1 bootstrap link. LiveEnrollmentTest redeems that link,
# which is what proves the Go side and the Kotlin side agree on its encoding — a disagreement
# there yields a pin wrong by one character, which fails much later and looks like anything but.
ENROLL_URI=$(printf '%s' "$MINTED" \
| python -c 'import json,sys;print(json.load(sys.stdin).get("enroll_uri",""))')
echo "· enrolling over ${CTL_URL} ..." echo "· enrolling over ${CTL_URL} ..."
CRED=$(curl -sk -X POST "${CTL_URL}/v1/enroll" -H "Authorization: Bearer ${TOKEN}" \ # A second token, because the one above is single-use and may be spent by LiveEnrollmentTest.
| python -c 'import json,sys;print(json.load(sys.stdin)["credential"])') TOKEN2=$(ssh -o BatchMode=yes "$SSH_HOST" \
'curl -s -X POST http://127.0.0.1:8444/admin/enroll-tokens' \
| python -c 'import json,sys;print(json.load(sys.stdin)["token"])')
CRED=$(curl -sk -X POST "${CTL_URL}/v1/enroll" -H "Authorization: Bearer ${TOKEN2}" \
-H "X-Echolot-App-Version: 0.2.0" \
| python -c 'import json,sys;d=json.load(sys.stdin);print(d.get("device_credential") or d["credential"])')
echo "· computing SPKI pin from served cert ..." echo "· computing SPKI pin from served cert ..."
PIN=$(echo | openssl s_client -connect "${CTL_HOST}:${CTL_PORT}" 2>/dev/null \ PIN=$(echo | openssl s_client -connect "${CTL_HOST}:${CTL_PORT}" 2>/dev/null \
@@ -43,5 +53,6 @@ ECHOLOT_LIVE_PIN="$PIN" \
ECHOLOT_LIVE_CRED="$CRED" \ ECHOLOT_LIVE_CRED="$CRED" \
ECHOLOT_LIVE_UDP="${CTL_HOST}:${UDP_PORT}" \ ECHOLOT_LIVE_UDP="${CTL_HOST}:${UDP_PORT}" \
ECHOLOT_LIVE_TARGET="${ECHOLOT_LIVE_TARGET:-fmr}" \ ECHOLOT_LIVE_TARGET="${ECHOLOT_LIVE_TARGET:-fmr}" \
ECHOLOT_ENROLL_URI="$ENROLL_URI" \
./gradlew "$TASK" --tests "$FILTER" --info --rerun-tasks --console=plain \ ./gradlew "$TASK" --tests "$FILTER" --info --rerun-tasks --console=plain \
2>&1 | grep -E "profile:|capabilities:|session:|echo |primed|mtu probe|downtrain|big_send|largest|observations bytes|Live[A-Za-z]*Test|BUILD|FAIL|PASS|^e:" || true 2>&1 | grep -E "profile:|capabilities:|session:|echo |primed|mtu probe|downtrain|big_send|largest|observations bytes|link:|parsed:|enrolled:|refused|Live[A-Za-z]*Test|BUILD|FAIL|PASS|^e:" || true
+53 -7
View File
@@ -31,6 +31,7 @@ import (
"os/signal" "os/signal"
"path/filepath" "path/filepath"
"strconv" "strconv"
"strings"
"sync/atomic" "sync/atomic"
"syscall" "syscall"
"time" "time"
@@ -111,6 +112,14 @@ func serve(cfg *config.Config) error {
} }
caps := []string{"udp-probe", "delayed-echo", "connect-back", "http-echo", "downtrain", "big-send"} caps := []string{"udp-probe", "delayed-echo", "connect-back", "http-echo", "downtrain", "big-send"}
// Crafted fragments need a raw socket. Advertised only when one can actually be opened —
// a capability we cannot deliver turns a missing feature into a failed measurement.
rawFrag := dataplane.RawFragSupported()
if rawFrag {
caps = append(caps, "frag-send")
} else {
slog.Info("frag-send unavailable: no raw socket (needs CAP_NET_RAW)")
}
if len(config.Addrs(cfg.TCPListen)) > 0 { if len(config.Addrs(cfg.TCPListen)) > 0 {
caps = append(caps, "tcp-echo", "tls-echo") caps = append(caps, "tcp-echo", "tls-echo")
} }
@@ -145,12 +154,18 @@ func serve(cfg *config.Config) error {
Store: st, Sessions: sessions, Name: cfg.Name, Store: st, Sessions: sessions, Name: cfg.Name,
UDPPort: mustPort(firstAddr(cfg.UDPListen)), TCPPort: mustPort(firstAddr(cfg.TCPListen)), UDPPort: mustPort(firstAddr(cfg.UDPListen)), TCPPort: mustPort(firstAddr(cfg.TCPListen)),
StunPort: mustPort(firstAddr(cfg.StunListen)), PinB64: pin, CertChain: cert.Certificate, StunPort: mustPort(firstAddr(cfg.StunListen)), PinB64: pin, CertChain: cert.Certificate,
DelayedEcho: dp.SendDelayedEcho, DelayedEcho: dp.SendDelayedEcho,
DownTrain: dp.DownTrain, DownTrain: dp.DownTrain,
BigSend: dp.BigSend, BigSend: dp.BigSend,
TCPRecent: func(ip string) any { return tcpSrv.RecentFor(ip) }, TCPRecent: func(ip string) any { return tcpSrv.RecentFor(ip) },
Runs: runStore, Runs: runStore,
AppRange: appRange, AppRange: appRange,
PublicControlURL: publicControlURL(cfg),
}
// Left nil when there is no raw socket, so the handler answers "not implemented" with a
// reason rather than failing somewhere deeper.
if rawFrag {
ctl.FragSend = dp.FragSend
} }
ctx, stop := signal.NotifyContext(context.Background(), syscall.SIGINT, syscall.SIGTERM) ctx, stop := signal.NotifyContext(context.Background(), syscall.SIGINT, syscall.SIGTERM)
@@ -232,7 +247,17 @@ func serve(cfg *config.Config) error {
http.Error(w, err.Error(), 500) http.Error(w, err.Error(), 500)
return return
} }
fmt.Fprintf(w, `{"token":%q,"expires_in_s":86400}`+"\n", tok) // The whole bootstrap, not just the token: this is what gets pasted or turned into a
// QR code, and assembling it here is what keeps an operator from transcribing a pin by
// hand — a pin wrong by one character fails as an inscrutable TLS error days later.
w.Header().Set("Content-Type", "application/json")
enc := json.NewEncoder(w)
enc.SetEscapeHTML(false) // the link is full of / and =; escaping them helps nobody
_ = enc.Encode(map[string]any{
"token": tok,
"expires_in_s": 86400,
"enroll_uri": ctl.EnrollmentLink(tok),
})
}) })
adminSrv := &http.Server{Addr: cfg.AdminListen, Handler: admin, ReadHeaderTimeout: 10 * time.Second} adminSrv := &http.Server{Addr: cfg.AdminListen, Handler: admin, ReadHeaderTimeout: 10 * time.Second}
go func() { errCh <- fmt.Errorf("admin: %w", adminSrv.ListenAndServe()) }() go func() { errCh <- fmt.Errorf("admin: %w", adminSrv.ListenAndServe()) }()
@@ -446,3 +471,24 @@ func loadOrCreateCert(cfg *config.Config) (tls.Certificate, error) {
slog.Info("generated self-signed certificate", "cert", certPath) slog.Info("generated self-signed certificate", "cert", certPath)
return tls.X509KeyPair(certPem, keyPem) return tls.X509KeyPair(certPem, keyPem)
} }
// publicControlURL is where clients should reach this server's control plane.
//
// Configured wins; otherwise the first control listen address is used, which is correct for the
// plain case (bind an address, hand out that address). A wildcard bind has no single right answer,
// so it is left to the operator rather than guessed — a link pointing at 0.0.0.0 is worse than a
// link the operator was told to configure.
func publicControlURL(cfg *config.Config) string {
if cfg.PublicControlURL != "" {
return strings.TrimRight(cfg.PublicControlURL, "/")
}
addr := firstAddr(cfg.ControlListen)
if addr == "" {
return ""
}
if strings.HasPrefix(addr, ":") || strings.HasPrefix(addr, "0.0.0.0:") || strings.HasPrefix(addr, "[::]:") {
slog.Warn("control plane is bound to a wildcard address; set ECHOLOT_PUBLIC_URL "+
"so enrollment links point somewhere reachable", "listen", addr)
}
return "https://" + addr
}
+1 -1
View File
@@ -183,7 +183,7 @@ func Check(peer string, r Range, peerName string) (Verdict, string) {
return TooOld, fmt.Sprintf("%s %s is older than this build supports (needs %s). Update the %s.", return TooOld, fmt.Sprintf("%s %s is older than this build supports (needs %s). Update the %s.",
peerName, v, r, peerName) peerName, v, r, peerName)
case r.HasMax && !v.Less(r.Max): case r.HasMax && !v.Less(r.Max):
return TooNew, fmt.Sprintf("%s %s is newer than this build supports (accepts %s). Update this side, or point at a %s within range.", return TooNew, fmt.Sprintf("%s %s is newer than this build supports (accepts %s). Update this side, or use a version of the %s within that range.",
peerName, v, r, peerName) peerName, v, r, peerName)
} }
return OK, "" return OK, ""
+5
View File
@@ -63,6 +63,10 @@ type Config struct {
MinAppVersion string // ECHOLOT_MIN_APP_VERSION / --min-app-version MinAppVersion string // ECHOLOT_MIN_APP_VERSION / --min-app-version
MaxAppVersion string // ECHOLOT_MAX_APP_VERSION / --max-app-version (exclusive) MaxAppVersion string // ECHOLOT_MAX_APP_VERSION / --max-app-version (exclusive)
// Where clients reach the control plane, for enrollment links. Empty = derive from the
// first control listen address.
PublicControlURL string // ECHOLOT_PUBLIC_URL / --public-url
// Mode // Mode
Docker bool // --docker (or autodetected; env ECHOLOT_DOCKER=1 forces) Docker bool // --docker (or autodetected; env ECHOLOT_DOCKER=1 forces)
} }
@@ -111,6 +115,7 @@ func Load(args []string) (*Config, *Actions, error) {
fs.IntVar(&c.UploadRetentionDays, "upload-retention-days", envInt("UPLOAD_RETENTION_DAYS", 90), "delete uploaded runs older than this; 0 disables") fs.IntVar(&c.UploadRetentionDays, "upload-retention-days", envInt("UPLOAD_RETENTION_DAYS", 90), "delete uploaded runs older than this; 0 disables")
fs.IntVar(&c.UploadMaxRuns, "upload-max-runs", envInt("UPLOAD_MAX_RUNS", 200), "keep at most this many runs per device; 0 disables") fs.IntVar(&c.UploadMaxRuns, "upload-max-runs", envInt("UPLOAD_MAX_RUNS", 200), "keep at most this many runs per device; 0 disables")
fs.StringVar(&c.UploadMinAnon, "upload-min-anonymization", envOr("UPLOAD_MIN_ANONYMIZATION", "full"), "least anonymization accepted: full|balanced|strict") fs.StringVar(&c.UploadMinAnon, "upload-min-anonymization", envOr("UPLOAD_MIN_ANONYMIZATION", "full"), "least anonymization accepted: full|balanced|strict")
fs.StringVar(&c.PublicControlURL, "public-url", envOr("PUBLIC_URL", ""), "public control-plane URL for enrollment links, e.g. https://probe.example.net:8443")
fs.StringVar(&c.MinAppVersion, "min-app-version", envOr("MIN_APP_VERSION", "0.2.0"), "oldest app version this server will serve (SemVer, inclusive)") fs.StringVar(&c.MinAppVersion, "min-app-version", envOr("MIN_APP_VERSION", "0.2.0"), "oldest app version this server will serve (SemVer, inclusive)")
fs.StringVar(&c.MaxAppVersion, "max-app-version", envOr("MAX_APP_VERSION", "1.0.0"), "first app version this server will refuse (SemVer, exclusive); empty = unbounded") fs.StringVar(&c.MaxAppVersion, "max-app-version", envOr("MAX_APP_VERSION", "1.0.0"), "first app version this server will refuse (SemVer, exclusive); empty = unbounded")
fs.BoolVar(&c.Docker, "docker", envOr("DOCKER", "") == "1", "force container mode (config from env, no systemd/self-update)") fs.BoolVar(&c.Docker, "docker", envOr("DOCKER", "") == "1", "force container mode (config from env, no systemd/self-update)")
+74 -3
View File
@@ -20,6 +20,7 @@ import (
"net" "net"
"net/http" "net/http"
"net/netip" "net/netip"
"net/url"
"strconv" "strconv"
"strings" "strings"
"time" "time"
@@ -58,6 +59,9 @@ type Server struct {
BigSend func(sess *session.Session, g *session.Grant, sizes []int, df bool) ([]dataplane.BigSendResult, error) BigSend func(sess *session.Session, g *session.Grant, sizes []int, df bool) ([]dataplane.BigSendResult, error)
// Runs stores uploaded measurement documents (may be nil: uploads unsupported). // Runs stores uploaded measurement documents (may be nil: uploads unsupported).
Runs *runs.Store Runs *runs.Store
// FragSend emits one datagram as hand-built IP fragments in a chosen order (may be nil:
// needs a raw socket, so it is unavailable to an unprivileged server).
FragSend func(sess *session.Session, g *session.Grant, sizeBytes int, mode dataplane.FragMode, fragSize int) (dataplane.FragResult, error)
// EgressMTU reports the server's own measured egress path MTU (0 = unknown). With DF set // EgressMTU reports the server's own measured egress path MTU (0 = unknown). With DF set
// we cannot emit a datagram larger than this, so requested sizes above it are refused up // we cannot emit a datagram larger than this, so requested sizes above it are refused up
// front and reported as such — the client must not read that as a downstream path limit. // front and reported as such — the client must not read that as a downstream path limit.
@@ -72,6 +76,10 @@ type Server struct {
// server, not the client. // server, not the client.
ProvenGood func() (mtuOK, sysctlOK bool) ProvenGood func() (mtuOK, sysctlOK bool)
// PublicControlURL is where clients reach this server, for the enrollment link (§2.1).
// Empty means "derive from the address we are listening on", which is right for a plain
// deployment and wrong behind a proxy or a name — hence the override.
PublicControlURL string
// AppRange is the app-version window this server will serve. Zero value means the built-in // AppRange is the app-version window this server will serve. Zero value means the built-in
// default (see DefaultAppRange). // default (see DefaultAppRange).
AppRange compat.Range AppRange compat.Range
@@ -236,6 +244,8 @@ func (s *Server) actions(w http.ResponseWriter, r *http.Request) {
IntervalUs int `json:"interval_us"` IntervalUs int `json:"interval_us"`
SizesBytes []int `json:"sizes_bytes"` SizesBytes []int `json:"sizes_bytes"`
DF *bool `json:"df"` DF *bool `json:"df"`
Mode string `json:"mode"`
FragBytes int `json:"frag_bytes"`
} }
if err := json.NewDecoder(r.Body).Decode(&req); err != nil { if err := json.NewDecoder(r.Body).Decode(&req); err != nil {
writeJSON(w, http.StatusBadRequest, map[string]string{"error": "bad body"}) writeJSON(w, http.StatusBadRequest, map[string]string{"error": "bad body"})
@@ -368,6 +378,45 @@ func (s *Server) actions(w http.ResponseWriter, r *http.Request) {
"grant": map[string]any{"max_bytes": g.MaxBytes, "max_kbps": g.MaxKbps}, "grant": map[string]any{"max_bytes": g.MaxBytes, "max_kbps": g.MaxKbps},
}) })
case "frag_send":
if s.FragSend == nil {
writeJSON(w, http.StatusNotImplemented, map[string]string{
"error": "frag_send needs a raw socket, which this server does not have",
})
return
}
size := clamp(req.SizeBytes, 1600, 8000) // must exceed the path MTU or nothing fragments
mode := dataplane.FragMode(req.Mode)
switch mode {
case dataplane.FragInOrder, dataplane.FragReversed, dataplane.FragFirstLast:
default:
mode = dataplane.FragInOrder
}
fragBytes := clamp(req.FragBytes, 8, 1400)
g := sess.NewGrant(actionID, int64(size), 0, session.DefaultGrantLimits)
if g == nil {
writeJSON(w, http.StatusConflict, noDataPlaneYet)
return
}
// Synchronous: the whole burst is a few kB and at most a few hundred milliseconds, and
// the caller wants to know it was actually emitted before it starts listening. An
// asynchronous send would make "nothing arrived" ambiguous between a path drop and a
// send that never happened — the one distinction this test exists to make.
result, err := s.FragSend(sess, g, size, mode, fragBytes)
slog.Info("frag_send finished", "action", actionID, "mode", mode,
"size", size, "fragments", result.Fragments, "err", err)
if err != nil {
writeJSON(w, http.StatusConflict, map[string]any{
"error": err.Error(), "action_id": actionID, "result": result,
})
return
}
writeJSON(w, http.StatusAccepted, map[string]any{
"action_id": actionID, "mode": string(mode), "size_bytes": size,
"frag_bytes": fragBytes, "fragments": result.Fragments,
"grant": map[string]any{"max_bytes": g.MaxBytes, "max_kbps": g.MaxKbps},
})
default: default:
writeJSON(w, http.StatusBadRequest, map[string]string{"error": "unknown or unimplemented action"}) writeJSON(w, http.StatusBadRequest, map[string]string{"error": "unknown or unimplemented action"})
} }
@@ -430,7 +479,12 @@ func bearer(r *http.Request) string {
func writeJSON(w http.ResponseWriter, code int, v any) { func writeJSON(w http.ResponseWriter, code int, v any) {
w.Header().Set("Content-Type", "application/json") w.Header().Set("Content-Type", "application/json")
w.WriteHeader(code) w.WriteHeader(code)
_ = json.NewEncoder(w).Encode(v) enc := json.NewEncoder(w)
// Go escapes <, > and & by default, for JSON embedded in HTML. This is an API, and the
// escaping is actively harmful here: a refusal message reading "needs >= 0.2.0" is what
// the user ends up seeing. Nothing we emit is ever interpolated into a page.
enc.SetEscapeHTML(false)
_ = enc.Encode(v)
} }
// enroll redeems a single-use enrollment token for a device credential (§2.1). // enroll redeems a single-use enrollment token for a device credential (§2.1).
@@ -451,8 +505,12 @@ func (s *Server) enroll(w http.ResponseWriter, r *http.Request) {
} }
slog.Info("device enrolled", "device", dev.ID, "name", dev.Name) slog.Info("device enrolled", "device", dev.ID, "name", dev.Name)
writeJSON(w, http.StatusCreated, map[string]string{ writeJSON(w, http.StatusCreated, map[string]string{
"device_id": dev.ID, "device_id": dev.ID,
"credential": dev.Credential, // returned exactly once // The spec (§2.1) names this device_credential; the first implementation shipped
// "credential". Both are sent while deployed 0.5.x clients still read the old name;
// the client prefers the spec's. Drop "credential" once nothing reads it.
"device_credential": dev.Credential, // returned exactly once
"credential": dev.Credential, // deprecated alias, see above
}) })
} }
@@ -657,3 +715,16 @@ func maxOrEmpty(r compat.Range) string {
} }
return r.Max.String() return r.Max.String()
} }
// EnrollmentLink builds the §2.1 bootstrap string for a freshly minted token.
//
// The server assembles it rather than the operator, because it is the only party that knows all
// three parts at once — its own URL, its own SPKI pin, and the token. An operator copying a pin
// by hand is the step that goes wrong, and a pin wrong by one character does not fail loudly.
func (s *Server) EnrollmentLink(token string) string {
u := s.PublicControlURL
return "echolot://enroll?v=1" +
"&u=" + url.QueryEscape(strings.TrimRight(u, "/")) +
"&p=" + url.QueryEscape("pin-sha256:"+s.PinB64) +
"&t=" + url.QueryEscape(token)
}
+263
View File
@@ -0,0 +1,263 @@
// SPDX-FileCopyrightText: 2026 Echolot contributors
// SPDX-License-Identifier: GPL-3.0-or-later
//go:build linux
package dataplane
import (
"encoding/binary"
"fmt"
"net/netip"
"sync/atomic"
"syscall"
"time"
"echo-lot.app/server/internal/session"
)
// Crafted IPv4 fragmentation (spec §5 frag_send).
//
// Letting the kernel fragment an oversized datagram — which is what big_send with df=false does —
// answers one question: do fragments get through at all. It cannot answer the more interesting
// one, because the kernel always emits fragments in order, first one first.
//
// The classic middlebox fault is precisely about that ordering. Only the *first* fragment carries
// the UDP header, and therefore the ports; a stateful firewall or NAT that has not seen it has no
// flow to match later fragments against. Plenty of implementations drop them. Others hold them
// briefly and reassemble; others leak. The difference is invisible to any test that sends
// fragments in order, and it shows up in the real world as "large DNS answers fail on this
// network" or "the VPN works until the MTU drops".
//
// So this builds the fragments by hand and controls their order and timing. That needs a raw
// socket (CAP_NET_RAW); when we do not have one the capability is not advertised, rather than
// advertised and failing later.
// FragMode is how a fragmented datagram is put on the wire.
type FragMode string
const (
// FragInOrder is the baseline: first fragment first, as the kernel would. A path that fails
// this fails everything, and it tells the others apart from a path that drops all fragments.
FragInOrder FragMode = "in_order"
// FragReversed sends the last fragment first. This is the one that finds stateful devices
// which need the first fragment to build state.
FragReversed FragMode = "reversed"
// FragFirstLast holds the first fragment back until the others have arrived, which tests
// whether the path buffers non-first fragments at all and for how long.
FragFirstLast FragMode = "first_last"
)
var fragIPID atomic.Uint32
// RawFragSupported reports whether crafted fragments can actually be sent here.
//
// Checked by opening the socket rather than by inspecting capabilities: the question is "will
// this work", and a permission model has more ways to say no than a capability bit has to say yes
// (user namespaces, seccomp, LSM). Advertising a capability we cannot deliver would turn a
// missing feature into a failed measurement.
func RawFragSupported() bool {
fd, err := syscall.Socket(syscall.AF_INET, syscall.SOCK_RAW, syscall.IPPROTO_RAW)
if err != nil {
return false
}
_ = syscall.Close(fd)
return true
}
// FragResult is what happened to one crafted fragment burst.
type FragResult struct {
Mode FragMode `json:"mode"`
SizeBytes int `json:"size_bytes"`
Fragments int `json:"fragments"`
Sent bool `json:"sent"`
Err string `json:"err,omitempty"`
}
// FragSend emits one ELT1 packet of sizeBytes as hand-built IPv4 fragments, in the given order.
//
// The datagram is assembled whole and then cut up, so what the client reassembles — if it
// reassembles — is a normal, HMAC-valid packet indistinguishable from any other. That matters:
// the client must not be able to tell a crafted fragment burst from a kernel one, or it would be
// measuring our sender rather than the path.
func (s *Server) FragSend(
sess *session.Session, g *session.Grant, sizeBytes int, mode FragMode, fragSize int,
) (FragResult, error) {
res := FragResult{Mode: mode, SizeBytes: sizeBytes}
target := sess.DataSource()
if !target.IsValid() {
return res, fmt.Errorf("no observed data-plane source")
}
if !target.Addr().Unmap().Is4() {
// IPv6 has no in-network fragmentation: only the source may fragment, via an extension
// header. Worth building, but it is a different mechanism and belongs in its own code
// path rather than pretending this one covers it.
return res, fmt.Errorf("crafted fragmentation is IPv4-only for now")
}
conn := s.connFor(target, sess.DataLocal())
if conn == nil {
return res, fmt.Errorf("no data-plane socket matches target family")
}
local := sess.DataLocal()
if !local.IsValid() {
return res, fmt.Errorf("session has no recorded local address")
}
if sizeBytes < HeaderSize+8 {
sizeBytes = HeaderSize + 8
}
if sizeBytes > 8000 {
sizeBytes = 8000
}
if !g.Allow(sizeBytes) {
return res, fmt.Errorf("grant exhausted")
}
// The ELT1 packet, signed exactly as any other, then wrapped in UDP.
payload := make([]byte, sizeBytes-HeaderSize)
binary.BigEndian.PutUint32(payload[0:4], uint32(sizeBytes))
copy(payload[4:], mode)
elt := s.buildPacket(sess, TypeFragData, 0, payload)
udp := buildUDP(local, target, elt)
// Fragment offsets are in 8-byte units, so every fragment except the last must be a multiple
// of 8. A payload that is not is not an error — it is a fragment that no host will reassemble.
if fragSize <= 0 {
fragSize = 576
}
fragSize = (fragSize / 8) * 8
if fragSize < 8 {
fragSize = 8
}
fragments := splitIPv4(local.Addr(), target.Addr(), udp, fragSize, uint16(fragIPID.Add(1)))
res.Fragments = len(fragments)
fd, err := syscall.Socket(syscall.AF_INET, syscall.SOCK_RAW, syscall.IPPROTO_RAW)
if err != nil {
res.Err = err.Error()
return res, err
}
defer syscall.Close(fd)
if err := syscall.SetsockoptInt(fd, syscall.IPPROTO_IP, syscall.IP_HDRINCL, 1); err != nil {
res.Err = err.Error()
return res, err
}
dst := syscall.SockaddrInet4{}
copy(dst.Addr[:], target.Addr().Unmap().AsSlice())
send := func(pkt []byte) error { return syscall.Sendto(fd, pkt, 0, &dst) }
switch mode {
case FragReversed:
for i := len(fragments) - 1; i >= 0; i-- {
if err := send(fragments[i]); err != nil {
res.Err = err.Error()
return res, err
}
time.Sleep(time.Millisecond)
}
case FragFirstLast:
for i := 1; i < len(fragments); i++ {
if err := send(fragments[i]); err != nil {
res.Err = err.Error()
return res, err
}
time.Sleep(time.Millisecond)
}
// Long enough to be a real test of whether anything holds fragments, short enough to stay
// inside the usual 30-second reassembly timeout by a wide margin.
time.Sleep(250 * time.Millisecond)
if err := send(fragments[0]); err != nil {
res.Err = err.Error()
return res, err
}
default:
for _, f := range fragments {
if err := send(f); err != nil {
res.Err = err.Error()
return res, err
}
time.Sleep(time.Millisecond)
}
}
res.Sent = true
return res, nil
}
// buildUDP wraps a payload in a UDP header with a computed checksum.
//
// The checksum is optional in IPv4 and it would be less code to send zero, but a zero-checksum
// datagram is dropped by some middleboxes — and that drop would be recorded as a fragmentation
// failure, which is exactly the wrong conclusion.
func buildUDP(src, dst netip.AddrPort, payload []byte) []byte {
out := make([]byte, 8+len(payload))
binary.BigEndian.PutUint16(out[0:2], src.Port())
binary.BigEndian.PutUint16(out[2:4], dst.Port())
binary.BigEndian.PutUint16(out[4:6], uint16(8+len(payload)))
copy(out[8:], payload)
// Pseudo-header + UDP header + data, per RFC 768.
var sum uint32
s4, d4 := src.Addr().Unmap().As4(), dst.Addr().Unmap().As4()
for _, b := range [][]byte{s4[:], d4[:]} {
sum += uint32(binary.BigEndian.Uint16(b[0:2]))
sum += uint32(binary.BigEndian.Uint16(b[2:4]))
}
sum += uint32(syscall.IPPROTO_UDP)
sum += uint32(len(out))
for i := 0; i+1 < len(out); i += 2 {
sum += uint32(binary.BigEndian.Uint16(out[i : i+2]))
}
if len(out)%2 == 1 {
sum += uint32(out[len(out)-1]) << 8
}
for sum>>16 != 0 {
sum = (sum & 0xFFFF) + (sum >> 16)
}
ck := ^uint16(sum)
if ck == 0 {
ck = 0xFFFF // 0 means "no checksum" in IPv4; the all-ones form is the same value
}
binary.BigEndian.PutUint16(out[6:8], ck)
return out
}
// splitIPv4 cuts a UDP datagram into IPv4 fragments of at most fragSize payload bytes each.
//
// Every fragment carries the same IP ID — that is what marks them as one datagram — and every one
// but the last sets MF. The kernel fills in the header checksum and total length for us under
// IP_HDRINCL (raw(7)); the ID it only fills when zero, which is why it is set explicitly here.
func splitIPv4(src, dst netip.Addr, udp []byte, fragSize int, id uint16) [][]byte {
s4, d4 := src.Unmap().As4(), dst.Unmap().As4()
var out [][]byte
for off := 0; off < len(udp); off += fragSize {
end := off + fragSize
if end > len(udp) {
end = len(udp)
}
chunk := udp[off:end]
more := end < len(udp)
hdr := make([]byte, 20, 20+len(chunk))
hdr[0] = 0x45 // IPv4, 5 words of header
hdr[1] = 0 // DSCP/ECN
binary.BigEndian.PutUint16(hdr[2:4], uint16(20+len(chunk)))
binary.BigEndian.PutUint16(hdr[4:6], id)
flagsOff := uint16(off / 8)
if more {
flagsOff |= 0x2000 // MF
}
binary.BigEndian.PutUint16(hdr[6:8], flagsOff)
hdr[8] = 64 // TTL
hdr[9] = syscall.IPPROTO_UDP
// hdr[10:12] checksum left zero: the kernel computes it under IP_HDRINCL.
copy(hdr[12:16], s4[:])
copy(hdr[16:20], d4[:])
out = append(out, append(hdr, chunk...))
}
return out
}
@@ -0,0 +1,160 @@
// SPDX-FileCopyrightText: 2026 Echolot contributors
// SPDX-License-Identifier: GPL-3.0-or-later
//go:build linux
package dataplane
import (
"encoding/binary"
"net/netip"
"testing"
)
// Fragment headers are the kind of thing that is either exactly right or silently useless: a
// wrong offset unit, a missing MF bit or a bad checksum produces packets that leave the machine
// and are dropped by the receiver's IP stack without a word. Nothing downstream would notice —
// the client would simply record "fragments do not get through", which is a wrong answer rather
// than a missing one. Hence these check the bytes.
func testAddrs() (netip.AddrPort, netip.AddrPort) {
return netip.MustParseAddrPort("192.0.2.1:8442"), netip.MustParseAddrPort("198.51.100.9:41000")
}
func TestSplitCoversThePayloadExactlyOnce(t *testing.T) {
src, dst := testAddrs()
udp := buildUDP(src, dst, make([]byte, 2000))
frags := splitIPv4(src.Addr(), dst.Addr(), udp, 576, 0x1234)
if len(frags) < 3 {
t.Fatalf("expected several fragments for %d bytes, got %d", len(udp), len(frags))
}
// Reassemble the way a receiver would: place each fragment's payload at its offset.
rebuilt := make([]byte, len(udp))
covered := make([]bool, len(udp))
for _, f := range frags {
flagsOff := binary.BigEndian.Uint16(f[6:8])
off := int(flagsOff&0x1FFF) * 8
body := f[20:]
if off+len(body) > len(udp) {
t.Fatalf("fragment at offset %d overruns the datagram", off)
}
for i, b := range body {
if covered[off+i] {
t.Fatalf("byte %d delivered twice", off+i)
}
covered[off+i] = true
rebuilt[off+i] = b
}
}
for i, c := range covered {
if !c {
t.Fatalf("byte %d was never sent", i)
}
}
for i := range udp {
if rebuilt[i] != udp[i] {
t.Fatalf("reassembled byte %d differs", i)
}
}
}
func TestFragmentHeadersAreWellFormed(t *testing.T) {
src, dst := testAddrs()
udp := buildUDP(src, dst, make([]byte, 3000))
frags := splitIPv4(src.Addr(), dst.Addr(), udp, 800, 0xBEEF)
for i, f := range frags {
if got := f[0]; got != 0x45 {
t.Errorf("fragment %d: version/IHL = %#x, want 0x45", i, got)
}
if got := f[9]; got != 17 {
t.Errorf("fragment %d: protocol = %d, want 17 (UDP)", i, got)
}
if got := binary.BigEndian.Uint16(f[4:6]); got != 0xBEEF {
t.Errorf("fragment %d: IP ID = %#x — all fragments of one datagram must share it", i, got)
}
if got := binary.BigEndian.Uint16(f[2:4]); int(got) != len(f) {
t.Errorf("fragment %d: total length = %d, actual %d", i, got, len(f))
}
flagsOff := binary.BigEndian.Uint16(f[6:8])
mf := flagsOff&0x2000 != 0
wantMF := i < len(frags)-1
if mf != wantMF {
t.Errorf("fragment %d: MF = %v, want %v", i, mf, wantMF)
}
}
}
// Offsets are counted in 8-byte units, so every fragment but the last must be a multiple of 8.
// A 100-byte "fragment size" that silently becomes 100 bytes on the wire produces a datagram no
// host will ever reassemble.
func TestNonFinalFragmentsAreEightByteMultiples(t *testing.T) {
src, dst := testAddrs()
udp := buildUDP(src, dst, make([]byte, 2500))
for _, size := range []int{8, 100, 576, 999, 1400} {
frags := splitIPv4(src.Addr(), dst.Addr(), udp, (size/8)*8, 1)
for i, f := range frags[:len(frags)-1] {
if body := len(f) - 20; body%8 != 0 {
t.Errorf("size %d: non-final fragment %d carries %d bytes, not a multiple of 8",
size, i, body)
}
}
}
}
// The UDP checksum is optional in IPv4, and sending zero would be less code — but a
// zero-checksum datagram is dropped by some middleboxes, and that drop would be recorded as a
// fragmentation failure. So it must be present and correct.
func TestUDPChecksumVerifies(t *testing.T) {
src, dst := testAddrs()
for _, n := range []int{0, 1, 7, 8, 100, 1001} { // odd lengths exercise the tail-byte path
udp := buildUDP(src, dst, make([]byte, n))
if got := binary.BigEndian.Uint16(udp[6:8]); got == 0 {
t.Fatalf("payload %d: checksum is zero, which means 'not computed'", n)
}
if sum := verifyUDPChecksum(src.Addr(), dst.Addr(), udp); sum != 0xFFFF {
t.Errorf("payload %d: checksum does not verify (one's complement sum %#x)", n, sum)
}
if got := binary.BigEndian.Uint16(udp[4:6]); int(got) != len(udp) {
t.Errorf("payload %d: UDP length field %d, actual %d", n, got, len(udp))
}
}
}
func TestUDPPortsComeFromTheSessionAddresses(t *testing.T) {
src, dst := testAddrs()
udp := buildUDP(src, dst, []byte("x"))
if got := binary.BigEndian.Uint16(udp[0:2]); got != src.Port() {
t.Errorf("source port = %d, want %d", got, src.Port())
}
// The destination port must be the client's observed source port, or the datagram arrives
// at the machine and is discarded before any socket sees it.
if got := binary.BigEndian.Uint16(udp[2:4]); got != dst.Port() {
t.Errorf("destination port = %d, want %d", got, dst.Port())
}
}
// Recomputes the one's complement sum over the pseudo-header and datagram; a correct checksum
// makes the total 0xFFFF.
func verifyUDPChecksum(src, dst netip.Addr, udp []byte) uint16 {
var sum uint32
s4, d4 := src.Unmap().As4(), dst.Unmap().As4()
for _, b := range [][]byte{s4[:], d4[:]} {
sum += uint32(binary.BigEndian.Uint16(b[0:2]))
sum += uint32(binary.BigEndian.Uint16(b[2:4]))
}
sum += 17
sum += uint32(len(udp))
for i := 0; i+1 < len(udp); i += 2 {
sum += uint32(binary.BigEndian.Uint16(udp[i : i+2]))
}
if len(udp)%2 == 1 {
sum += uint32(udp[len(udp)-1]) << 8
}
for sum>>16 != 0 {
sum = (sum & 0xFFFF) + (sum >> 16)
}
return uint16(sum)
}
+41
View File
@@ -0,0 +1,41 @@
// SPDX-FileCopyrightText: 2026 Echolot contributors
// SPDX-License-Identifier: GPL-3.0-or-later
//go:build !linux
package dataplane
import (
"fmt"
"echo-lot.app/server/internal/session"
)
// Crafting IP fragments needs a raw socket and Linux's IP_HDRINCL semantics. Off Linux the
// capability is simply not advertised, so a client never asks for it — better than answering
// with a measurement we cannot actually make.
type FragMode string
const (
FragInOrder FragMode = "in_order"
FragReversed FragMode = "reversed"
FragFirstLast FragMode = "first_last"
)
type FragResult struct {
Mode FragMode `json:"mode"`
SizeBytes int `json:"size_bytes"`
Fragments int `json:"fragments"`
Sent bool `json:"sent"`
Err string `json:"err,omitempty"`
}
func RawFragSupported() bool { return false }
func (s *Server) FragSend(
sess *session.Session, g *session.Grant, sizeBytes int, mode FragMode, fragSize int,
) (FragResult, error) {
return FragResult{Mode: mode, SizeBytes: sizeBytes},
fmt.Errorf("crafted fragmentation is only implemented on Linux")
}
+14 -2
View File
@@ -35,6 +35,8 @@ const (
// Server->client under an asymmetric grant (spec §3.4/§5). // Server->client under an asymmetric grant (spec §3.4/§5).
TypeDownTrainData = 0x06 TypeDownTrainData = 0x06
TypeBigSend = 0x0C TypeBigSend = 0x0C
// TypeFragData is delivered only after IP reassembly, so its arrival IS the measurement.
TypeFragData = 0x0D
) )
type Server struct { type Server struct {
@@ -232,6 +234,17 @@ func (s *Server) send(conn *net.UDPConn, raddr netip.AddrPort, sess *session.Ses
// EMSGSIZE means our own egress MTU refused the datagram, which is a different fact from the // EMSGSIZE means our own egress MTU refused the datagram, which is a different fact from the
// client not receiving it. // client not receiving it.
func (s *Server) sendErr(conn *net.UDPConn, raddr netip.AddrPort, sess *session.Session, typ byte, seq uint32, payload []byte) error { func (s *Server) sendErr(conn *net.UDPConn, raddr netip.AddrPort, sess *session.Session, typ byte, seq uint32, payload []byte) error {
pkt := s.buildPacket(sess, typ, seq, payload)
_, err := conn.WriteToUDPAddrPort(pkt, raddr)
return err
}
// buildPacket assembles and signs an ELT1 packet without sending it.
//
// Split out for the crafted-fragment path, which needs the bytes so it can cut them up itself.
// What arrives after reassembly must be indistinguishable from an ordinary packet, or the client
// would be measuring our sender rather than the path — so it goes through exactly this function.
func (s *Server) buildPacket(sess *session.Session, typ byte, seq uint32, payload []byte) []byte {
pkt := make([]byte, HeaderSize+len(payload)) pkt := make([]byte, HeaderSize+len(payload))
copy(pkt[0:4], Magic) copy(pkt[0:4], Magic)
pkt[4] = typ pkt[4] = typ
@@ -247,8 +260,7 @@ func (s *Server) sendErr(conn *net.UDPConn, raddr netip.AddrPort, sess *session.
mac.Write(pkt[0:28]) mac.Write(pkt[0:28])
mac.Write(payload) mac.Write(payload)
copy(pkt[28:32], mac.Sum(nil)[:4]) copy(pkt[28:32], mac.Sum(nil)[:4])
_, err := conn.WriteToUDPAddrPort(pkt, raddr) return pkt
return err
} }
func hexByte(hi, lo byte) byte { func hexByte(hi, lo byte) byte {