diff --git a/CLAUDE.md b/CLAUDE.md index 658475c..bf03e5b 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -52,6 +52,29 @@ echolot-prober/ the capability prober (self-contained Gradle buil ui/ProberScreen.kt result cards colored by verdict ``` +## Client modules (echolot-app/) + +Pure Kotlin/JVM where possible, so the interesting logic is unit-testable without a device and can +be exercised against the live server from the PC: + +- `core-protocol` — control plane (pinned TLS) + ELT1 UDP data plane. **One `ProbeSession` per + server session, for its whole lifetime**: a second one restarts sequence numbers, the server's + anti-replay window discards every packet, and granted sends then target the closed socket. +- `core-measurement` — the schema types. `core-engine` — composes probes into documents. +- `core-privacy` — the §8 anonymizer (`full` / `balanced` / `strict`). Field classification lives + in one table (`Classification.kt`); keep it there rather than annotating models. +- `core-archive` — on-device run storage + retention. `enabled` is separate from the three + ceilings: all-zeros means "no limits", not "keep nothing". + +**The local archive keeps the unredacted document; anonymization happens per upload, on the way +out.** Never redact what is stored locally. + +### Live testing without a device +`echolot-app/scripts/test-fmr.sh [gradle-task] [test-filter]` mints an enrollment token over SSH, +enrolls, computes the SPKI pin from the served cert and runs a `Live*Test` against fmr. This covers +the whole server-facing vertical (granted sends, downstream MTU, uploads) with no phone involved — +use it before asking the user to test on hardware. + ## Conventions - Every probe returns a `ProbeResult` — never throws to the caller (MainActivity wraps anyway). diff --git a/docs/build-status.md b/docs/build-status.md index a64b0e5..eb33e49 100644 --- a/docs/build-status.md +++ b/docs/build-status.md @@ -543,3 +543,81 @@ Best available behavior, now implemented: the banner still opens Shizuku, but th exact steps there ("Pairing", then "Start"), and a second tap target opens **Developer options** (`Settings.ACTION_APPLICATION_DEVELOPMENT_SETTINGS` — public and exported) since Wireless debugging must be enabled first for Shizuku's wireless start to work at all. + +### Downstream measurements: asymmetric grants, DF-mode big_send (server-v0.4.0 … v0.4.2, 2026-08-01) +The client can measure a round trip and the largest packet it can *send*. It cannot measure the +largest packet it can *receive*, or downstream-only loss — those need the server to push, which is +exactly what §3.4 gates behind an asymmetric grant. Implemented and verified live from the PC: + +- **`session.Grant`** — created per action, bound at creation to the session's *observed* + data-plane source (no grant without a verified destination), clamped to server limits, with a + byte budget, an average-rate ceiling and an expiry. Unit-tested for each of those refusals. +- **`downtrain`** — N packets of size S every I µs; the client derives downstream loss, + reordering and inter-arrival spacing. +- **`big_send`** — one datagram per requested size. **DF is on by default**, so the largest size + that arrives *is* the downstream path MTU. Without DF the kernel fragments and the result only + says whether fragments get through — a different fact, and the reason the schema has both + `mtu.pmtud_down` and `mtu.frag_delivery`. Sizes above the server's own egress MTU (from the + startup self-test) are refused up front and reported as `max_df_bytes`, so an absence caused by + our kernel is never read as a limit of the client's path. + +Live from the PC against fmr: downstream path MTU **1500** (1472 payload, DF), fragmented delivery +up to **4000**, downstream train **100/100, 0 % loss, 0 reordered**, inter-arrival 3.3 ms for a +3000 µs send interval. + +#### Two bugs this shook out, both invisible in a single-homed lab +1. **Granted sends went out from the wrong local address** (fixed in server-v0.4.2). fmr binds two + IPv4 addresses; `connFor` returned whichever socket of the right family came first in the bind + list. A train for a session established on `.150` left from `.151` and every packet was dropped + by the client's NAT, which has no mapping for that pair. tcpdump showed all 50 leaving, the + client saw none — reported as *100 % downstream loss*, a confident measurement of something + that never happened. Sessions now record which of our own bound addresses received their + traffic and granted sends go back through that socket; `connfor_test.go` pins both that and the + family fallback. +2. **A second `ProbeSession` on one server session is silently dead.** Sequence numbers restart at + zero client-side while the server's anti-replay window keeps counting, so every packet is + discarded as a replay — and because the server then never records the new source, the grant + still targets the closed socket. `ServerMeasurement` now uses one ProbeSession for the whole + run; `ProbeSession`'s doc comment states the constraint. + +### Run archive, anonymizer and uploads (2026-08-01) +Three pieces, deliberately separate: + +- **`core-archive`** — one JSON file per run plus an index entry, in a plain directory the user can + inspect or delete with a file manager. Retention (max runs / max age / max total bytes) is + enforced on every save rather than by a sweeper. `enabled` is a separate flag from the three + ceilings because "no limits" and "keep nothing" are opposite intentions; collapsing them onto + all-zeros is how a user who turns the caps off ends up with an empty history. 13 tests. +- **`core-privacy`** — the schema §8 anonymizer, three levels. `full` (your own server) changes + nothing; `balanced` pseudonymizes SSIDs/hostnames, keeps the OUI half of a MAC and the /16 of a + public IP, keeps RFC1918 verbatim (it describes topology, not a person), and *drops* neighbour + inventories (SSDP/ARP/scan results) rather than mangling them; `strict` keeps only metrics, + statuses and finding codes. Pseudonyms are consistent within a document and — by default — not + across documents, so an upload endpoint cannot link a device's runs; a stable salt is opt-in for + people diffing their own history. Classification is one readable table, not annotations spread + across modules. 14 tests, each pinning a property someone's privacy depends on. +- **Server-side upload policy** — `off | anonymous | account`, plus max size, retention days, max + runs per device, and the *least* anonymization accepted. The profile advertises all of it so the + app presents the choice honestly instead of discovering the rules by being rejected. `account` + refuses today rather than falling back to anonymous: picking the strict setting before OIDC + lands must not silently mean the loose one. + +**The archive holds the unredacted document; redaction happens on the way out, per upload.** The +local archive is the user's own data on their own device, and redacting it would destroy exactly +the detail that makes a week-old run worth keeping. + +App-side: settings screen (archive limits, privacy level with a plain-language description of what +each keeps, auto-upload off by default, server URL/pin/credential), history screen showing whether +each run left the device, and a **preview of the exact bytes an upload would send** — an anonymizer +the user cannot inspect is only a promise. + +Live round trip against fmr: uploaded a run, listed it, fetched it back and asserted the SSID, the +SSDP neighbour name and the free-text note are absent from what the server stores while the +finding code and the metrics survive, then deleted it. + +### Still open +- `mtu.pmtud_up` (DF + errqueue), `frag_send`, `throughput`, TRAIN_REPORT retrieval. +- Enrollment UI in the app (server URL/pin/credential are typed in by hand today). +- Accounts/OIDC on the server, which is what `uploads=account` is waiting for. +- Nothing in this entry has been exercised on a phone yet — all of it was verified from the PC + against the live server. On-device verification is the next step. diff --git a/echolot-app/core-engine/src/main/kotlin/app/echo_lot/engine/DownstreamMeasurement.kt b/echolot-app/core-engine/src/main/kotlin/app/echo_lot/engine/DownstreamMeasurement.kt new file mode 100644 index 0000000..942c7dc --- /dev/null +++ b/echolot-app/core-engine/src/main/kotlin/app/echo_lot/engine/DownstreamMeasurement.kt @@ -0,0 +1,344 @@ +// SPDX-FileCopyrightText: 2026 Echolot contributors +// SPDX-License-Identifier: GPL-3.0-or-later + +package app.echo_lot.engine + +import app.echo_lot.measurement.* +import app.echo_lot.protocol.ControlClient +import app.echo_lot.protocol.ProbeSession +import app.echo_lot.protocol.Wire +import kotlinx.serialization.SerialName +import kotlinx.serialization.Serializable +import kotlinx.serialization.json.Json +import kotlinx.serialization.json.JsonObject +import kotlinx.serialization.json.encodeToJsonElement + +/** + * The measurements only the far end can make: what the *downstream* path does to traffic the + * client never asked for packet-by-packet. + * + * A client alone can measure a round trip, and it can find the largest packet it can *send*. It + * cannot find the largest packet it can *receive*, or whether the network drops downstream + * packets independently of upstream ones — those need a server willing to push, which is why the + * protocol gates them behind an asymmetric grant (probe-protocol.md §3.4). + * + * Three separate facts come out, and keeping them separate is the point: + * - `mtu.pmtud_down` — the largest datagram that arrives *unfragmented*. This is the number + * that matters for anything setting DF, and it is only meaningful because the server sets DF. + * - `mtu.frag_delivery` — whether larger datagrams arrive once the network is allowed to + * fragment them. A path can be fine for one and broken for the other; conflating them is how + * you get "MTU is 4000" on a link that drops every DF packet over 1400. + * - `train.udp_downstream` — loss, reordering and arrival spacing in the download direction. + */ +class DownstreamMeasurement(private val ids: IdSource) { + + private val json = Json { encodeDefaults = true; explicitNulls = true } + + /** How long to wait for a granted burst after the server accepts the action. */ + private val collectWindowMs = 4_000L + + /** + * Runs all three against an already-primed session. + * + * [session] must already have sent at least one ECHO: the grant is bound to the source the + * server has actually observed, so an unprimed session gets a 409 rather than a grant. That + * is the anti-amplification rule doing its job, not an error to work around. + */ + fun run( + credential: String, + sessionId: String, + control: ControlClient, + probe: ProbeSession, + sessionRef: String, + sizes: List = DEFAULT_SIZES, + trainCount: Int = 100, + trainSizeBytes: Int = 300, + trainIntervalUs: Int = 3_000, + ): Pair, List> { + val tests = ArrayList() + val findings = ArrayList() + + val df = bigSend(credential, sessionId, control, probe, sessionRef, sizes, df = true) + val frag = bigSend(credential, sessionId, control, probe, sessionRef, sizes, df = false) + val train = downTrain( + credential, sessionId, control, probe, sessionRef, trainCount, trainSizeBytes, trainIntervalUs, + ) + + tests.add(df.test); tests.add(frag.test); tests.add(train.test) + + // A downstream MTU below the classic 1500-byte Ethernet payload is worth saying out loud: + // it is the usual cause of "small requests work, large responses hang". + val pathMtu = df.largestDelivered + if (pathMtu != null && pathMtu > 0) { + val ipMtu = pathMtu + IP_UDP_OVERHEAD4 + if (ipMtu < 1500) { + findings.add( + finding( + "mtu.reduced_downstream", Category.MTU, Severity.LOW, df.test.id, + "Downstream path MTU is $ipMtu bytes, below 1500", + "The largest datagram that reached this device without fragmenting was " + + "$pathMtu bytes of payload ($ipMtu on the wire). Tunnels (PPPoE, VPN, " + + "IPv6-in-IPv4) commonly do this; it is only a fault when something " + + "on the path also blocks the ICMP messages that let senders discover it.", + ), + ) + } + // The dangerous combination: unfragmented large packets vanish AND fragments do too, + // so a sender that never gets told will retransmit into a black hole. + val fragLargest = frag.largestDelivered ?: 0 + if (fragLargest <= pathMtu && sizes.any { it > pathMtu }) { + findings.add( + finding( + "mtu.downstream_blackhole", Category.MTU, Severity.MEDIUM, frag.test.id, + "Datagrams above $pathMtu bytes are dropped downstream, fragmented or not", + "Nothing larger than $pathMtu bytes arrived, even when the network was " + + "free to fragment it. Traffic that relies on large responses will " + + "stall rather than fail cleanly.", + ), + ) + } + } + + if (train.received == 0) { + findings.add( + finding( + "connectivity.downstream_blocked", Category.CONNECTIVITY, Severity.HIGH, train.test.id, + "No server-initiated packets arrived", + "The server sent ${train.sent} packets toward this device and none arrived, " + + "while the round-trip echo worked. Something on the path forwards replies " + + "but drops traffic the device did not individually solicit.", + ), + ) + } else if (train.lossPct >= 5.0) { + findings.add( + finding( + "connectivity.downstream_loss", Category.CONNECTIVITY, Severity.MEDIUM, train.test.id, + "Downstream loss of ${round1(train.lossPct)}%", + "${train.sent - train.received} of ${train.sent} packets sent toward this " + + "device were lost. Downstream loss is invisible to a round-trip test, " + + "which reports only that *something* was lost somewhere.", + ), + ) + } + if (train.reordered > 0) { + findings.add( + finding( + "connectivity.downstream_reorder", Category.CONNECTIVITY, Severity.LOW, train.test.id, + "${train.reordered} downstream packet(s) arrived out of order", + "Packets arrived in a different order than they were sent. Usually per-packet " + + "load balancing across links; harmless for most traffic, not for all of it.", + ), + ) + } + return tests to findings + } + + // ---- big_send --------------------------------------------------------------------- + + private class SizeResult(val test: Test, val largestDelivered: Int?) + + private fun bigSend( + credential: String, sessionId: String, control: ControlClient, probe: ProbeSession, + sessionRef: String, sizes: List, df: Boolean, + ): SizeResult { + val testId = ids.uuid() + val started = ids.monoNs() + val requested = sizes.joinToString(",") + + val reply = runCatching { + control.action( + credential, sessionId, + """{"action":"big_send","df":$df,"sizes_bytes":[$requested]}""", + ) + } + if (reply.isFailure) { + return SizeResult( + Test( + id = testId, type = if (df) TestType.MTU_PMTUD_DOWN else TestType.MTU_FRAG_DELIVERY, + sessionRef = sessionRef, tier = Tier.APP, + startedMonoNs = started, endedMonoNs = ids.monoNs(), + status = TestStatus.UNSUPPORTED, + error = TestError("action_refused", reply.exceptionOrNull()?.message ?: "big_send refused"), + ), + null, + ) + } + + // The server tells us which sizes it actually put on the wire. With DF it refuses + // anything above its own egress MTU, and treating those as "lost downstream" would + // blame the client's network for our own limit. + val accepted = parseIntArray(reply.getOrNull(), "sizes_bytes").ifEmpty { sizes } + val serverMaxDf = parseInt(reply.getOrNull(), "max_df_bytes") + + val arrived = probe.collectGranted(collectWindowMs) + .filter { it.type == Wire.TYPE_BIG_SEND } + .map { it.sizeBytes } + .distinct() + .sorted() + val largest = arrived.maxOrNull() + + val metrics = json.encodeToJsonElement( + BigSendMetrics( + requestedBytes = sizes, + sentBytes = accepted, + deliveredBytes = arrived, + largestDeliveredBytes = largest, + dontFragment = df, + serverMaxDfBytes = serverMaxDf, + // Only meaningful for the DF run; the IP-level MTU is the payload plus headers. + pathMtuBytes = if (df && largest != null) largest + IP_UDP_OVERHEAD4 else null, + ), + ) as JsonObject + + val status = when { + arrived.isEmpty() -> TestStatus.FAILED + arrived.size < accepted.size -> TestStatus.PARTIAL + else -> TestStatus.OK + } + return SizeResult( + Test( + id = testId, type = if (df) TestType.MTU_PMTUD_DOWN else TestType.MTU_FRAG_DELIVERY, + sessionRef = sessionRef, tier = Tier.APP, + startedMonoNs = started, endedMonoNs = ids.monoNs(), + status = status, metrics = metrics, + ), + largest, + ) + } + + // ---- downtrain -------------------------------------------------------------------- + + private class TrainResult( + val test: Test, val sent: Int, val received: Int, val lossPct: Double, val reordered: Int, + ) + + private fun downTrain( + credential: String, sessionId: String, control: ControlClient, probe: ProbeSession, + sessionRef: String, count: Int, sizeBytes: Int, intervalUs: Int, + ): TrainResult { + val testId = ids.uuid() + val started = ids.monoNs() + + val reply = runCatching { + control.action( + credential, sessionId, + """{"action":"downtrain","count":$count,"size_bytes":$sizeBytes,"interval_us":$intervalUs}""", + ) + } + if (reply.isFailure) { + return TrainResult( + Test( + id = testId, type = TestType.TRAIN_UDP_DOWNSTREAM, sessionRef = sessionRef, + tier = Tier.APP, startedMonoNs = started, endedMonoNs = ids.monoNs(), + status = TestStatus.UNSUPPORTED, + error = TestError("action_refused", reply.exceptionOrNull()?.message ?: "downtrain refused"), + ), + 0, 0, 0.0, 0, + ) + } + val sent = parseInt(reply.getOrNull(), "count") ?: count + + val got = probe.collectGranted(collectWindowMs).filter { it.type == Wire.TYPE_DOWNTRAIN_DATA } + val received = got.size + val lossPct = if (sent == 0) 0.0 else (sent - received) * 100.0 / sent + + // Reordering: a packet whose sequence is below the highest already seen. Counting + // inversions rather than "not sorted" keeps one late packet from being reported as + // dozens of reorder events. + var highest = -1 + var reordered = 0 + for (p in got) { + if (p.seq < highest) reordered++ else highest = p.seq + } + + // Columnar evidence per the schema: what arrived, when, and how big — so every metric + // above is recomputable by a reader who does not trust our arithmetic. + val evidence = TrainEvidence( + epochMonoNs = started, + seq = got.map { it.seq }, + tTxNs = got.map { null }, + tRxNs = got.map { it.tRxNs }, + sizeBytes = got.map { it.sizeBytes }, + ).toEvidence() + + val interArrival = got.zipWithNext { a, b -> (b.tRxNs - a.tRxNs) / 1_000_000.0 } + val metrics = json.encodeToJsonElement( + DownTrainMetrics( + sent = sent, received = received, lossPct = round1(lossPct), + reorderedPackets = reordered, + sizeBytes = sizeBytes, + interArrivalMsAvg = interArrival.average().takeIf { interArrival.isNotEmpty() }?.let(::round1), + interArrivalMsMax = interArrival.maxOrNull()?.let(::round1), + sendIntervalUs = intervalUs, + ), + ) as JsonObject + + val status = when { + received == 0 -> TestStatus.FAILED + received < sent -> TestStatus.PARTIAL + else -> TestStatus.OK + } + return TrainResult( + Test( + id = testId, type = TestType.TRAIN_UDP_DOWNSTREAM, sessionRef = sessionRef, tier = Tier.APP, + startedMonoNs = started, endedMonoNs = ids.monoNs(), + status = status, evidence = evidence, metrics = metrics, + ), + sent, received, lossPct, reordered, + ) + } + + // ---- helpers ---------------------------------------------------------------------- + + private fun finding(code: String, cat: Category, sev: Severity, testId: String, title: String, desc: String) = + Finding( + id = ids.uuid(), code = code, category = cat, severity = sev, confidence = Confidence.HIGH, + title = title, description = desc, evidenceRefs = listOf(EvidenceRef(testId)), + ) + + /** Minimal scalar extraction from the action reply; the shape is small and server-owned. */ + private fun parseInt(body: String?, key: String): Int? = + body?.let { Regex("\"$key\"\\s*:\\s*(-?\\d+)").find(it)?.groupValues?.get(1)?.toIntOrNull() } + + private fun parseIntArray(body: String?, key: String): List = + body?.let { b -> + Regex("\"$key\"\\s*:\\s*\\[([^\\]]*)\\]").find(b)?.groupValues?.get(1) + ?.split(",")?.mapNotNull { it.trim().toIntOrNull() } + } ?: emptyList() + + private companion object { + /** IPv4 (20) + UDP (8). The v6 case is 48; reported per-family once v6 sessions land. */ + const val IP_UDP_OVERHEAD4 = 28 + + /** Straddles the usual suspects: 1500 Ethernet, 1492 PPPoE, 1400-ish tunnels. */ + val DEFAULT_SIZES = listOf(600, 1200, 1372, 1400, 1450, 1472, 1500, 2000, 4000) + + fun round1(v: Double) = Math.round(v * 10.0) / 10.0 + } +} + +/** Metrics for mtu.pmtud_down / mtu.frag_delivery. */ +@Serializable +data class BigSendMetrics( + @SerialName("requested_bytes") val requestedBytes: List, + @SerialName("sent_bytes") val sentBytes: List, + @SerialName("delivered_bytes") val deliveredBytes: List, + @SerialName("largest_delivered_bytes") val largestDeliveredBytes: Int? = null, + @SerialName("dont_fragment") val dontFragment: Boolean, + /** The server's own DF ceiling; sizes above it were never sent and are not path evidence. */ + @SerialName("server_max_df_bytes") val serverMaxDfBytes: Int? = null, + @SerialName("path_mtu_bytes") val pathMtuBytes: Int? = null, +) + +/** Metrics for train.udp_downstream. */ +@Serializable +data class DownTrainMetrics( + val sent: Int, + val received: Int, + @SerialName("loss_pct") val lossPct: Double, + @SerialName("reordered_packets") val reorderedPackets: Int, + @SerialName("size_bytes") val sizeBytes: Int, + @SerialName("inter_arrival_ms_avg") val interArrivalMsAvg: Double? = null, + @SerialName("inter_arrival_ms_max") val interArrivalMsMax: Double? = null, + @SerialName("send_interval_us") val sendIntervalUs: Int, +) diff --git a/echolot-app/core-engine/src/main/kotlin/app/echo_lot/engine/ServerMeasurement.kt b/echolot-app/core-engine/src/main/kotlin/app/echo_lot/engine/ServerMeasurement.kt index bc1ba68..90f4939 100644 --- a/echolot-app/core-engine/src/main/kotlin/app/echo_lot/engine/ServerMeasurement.kt +++ b/echolot-app/core-engine/src/main/kotlin/app/echo_lot/engine/ServerMeasurement.kt @@ -36,6 +36,12 @@ class ServerMeasurement( val udpPort: Int, val echoCount: Int = 20, val echoPaddingBytes: Int = 64, + /** + * Whether to ask the server to push traffic back (downstream MTU and downstream train). + * Costs a few hundred kB of download and needs a server that advertises the grants, so + * it is a flag rather than an assumption. + */ + val downstream: Boolean = true, ) fun run(cfg: Config): MeasurementDocument { @@ -57,11 +63,32 @@ class ServerMeasurement( target = SessionTarget(ip4 = cfg.udpHost, udpPort = cfg.udpPort), ) - val (test, findings) = echoTrain(cfg, control, session, startMono) + val tests = ArrayList() + val allFindings = ArrayList() + + // One ProbeSession for the whole run. A second one would open a new socket and restart + // the sequence counter, which the server's anti-replay window correctly rejects — so the + // re-primed source is never recorded and every granted send goes to the old, closed port. + // Session identity lives on the server; the socket must live as long as it does. + ProbeSession(cfg.credential, session, cfg.udpHost, cfg.udpPort).use { ps -> + val (test, findings) = echoTrain(cfg, ps, startMono) + tests.add(test) + allFindings.addAll(findings) + + // Downstream needs a session the server has already seen traffic from — the echo + // train just provided that — and a server that advertises the grants. Skipped + // quietly against an older server rather than reported as a failure of the network. + if (cfg.downstream && profile.supports("downtrain") && profile.supports("big-send")) { + val (dsTests, dsFindings) = DownstreamMeasurement(ids) + .run(cfg.credential, session.sessionId, control, ps, sessionRef = "sess-1") + tests.addAll(dsTests) + allFindings.addAll(dsFindings) + } + } control.deleteSession(cfg.credential, session.sessionId) - val summary = Verdicts.derive(listOf(test), findings) + val summary = Verdicts.derive(tests, allFindings) return MeasurementDocument( run = Run( id = runId, trigger = Trigger.MANUAL, startedAt = startWall, endedAt = ids.nowWall(), @@ -70,14 +97,14 @@ class ServerMeasurement( tiers = Tiers(app = true), ), serverSessions = listOf(serverSession), - tests = listOf(test), - findings = findings, + tests = tests, + findings = allFindings, summary = summary, ) } private fun echoTrain( - cfg: Config, control: ControlClient, session: app.echo_lot.protocol.SessionResponse, startMono: Long, + cfg: Config, ps: ProbeSession, startMono: Long, ): Pair> { val testId = ids.uuid() val seqs = ArrayList() @@ -87,20 +114,18 @@ class ServerMeasurement( val rtts = ArrayList() val observedPorts = LinkedHashSet() - ProbeSession(cfg.credential, session, cfg.udpHost, cfg.udpPort).use { ps -> - for (i in 0 until cfg.echoCount) { - val txMono = ids.monoNs() - startMono - val r = ps.echo(cfg.echoPaddingBytes) - seqs.add(i) - tTx.add(txMono) - sizes.add(Wire_HEADER + cfg.echoPaddingBytes) - if (r != null) { - tRx.add(ids.monoNs() - startMono) - rtts.add(r.rttMs) - r.observation?.observedPort?.let { observedPorts.add(it) } - } else { - tRx.add(null) - } + for (i in 0 until cfg.echoCount) { + val txMono = ids.monoNs() - startMono + val r = ps.echo(cfg.echoPaddingBytes) + seqs.add(i) + tTx.add(txMono) + sizes.add(Wire_HEADER + cfg.echoPaddingBytes) + if (r != null) { + tRx.add(ids.monoNs() - startMono) + rtts.add(r.rttMs) + r.observation?.observedPort?.let { observedPorts.add(it) } + } else { + tRx.add(null) } } diff --git a/echolot-app/core-engine/src/test/kotlin/app/echo_lot/engine/LiveDownstreamTest.kt b/echolot-app/core-engine/src/test/kotlin/app/echo_lot/engine/LiveDownstreamTest.kt new file mode 100644 index 0000000..548125f --- /dev/null +++ b/echolot-app/core-engine/src/test/kotlin/app/echo_lot/engine/LiveDownstreamTest.kt @@ -0,0 +1,68 @@ +// SPDX-FileCopyrightText: 2026 Echolot contributors +// SPDX-License-Identifier: GPL-3.0-or-later + +package app.echo_lot.engine + +import app.echo_lot.measurement.TestStatus +import app.echo_lot.measurement.TestType +import app.echo_lot.protocol.ControlClient +import app.echo_lot.protocol.ProbeSession +import kotlin.test.Test as JTest +import kotlin.test.assertEquals +import kotlin.test.assertNotNull +import kotlin.test.assertTrue + +/** + * Runs DownstreamMeasurement against a LIVE server and checks the *documents* it produces, not + * just that packets moved: the tests must carry recomputable metrics and land on the right test + * types, because that is what an archived run is read back as. Self-skips without ECHOLOT_LIVE_*. + */ +class LiveDownstreamTest { + + private val url = System.getenv("ECHOLOT_LIVE_URL") + private val pin = System.getenv("ECHOLOT_LIVE_PIN") + private val cred = System.getenv("ECHOLOT_LIVE_CRED") + private val udp = System.getenv("ECHOLOT_LIVE_UDP") + private val target = System.getenv("ECHOLOT_LIVE_TARGET") ?: "fmr" + + @JTest + fun producesDownstreamTestsAndFindings() { + if (url == null || pin == null || cred == null || udp == null) { + println("LiveDownstreamTest skipped (no ECHOLOT_LIVE_* env)"); return + } + val control = ControlClient(url, setOf(pin)) + val session = control.createSession(cred, target) + val (host, port) = udp.split(":").let { it[0] to it[1].toInt() } + + val (tests, findings) = ProbeSession(cred, session, host, port).use { ps -> + ps.echo() // prime: the grant binds to the source the server has actually observed + DownstreamMeasurement(SystemIdSource()) + .run(cred, session.sessionId, control, ps, sessionRef = "sess-1") + } + control.deleteSession(cred, session.sessionId) + + for (t in tests) println("${t.type} status=${t.status} metrics=${t.metrics}") + for (f in findings) println("finding ${f.code} [${f.severity}] ${f.title}") + + assertEquals(3, tests.size, "expected pmtud_down, frag_delivery and a downstream train") + val byType = tests.associateBy { it.type } + + val pmtud = assertNotNull(byType[TestType.MTU_PMTUD_DOWN], "no mtu.pmtud_down test") + assertTrue(pmtud.status == TestStatus.OK || pmtud.status == TestStatus.PARTIAL, + "DF probe did not deliver anything: ${pmtud.status}") + val pathMtu = pmtud.metrics?.get("path_mtu_bytes")?.toString()?.toIntOrNull() + assertNotNull(pathMtu, "pmtud_down must report a path MTU") + assertTrue(pathMtu in 576..9000, "implausible downstream path MTU: $pathMtu") + println("downstream path MTU = $pathMtu bytes") + + val frag = assertNotNull(byType[TestType.MTU_FRAG_DELIVERY], "no mtu.frag_delivery test") + assertNotNull(frag.metrics?.get("largest_delivered_bytes")) + + val train = assertNotNull(byType[TestType.TRAIN_UDP_DOWNSTREAM], "no downstream train") + assertNotNull(train.evidence, "a train without columnar evidence is not recomputable") + val received = train.metrics?.get("received")?.toString()?.toIntOrNull() ?: 0 + assertTrue(received > 0, "no downstream train packets arrived") + println("downstream train: $received received, loss=${train.metrics?.get("loss_pct")}, " + + "reordered=${train.metrics?.get("reordered_packets")}") + } +} diff --git a/echolot-app/core-engine/src/test/kotlin/app/echo_lot/engine/LiveMeasurementTest.kt b/echolot-app/core-engine/src/test/kotlin/app/echo_lot/engine/LiveMeasurementTest.kt index 1619b38..d2ce1a3 100644 --- a/echolot-app/core-engine/src/test/kotlin/app/echo_lot/engine/LiveMeasurementTest.kt +++ b/echolot-app/core-engine/src/test/kotlin/app/echo_lot/engine/LiveMeasurementTest.kt @@ -47,8 +47,11 @@ class LiveMeasurementTest { assertEquals(1, doc.serverSessions.size) assertTrue(doc.serverSessions[0].capabilities.contains("udp-probe")) - val test = doc.tests.single() - assertEquals(TestType.TRAIN_UDP_UPDOWN, test.type) + // A full run is the echo train plus the three downstream tests; assert on the one this + // test is about rather than on the count, so adding a measurement is not a test edit. + for (t in doc.tests) println(" ${t.type} → ${t.status}") + for (f in doc.findings) println(" finding ${f.code} [${f.severity}] ${f.title}") + val test = doc.tests.first { it.type == TestType.TRAIN_UDP_UPDOWN } assertTrue(test.status == TestStatus.OK || test.status == TestStatus.PARTIAL, "expected replies from live server, got ${test.status}") diff --git a/echolot-app/core-measurement/bin/main/app/echo_lot/measurement/Test.kt b/echolot-app/core-measurement/bin/main/app/echo_lot/measurement/Test.kt index c26f202..3393dcd 100644 --- a/echolot-app/core-measurement/bin/main/app/echo_lot/measurement/Test.kt +++ b/echolot-app/core-measurement/bin/main/app/echo_lot/measurement/Test.kt @@ -69,6 +69,8 @@ object TestType { const val TRACEROUTE_ICMP6 = "traceroute.icmp6" // train const val TRAIN_UDP_UPDOWN = "train.udp_updown" + /** Server-to-client train under a §3.4 grant: the direction a round trip cannot separate. */ + const val TRAIN_UDP_DOWNSTREAM = "train.udp_downstream" // mtu const val MTU_PMTUD_UP = "mtu.pmtud_up" const val MTU_PMTUD_DOWN = "mtu.pmtud_down" diff --git a/echolot-app/core-measurement/src/main/kotlin/app/echo_lot/measurement/Test.kt b/echolot-app/core-measurement/src/main/kotlin/app/echo_lot/measurement/Test.kt index c26f202..3393dcd 100644 --- a/echolot-app/core-measurement/src/main/kotlin/app/echo_lot/measurement/Test.kt +++ b/echolot-app/core-measurement/src/main/kotlin/app/echo_lot/measurement/Test.kt @@ -69,6 +69,8 @@ object TestType { const val TRACEROUTE_ICMP6 = "traceroute.icmp6" // train const val TRAIN_UDP_UPDOWN = "train.udp_updown" + /** Server-to-client train under a §3.4 grant: the direction a round trip cannot separate. */ + const val TRAIN_UDP_DOWNSTREAM = "train.udp_downstream" // mtu const val MTU_PMTUD_UP = "mtu.pmtud_up" const val MTU_PMTUD_DOWN = "mtu.pmtud_down" diff --git a/echolot-app/core-protocol/src/main/kotlin/app/echo_lot/protocol/ProbeSession.kt b/echolot-app/core-protocol/src/main/kotlin/app/echo_lot/protocol/ProbeSession.kt index 6266726..76803a2 100644 --- a/echolot-app/core-protocol/src/main/kotlin/app/echo_lot/protocol/ProbeSession.kt +++ b/echolot-app/core-protocol/src/main/kotlin/app/echo_lot/protocol/ProbeSession.kt @@ -12,6 +12,11 @@ import java.util.Base64 * A data-plane session (probe-protocol.md §3): derives the session key, then sends signed ELT1 * packets to the server's UDP endpoint and reads back verified responses. One session ↔ one * server target. Blocking; the caller owns threading. + * + * One instance per server session, for its whole lifetime. Sequence numbers start at zero here + * while the server's anti-replay window (§3.2) keeps counting, so a second instance sharing a + * session id has all its packets discarded as replays — and, because the server then never + * records the new source, any granted send still targets the socket that was closed. */ class ProbeSession( private val credential: String, diff --git a/web/README.md b/web/README.md new file mode 100644 index 0000000..f554032 --- /dev/null +++ b/web/README.md @@ -0,0 +1,74 @@ +# Echolot website (`web/`) + +Minimal single-page site for [echo-lot.app](https://echo-lot.app), served from Cloudflare +Workers. Static files in `public/` are served straight from the edge; the tiny Worker in +`src/index.js` only runs for paths that aren't files: + +| Path | Behavior | +| ------------- | ------------------------------------------------------------------------ | +| `/apk` | 302 → newest `.apk` asset of the latest Gitea release (QR-code friendly) | +| `/apk.sha256` | 302 → the matching `.sha256` asset | +| `/api/latest` | JSON `{version, published_at, apk, sha256}` — the homepage's version readout | +| `/fdroid`, `/source` | 302 → the URLs configured in `wrangler.jsonc` vars | + +The latest release is resolved from the Gitea API **at request time** (edge-cached 5 min), so +publishing a release — `git tag v0.2.0 && git push origin v0.2.0`, which triggers +`.gitea/workflows/release.yml` — is the only release step. The site never needs a redeploy for +a new version, and empty/unreachable values fall back to the homepage instead of 404ing. + +Light/dark follows the OS (`prefers-color-scheme`), no toggle, no JS required for it. Colors +come from the branding palette (teal = instrument, single amber point = finding). + +`public/assets/` (favicon, wordmark, social preview) are **copies** of `../assets/branding/` — +that directory is the source of truth; re-copy after any branding change. + +## Deploy + +Everything is driven by [wrangler](https://developers.cloudflare.com/workers/wrangler/), config +in `wrangler.jsonc`. No build step, no node_modules to commit. + +### One-time setup + +1. In the Cloudflare dashboard, add **echo-lot.app** as a zone (and point the domain's + nameservers at Cloudflare). The `routes` in `wrangler.jsonc` use `custom_domain: true`, so + wrangler creates the DNS records for `echo-lot.app` and `www` automatically on first deploy — + the zone just has to exist in the same account. +2. Auth, either flavor: + - **Interactive:** `npx wrangler login` (opens the browser once, stores an OAuth token). + - **API token (also what CI uses):** dashboard → My Profile → API Tokens → create from the + **"Edit Cloudflare Workers"** template. Then: + + ``` + $env:CLOUDFLARE_API_TOKEN = "..." # PowerShell; export ... on POSIX + $env:CLOUDFLARE_ACCOUNT_ID = "..." # dashboard → Workers & Pages, right sidebar + ``` + +### Deploy + +``` +cd web +npx wrangler@4 deploy +``` + +That's it — uploads `src/index.js` + the `public/` assets, wires the custom domains. Useful +extras: `npx wrangler dev` (local preview at localhost:8787), `npx wrangler tail` (live logs), +`npx wrangler versions list`. + +### CI deploy (Gitea Actions) + +`.gitea/workflows/deploy-site.yml` runs `wrangler deploy` on every push to `main`/`master` that +touches `web/`. It stays inert until you add two repo secrets (Settings → Actions → Secrets): +`CLOUDFLARE_API_TOKEN` and `CLOUDFLARE_ACCOUNT_ID` (same values as above). + +Cloudflare's raw REST API (`PUT /accounts/:id/workers/scripts/...`) exists, but the assets +upload needs a manifest/session dance that wrangler already implements — use wrangler even in +automation. + +## Config knobs (`wrangler.jsonc` → `vars`) + +- `GITEA_REPO_API` — Gitea repo API base; releases must be publicly readable. +- `DOWNLOAD_URL` — manual `/apk` fallback while Gitea is unreachable. +- `FDROID_URL` — set when the F-Droid listing exists; until then `/fdroid` loops home. +- `SOURCE_URL` — public source mirror for the footer + `/source`. + +Vars are plain (non-secret) config; change + `wrangler deploy` to apply. diff --git a/web/public/assets/icon.png b/web/public/assets/icon.png new file mode 100644 index 0000000..15ae7f7 Binary files /dev/null and b/web/public/assets/icon.png differ diff --git a/web/public/assets/icon.svg b/web/public/assets/icon.svg new file mode 100644 index 0000000..cc36d74 --- /dev/null +++ b/web/public/assets/icon.svg @@ -0,0 +1,24 @@ + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/web/public/assets/social-preview.png b/web/public/assets/social-preview.png new file mode 100644 index 0000000..54654d1 Binary files /dev/null and b/web/public/assets/social-preview.png differ diff --git a/web/public/assets/wordmark-on-dark.svg b/web/public/assets/wordmark-on-dark.svg new file mode 100644 index 0000000..bf9b2a7 --- /dev/null +++ b/web/public/assets/wordmark-on-dark.svg @@ -0,0 +1,17 @@ + + + + + + + + + + + + + + + + + diff --git a/web/public/assets/wordmark-on-light.svg b/web/public/assets/wordmark-on-light.svg new file mode 100644 index 0000000..49a0cf6 --- /dev/null +++ b/web/public/assets/wordmark-on-light.svg @@ -0,0 +1,17 @@ + + + + + + + + + + + + + + + + + diff --git a/web/public/index.html b/web/public/index.html index 0a78ab7..6331895 100644 --- a/web/public/index.html +++ b/web/public/index.html @@ -5,103 +5,124 @@ -Echolot — depth soundings for your local network +Echolot — measure, don't guess - + - + + + + + +