Compare commits
4
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
0c5b021b63 | ||
|
|
277e33da75 | ||
|
|
14e5fad1b2 | ||
|
|
ce1aaa332a |
@@ -32,10 +32,25 @@ Keep prober result IDs aligned with the measurement-schema test-type registry.
|
|||||||
|
|
||||||
## Versioning
|
## Versioning
|
||||||
|
|
||||||
Prefer **patch** bumps (`server-v0.3.1`) for additive/incremental work; reserve **minor** bumps
|
Both artifacts are **SemVer**. Prefer **patch** bumps (`server-v0.3.1`) for additive/incremental
|
||||||
for real milestones. Don't burn through minor versions. Tags are namespaced: `server-v*` for the
|
work; reserve **minor** bumps for real milestones. Don't burn through minor versions. Tags are
|
||||||
Go server, `v*` for the app. Pushing a `server-v*` tag runs CI → binaries + Gitea release +
|
namespaced: `server-v*` for the Go server, `v*` for the app. Pushing a `server-v*` tag runs CI →
|
||||||
registry image; the server on fmr can `--self-update` from those releases.
|
binaries + Gitea release + registry image; the server on fmr can `--self-update` from those
|
||||||
|
releases.
|
||||||
|
|
||||||
|
The app's version lives once, as `appVersionName` in `app/build.gradle.kts`; **`versionCode` is
|
||||||
|
derived from it** (`major*1e6 + minor*1e4 + patch*10`). Never set it by hand — a second number a
|
||||||
|
human has to remember to bump eventually disagrees with the first.
|
||||||
|
|
||||||
|
**Versions are load-bearing** (probe-protocol.md §8): the server refuses apps outside its window
|
||||||
|
with `426`, and the app refuses servers outside its own. Two axes, kept separate:
|
||||||
|
- `protocol_version` — *can* they talk. The correctness axis; below 1.0.0 the **minor** is the
|
||||||
|
breaking axis.
|
||||||
|
- release-version window — *may* they, per policy. `[min, max)`, bounds at breaking boundaries so
|
||||||
|
a patch never strands a fleet. Client bounds: `Compat.kt`. Server: `ECHOLOT_MIN/MAX_APP_VERSION`.
|
||||||
|
|
||||||
|
Raise a minimum only when older peers are actively harmful, and say why in the constant's comment.
|
||||||
|
`GET /v1/profile` must stay ungated — it is how a refused client learns what it needs.
|
||||||
|
|
||||||
## Layout
|
## Layout
|
||||||
|
|
||||||
@@ -52,6 +67,29 @@ echolot-prober/ the capability prober (self-contained Gradle buil
|
|||||||
ui/ProberScreen.kt result cards colored by verdict
|
ui/ProberScreen.kt result cards colored by verdict
|
||||||
```
|
```
|
||||||
|
|
||||||
|
## Client modules (echolot-app/)
|
||||||
|
|
||||||
|
Pure Kotlin/JVM where possible, so the interesting logic is unit-testable without a device and can
|
||||||
|
be exercised against the live server from the PC:
|
||||||
|
|
||||||
|
- `core-protocol` — control plane (pinned TLS) + ELT1 UDP data plane. **One `ProbeSession` per
|
||||||
|
server session, for its whole lifetime**: a second one restarts sequence numbers, the server's
|
||||||
|
anti-replay window discards every packet, and granted sends then target the closed socket.
|
||||||
|
- `core-measurement` — the schema types. `core-engine` — composes probes into documents.
|
||||||
|
- `core-privacy` — the §8 anonymizer (`full` / `balanced` / `strict`). Field classification lives
|
||||||
|
in one table (`Classification.kt`); keep it there rather than annotating models.
|
||||||
|
- `core-archive` — on-device run storage + retention. `enabled` is separate from the three
|
||||||
|
ceilings: all-zeros means "no limits", not "keep nothing".
|
||||||
|
|
||||||
|
**The local archive keeps the unredacted document; anonymization happens per upload, on the way
|
||||||
|
out.** Never redact what is stored locally.
|
||||||
|
|
||||||
|
### Live testing without a device
|
||||||
|
`echolot-app/scripts/test-fmr.sh [gradle-task] [test-filter]` mints an enrollment token over SSH,
|
||||||
|
enrolls, computes the SPKI pin from the served cert and runs a `Live*Test` against fmr. This covers
|
||||||
|
the whole server-facing vertical (granted sends, downstream MTU, uploads) with no phone involved —
|
||||||
|
use it before asking the user to test on hardware.
|
||||||
|
|
||||||
## Conventions
|
## Conventions
|
||||||
|
|
||||||
- Every probe returns a `ProbeResult` — never throws to the caller (MainActivity wraps anyway).
|
- Every probe returns a `ProbeResult` — never throws to the caller (MainActivity wraps anyway).
|
||||||
@@ -107,6 +145,12 @@ First build downloads AGP/Compose/Shizuku from Google Maven + Maven Central.
|
|||||||
Shizuku, **toggle Wireless debugging off/on** — Shizuku keeps running (separate process), a fresh
|
Shizuku, **toggle Wireless debugging off/on** — Shizuku keeps running (separate process), a fresh
|
||||||
port + mDNS record appear, and the beacon/connector recover. Plan the Shizuku-tier dev loop
|
port + mDNS record appear, and the beacon/connector recover. Plan the Shizuku-tier dev loop
|
||||||
around this (or USB, if ever available).
|
around this (or USB, if ever available).
|
||||||
|
- **Empty-jar race with the IDE.** VSCodium's Java/Kotlin extension runs its own Gradle daemon on
|
||||||
|
the same project; when it overlaps a CLI build, a module's `build/libs/*.jar` can end up
|
||||||
|
containing only a manifest, and Gradle then considers `jar` up-to-date. Dependent modules fail
|
||||||
|
with "Unresolved reference" on symbols that plainly exist. Fix: `rm -f <module>/build/libs/*.jar`
|
||||||
|
and re-run the `jar` task. Suspect this whenever a reference resolves in one module but not in
|
||||||
|
its consumer.
|
||||||
- As of the AGP 9.2.0 / Gradle 9.6.0 / Kotlin 2.2.10 bump, JDK 17+ (including 25) works —
|
- As of the AGP 9.2.0 / Gradle 9.6.0 / Kotlin 2.2.10 bump, JDK 17+ (including 25) works —
|
||||||
AGP 9 requires Gradle 9.1.0+ and Kotlin 2.2.10+ as its minimum KGP version. On machines with
|
AGP 9 requires Gradle 9.1.0+ and Kotlin 2.2.10+ as its minimum KGP version. On machines with
|
||||||
Android Studio, its `jbr` directory works as `JAVA_HOME`.
|
Android Studio, its `jbr` directory works as `JAVA_HOME`.
|
||||||
|
|||||||
@@ -543,3 +543,81 @@ Best available behavior, now implemented: the banner still opens Shizuku, but th
|
|||||||
exact steps there ("Pairing", then "Start"), and a second tap target opens **Developer options**
|
exact steps there ("Pairing", then "Start"), and a second tap target opens **Developer options**
|
||||||
(`Settings.ACTION_APPLICATION_DEVELOPMENT_SETTINGS` — public and exported) since Wireless
|
(`Settings.ACTION_APPLICATION_DEVELOPMENT_SETTINGS` — public and exported) since Wireless
|
||||||
debugging must be enabled first for Shizuku's wireless start to work at all.
|
debugging must be enabled first for Shizuku's wireless start to work at all.
|
||||||
|
|
||||||
|
### Downstream measurements: asymmetric grants, DF-mode big_send (server-v0.4.0 … v0.4.2, 2026-08-01)
|
||||||
|
The client can measure a round trip and the largest packet it can *send*. It cannot measure the
|
||||||
|
largest packet it can *receive*, or downstream-only loss — those need the server to push, which is
|
||||||
|
exactly what §3.4 gates behind an asymmetric grant. Implemented and verified live from the PC:
|
||||||
|
|
||||||
|
- **`session.Grant`** — created per action, bound at creation to the session's *observed*
|
||||||
|
data-plane source (no grant without a verified destination), clamped to server limits, with a
|
||||||
|
byte budget, an average-rate ceiling and an expiry. Unit-tested for each of those refusals.
|
||||||
|
- **`downtrain`** — N packets of size S every I µs; the client derives downstream loss,
|
||||||
|
reordering and inter-arrival spacing.
|
||||||
|
- **`big_send`** — one datagram per requested size. **DF is on by default**, so the largest size
|
||||||
|
that arrives *is* the downstream path MTU. Without DF the kernel fragments and the result only
|
||||||
|
says whether fragments get through — a different fact, and the reason the schema has both
|
||||||
|
`mtu.pmtud_down` and `mtu.frag_delivery`. Sizes above the server's own egress MTU (from the
|
||||||
|
startup self-test) are refused up front and reported as `max_df_bytes`, so an absence caused by
|
||||||
|
our kernel is never read as a limit of the client's path.
|
||||||
|
|
||||||
|
Live from the PC against fmr: downstream path MTU **1500** (1472 payload, DF), fragmented delivery
|
||||||
|
up to **4000**, downstream train **100/100, 0 % loss, 0 reordered**, inter-arrival 3.3 ms for a
|
||||||
|
3000 µs send interval.
|
||||||
|
|
||||||
|
#### Two bugs this shook out, both invisible in a single-homed lab
|
||||||
|
1. **Granted sends went out from the wrong local address** (fixed in server-v0.4.2). fmr binds two
|
||||||
|
IPv4 addresses; `connFor` returned whichever socket of the right family came first in the bind
|
||||||
|
list. A train for a session established on `.150` left from `.151` and every packet was dropped
|
||||||
|
by the client's NAT, which has no mapping for that pair. tcpdump showed all 50 leaving, the
|
||||||
|
client saw none — reported as *100 % downstream loss*, a confident measurement of something
|
||||||
|
that never happened. Sessions now record which of our own bound addresses received their
|
||||||
|
traffic and granted sends go back through that socket; `connfor_test.go` pins both that and the
|
||||||
|
family fallback.
|
||||||
|
2. **A second `ProbeSession` on one server session is silently dead.** Sequence numbers restart at
|
||||||
|
zero client-side while the server's anti-replay window keeps counting, so every packet is
|
||||||
|
discarded as a replay — and because the server then never records the new source, the grant
|
||||||
|
still targets the closed socket. `ServerMeasurement` now uses one ProbeSession for the whole
|
||||||
|
run; `ProbeSession`'s doc comment states the constraint.
|
||||||
|
|
||||||
|
### Run archive, anonymizer and uploads (2026-08-01)
|
||||||
|
Three pieces, deliberately separate:
|
||||||
|
|
||||||
|
- **`core-archive`** — one JSON file per run plus an index entry, in a plain directory the user can
|
||||||
|
inspect or delete with a file manager. Retention (max runs / max age / max total bytes) is
|
||||||
|
enforced on every save rather than by a sweeper. `enabled` is a separate flag from the three
|
||||||
|
ceilings because "no limits" and "keep nothing" are opposite intentions; collapsing them onto
|
||||||
|
all-zeros is how a user who turns the caps off ends up with an empty history. 13 tests.
|
||||||
|
- **`core-privacy`** — the schema §8 anonymizer, three levels. `full` (your own server) changes
|
||||||
|
nothing; `balanced` pseudonymizes SSIDs/hostnames, keeps the OUI half of a MAC and the /16 of a
|
||||||
|
public IP, keeps RFC1918 verbatim (it describes topology, not a person), and *drops* neighbour
|
||||||
|
inventories (SSDP/ARP/scan results) rather than mangling them; `strict` keeps only metrics,
|
||||||
|
statuses and finding codes. Pseudonyms are consistent within a document and — by default — not
|
||||||
|
across documents, so an upload endpoint cannot link a device's runs; a stable salt is opt-in for
|
||||||
|
people diffing their own history. Classification is one readable table, not annotations spread
|
||||||
|
across modules. 14 tests, each pinning a property someone's privacy depends on.
|
||||||
|
- **Server-side upload policy** — `off | anonymous | account`, plus max size, retention days, max
|
||||||
|
runs per device, and the *least* anonymization accepted. The profile advertises all of it so the
|
||||||
|
app presents the choice honestly instead of discovering the rules by being rejected. `account`
|
||||||
|
refuses today rather than falling back to anonymous: picking the strict setting before OIDC
|
||||||
|
lands must not silently mean the loose one.
|
||||||
|
|
||||||
|
**The archive holds the unredacted document; redaction happens on the way out, per upload.** The
|
||||||
|
local archive is the user's own data on their own device, and redacting it would destroy exactly
|
||||||
|
the detail that makes a week-old run worth keeping.
|
||||||
|
|
||||||
|
App-side: settings screen (archive limits, privacy level with a plain-language description of what
|
||||||
|
each keeps, auto-upload off by default, server URL/pin/credential), history screen showing whether
|
||||||
|
each run left the device, and a **preview of the exact bytes an upload would send** — an anonymizer
|
||||||
|
the user cannot inspect is only a promise.
|
||||||
|
|
||||||
|
Live round trip against fmr: uploaded a run, listed it, fetched it back and asserted the SSID, the
|
||||||
|
SSDP neighbour name and the free-text note are absent from what the server stores while the
|
||||||
|
finding code and the metrics survive, then deleted it.
|
||||||
|
|
||||||
|
### Still open
|
||||||
|
- `mtu.pmtud_up` (DF + errqueue), `frag_send`, `throughput`, TRAIN_REPORT retrieval.
|
||||||
|
- Enrollment UI in the app (server URL/pin/credential are typed in by hand today).
|
||||||
|
- Accounts/OIDC on the server, which is what `uploads=account` is waiting for.
|
||||||
|
- Nothing in this entry has been exercised on a phone yet — all of it was verified from the PC
|
||||||
|
against the live server. On-device verification is the next step.
|
||||||
|
|||||||
+65
-2
@@ -192,14 +192,77 @@ Note: exact RDATA constants to be frozen in the implementation's `dns_reference.
|
|||||||
|
|
||||||
Same daemon, separate listener (default localhost-only): health + self-test (are both IPs live, is the canary zone delegated correctly, is UDP reachable from outside — tested via a public echolot "mirror" if configured), enrollment token management (create/expire/scope), device list + revocation, retention settings, QR rendering (client-side JS). Out of scope for this spec beyond the endpoints above.
|
Same daemon, separate listener (default localhost-only): health + self-test (are both IPs live, is the canary zone delegated correctly, is UDP reachable from outside — tested via a public echolot "mirror" if configured), enrollment token management (create/expire/scope), device list + revocation, retention settings, QR rendering (client-side JS). Out of scope for this spec beyond the endpoints above.
|
||||||
|
|
||||||
## 8. Cross-references to the measurement schema
|
## 8. Version compatibility
|
||||||
|
|
||||||
|
Both artifacts are versioned with **SemVer**: the Go server (`server-vX.Y.Z` tags) and the Android
|
||||||
|
app (`versionName`; `versionCode` is derived from it, never maintained separately). Two independent
|
||||||
|
things are checked, and conflating them is the mistake this section exists to prevent.
|
||||||
|
|
||||||
|
### 8.1 Protocol version — *can* these builds talk?
|
||||||
|
|
||||||
|
`protocol_version` is the version of **this document**. It is advertised in the profile
|
||||||
|
(`compat.protocol_version`) and is the correctness axis: a peer in a different breaking series
|
||||||
|
cannot be talked to, whatever its release version says. Below `1.0.0` the **minor** is the breaking
|
||||||
|
axis (SemVer §4); at and above it, the major is. A patch bump of the protocol never splits a fleet.
|
||||||
|
|
||||||
|
### 8.2 Release-version window — *should* they, per policy?
|
||||||
|
|
||||||
|
Each side declares the range of peer release versions it will work with, as `[min, max)` —
|
||||||
|
**minimum inclusive, maximum exclusive**, because the useful bound is always "the version that
|
||||||
|
broke it" and writing that literally is unambiguous. An empty maximum means unbounded.
|
||||||
|
|
||||||
|
The server advertises its window and enforces it:
|
||||||
|
|
||||||
|
```jsonc
|
||||||
|
"compat": {
|
||||||
|
"protocol_version": "1.0.0",
|
||||||
|
"schema_version": "1.0.0",
|
||||||
|
"app_min": "0.2.0",
|
||||||
|
"app_max": "1.0.0" // exclusive; "" = no upper bound
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
Operators override it with `ECHOLOT_MIN_APP_VERSION` / `ECHOLOT_MAX_APP_VERSION` (or
|
||||||
|
`--min-app-version` / `--max-app-version`). A malformed bound is **fatal at startup**, not ignored:
|
||||||
|
a typo must not silently disable a restriction the operator meant to set.
|
||||||
|
|
||||||
|
The app sends its version on every control-plane request:
|
||||||
|
|
||||||
|
```
|
||||||
|
X-Echolot-App-Version: 0.2.0
|
||||||
|
```
|
||||||
|
|
||||||
|
and carries its own bounds for the server (`MIN_SERVER` / `MAX_SERVER` in `Compat.kt`). It checks
|
||||||
|
the profile in **both** directions — is the server in our range, and are we in the server's — so a
|
||||||
|
mismatch is reported before a run starts rather than discovered halfway through one.
|
||||||
|
|
||||||
|
### 8.3 Rules
|
||||||
|
|
||||||
|
1. **`GET /v1/profile` is never gated.** It is where a refused client learns which version it needs.
|
||||||
|
Gating it leaves the user with a network error instead of an answer, defeating the check.
|
||||||
|
2. **Refusal is `426 Upgrade Required`**, with a body naming both versions and the accepted window:
|
||||||
|
```json
|
||||||
|
{ "error": "app 0.1.0 is older than this build supports (needs >= 0.2.0, < 1.0.0). Update the app.",
|
||||||
|
"app_version": "0.1.0", "accepts_app": ">= 0.2.0, < 1.0.0",
|
||||||
|
"server_version": "0.5.0", "protocol_version": "1.0.0" }
|
||||||
|
```
|
||||||
|
3. **An unparseable or absent version is `unknown`, and is allowed.** Development builds report
|
||||||
|
`dev`, and a client too old to send the header cannot be identified anyway. The check exists to
|
||||||
|
turn confusing failures into clear ones; refusing what it cannot identify does the opposite.
|
||||||
|
4. **Bounds move at breaking boundaries, not at releases.** Shipping a patch must never require
|
||||||
|
editing a range. A minimum is raised only when older peers are actually harmful — e.g. the app
|
||||||
|
requires server `>= 0.4.2` because earlier multi-homed servers sent granted traffic from an
|
||||||
|
address the session never used, which the client measured as 100 % downstream loss. A
|
||||||
|
confidently wrong measurement is worse than a refused one.
|
||||||
|
|
||||||
|
## 9. Cross-references to the measurement schema
|
||||||
|
|
||||||
- Observation-block fields (§3.3) appear as `*_seen_by_server` columns in `train` evidence (schema §6.2).
|
- Observation-block fields (§3.3) appear as `*_seen_by_server` columns in `train` evidence (schema §6.2).
|
||||||
- `TIMESYNC` (§3.2) produces the `time.server_offset` test; without it, cross-clock fields must not be compared (schema §6.2 note).
|
- `TIMESYNC` (§3.2) produces the `time.server_offset` test; without it, cross-clock fields must not be compared (schema §6.2 note).
|
||||||
- Capability strings (§2.3) are copied verbatim into `server_sessions[].capabilities` (schema §5); tests skipped for missing capability get `status: "unsupported"`.
|
- Capability strings (§2.3) are copied verbatim into `server_sessions[].capabilities` (schema §5); tests skipped for missing capability get `status: "unsupported"`.
|
||||||
- `session_id` maps to `server_sessions[].session_id`; `action_id`s appear in test `params`.
|
- `session_id` maps to `server_sessions[].session_id`; `action_id`s appear in test `params`.
|
||||||
|
|
||||||
## 9. Open items
|
## 10. Open items
|
||||||
|
|
||||||
1. Whether TRAIN_REPORT should also stream during long trains (partial reports every N packets) for live UI feedback — leaning yes, same type with a `flags` bit.
|
1. Whether TRAIN_REPORT should also stream during long trains (partial reports every N packets) for live UI feedback — leaning yes, same type with a `flags` bit.
|
||||||
2. Throughput methodology (fixed streams vs BBR-style ramp) — decide with `perf.*` test design.
|
2. Throughput methodology (fixed streams vs BBR-style ramp) — decide with `perf.*` test design.
|
||||||
|
|||||||
@@ -8,6 +8,20 @@ plugins {
|
|||||||
alias(libs.plugins.kotlin.serialization)
|
alias(libs.plugins.kotlin.serialization)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// The app's version is SemVer and lives here, once. versionCode is derived from it rather than
|
||||||
|
// maintained alongside: Play/F-Droid need a monotonically increasing integer, but a second number
|
||||||
|
// that a human has to remember to bump is a number that eventually disagrees with the first — and
|
||||||
|
// the version is now load-bearing, since the server decides whether to serve us by it.
|
||||||
|
//
|
||||||
|
// major*1_000_000 + minor*10_000 + patch*10 leaves room for 9 patch-level rebuilds (the trailing
|
||||||
|
// digit) without disturbing the mapping, and stays inside the 2_100_000_000 ceiling until major 2100.
|
||||||
|
val appVersionName = "0.2.0"
|
||||||
|
|
||||||
|
fun versionCodeOf(semver: String): Int {
|
||||||
|
val (major, minor, patch) = semver.substringBefore('-').split(".").map(String::toInt)
|
||||||
|
return major * 1_000_000 + minor * 10_000 + patch * 10
|
||||||
|
}
|
||||||
|
|
||||||
android {
|
android {
|
||||||
namespace = "app.echo_lot.app"
|
namespace = "app.echo_lot.app"
|
||||||
compileSdk = 36
|
compileSdk = 36
|
||||||
@@ -16,12 +30,15 @@ android {
|
|||||||
applicationId = "app.echo_lot.app"
|
applicationId = "app.echo_lot.app"
|
||||||
minSdk = 26
|
minSdk = 26
|
||||||
targetSdk = 36
|
targetSdk = 36
|
||||||
versionCode = 1
|
versionCode = versionCodeOf(appVersionName)
|
||||||
versionName = "0.1.0"
|
versionName = appVersionName
|
||||||
// Automation: `adb shell am start -n app.echo_lot.app/.MainActivity --ez autorun true`
|
// Automation: `adb shell am start -n app.echo_lot.app/.MainActivity --ez autorun true`
|
||||||
// runs a measurement immediately and POSTs the report here (dev collection endpoint).
|
// runs a measurement immediately and POSTs the report here (dev collection endpoint).
|
||||||
buildConfigField("String", "REPORT_UPLOAD_URL", "\"http://89.185.109.150:443/report\"")
|
buildConfigField("String", "REPORT_UPLOAD_URL", "\"http://89.185.109.150:443/report\"")
|
||||||
buildConfigField("String", "REPORT_UPLOAD_SECRET", "\"D4OmG5gGJsElqVVbtYIZbR\"")
|
buildConfigField("String", "REPORT_UPLOAD_SECRET", "\"D4OmG5gGJsElqVVbtYIZbR\"")
|
||||||
|
// The bare SemVer, without the debug build's "-dev" suffix stripped away by the server's
|
||||||
|
// parser anyway — sent to servers so they can apply their compatibility window.
|
||||||
|
buildConfigField("String", "APP_SEMVER", "\"$appVersionName\"")
|
||||||
}
|
}
|
||||||
buildTypes {
|
buildTypes {
|
||||||
release { isMinifyEnabled = false }
|
release { isMinifyEnabled = false }
|
||||||
@@ -48,6 +65,8 @@ dependencies {
|
|||||||
implementation(project(":core-engine"))
|
implementation(project(":core-engine"))
|
||||||
implementation(project(":core-probe"))
|
implementation(project(":core-probe"))
|
||||||
implementation(project(":core-shizuku"))
|
implementation(project(":core-shizuku"))
|
||||||
|
implementation(project(":core-privacy"))
|
||||||
|
implementation(project(":core-archive"))
|
||||||
|
|
||||||
implementation(libs.kotlinx.serialization.json)
|
implementation(libs.kotlinx.serialization.json)
|
||||||
implementation(libs.kotlinx.coroutines.android)
|
implementation(libs.kotlinx.coroutines.android)
|
||||||
|
|||||||
@@ -0,0 +1,107 @@
|
|||||||
|
// SPDX-FileCopyrightText: 2026 Echolot contributors
|
||||||
|
// SPDX-License-Identifier: GPL-3.0-or-later
|
||||||
|
|
||||||
|
package app.echo_lot.app
|
||||||
|
|
||||||
|
import androidx.compose.foundation.layout.Arrangement
|
||||||
|
import androidx.compose.foundation.layout.Column
|
||||||
|
import androidx.compose.foundation.layout.Row
|
||||||
|
import androidx.compose.foundation.layout.fillMaxWidth
|
||||||
|
import androidx.compose.foundation.layout.padding
|
||||||
|
import androidx.compose.foundation.lazy.LazyColumn
|
||||||
|
import androidx.compose.foundation.lazy.items
|
||||||
|
import androidx.compose.material3.Card
|
||||||
|
import androidx.compose.material3.MaterialTheme
|
||||||
|
import androidx.compose.material3.Text
|
||||||
|
import androidx.compose.material3.TextButton
|
||||||
|
import androidx.compose.runtime.Composable
|
||||||
|
import androidx.compose.ui.Alignment
|
||||||
|
import androidx.compose.ui.Modifier
|
||||||
|
import androidx.compose.ui.graphics.Color
|
||||||
|
import androidx.compose.ui.unit.dp
|
||||||
|
import app.echo_lot.archive.ArchivedRun
|
||||||
|
import java.time.Instant
|
||||||
|
import java.time.ZoneId
|
||||||
|
import java.time.format.DateTimeFormatter
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Archived runs, newest first.
|
||||||
|
*
|
||||||
|
* Each row states plainly whether the run left the device, because "is this backed up / did I
|
||||||
|
* share this?" is the question a history list actually gets asked.
|
||||||
|
*/
|
||||||
|
@Composable
|
||||||
|
fun HistoryScreen(
|
||||||
|
runs: List<ArchivedRun>,
|
||||||
|
status: String?,
|
||||||
|
onOpen: (String) -> Unit,
|
||||||
|
onUpload: (String) -> Unit,
|
||||||
|
onDelete: (String) -> Unit,
|
||||||
|
onBack: () -> Unit,
|
||||||
|
) {
|
||||||
|
Column(Modifier.fillMaxWidth().padding(16.dp), verticalArrangement = Arrangement.spacedBy(10.dp)) {
|
||||||
|
Row(verticalAlignment = Alignment.CenterVertically) {
|
||||||
|
TextButton(onClick = onBack) { Text("‹ Back") }
|
||||||
|
Text("History", style = MaterialTheme.typography.titleLarge)
|
||||||
|
}
|
||||||
|
status?.let { Text(it, style = MaterialTheme.typography.bodySmall) }
|
||||||
|
|
||||||
|
if (runs.isEmpty()) {
|
||||||
|
Text(
|
||||||
|
"No archived runs yet. Finished runs are kept here automatically unless you turn " +
|
||||||
|
"archiving off in settings.",
|
||||||
|
style = MaterialTheme.typography.bodyMedium,
|
||||||
|
)
|
||||||
|
return@Column
|
||||||
|
}
|
||||||
|
|
||||||
|
LazyColumn(verticalArrangement = Arrangement.spacedBy(8.dp)) {
|
||||||
|
items(runs, key = { it.id }) { r ->
|
||||||
|
Card(Modifier.fillMaxWidth()) {
|
||||||
|
Column(Modifier.padding(12.dp), verticalArrangement = Arrangement.spacedBy(4.dp)) {
|
||||||
|
Row(verticalAlignment = Alignment.CenterVertically) {
|
||||||
|
Text(
|
||||||
|
r.verdict?.uppercase() ?: "—",
|
||||||
|
color = verdictTint(r.verdict),
|
||||||
|
style = MaterialTheme.typography.titleMedium,
|
||||||
|
)
|
||||||
|
Text(
|
||||||
|
" " + humanTime(r.savedAtEpochMs),
|
||||||
|
style = MaterialTheme.typography.bodyMedium,
|
||||||
|
)
|
||||||
|
}
|
||||||
|
Text(
|
||||||
|
"${r.findingCount} finding(s) · ${r.sizeBytes / 1024} kB · ${r.anonymization}",
|
||||||
|
style = MaterialTheme.typography.bodySmall,
|
||||||
|
)
|
||||||
|
Text(
|
||||||
|
if (r.uploaded) "uploaded to ${r.uploadedTo ?: "a server"}"
|
||||||
|
else "on this device only",
|
||||||
|
style = MaterialTheme.typography.bodySmall,
|
||||||
|
color = if (r.uploaded) Color(0xFF7FD17F) else Color(0xFFBBBBBB),
|
||||||
|
)
|
||||||
|
Row(horizontalArrangement = Arrangement.spacedBy(4.dp)) {
|
||||||
|
TextButton(onClick = { onOpen(r.id) }) { Text("Export") }
|
||||||
|
TextButton(onClick = { onUpload(r.id) }) {
|
||||||
|
Text(if (r.uploaded) "Upload again" else "Upload")
|
||||||
|
}
|
||||||
|
TextButton(onClick = { onDelete(r.id) }) { Text("Delete") }
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
private fun verdictTint(v: String?): Color = when (v?.lowercase()) {
|
||||||
|
"green", "ok", "pass" -> Color(0xFF7FD17F)
|
||||||
|
"yellow", "warn" -> Color(0xFFE0C060)
|
||||||
|
"red", "fail" -> Color(0xFFE07070)
|
||||||
|
else -> Color(0xFFBBBBBB)
|
||||||
|
}
|
||||||
|
|
||||||
|
private val stamp: DateTimeFormatter =
|
||||||
|
DateTimeFormatter.ofPattern("yyyy-MM-dd HH:mm").withZone(ZoneId.systemDefault())
|
||||||
|
|
||||||
|
private fun humanTime(epochMs: Long): String = stamp.format(Instant.ofEpochMilli(epochMs))
|
||||||
@@ -18,6 +18,10 @@ import androidx.compose.foundation.shape.RoundedCornerShape
|
|||||||
import androidx.compose.foundation.verticalScroll
|
import androidx.compose.foundation.verticalScroll
|
||||||
import androidx.compose.material3.*
|
import androidx.compose.material3.*
|
||||||
import androidx.compose.runtime.Composable
|
import androidx.compose.runtime.Composable
|
||||||
|
import androidx.compose.runtime.getValue
|
||||||
|
import androidx.compose.runtime.mutableStateOf
|
||||||
|
import androidx.compose.runtime.remember
|
||||||
|
import androidx.compose.runtime.setValue
|
||||||
import androidx.compose.ui.Alignment
|
import androidx.compose.ui.Alignment
|
||||||
import androidx.compose.ui.Modifier
|
import androidx.compose.ui.Modifier
|
||||||
import androidx.compose.ui.graphics.Color
|
import androidx.compose.ui.graphics.Color
|
||||||
@@ -26,9 +30,14 @@ import androidx.compose.ui.text.font.FontWeight
|
|||||||
import androidx.compose.ui.unit.dp
|
import androidx.compose.ui.unit.dp
|
||||||
import androidx.compose.ui.unit.sp
|
import androidx.compose.ui.unit.sp
|
||||||
import androidx.core.content.ContextCompat
|
import androidx.core.content.ContextCompat
|
||||||
|
import androidx.lifecycle.lifecycleScope
|
||||||
|
import kotlinx.coroutines.launch
|
||||||
import androidx.lifecycle.viewmodel.compose.viewModel
|
import androidx.lifecycle.viewmodel.compose.viewModel
|
||||||
import app.echo_lot.measurement.*
|
import app.echo_lot.measurement.*
|
||||||
|
|
||||||
|
/** The app's three top-level screens. */
|
||||||
|
private enum class Screen { RUN, HISTORY, SETTINGS }
|
||||||
|
|
||||||
class MainActivity : ComponentActivity() {
|
class MainActivity : ComponentActivity() {
|
||||||
|
|
||||||
private val permissionLauncher =
|
private val permissionLauncher =
|
||||||
@@ -41,13 +50,17 @@ class MainActivity : ComponentActivity() {
|
|||||||
MaterialTheme(colorScheme = darkColorScheme()) {
|
MaterialTheme(colorScheme = darkColorScheme()) {
|
||||||
Surface(color = MaterialTheme.colorScheme.background) {
|
Surface(color = MaterialTheme.colorScheme.background) {
|
||||||
val vm: RunViewModel = viewModel()
|
val vm: RunViewModel = viewModel()
|
||||||
|
// Three flat screens, so a plain state variable beats a navigation library:
|
||||||
|
// there is no back stack to model beyond "return to the run screen".
|
||||||
|
var screen by remember { mutableStateOf(Screen.RUN) }
|
||||||
|
var preview by remember { mutableStateOf<String?>(null) }
|
||||||
// Automation entry point:
|
// Automation entry point:
|
||||||
// adb shell am start -n app.echo_lot.app/.MainActivity --ez autorun true
|
// adb shell am start -n app.echo_lot.app/.MainActivity --ez autorun true
|
||||||
// starts a run immediately and uploads the report, so an unattended
|
// starts a run immediately and uploads the report, so an unattended
|
||||||
// measurement needs no UI tapping and no adb round-trip to collect.
|
// measurement needs no UI tapping and no adb round-trip to collect.
|
||||||
val autorun = intent?.getBooleanExtra("autorun", false) == true
|
val autorun = intent?.getBooleanExtra("autorun", false) == true
|
||||||
androidx.compose.runtime.LaunchedEffect(autorun) {
|
androidx.compose.runtime.LaunchedEffect(autorun) {
|
||||||
if (autorun) vm.run(upload = true)
|
if (autorun) vm.run(devUpload = true)
|
||||||
}
|
}
|
||||||
// In autorun the app is a batch job: once the run is done AND the upload
|
// In autorun the app is a batch job: once the run is done AND the upload
|
||||||
// succeeded, show the result briefly, then close so the device is left as it
|
// succeeded, show the result briefly, then close so the device is left as it
|
||||||
@@ -61,7 +74,44 @@ class MainActivity : ComponentActivity() {
|
|||||||
finish()
|
finish()
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
EcholotScreen(
|
when (screen) {
|
||||||
|
Screen.SETTINGS -> SettingsScreen(
|
||||||
|
settings = vm.settings,
|
||||||
|
archivedRuns = vm.state.history.size,
|
||||||
|
archivedBytes = vm.archivedBytes(),
|
||||||
|
onApplyRetention = vm::applyRetention,
|
||||||
|
onDeleteAll = vm::deleteAllRuns,
|
||||||
|
onPreviewUpload = {
|
||||||
|
// Preview the newest run, since that is the one the user just made
|
||||||
|
// and the one they are deciding about.
|
||||||
|
vm.state.history.firstOrNull()?.let { r ->
|
||||||
|
lifecycleScope.launch { preview = vm.uploadPreview(r.id) }
|
||||||
|
}
|
||||||
|
},
|
||||||
|
onCheckServer = vm::checkServer,
|
||||||
|
serverStatus = vm.state.archiveStatus,
|
||||||
|
onBack = { screen = Screen.RUN },
|
||||||
|
)
|
||||||
|
Screen.HISTORY -> HistoryScreen(
|
||||||
|
runs = vm.state.history,
|
||||||
|
status = vm.state.archiveStatus,
|
||||||
|
onOpen = { id ->
|
||||||
|
lifecycleScope.launch {
|
||||||
|
vm.readRun(id)?.let { text ->
|
||||||
|
startActivity(
|
||||||
|
Intent.createChooser(
|
||||||
|
Report.shareJson(this@MainActivity, id, text),
|
||||||
|
"Export Echolot run",
|
||||||
|
)
|
||||||
|
)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
},
|
||||||
|
onUpload = vm::uploadRun,
|
||||||
|
onDelete = vm::deleteRun,
|
||||||
|
onBack = { screen = Screen.RUN },
|
||||||
|
)
|
||||||
|
Screen.RUN -> EcholotScreen(
|
||||||
state = vm.state,
|
state = vm.state,
|
||||||
onRun = { vm.run() },
|
onRun = { vm.run() },
|
||||||
onCancel = vm::cancel,
|
onCancel = vm::cancel,
|
||||||
@@ -86,8 +136,14 @@ class MainActivity : ComponentActivity() {
|
|||||||
}
|
}
|
||||||
},
|
},
|
||||||
onExport = { doc -> startActivity(Intent.createChooser(Report.share(this, doc), "Export Echolot run")) },
|
onExport = { doc -> startActivity(Intent.createChooser(Report.share(this, doc), "Export Echolot run")) },
|
||||||
|
onOpenSettings = { screen = Screen.SETTINGS },
|
||||||
|
onOpenHistory = { vm.refreshHistory(); screen = Screen.HISTORY },
|
||||||
)
|
)
|
||||||
}
|
}
|
||||||
|
preview?.let { text ->
|
||||||
|
UploadPreviewDialog(text) { preview = null }
|
||||||
|
}
|
||||||
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -123,6 +179,8 @@ private fun EcholotScreen(
|
|||||||
onShizukuAction: () -> Unit,
|
onShizukuAction: () -> Unit,
|
||||||
onDeveloperOptions: () -> Unit,
|
onDeveloperOptions: () -> Unit,
|
||||||
onExport: (MeasurementDocument) -> Unit,
|
onExport: (MeasurementDocument) -> Unit,
|
||||||
|
onOpenSettings: () -> Unit,
|
||||||
|
onOpenHistory: () -> Unit,
|
||||||
) {
|
) {
|
||||||
Column(
|
Column(
|
||||||
Modifier
|
Modifier
|
||||||
@@ -135,8 +193,15 @@ private fun EcholotScreen(
|
|||||||
.verticalScroll(rememberScrollState()),
|
.verticalScroll(rememberScrollState()),
|
||||||
verticalArrangement = Arrangement.spacedBy(12.dp),
|
verticalArrangement = Arrangement.spacedBy(12.dp),
|
||||||
) {
|
) {
|
||||||
|
Row(Modifier.fillMaxWidth(), verticalAlignment = Alignment.CenterVertically) {
|
||||||
|
Column(Modifier.weight(1f)) {
|
||||||
Text("Echolot", fontSize = 26.sp, fontWeight = FontWeight.SemiBold)
|
Text("Echolot", fontSize = 26.sp, fontWeight = FontWeight.SemiBold)
|
||||||
Text("measure, don't guess", color = MaterialTheme.colorScheme.onSurfaceVariant, fontSize = 13.sp)
|
Text("measure, don't guess",
|
||||||
|
color = MaterialTheme.colorScheme.onSurfaceVariant, fontSize = 13.sp)
|
||||||
|
}
|
||||||
|
TextButton(onClick = onOpenHistory) { Text("History") }
|
||||||
|
TextButton(onClick = onOpenSettings) { Text("Settings") }
|
||||||
|
}
|
||||||
|
|
||||||
// Shell-tier readiness, before the run. Nothing is shown when Shizuku isn't installed —
|
// Shell-tier readiness, before the run. Nothing is shown when Shizuku isn't installed —
|
||||||
// only users who actually use it get reminded that it must be running.
|
// only users who actually use it get reminded that it must be running.
|
||||||
@@ -185,6 +250,10 @@ private fun EcholotScreen(
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
state.archiveStatus?.let {
|
||||||
|
Text(it, fontSize = 12.sp, color = MaterialTheme.colorScheme.onSurfaceVariant)
|
||||||
|
}
|
||||||
|
|
||||||
state.uploadStatus?.let {
|
state.uploadStatus?.let {
|
||||||
Text(it, fontSize = 12.sp, color = MaterialTheme.colorScheme.onSurfaceVariant)
|
Text(it, fontSize = 12.sp, color = MaterialTheme.colorScheme.onSurfaceVariant)
|
||||||
}
|
}
|
||||||
@@ -338,3 +407,24 @@ private fun Dot(color: Color) {
|
|||||||
private fun SectionTitle(text: String) {
|
private fun SectionTitle(text: String) {
|
||||||
Text(text, fontWeight = FontWeight.SemiBold, fontSize = 15.sp, modifier = Modifier.padding(top = 8.dp))
|
Text(text, fontWeight = FontWeight.SemiBold, fontSize = 15.sp, modifier = Modifier.padding(top = 8.dp))
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Shows the exact JSON an upload would send.
|
||||||
|
*
|
||||||
|
* This exists because an anonymizer the user cannot inspect is just a promise. Being able to
|
||||||
|
* read the outgoing document — and find their own SSID absent from it — is what makes the
|
||||||
|
* privacy setting checkable rather than merely stated.
|
||||||
|
*/
|
||||||
|
@Composable
|
||||||
|
private fun UploadPreviewDialog(text: String, onDismiss: () -> Unit) {
|
||||||
|
AlertDialog(
|
||||||
|
onDismissRequest = onDismiss,
|
||||||
|
confirmButton = { TextButton(onClick = onDismiss) { Text("Close") } },
|
||||||
|
title = { Text("This is what would be uploaded") },
|
||||||
|
text = {
|
||||||
|
Column(Modifier.heightIn(max = 420.dp).verticalScroll(rememberScrollState())) {
|
||||||
|
Text(text, fontSize = 10.sp, fontFamily = FontFamily.Monospace)
|
||||||
|
}
|
||||||
|
},
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|||||||
@@ -17,10 +17,18 @@ object Report {
|
|||||||
fun toJson(doc: MeasurementDocument): String =
|
fun toJson(doc: MeasurementDocument): String =
|
||||||
json.encodeToString(MeasurementDocument.serializer(), doc)
|
json.encodeToString(MeasurementDocument.serializer(), doc)
|
||||||
|
|
||||||
fun share(ctx: Context, doc: MeasurementDocument): Intent {
|
fun share(ctx: Context, doc: MeasurementDocument): Intent =
|
||||||
|
shareJson(ctx, doc.run.id, toJson(doc))
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Shares an already-serialized run — an archived one, whose bytes must go out exactly as
|
||||||
|
* stored rather than being re-serialized through the model (which would silently drop
|
||||||
|
* anything a newer schema version added).
|
||||||
|
*/
|
||||||
|
fun shareJson(ctx: Context, runId: String, json: String): Intent {
|
||||||
val dir = File(ctx.cacheDir, "reports").apply { mkdirs() }
|
val dir = File(ctx.cacheDir, "reports").apply { mkdirs() }
|
||||||
val file = File(dir, "echolot-run-${doc.run.id}.json")
|
val file = File(dir, "echolot-run-$runId.json")
|
||||||
file.writeText(toJson(doc))
|
file.writeText(json)
|
||||||
val uri = FileProvider.getUriForFile(ctx, "${ctx.packageName}.fileprovider", file)
|
val uri = FileProvider.getUriForFile(ctx, "${ctx.packageName}.fileprovider", file)
|
||||||
return Intent(Intent.ACTION_SEND).apply {
|
return Intent(Intent.ACTION_SEND).apply {
|
||||||
type = "application/json"
|
type = "application/json"
|
||||||
|
|||||||
@@ -0,0 +1,148 @@
|
|||||||
|
// SPDX-FileCopyrightText: 2026 Echolot contributors
|
||||||
|
// SPDX-License-Identifier: GPL-3.0-or-later
|
||||||
|
|
||||||
|
package app.echo_lot.app
|
||||||
|
|
||||||
|
import android.content.Context
|
||||||
|
import app.echo_lot.archive.ArchivedRun
|
||||||
|
import app.echo_lot.archive.RunArchive
|
||||||
|
import app.echo_lot.measurement.MeasurementDocument
|
||||||
|
import app.echo_lot.privacy.Anonymizer
|
||||||
|
import app.echo_lot.privacy.PrivacyLevel
|
||||||
|
import app.echo_lot.privacy.Salt
|
||||||
|
import app.echo_lot.protocol.Compat
|
||||||
|
import app.echo_lot.protocol.ControlClient
|
||||||
|
import app.echo_lot.protocol.UploadRefused
|
||||||
|
import app.echo_lot.protocol.VersionRefused
|
||||||
|
import kotlinx.serialization.json.Json
|
||||||
|
import kotlinx.serialization.json.JsonObject
|
||||||
|
import kotlinx.serialization.json.jsonObject
|
||||||
|
import java.io.File
|
||||||
|
import java.security.SecureRandom
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Ties together the three things that happen to a finished run: it gets archived, it may get
|
||||||
|
* anonymized, and it may get uploaded — in that order, and with the archive always holding the
|
||||||
|
* *unredacted* document.
|
||||||
|
*
|
||||||
|
* That ordering is the important decision. The local archive is the user's own data on their own
|
||||||
|
* device, and redacting it would destroy exactly the detail that makes a week-old run worth
|
||||||
|
* keeping; the anonymizer exists for the moment data crosses to someone else's machine. So
|
||||||
|
* redaction happens on the way out, per upload, and the archive is never the lossy copy.
|
||||||
|
*/
|
||||||
|
class RunStore(context: Context, private val settings: Settings) {
|
||||||
|
|
||||||
|
private val archive = RunArchive(File(context.filesDir, "runs"))
|
||||||
|
private val json = Json { encodeDefaults = true; explicitNulls = true }
|
||||||
|
|
||||||
|
fun list(): List<ArchivedRun> = archive.list()
|
||||||
|
fun read(id: String): String? = archive.read(id)
|
||||||
|
fun delete(id: String) = archive.delete(id)
|
||||||
|
fun deleteAll(): Int = archive.deleteAll()
|
||||||
|
fun totalBytes(): Long = archive.totalBytes()
|
||||||
|
|
||||||
|
/** Archives a finished run under the user's retention policy. Null when archiving is off. */
|
||||||
|
fun archive(doc: MeasurementDocument): ArchivedRun? =
|
||||||
|
archive.save(Report.toJson(doc), settings.retention())
|
||||||
|
|
||||||
|
/** Applies retention now — e.g. after the user tightens the limits in settings. */
|
||||||
|
fun purgeNow() = archive.purge(settings.retention())
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Produces exactly the bytes an upload would send, so the UI can show the user their own
|
||||||
|
* document as the server will see it *before* it goes. "Preview what you're about to share"
|
||||||
|
* is the only honest way to present an anonymizer: its correctness is not something a user
|
||||||
|
* should have to take on faith.
|
||||||
|
*/
|
||||||
|
fun redactedForUpload(docJson: String, level: PrivacyLevel = settings.privacyLevel): String {
|
||||||
|
val parsed = runCatching { json.parseToJsonElement(docJson).jsonObject }.getOrNull()
|
||||||
|
?: return docJson
|
||||||
|
return json.encodeToString(JsonObject.serializer(), Anonymizer(level, salt()).anonymize(parsed))
|
||||||
|
}
|
||||||
|
|
||||||
|
private fun salt(): Salt =
|
||||||
|
if (settings.stableSalt) Salt.stable(settings.saltSecret())
|
||||||
|
else Salt.perRun(ByteArray(32).also { SecureRandom().nextBytes(it) })
|
||||||
|
|
||||||
|
sealed interface UploadOutcome {
|
||||||
|
data class Sent(val serverName: String, val detail: String) : UploadOutcome
|
||||||
|
/** The operator's policy says no. Not retryable, and not the user's fault. */
|
||||||
|
data class Refused(val reason: String) : UploadOutcome
|
||||||
|
/**
|
||||||
|
* The two builds do not go together. Kept apart from [Refused] and [Failed] because the
|
||||||
|
* remedy is different and specific — install a particular version — and a message that
|
||||||
|
* says so is worth more than one that says "upload failed".
|
||||||
|
*/
|
||||||
|
data class Incompatible(val reason: String) : UploadOutcome
|
||||||
|
data class Failed(val detail: String) : UploadOutcome
|
||||||
|
data object NotConfigured : UploadOutcome
|
||||||
|
}
|
||||||
|
|
||||||
|
private fun client() = ControlClient(
|
||||||
|
settings.serverUrl, setOf(settings.serverPin), BuildConfig.APP_SEMVER,
|
||||||
|
)
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Checks the configured server without uploading anything: reachable, pinned, compatible, and
|
||||||
|
* willing to accept runs. Lets the user find out in settings rather than from a failed run.
|
||||||
|
*/
|
||||||
|
fun checkServer(): String {
|
||||||
|
if (!settings.serverConfigured) return "Fill in the server URL, pin and credential first."
|
||||||
|
return try {
|
||||||
|
val profile = client().profile(settings.serverCredential)
|
||||||
|
val compat = Compat.check(profile, BuildConfig.APP_SEMVER)
|
||||||
|
val head = "${profile.name} · server ${profile.serverVersion} · " +
|
||||||
|
"protocol ${profile.compat.protocolVersion.ifBlank { "unstated" }}"
|
||||||
|
when {
|
||||||
|
compat.message != null -> head + "\n" + compat.message
|
||||||
|
else -> {
|
||||||
|
val uploads = profile.uploads.refusalReason()
|
||||||
|
?: "uploads accepted (min anonymization: ${profile.uploads.minAnonymization})"
|
||||||
|
head + "\nCompatible. " + uploads
|
||||||
|
}
|
||||||
|
}
|
||||||
|
} catch (e: VersionRefused) {
|
||||||
|
"This server will not serve this app: ${e.message}"
|
||||||
|
} catch (t: Throwable) {
|
||||||
|
"Could not reach the server: ${t.message ?: t.javaClass.simpleName}"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Uploads one archived run to the configured server, redacting first.
|
||||||
|
*
|
||||||
|
* The server's advertised minimum wins over the user's preference when it is stricter — a
|
||||||
|
* server may demand more anonymization than the user chose, never less. Blocking; callers
|
||||||
|
* run it off the main thread.
|
||||||
|
*/
|
||||||
|
fun upload(runId: String): UploadOutcome {
|
||||||
|
if (!settings.serverConfigured) return UploadOutcome.NotConfigured
|
||||||
|
val docJson = read(runId) ?: return UploadOutcome.Failed("run $runId is not in the archive")
|
||||||
|
return try {
|
||||||
|
val client = client()
|
||||||
|
val profile = client.profile(settings.serverCredential)
|
||||||
|
|
||||||
|
// Compatibility before policy: an incompatible server may well advertise an upload
|
||||||
|
// policy it would never actually apply to us.
|
||||||
|
val compat = Compat.check(profile, BuildConfig.APP_SEMVER)
|
||||||
|
if (!compat.usable) return UploadOutcome.Incompatible(compat.message ?: "incompatible versions")
|
||||||
|
|
||||||
|
profile.uploads.refusalReason()?.let { return UploadOutcome.Refused(it) }
|
||||||
|
|
||||||
|
val level = PrivacyLevel.max(
|
||||||
|
settings.privacyLevel,
|
||||||
|
PrivacyLevel.fromWire(profile.uploads.minAnonymization),
|
||||||
|
)
|
||||||
|
val body = redactedForUpload(docJson, level)
|
||||||
|
val reply = client.uploadRun(settings.serverCredential, body)
|
||||||
|
archive.markUploaded(runId, profile.name)
|
||||||
|
UploadOutcome.Sent(profile.name, "as $level, ${body.toByteArray().size} bytes: ${reply.take(120)}")
|
||||||
|
} catch (e: VersionRefused) {
|
||||||
|
UploadOutcome.Incompatible(e.message ?: "the server refused this app's version")
|
||||||
|
} catch (e: UploadRefused) {
|
||||||
|
UploadOutcome.Refused(e.message ?: "refused by the server")
|
||||||
|
} catch (t: Throwable) {
|
||||||
|
UploadOutcome.Failed(t.message ?: t.javaClass.simpleName)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -38,6 +38,10 @@ data class UiState(
|
|||||||
val stepsDone: Int = 0,
|
val stepsDone: Int = 0,
|
||||||
val stepsTotal: Int = 0,
|
val stepsTotal: Int = 0,
|
||||||
val etaSeconds: Int = 0,
|
val etaSeconds: Int = 0,
|
||||||
|
/** Where the finished run went: archived locally, uploaded, or neither (and why). */
|
||||||
|
val archiveStatus: String? = null,
|
||||||
|
/** History, newest first. Refreshed after every run and whenever the history screen opens. */
|
||||||
|
val history: List<app.echo_lot.archive.ArchivedRun> = emptyList(),
|
||||||
/** Shell-tier readiness, shown before a run; null message = say nothing (Shizuku not installed). */
|
/** Shell-tier readiness, shown before a run; null message = say nothing (Shizuku not installed). */
|
||||||
val shizukuNotice: String? = null,
|
val shizukuNotice: String? = null,
|
||||||
val shizukuReady: Boolean = false,
|
val shizukuReady: Boolean = false,
|
||||||
@@ -56,6 +60,9 @@ class RunViewModel(app: Application) : AndroidViewModel(app) {
|
|||||||
var state by mutableStateOf(UiState())
|
var state by mutableStateOf(UiState())
|
||||||
private set
|
private set
|
||||||
|
|
||||||
|
val settings = Settings(app)
|
||||||
|
private val store = RunStore(app, settings)
|
||||||
|
|
||||||
private var runJob: kotlinx.coroutines.Job? = null
|
private var runJob: kotlinx.coroutines.Job? = null
|
||||||
private val stopShizukuObserver: () -> Unit
|
private val stopShizukuObserver: () -> Unit
|
||||||
|
|
||||||
@@ -92,22 +99,120 @@ class RunViewModel(app: Application) : AndroidViewModel(app) {
|
|||||||
}
|
}
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* Runs one measurement. With [upload] (autorun mode) the finished document is POSTed to the
|
* Runs one measurement, then archives it and — if the user has turned that on — uploads it.
|
||||||
* collection endpoint so an unattended run can be retrieved without adb.
|
*
|
||||||
|
* [devUpload] is the separate autorun/adb path (BuildConfig collection endpoint), kept apart
|
||||||
|
* from the user-facing upload so a debugging convenience can never be mistaken for, or
|
||||||
|
* silently satisfy, the consent-gated one.
|
||||||
*/
|
*/
|
||||||
fun run(upload: Boolean = false) {
|
fun run(devUpload: Boolean = false) {
|
||||||
if (state.running) return
|
if (state.running) return
|
||||||
collected.clear()
|
collected.clear()
|
||||||
state = state.copy(running = true, currentStep = "starting", document = null, uploadStatus = null)
|
state = state.copy(running = true, currentStep = "starting", document = null,
|
||||||
|
uploadStatus = null, archiveStatus = null)
|
||||||
runJob = viewModelScope.launch {
|
runJob = viewModelScope.launch {
|
||||||
val doc = withContext(Dispatchers.IO) { measure() }
|
val doc = withContext(Dispatchers.IO) { measure() }
|
||||||
|
|
||||||
|
step("archiving")
|
||||||
|
val archived = withContext(Dispatchers.IO) { store.archive(doc) }
|
||||||
|
var archiveStatus = if (archived != null) {
|
||||||
|
"archived locally (${store.list().size} runs kept)"
|
||||||
|
} else {
|
||||||
|
"not archived — archiving is off in settings"
|
||||||
|
}
|
||||||
|
|
||||||
var status: String? = null
|
var status: String? = null
|
||||||
if (upload) {
|
if (devUpload) {
|
||||||
state = state.copy(currentStep = "uploading report")
|
state = state.copy(currentStep = "uploading report")
|
||||||
val r = withContext(Dispatchers.IO) { ReportUploader.upload(doc) }
|
val r = withContext(Dispatchers.IO) { ReportUploader.upload(doc) }
|
||||||
status = if (r.ok) "uploaded ✓ ${r.detail}" else "upload failed: ${r.detail}"
|
status = if (r.ok) "uploaded ✓ ${r.detail}" else "upload failed: ${r.detail}"
|
||||||
}
|
}
|
||||||
state = UiState(running = false, currentStep = null, document = doc, uploadStatus = status)
|
if (archived != null && settings.autoUpload) {
|
||||||
|
state = state.copy(currentStep = "uploading to server")
|
||||||
|
val outcome = withContext(Dispatchers.IO) { store.upload(archived.id) }
|
||||||
|
archiveStatus += " · " + describe(outcome)
|
||||||
|
}
|
||||||
|
|
||||||
|
state = UiState(
|
||||||
|
running = false, currentStep = null, document = doc,
|
||||||
|
uploadStatus = status, archiveStatus = archiveStatus,
|
||||||
|
history = withContext(Dispatchers.IO) { store.list() },
|
||||||
|
)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
private fun describe(o: RunStore.UploadOutcome): String = when (o) {
|
||||||
|
is RunStore.UploadOutcome.Sent -> "uploaded to ${o.serverName} (${o.detail})"
|
||||||
|
is RunStore.UploadOutcome.Refused -> "server refused the upload: ${o.reason}"
|
||||||
|
is RunStore.UploadOutcome.Incompatible -> "version mismatch: ${o.reason}"
|
||||||
|
is RunStore.UploadOutcome.Failed -> "upload failed: ${o.detail}"
|
||||||
|
RunStore.UploadOutcome.NotConfigured -> "no server configured, so nothing was uploaded"
|
||||||
|
}
|
||||||
|
|
||||||
|
// ---- history ---------------------------------------------------------------------
|
||||||
|
|
||||||
|
fun refreshHistory() {
|
||||||
|
viewModelScope.launch {
|
||||||
|
state = state.copy(history = withContext(Dispatchers.IO) { store.list() })
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
fun deleteRun(id: String) {
|
||||||
|
viewModelScope.launch {
|
||||||
|
withContext(Dispatchers.IO) { store.delete(id) }
|
||||||
|
state = state.copy(history = withContext(Dispatchers.IO) { store.list() })
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
fun deleteAllRuns() {
|
||||||
|
viewModelScope.launch {
|
||||||
|
val n = withContext(Dispatchers.IO) { store.deleteAll() }
|
||||||
|
state = state.copy(
|
||||||
|
history = emptyList(),
|
||||||
|
archiveStatus = "deleted $n archived run(s)",
|
||||||
|
)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Uploads one already-archived run on demand, regardless of the auto-upload setting. */
|
||||||
|
fun uploadRun(id: String) {
|
||||||
|
viewModelScope.launch {
|
||||||
|
state = state.copy(archiveStatus = "uploading …")
|
||||||
|
val outcome = withContext(Dispatchers.IO) { store.upload(id) }
|
||||||
|
state = state.copy(
|
||||||
|
archiveStatus = describe(outcome),
|
||||||
|
history = withContext(Dispatchers.IO) { store.list() },
|
||||||
|
)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/** The archived document as stored, for export. */
|
||||||
|
suspend fun readRun(id: String): String? = withContext(Dispatchers.IO) { store.read(id) }
|
||||||
|
|
||||||
|
/** The exact bytes an upload would send, for the settings screen's preview. */
|
||||||
|
suspend fun uploadPreview(id: String): String? = withContext(Dispatchers.IO) {
|
||||||
|
store.read(id)?.let { store.redactedForUpload(it) }
|
||||||
|
}
|
||||||
|
|
||||||
|
fun archivedBytes(): Long = store.totalBytes()
|
||||||
|
|
||||||
|
/** Settings-screen action: report what the configured server is and whether we can use it. */
|
||||||
|
fun checkServer() {
|
||||||
|
viewModelScope.launch {
|
||||||
|
state = state.copy(archiveStatus = "checking server …")
|
||||||
|
state = state.copy(archiveStatus = withContext(Dispatchers.IO) { store.checkServer() })
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Re-applies retention after the user changes the limits. */
|
||||||
|
fun applyRetention() {
|
||||||
|
viewModelScope.launch {
|
||||||
|
val result = withContext(Dispatchers.IO) { store.purgeNow() }
|
||||||
|
state = state.copy(
|
||||||
|
history = withContext(Dispatchers.IO) { store.list() },
|
||||||
|
archiveStatus = if (result.isEmpty) "nothing to purge"
|
||||||
|
else "purged ${result.removed.size} run(s), freed ${result.freedBytes / 1024} kB",
|
||||||
|
)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -122,6 +227,8 @@ class RunViewModel(app: Application) : AndroidViewModel(app) {
|
|||||||
state = UiState(
|
state = UiState(
|
||||||
running = false, currentStep = null, document = doc,
|
running = false, currentStep = null, document = doc,
|
||||||
uploadStatus = "cancelled after ${collected.size} test(s) — shown locally, not uploaded",
|
uploadStatus = "cancelled after ${collected.size} test(s) — shown locally, not uploaded",
|
||||||
|
archiveStatus = "partial run — not archived",
|
||||||
|
history = state.history,
|
||||||
)
|
)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
@@ -0,0 +1,124 @@
|
|||||||
|
// SPDX-FileCopyrightText: 2026 Echolot contributors
|
||||||
|
// SPDX-License-Identifier: GPL-3.0-or-later
|
||||||
|
|
||||||
|
package app.echo_lot.app
|
||||||
|
|
||||||
|
import android.content.Context
|
||||||
|
import android.content.SharedPreferences
|
||||||
|
import app.echo_lot.archive.RetentionPolicy
|
||||||
|
import app.echo_lot.privacy.PrivacyLevel
|
||||||
|
import java.security.SecureRandom
|
||||||
|
|
||||||
|
/**
|
||||||
|
* User settings for archiving, uploading and anonymization.
|
||||||
|
*
|
||||||
|
* Defaults are the conservative reading of "an engineer's tool that still respects the person
|
||||||
|
* holding it": keep history (that is the point of the archive), never upload without being asked,
|
||||||
|
* and when uploading, strip identifiers unless the user says this is their own server.
|
||||||
|
*
|
||||||
|
* SharedPreferences rather than DataStore because these are a dozen scalars read synchronously at
|
||||||
|
* the start of a run; a coroutine-flow store would add a dependency and a lifecycle for nothing.
|
||||||
|
*/
|
||||||
|
class Settings(context: Context) {
|
||||||
|
|
||||||
|
private val prefs: SharedPreferences =
|
||||||
|
context.getSharedPreferences("echolot-settings", Context.MODE_PRIVATE)
|
||||||
|
|
||||||
|
// ---- archive ---------------------------------------------------------------------
|
||||||
|
|
||||||
|
var archiveEnabled: Boolean
|
||||||
|
get() = prefs.getBoolean(ARCHIVE_ENABLED, true)
|
||||||
|
set(v) = prefs.edit().putBoolean(ARCHIVE_ENABLED, v).apply()
|
||||||
|
|
||||||
|
/** 0 = no ceiling. */
|
||||||
|
var maxRuns: Int
|
||||||
|
get() = prefs.getInt(MAX_RUNS, 100)
|
||||||
|
set(v) = prefs.edit().putInt(MAX_RUNS, v.coerceAtLeast(0)).apply()
|
||||||
|
|
||||||
|
var maxAgeDays: Int
|
||||||
|
get() = prefs.getInt(MAX_AGE_DAYS, 90)
|
||||||
|
set(v) = prefs.edit().putInt(MAX_AGE_DAYS, v.coerceAtLeast(0)).apply()
|
||||||
|
|
||||||
|
var maxTotalMb: Int
|
||||||
|
get() = prefs.getInt(MAX_TOTAL_MB, 64)
|
||||||
|
set(v) = prefs.edit().putInt(MAX_TOTAL_MB, v.coerceAtLeast(0)).apply()
|
||||||
|
|
||||||
|
fun retention(): RetentionPolicy = RetentionPolicy(
|
||||||
|
enabled = archiveEnabled,
|
||||||
|
maxRuns = maxRuns,
|
||||||
|
maxAgeDays = maxAgeDays,
|
||||||
|
maxTotalBytes = maxTotalMb.toLong() * 1024 * 1024,
|
||||||
|
)
|
||||||
|
|
||||||
|
// ---- upload ----------------------------------------------------------------------
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Off by default. Measurement data describes the network the user is standing in; sending it
|
||||||
|
* anywhere is a decision they make, not one they discover after the fact.
|
||||||
|
*/
|
||||||
|
var autoUpload: Boolean
|
||||||
|
get() = prefs.getBoolean(AUTO_UPLOAD, false)
|
||||||
|
set(v) = prefs.edit().putBoolean(AUTO_UPLOAD, v).apply()
|
||||||
|
|
||||||
|
/** Anonymization applied before a run leaves the device. Never applied to the local archive. */
|
||||||
|
var privacyLevel: PrivacyLevel
|
||||||
|
get() = PrivacyLevel.fromWire(prefs.getString(PRIVACY_LEVEL, PrivacyLevel.BALANCED.wire))
|
||||||
|
set(v) = prefs.edit().putString(PRIVACY_LEVEL, v.wire).apply()
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Whether pseudonyms stay stable across runs. That makes history diffable ("same SSID as
|
||||||
|
* last week") and is what someone wants on their own server — but it also produces an
|
||||||
|
* identifier that links a device's uploads, so it is off unless chosen.
|
||||||
|
*/
|
||||||
|
var stableSalt: Boolean
|
||||||
|
get() = prefs.getBoolean(STABLE_SALT, false)
|
||||||
|
set(v) = prefs.edit().putBoolean(STABLE_SALT, v).apply()
|
||||||
|
|
||||||
|
/**
|
||||||
|
* The device-local secret behind stable pseudonyms. Generated once, never leaves the device,
|
||||||
|
* and clearing it (via [resetSalt]) breaks the link to everything uploaded before.
|
||||||
|
*/
|
||||||
|
fun saltSecret(): ByteArray {
|
||||||
|
prefs.getString(SALT_SECRET, null)?.let { return hex(it) }
|
||||||
|
val fresh = ByteArray(32).also { SecureRandom().nextBytes(it) }
|
||||||
|
prefs.edit().putString(SALT_SECRET, fresh.joinToString("") { "%02x".format(it) }).apply()
|
||||||
|
return fresh
|
||||||
|
}
|
||||||
|
|
||||||
|
fun resetSalt() = prefs.edit().remove(SALT_SECRET).apply()
|
||||||
|
|
||||||
|
// ---- server ----------------------------------------------------------------------
|
||||||
|
|
||||||
|
var serverUrl: String
|
||||||
|
get() = prefs.getString(SERVER_URL, "") ?: ""
|
||||||
|
set(v) = prefs.edit().putString(SERVER_URL, v.trim()).apply()
|
||||||
|
|
||||||
|
var serverPin: String
|
||||||
|
get() = prefs.getString(SERVER_PIN, "") ?: ""
|
||||||
|
set(v) = prefs.edit().putString(SERVER_PIN, v.trim()).apply()
|
||||||
|
|
||||||
|
var serverCredential: String
|
||||||
|
get() = prefs.getString(SERVER_CRED, "") ?: ""
|
||||||
|
set(v) = prefs.edit().putString(SERVER_CRED, v.trim()).apply()
|
||||||
|
|
||||||
|
val serverConfigured: Boolean
|
||||||
|
get() = serverUrl.isNotBlank() && serverPin.isNotBlank() && serverCredential.isNotBlank()
|
||||||
|
|
||||||
|
private fun hex(s: String) = ByteArray(s.length / 2) {
|
||||||
|
((Character.digit(s[it * 2], 16) shl 4) or Character.digit(s[it * 2 + 1], 16)).toByte()
|
||||||
|
}
|
||||||
|
|
||||||
|
private companion object {
|
||||||
|
const val ARCHIVE_ENABLED = "archive_enabled"
|
||||||
|
const val MAX_RUNS = "archive_max_runs"
|
||||||
|
const val MAX_AGE_DAYS = "archive_max_age_days"
|
||||||
|
const val MAX_TOTAL_MB = "archive_max_total_mb"
|
||||||
|
const val AUTO_UPLOAD = "auto_upload"
|
||||||
|
const val PRIVACY_LEVEL = "privacy_level"
|
||||||
|
const val STABLE_SALT = "stable_salt"
|
||||||
|
const val SALT_SECRET = "salt_secret"
|
||||||
|
const val SERVER_URL = "server_url"
|
||||||
|
const val SERVER_PIN = "server_pin"
|
||||||
|
const val SERVER_CRED = "server_credential"
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,230 @@
|
|||||||
|
// SPDX-FileCopyrightText: 2026 Echolot contributors
|
||||||
|
// SPDX-License-Identifier: GPL-3.0-or-later
|
||||||
|
|
||||||
|
package app.echo_lot.app
|
||||||
|
|
||||||
|
import androidx.compose.foundation.layout.Arrangement
|
||||||
|
import androidx.compose.foundation.layout.Column
|
||||||
|
import androidx.compose.foundation.layout.Row
|
||||||
|
import androidx.compose.foundation.layout.Spacer
|
||||||
|
import androidx.compose.foundation.layout.fillMaxWidth
|
||||||
|
import androidx.compose.foundation.layout.height
|
||||||
|
import androidx.compose.foundation.layout.padding
|
||||||
|
import androidx.compose.foundation.rememberScrollState
|
||||||
|
import androidx.compose.foundation.verticalScroll
|
||||||
|
import androidx.compose.material3.Button
|
||||||
|
import androidx.compose.material3.Card
|
||||||
|
import androidx.compose.material3.FilterChip
|
||||||
|
import androidx.compose.material3.MaterialTheme
|
||||||
|
import androidx.compose.material3.OutlinedTextField
|
||||||
|
import androidx.compose.material3.Switch
|
||||||
|
import androidx.compose.material3.Text
|
||||||
|
import androidx.compose.material3.TextButton
|
||||||
|
import androidx.compose.runtime.Composable
|
||||||
|
import androidx.compose.runtime.getValue
|
||||||
|
import androidx.compose.runtime.mutableStateOf
|
||||||
|
import androidx.compose.runtime.remember
|
||||||
|
import androidx.compose.runtime.setValue
|
||||||
|
import androidx.compose.ui.Alignment
|
||||||
|
import androidx.compose.ui.Modifier
|
||||||
|
import androidx.compose.ui.text.font.FontFamily
|
||||||
|
import androidx.compose.ui.unit.dp
|
||||||
|
import app.echo_lot.privacy.PrivacyLevel
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Archiving, upload and anonymization settings.
|
||||||
|
*
|
||||||
|
* The screen is written to make the consequences legible rather than to look tidy: every toggle
|
||||||
|
* says what it means for the user's data in a sentence, and the privacy levels are described by
|
||||||
|
* what survives them, because "balanced" on its own tells nobody anything.
|
||||||
|
*/
|
||||||
|
@Composable
|
||||||
|
fun SettingsScreen(
|
||||||
|
settings: Settings,
|
||||||
|
archivedRuns: Int,
|
||||||
|
archivedBytes: Long,
|
||||||
|
onApplyRetention: () -> Unit,
|
||||||
|
onDeleteAll: () -> Unit,
|
||||||
|
onPreviewUpload: () -> Unit,
|
||||||
|
onCheckServer: () -> Unit,
|
||||||
|
serverStatus: String?,
|
||||||
|
onBack: () -> Unit,
|
||||||
|
) {
|
||||||
|
// SharedPreferences is not observable, so mirror each value into Compose state and write
|
||||||
|
// through on change. A dozen scalars; a store with flows would be ceremony for nothing.
|
||||||
|
var archiveEnabled by remember { mutableStateOf(settings.archiveEnabled) }
|
||||||
|
var maxRuns by remember { mutableStateOf(settings.maxRuns.toString()) }
|
||||||
|
var maxAgeDays by remember { mutableStateOf(settings.maxAgeDays.toString()) }
|
||||||
|
var maxTotalMb by remember { mutableStateOf(settings.maxTotalMb.toString()) }
|
||||||
|
var autoUpload by remember { mutableStateOf(settings.autoUpload) }
|
||||||
|
var privacy by remember { mutableStateOf(settings.privacyLevel) }
|
||||||
|
var stableSalt by remember { mutableStateOf(settings.stableSalt) }
|
||||||
|
var serverUrl by remember { mutableStateOf(settings.serverUrl) }
|
||||||
|
var serverPin by remember { mutableStateOf(settings.serverPin) }
|
||||||
|
var serverCred by remember { mutableStateOf(settings.serverCredential) }
|
||||||
|
|
||||||
|
Column(
|
||||||
|
Modifier.fillMaxWidth().verticalScroll(rememberScrollState()).padding(16.dp),
|
||||||
|
verticalArrangement = Arrangement.spacedBy(12.dp),
|
||||||
|
) {
|
||||||
|
Row(verticalAlignment = Alignment.CenterVertically) {
|
||||||
|
TextButton(onClick = onBack) { Text("‹ Back") }
|
||||||
|
Text("Settings", style = MaterialTheme.typography.titleLarge)
|
||||||
|
}
|
||||||
|
|
||||||
|
// ---- archive ----
|
||||||
|
Card(Modifier.fillMaxWidth()) {
|
||||||
|
Column(Modifier.padding(14.dp), verticalArrangement = Arrangement.spacedBy(8.dp)) {
|
||||||
|
Text("Archive", style = MaterialTheme.typography.titleMedium)
|
||||||
|
Toggle(
|
||||||
|
label = "Keep finished runs on this device",
|
||||||
|
detail = "History is what makes a run comparable later. Archived runs are " +
|
||||||
|
"stored complete and unredacted — anonymization only applies to uploads.",
|
||||||
|
checked = archiveEnabled,
|
||||||
|
) { archiveEnabled = it; settings.archiveEnabled = it }
|
||||||
|
|
||||||
|
Text(
|
||||||
|
"Purge automatically when a run exceeds any of these. 0 turns that limit off.",
|
||||||
|
style = MaterialTheme.typography.bodySmall,
|
||||||
|
)
|
||||||
|
NumberField("Keep at most (runs)", maxRuns) {
|
||||||
|
maxRuns = it; settings.maxRuns = it.toIntOrNull() ?: 0
|
||||||
|
}
|
||||||
|
NumberField("Delete older than (days)", maxAgeDays) {
|
||||||
|
maxAgeDays = it; settings.maxAgeDays = it.toIntOrNull() ?: 0
|
||||||
|
}
|
||||||
|
NumberField("Keep at most (MB)", maxTotalMb) {
|
||||||
|
maxTotalMb = it; settings.maxTotalMb = it.toIntOrNull() ?: 0
|
||||||
|
}
|
||||||
|
Text(
|
||||||
|
"$archivedRuns run(s), ${archivedBytes / 1024} kB stored",
|
||||||
|
style = MaterialTheme.typography.bodySmall,
|
||||||
|
)
|
||||||
|
Row(horizontalArrangement = Arrangement.spacedBy(8.dp)) {
|
||||||
|
Button(onClick = onApplyRetention) { Text("Apply now") }
|
||||||
|
TextButton(onClick = onDeleteAll) { Text("Delete all runs") }
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// ---- privacy ----
|
||||||
|
Card(Modifier.fillMaxWidth()) {
|
||||||
|
Column(Modifier.padding(14.dp), verticalArrangement = Arrangement.spacedBy(8.dp)) {
|
||||||
|
Text("What leaves the device", style = MaterialTheme.typography.titleMedium)
|
||||||
|
Text(
|
||||||
|
"Applied to uploads only. Measurements, verdicts and finding codes survive " +
|
||||||
|
"every level — only the parts that identify you or your network change.",
|
||||||
|
style = MaterialTheme.typography.bodySmall,
|
||||||
|
)
|
||||||
|
Row(horizontalArrangement = Arrangement.spacedBy(8.dp)) {
|
||||||
|
for (level in PrivacyLevel.entries) {
|
||||||
|
FilterChip(
|
||||||
|
selected = privacy == level,
|
||||||
|
onClick = { privacy = level; settings.privacyLevel = level },
|
||||||
|
label = { Text(level.wire) },
|
||||||
|
)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
Text(privacyExplanation(privacy), style = MaterialTheme.typography.bodySmall)
|
||||||
|
|
||||||
|
Toggle(
|
||||||
|
label = "Stable pseudonyms across runs",
|
||||||
|
detail = "Lets you compare uploaded runs over time (same SSID reads the same " +
|
||||||
|
"each time). It also links your uploads together, so leave it off on a " +
|
||||||
|
"server you don't run yourself.",
|
||||||
|
checked = stableSalt,
|
||||||
|
) { stableSalt = it; settings.stableSalt = it }
|
||||||
|
|
||||||
|
TextButton(onClick = onPreviewUpload) { Text("Preview what an upload would send") }
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// ---- upload ----
|
||||||
|
Card(Modifier.fillMaxWidth()) {
|
||||||
|
Column(Modifier.padding(14.dp), verticalArrangement = Arrangement.spacedBy(8.dp)) {
|
||||||
|
Text("Upload", style = MaterialTheme.typography.titleMedium)
|
||||||
|
Toggle(
|
||||||
|
label = "Upload finished runs automatically",
|
||||||
|
detail = "Sends each completed run to the server below, anonymized to the " +
|
||||||
|
"level above. The server may require more anonymization than you chose; " +
|
||||||
|
"it can never require less.",
|
||||||
|
checked = autoUpload,
|
||||||
|
) { autoUpload = it; settings.autoUpload = it }
|
||||||
|
|
||||||
|
OutlinedTextField(
|
||||||
|
value = serverUrl, onValueChange = { serverUrl = it; settings.serverUrl = it },
|
||||||
|
label = { Text("Server URL") }, singleLine = true, modifier = Modifier.fillMaxWidth(),
|
||||||
|
)
|
||||||
|
OutlinedTextField(
|
||||||
|
value = serverPin, onValueChange = { serverPin = it; settings.serverPin = it },
|
||||||
|
label = { Text("Certificate pin (SPKI, base64)") }, singleLine = true,
|
||||||
|
textStyle = MaterialTheme.typography.bodySmall.copy(fontFamily = FontFamily.Monospace),
|
||||||
|
modifier = Modifier.fillMaxWidth(),
|
||||||
|
)
|
||||||
|
OutlinedTextField(
|
||||||
|
value = serverCred,
|
||||||
|
onValueChange = { serverCred = it; settings.serverCredential = it },
|
||||||
|
label = { Text("Device credential") }, singleLine = true,
|
||||||
|
textStyle = MaterialTheme.typography.bodySmall.copy(fontFamily = FontFamily.Monospace),
|
||||||
|
modifier = Modifier.fillMaxWidth(),
|
||||||
|
)
|
||||||
|
Row(verticalAlignment = Alignment.CenterVertically) {
|
||||||
|
Button(onClick = onCheckServer) { Text("Check server") }
|
||||||
|
Text(
|
||||||
|
" " + if (settings.serverConfigured) "Configured."
|
||||||
|
else "Uploads stay off until all three fields are set.",
|
||||||
|
style = MaterialTheme.typography.bodySmall,
|
||||||
|
)
|
||||||
|
}
|
||||||
|
// Version compatibility is checked here rather than discovered mid-run: a server
|
||||||
|
// that will refuse this build should say so before a measurement is wasted.
|
||||||
|
serverStatus?.let {
|
||||||
|
Text(it, style = MaterialTheme.typography.bodySmall)
|
||||||
|
}
|
||||||
|
Text(
|
||||||
|
"This app is ${BuildConfig.APP_SEMVER} and speaks probe protocol " +
|
||||||
|
"${app.echo_lot.protocol.Compat.PROTOCOL_VERSION}. It works with servers " +
|
||||||
|
"${app.echo_lot.protocol.Compat.serverRange}.",
|
||||||
|
style = MaterialTheme.typography.bodySmall,
|
||||||
|
)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
Spacer(Modifier.height(24.dp))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
private fun privacyExplanation(level: PrivacyLevel): String = when (level) {
|
||||||
|
PrivacyLevel.FULL ->
|
||||||
|
"Nothing is removed: SSIDs, MAC addresses, hostnames and discovered neighbours are sent " +
|
||||||
|
"as measured. Appropriate for a server you run yourself."
|
||||||
|
PrivacyLevel.BALANCED ->
|
||||||
|
"Network names and hostnames become pseudonyms, MAC addresses keep only their vendor " +
|
||||||
|
"prefix, public IP addresses keep only their /16, and discovered neighbours (SSDP, " +
|
||||||
|
"ARP, nearby networks) are dropped entirely. Private addresses stay readable, since " +
|
||||||
|
"192.168.1.1 describes the topology and not the person."
|
||||||
|
PrivacyLevel.STRICT ->
|
||||||
|
"Only numbers: test results, metrics and finding codes. No network description, no raw " +
|
||||||
|
"evidence, no finding text. Nothing left can identify a network."
|
||||||
|
}
|
||||||
|
|
||||||
|
@Composable
|
||||||
|
private fun Toggle(label: String, detail: String, checked: Boolean, onChange: (Boolean) -> Unit) {
|
||||||
|
Row(Modifier.fillMaxWidth(), verticalAlignment = Alignment.Top) {
|
||||||
|
Column(Modifier.weight(1f)) {
|
||||||
|
Text(label, style = MaterialTheme.typography.bodyMedium)
|
||||||
|
Text(detail, style = MaterialTheme.typography.bodySmall)
|
||||||
|
}
|
||||||
|
Switch(checked = checked, onCheckedChange = onChange)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
@Composable
|
||||||
|
private fun NumberField(label: String, value: String, onChange: (String) -> Unit) {
|
||||||
|
OutlinedTextField(
|
||||||
|
value = value,
|
||||||
|
onValueChange = { s -> onChange(s.filter { it.isDigit() }.take(7)) },
|
||||||
|
label = { Text(label) },
|
||||||
|
singleLine = true,
|
||||||
|
modifier = Modifier.fillMaxWidth(),
|
||||||
|
)
|
||||||
|
}
|
||||||
@@ -0,0 +1,23 @@
|
|||||||
|
// SPDX-FileCopyrightText: 2026 Echolot contributors
|
||||||
|
// SPDX-License-Identifier: GPL-3.0-or-later
|
||||||
|
|
||||||
|
plugins {
|
||||||
|
alias(libs.plugins.kotlin.jvm)
|
||||||
|
alias(libs.plugins.kotlin.serialization)
|
||||||
|
}
|
||||||
|
|
||||||
|
// The on-device run archive: measurement documents on disk, with retention.
|
||||||
|
// Pure Kotlin/JVM (it takes a directory, not a Context) so the retention rules
|
||||||
|
// — the part with edge cases — are unit-testable without a device.
|
||||||
|
dependencies {
|
||||||
|
implementation(libs.kotlinx.serialization.json)
|
||||||
|
testImplementation(kotlin("test"))
|
||||||
|
}
|
||||||
|
|
||||||
|
kotlin {
|
||||||
|
jvmToolchain(21)
|
||||||
|
compilerOptions { jvmTarget.set(org.jetbrains.kotlin.gradle.dsl.JvmTarget.JVM_17) }
|
||||||
|
}
|
||||||
|
java { sourceCompatibility = JavaVersion.VERSION_17; targetCompatibility = JavaVersion.VERSION_17 }
|
||||||
|
|
||||||
|
tasks.test { useJUnitPlatform() }
|
||||||
@@ -0,0 +1,209 @@
|
|||||||
|
// SPDX-FileCopyrightText: 2026 Echolot contributors
|
||||||
|
// SPDX-License-Identifier: GPL-3.0-or-later
|
||||||
|
|
||||||
|
// Package archive keeps completed measurement runs on the device.
|
||||||
|
//
|
||||||
|
// The point of an archive is the second run: "this network was fine on Tuesday" is only
|
||||||
|
// answerable if Tuesday was kept. But an app that silently accumulates network dumps forever is
|
||||||
|
// its own privacy problem, so retention is a first-class part of the type rather than a cleanup
|
||||||
|
// job somebody remembers to write — every save enforces it.
|
||||||
|
//
|
||||||
|
// Storage is one JSON file per run plus a small index entry, in a plain directory. Nothing here
|
||||||
|
// needs a database, and a plain directory is something a user can inspect, copy off, or delete
|
||||||
|
// with a file manager. Files are written to a temp name and renamed, so a run interrupted mid-
|
||||||
|
// write never leaves a half-document that reads as real.
|
||||||
|
package app.echo_lot.archive
|
||||||
|
|
||||||
|
import kotlinx.serialization.SerialName
|
||||||
|
import kotlinx.serialization.Serializable
|
||||||
|
import kotlinx.serialization.json.Json
|
||||||
|
import kotlinx.serialization.json.JsonObject
|
||||||
|
import kotlinx.serialization.json.jsonObject
|
||||||
|
import kotlinx.serialization.json.jsonPrimitive
|
||||||
|
import java.io.File
|
||||||
|
|
||||||
|
/** Index entry for one archived run — enough for a history list without opening the documents. */
|
||||||
|
@Serializable
|
||||||
|
data class ArchivedRun(
|
||||||
|
val id: String,
|
||||||
|
@SerialName("saved_at_epoch_ms") val savedAtEpochMs: Long,
|
||||||
|
@SerialName("started_at") val startedAt: String? = null,
|
||||||
|
val verdict: String? = null,
|
||||||
|
@SerialName("finding_count") val findingCount: Int = 0,
|
||||||
|
@SerialName("size_bytes") val sizeBytes: Long = 0,
|
||||||
|
val anonymization: String = "full",
|
||||||
|
/** Whether this run has been accepted by a server, so history can show what is backed up. */
|
||||||
|
val uploaded: Boolean = false,
|
||||||
|
@SerialName("uploaded_to") val uploadedTo: String? = null,
|
||||||
|
)
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Retention limits. All three are independent ceilings; a run is dropped when it violates any of
|
||||||
|
* them. Zero disables that limit.
|
||||||
|
*
|
||||||
|
* The default keeps a hundred runs or three months, whichever comes first. That is enough to see
|
||||||
|
* a pattern ("it degrades every evening") without turning the phone into an archive of every
|
||||||
|
* network its owner ever walked past.
|
||||||
|
*/
|
||||||
|
@Serializable
|
||||||
|
data class RetentionPolicy(
|
||||||
|
/**
|
||||||
|
* Whether to archive at all. Separate from the limits because "no limits" (every limit zero)
|
||||||
|
* and "keep nothing" are opposite intentions, and collapsing them onto the same value is how
|
||||||
|
* a user who turns all the caps off ends up with an empty history.
|
||||||
|
*/
|
||||||
|
val enabled: Boolean = true,
|
||||||
|
@SerialName("max_runs") val maxRuns: Int = 100,
|
||||||
|
@SerialName("max_age_days") val maxAgeDays: Int = 90,
|
||||||
|
@SerialName("max_total_bytes") val maxTotalBytes: Long = 64L * 1024 * 1024,
|
||||||
|
) {
|
||||||
|
companion object {
|
||||||
|
/** Archiving off: runs are shown once and never written. */
|
||||||
|
val KeepNothing = RetentionPolicy(enabled = false)
|
||||||
|
|
||||||
|
/** Archiving on with no ceilings. Every run is kept until the user deletes it. */
|
||||||
|
val Unlimited = RetentionPolicy(maxRuns = 0, maxAgeDays = 0, maxTotalBytes = 0)
|
||||||
|
|
||||||
|
val Default = RetentionPolicy()
|
||||||
|
}
|
||||||
|
|
||||||
|
val keepsAnything: Boolean get() = enabled
|
||||||
|
}
|
||||||
|
|
||||||
|
/** What a purge removed, so the UI can say "dropped 3 old runs" instead of silently deleting. */
|
||||||
|
data class PurgeResult(val removed: List<String>, val freedBytes: Long) {
|
||||||
|
val isEmpty: Boolean get() = removed.isEmpty()
|
||||||
|
}
|
||||||
|
|
||||||
|
class RunArchive(private val dir: File, private val now: () -> Long = System::currentTimeMillis) {
|
||||||
|
|
||||||
|
private val json = Json { ignoreUnknownKeys = true; encodeDefaults = true }
|
||||||
|
|
||||||
|
init {
|
||||||
|
dir.mkdirs()
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Writes one run and applies retention. Returns the index entry, or null when the policy
|
||||||
|
* keeps nothing at all — in which case nothing is written, rather than written and instantly
|
||||||
|
* deleted (the difference matters on flash storage and to anyone watching the filesystem).
|
||||||
|
*/
|
||||||
|
fun save(runJson: String, policy: RetentionPolicy = RetentionPolicy.Default): ArchivedRun? {
|
||||||
|
if (!policy.keepsAnything) return null
|
||||||
|
val doc = runCatching { json.parseToJsonElement(runJson).jsonObject }.getOrNull() ?: return null
|
||||||
|
val meta = indexOf(doc, runJson.toByteArray().size.toLong()) ?: return null
|
||||||
|
|
||||||
|
writeAtomically(File(dir, meta.id + EXT), runJson)
|
||||||
|
writeAtomically(File(dir, meta.id + META_EXT), json.encodeToString(ArchivedRun.serializer(), meta))
|
||||||
|
purge(policy)
|
||||||
|
return meta
|
||||||
|
}
|
||||||
|
|
||||||
|
/** History, newest first. */
|
||||||
|
fun list(): List<ArchivedRun> =
|
||||||
|
(dir.listFiles { f -> f.name.endsWith(META_EXT) } ?: emptyArray())
|
||||||
|
.mapNotNull { f ->
|
||||||
|
runCatching { json.decodeFromString(ArchivedRun.serializer(), f.readText()) }.getOrNull()
|
||||||
|
}
|
||||||
|
.sortedByDescending { it.savedAtEpochMs }
|
||||||
|
|
||||||
|
fun read(id: String): String? = File(dir, safe(id) + EXT).takeIf { it.isFile }?.readText()
|
||||||
|
|
||||||
|
fun delete(id: String): Boolean {
|
||||||
|
val s = safe(id)
|
||||||
|
val doc = File(dir, s + EXT).delete()
|
||||||
|
File(dir, s + META_EXT).delete()
|
||||||
|
return doc
|
||||||
|
}
|
||||||
|
|
||||||
|
fun deleteAll(): Int = list().count { delete(it.id) }
|
||||||
|
|
||||||
|
/** Records that a server accepted this run, so history can distinguish backed-up from local. */
|
||||||
|
fun markUploaded(id: String, serverName: String) {
|
||||||
|
val f = File(dir, safe(id) + META_EXT)
|
||||||
|
val meta = runCatching { json.decodeFromString(ArchivedRun.serializer(), f.readText()) }.getOrNull()
|
||||||
|
?: return
|
||||||
|
writeAtomically(
|
||||||
|
f,
|
||||||
|
json.encodeToString(
|
||||||
|
ArchivedRun.serializer(),
|
||||||
|
meta.copy(uploaded = true, uploadedTo = serverName),
|
||||||
|
),
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
fun totalBytes(): Long = list().sumOf { it.sizeBytes }
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Enforces the policy. Age first, then total size, then count: dropping stale runs may already
|
||||||
|
* satisfy the other two, and it is the limit a user reasons about ("keep three months"), so it
|
||||||
|
* should not be pre-empted by a size sweep deleting last week instead.
|
||||||
|
*/
|
||||||
|
fun purge(policy: RetentionPolicy): PurgeResult {
|
||||||
|
val removed = ArrayList<String>()
|
||||||
|
var freed = 0L
|
||||||
|
fun drop(r: ArchivedRun) {
|
||||||
|
if (delete(r.id)) {
|
||||||
|
removed.add(r.id)
|
||||||
|
freed += r.sizeBytes
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
var kept = list()
|
||||||
|
if (policy.maxAgeDays > 0) {
|
||||||
|
val cutoff = now() - policy.maxAgeDays * 24L * 60 * 60 * 1000
|
||||||
|
val (fresh, stale) = kept.partition { it.savedAtEpochMs >= cutoff }
|
||||||
|
stale.forEach(::drop)
|
||||||
|
kept = fresh
|
||||||
|
}
|
||||||
|
if (policy.maxTotalBytes > 0) {
|
||||||
|
var total = kept.sumOf { it.sizeBytes }
|
||||||
|
// Oldest first until we are under the ceiling.
|
||||||
|
for (r in kept.reversed()) {
|
||||||
|
if (total <= policy.maxTotalBytes) break
|
||||||
|
drop(r)
|
||||||
|
total -= r.sizeBytes
|
||||||
|
}
|
||||||
|
kept = kept.filter { it.id !in removed }
|
||||||
|
}
|
||||||
|
if (policy.maxRuns > 0 && kept.size > policy.maxRuns) {
|
||||||
|
kept.drop(policy.maxRuns).forEach(::drop) // list() is newest-first
|
||||||
|
}
|
||||||
|
return PurgeResult(removed, freed)
|
||||||
|
}
|
||||||
|
|
||||||
|
// ---- internals ---------------------------------------------------------------------
|
||||||
|
|
||||||
|
private fun indexOf(doc: JsonObject, size: Long): ArchivedRun? {
|
||||||
|
val run = doc["run"]?.jsonObject ?: return null
|
||||||
|
val id = run["id"]?.jsonPrimitive?.content?.let(::safe)?.takeIf { it.isNotEmpty() } ?: return null
|
||||||
|
return ArchivedRun(
|
||||||
|
id = id,
|
||||||
|
savedAtEpochMs = now(),
|
||||||
|
startedAt = run["started_at"]?.jsonPrimitive?.content,
|
||||||
|
verdict = doc["summary"]?.jsonObject?.get("verdict")?.jsonPrimitive?.content,
|
||||||
|
findingCount = (doc["findings"] as? kotlinx.serialization.json.JsonArray)?.size ?: 0,
|
||||||
|
sizeBytes = size,
|
||||||
|
anonymization = run["privacy"]?.jsonObject?.get("anonymization")?.jsonPrimitive?.content ?: "full",
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
private fun writeAtomically(target: File, content: String) {
|
||||||
|
val tmp = File(target.parentFile, target.name + ".tmp")
|
||||||
|
tmp.writeText(content)
|
||||||
|
if (!tmp.renameTo(target)) {
|
||||||
|
target.delete()
|
||||||
|
tmp.renameTo(target)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Run ids reach the filesystem; keep them to characters that cannot climb out of [dir]. */
|
||||||
|
private fun safe(id: String): String = buildString {
|
||||||
|
for (c in id) if (c.isLetterOrDigit() || c == '-' || c == '_') append(c)
|
||||||
|
}.take(64)
|
||||||
|
|
||||||
|
private companion object {
|
||||||
|
const val EXT = ".json"
|
||||||
|
const val META_EXT = ".meta.json"
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,170 @@
|
|||||||
|
// SPDX-FileCopyrightText: 2026 Echolot contributors
|
||||||
|
// SPDX-License-Identifier: GPL-3.0-or-later
|
||||||
|
|
||||||
|
package app.echo_lot.archive
|
||||||
|
|
||||||
|
import java.io.File
|
||||||
|
import java.nio.file.Files
|
||||||
|
import kotlin.test.AfterTest
|
||||||
|
import kotlin.test.Test
|
||||||
|
import kotlin.test.assertEquals
|
||||||
|
import kotlin.test.assertFalse
|
||||||
|
import kotlin.test.assertNotNull
|
||||||
|
import kotlin.test.assertNull
|
||||||
|
import kotlin.test.assertTrue
|
||||||
|
|
||||||
|
class RunArchiveTest {
|
||||||
|
|
||||||
|
private val dir: File = Files.createTempDirectory("echolot-archive").toFile()
|
||||||
|
private var clock = 1_000_000_000_000L // fixed: retention is time arithmetic, not wall time
|
||||||
|
|
||||||
|
private fun archive() = RunArchive(dir) { clock }
|
||||||
|
|
||||||
|
@AfterTest fun cleanup() { dir.deleteRecursively() }
|
||||||
|
|
||||||
|
private fun doc(id: String, findings: Int = 1, pad: Int = 0): String {
|
||||||
|
val f = (1..findings).joinToString(",") { """{"id":"f$it"}""" }
|
||||||
|
return """{"run":{"id":"$id","started_at":"2026-08-01T10:00:00Z","privacy":{"anonymization":"balanced"}},""" +
|
||||||
|
""""findings":[$f],"summary":{"verdict":"warn"},"pad":"${"x".repeat(pad)}"}"""
|
||||||
|
}
|
||||||
|
|
||||||
|
@Test
|
||||||
|
fun savedRunsComeBackNewestFirst() {
|
||||||
|
val a = archive()
|
||||||
|
for (i in 1..3) {
|
||||||
|
a.save(doc("run-$i"))
|
||||||
|
clock += 60_000
|
||||||
|
}
|
||||||
|
assertEquals(listOf("run-3", "run-2", "run-1"), a.list().map { it.id })
|
||||||
|
}
|
||||||
|
|
||||||
|
@Test
|
||||||
|
fun theIndexSummarisesTheDocument() {
|
||||||
|
val meta = assertNotNull(archive().save(doc("run-1", findings = 4)))
|
||||||
|
assertEquals("warn", meta.verdict)
|
||||||
|
assertEquals(4, meta.findingCount)
|
||||||
|
assertEquals("balanced", meta.anonymization)
|
||||||
|
assertEquals("2026-08-01T10:00:00Z", meta.startedAt)
|
||||||
|
assertFalse(meta.uploaded)
|
||||||
|
}
|
||||||
|
|
||||||
|
@Test
|
||||||
|
fun theDocumentComesBackByteForByte() {
|
||||||
|
val a = archive()
|
||||||
|
val original = doc("run-1")
|
||||||
|
a.save(original)
|
||||||
|
assertEquals(original, a.read("run-1"))
|
||||||
|
}
|
||||||
|
|
||||||
|
@Test
|
||||||
|
fun countLimitKeepsTheNewest() {
|
||||||
|
val a = archive()
|
||||||
|
val policy = RetentionPolicy(maxRuns = 3, maxAgeDays = 0, maxTotalBytes = 0)
|
||||||
|
for (i in 1..7) {
|
||||||
|
a.save(doc("run-$i"), policy)
|
||||||
|
clock += 60_000
|
||||||
|
}
|
||||||
|
assertEquals(listOf("run-7", "run-6", "run-5"), a.list().map { it.id })
|
||||||
|
assertNull(a.read("run-1"), "purged run's document should be gone, not just its index entry")
|
||||||
|
}
|
||||||
|
|
||||||
|
@Test
|
||||||
|
fun ageLimitDropsRunsPastTheWindow() {
|
||||||
|
val a = archive()
|
||||||
|
val policy = RetentionPolicy(maxRuns = 0, maxAgeDays = 7, maxTotalBytes = 0)
|
||||||
|
a.save(doc("old"), policy)
|
||||||
|
clock += 30L * 24 * 60 * 60 * 1000 // a month later
|
||||||
|
a.save(doc("new"), policy)
|
||||||
|
assertEquals(listOf("new"), a.list().map { it.id })
|
||||||
|
}
|
||||||
|
|
||||||
|
@Test
|
||||||
|
fun sizeLimitDropsOldestUntilUnderTheCeiling() {
|
||||||
|
val a = archive()
|
||||||
|
val one = doc("x", pad = 900).toByteArray().size.toLong()
|
||||||
|
val policy = RetentionPolicy(maxRuns = 0, maxAgeDays = 0, maxTotalBytes = one * 2 + 10)
|
||||||
|
for (i in 1..5) {
|
||||||
|
a.save(doc("run-$i", pad = 900), policy)
|
||||||
|
clock += 60_000
|
||||||
|
}
|
||||||
|
val kept = a.list()
|
||||||
|
assertTrue(kept.size <= 2, "size ceiling not enforced: kept ${kept.size}")
|
||||||
|
assertEquals("run-5", kept.first().id, "the newest run must always survive")
|
||||||
|
assertTrue(a.totalBytes() <= policy.maxTotalBytes)
|
||||||
|
}
|
||||||
|
|
||||||
|
// A policy that keeps nothing must not write-then-delete: the run should never touch storage.
|
||||||
|
@Test
|
||||||
|
fun keepNothingWritesNothing() {
|
||||||
|
val a = archive()
|
||||||
|
assertNull(a.save(doc("run-1"), RetentionPolicy.KeepNothing))
|
||||||
|
assertTrue(a.list().isEmpty())
|
||||||
|
assertEquals(0, dir.listFiles()?.size ?: 0, "files were written for a keep-nothing policy")
|
||||||
|
}
|
||||||
|
|
||||||
|
// "no ceilings" and "keep nothing" must not be the same policy, however a user arrives at
|
||||||
|
// one: turning every limit off should keep everything, not wipe the history.
|
||||||
|
@Test
|
||||||
|
fun unlimitedKeepsEverythingWhileKeepNothingKeepsNone() {
|
||||||
|
val a = archive()
|
||||||
|
for (i in 1..5) {
|
||||||
|
a.save(doc("run-$i"), RetentionPolicy.Unlimited)
|
||||||
|
clock += 60_000
|
||||||
|
}
|
||||||
|
assertEquals(5, a.list().size)
|
||||||
|
assertNull(a.save(doc("run-6"), RetentionPolicy.KeepNothing))
|
||||||
|
assertEquals(5, a.list().size, "keep-nothing must not touch what is already archived")
|
||||||
|
}
|
||||||
|
|
||||||
|
@Test
|
||||||
|
fun uploadStateIsRecorded() {
|
||||||
|
val a = archive()
|
||||||
|
a.save(doc("run-1"))
|
||||||
|
a.markUploaded("run-1", "fmr")
|
||||||
|
val meta = a.list().single()
|
||||||
|
assertTrue(meta.uploaded)
|
||||||
|
assertEquals("fmr", meta.uploadedTo)
|
||||||
|
assertEquals("run-1", meta.id, "marking upload must not disturb the rest of the entry")
|
||||||
|
}
|
||||||
|
|
||||||
|
@Test
|
||||||
|
fun deleteRemovesBothFiles() {
|
||||||
|
val a = archive()
|
||||||
|
a.save(doc("run-1"))
|
||||||
|
assertTrue(a.delete("run-1"))
|
||||||
|
assertTrue(a.list().isEmpty())
|
||||||
|
assertNull(a.read("run-1"))
|
||||||
|
assertEquals(0, dir.listFiles()?.size ?: 0)
|
||||||
|
}
|
||||||
|
|
||||||
|
@Test
|
||||||
|
fun malformedInputIsRejectedRatherThanArchived() {
|
||||||
|
val a = archive()
|
||||||
|
assertNull(a.save("not json"))
|
||||||
|
assertNull(a.save("""{"summary":{"verdict":"ok"}}"""), "a document with no run id has no identity")
|
||||||
|
assertTrue(a.list().isEmpty())
|
||||||
|
}
|
||||||
|
|
||||||
|
// Run ids come from a document that may have been produced elsewhere; they must not be able
|
||||||
|
// to write outside the archive directory.
|
||||||
|
@Test
|
||||||
|
fun runIdsCannotEscapeTheArchiveDirectory() {
|
||||||
|
val a = archive()
|
||||||
|
a.save(doc("../../evil"))
|
||||||
|
val strays = dir.parentFile.listFiles { f -> f.name.contains("evil") } ?: emptyArray()
|
||||||
|
assertTrue(strays.isEmpty(), "wrote outside the archive: ${strays.toList()}")
|
||||||
|
}
|
||||||
|
|
||||||
|
@Test
|
||||||
|
fun purgeReportsWhatItRemoved() {
|
||||||
|
val a = archive()
|
||||||
|
for (i in 1..5) {
|
||||||
|
a.save(doc("run-$i"), RetentionPolicy.Unlimited)
|
||||||
|
clock += 60_000
|
||||||
|
}
|
||||||
|
val result = a.purge(RetentionPolicy(maxRuns = 2, maxAgeDays = 0, maxTotalBytes = 0))
|
||||||
|
assertEquals(3, result.removed.size)
|
||||||
|
assertTrue(result.freedBytes > 0)
|
||||||
|
assertEquals(2, a.list().size)
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -12,6 +12,7 @@ plugins {
|
|||||||
dependencies {
|
dependencies {
|
||||||
implementation(project(":core-protocol"))
|
implementation(project(":core-protocol"))
|
||||||
implementation(project(":core-measurement"))
|
implementation(project(":core-measurement"))
|
||||||
|
implementation(project(":core-privacy"))
|
||||||
implementation(libs.kotlinx.serialization.json)
|
implementation(libs.kotlinx.serialization.json)
|
||||||
testImplementation(kotlin("test"))
|
testImplementation(kotlin("test"))
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -0,0 +1,344 @@
|
|||||||
|
// SPDX-FileCopyrightText: 2026 Echolot contributors
|
||||||
|
// SPDX-License-Identifier: GPL-3.0-or-later
|
||||||
|
|
||||||
|
package app.echo_lot.engine
|
||||||
|
|
||||||
|
import app.echo_lot.measurement.*
|
||||||
|
import app.echo_lot.protocol.ControlClient
|
||||||
|
import app.echo_lot.protocol.ProbeSession
|
||||||
|
import app.echo_lot.protocol.Wire
|
||||||
|
import kotlinx.serialization.SerialName
|
||||||
|
import kotlinx.serialization.Serializable
|
||||||
|
import kotlinx.serialization.json.Json
|
||||||
|
import kotlinx.serialization.json.JsonObject
|
||||||
|
import kotlinx.serialization.json.encodeToJsonElement
|
||||||
|
|
||||||
|
/**
|
||||||
|
* The measurements only the far end can make: what the *downstream* path does to traffic the
|
||||||
|
* client never asked for packet-by-packet.
|
||||||
|
*
|
||||||
|
* A client alone can measure a round trip, and it can find the largest packet it can *send*. It
|
||||||
|
* cannot find the largest packet it can *receive*, or whether the network drops downstream
|
||||||
|
* packets independently of upstream ones — those need a server willing to push, which is why the
|
||||||
|
* protocol gates them behind an asymmetric grant (probe-protocol.md §3.4).
|
||||||
|
*
|
||||||
|
* Three separate facts come out, and keeping them separate is the point:
|
||||||
|
* - `mtu.pmtud_down` — the largest datagram that arrives *unfragmented*. This is the number
|
||||||
|
* that matters for anything setting DF, and it is only meaningful because the server sets DF.
|
||||||
|
* - `mtu.frag_delivery` — whether larger datagrams arrive once the network is allowed to
|
||||||
|
* fragment them. A path can be fine for one and broken for the other; conflating them is how
|
||||||
|
* you get "MTU is 4000" on a link that drops every DF packet over 1400.
|
||||||
|
* - `train.udp_downstream` — loss, reordering and arrival spacing in the download direction.
|
||||||
|
*/
|
||||||
|
class DownstreamMeasurement(private val ids: IdSource) {
|
||||||
|
|
||||||
|
private val json = Json { encodeDefaults = true; explicitNulls = true }
|
||||||
|
|
||||||
|
/** How long to wait for a granted burst after the server accepts the action. */
|
||||||
|
private val collectWindowMs = 4_000L
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Runs all three against an already-primed session.
|
||||||
|
*
|
||||||
|
* [session] must already have sent at least one ECHO: the grant is bound to the source the
|
||||||
|
* server has actually observed, so an unprimed session gets a 409 rather than a grant. That
|
||||||
|
* is the anti-amplification rule doing its job, not an error to work around.
|
||||||
|
*/
|
||||||
|
fun run(
|
||||||
|
credential: String,
|
||||||
|
sessionId: String,
|
||||||
|
control: ControlClient,
|
||||||
|
probe: ProbeSession,
|
||||||
|
sessionRef: String,
|
||||||
|
sizes: List<Int> = DEFAULT_SIZES,
|
||||||
|
trainCount: Int = 100,
|
||||||
|
trainSizeBytes: Int = 300,
|
||||||
|
trainIntervalUs: Int = 3_000,
|
||||||
|
): Pair<List<Test>, List<Finding>> {
|
||||||
|
val tests = ArrayList<Test>()
|
||||||
|
val findings = ArrayList<Finding>()
|
||||||
|
|
||||||
|
val df = bigSend(credential, sessionId, control, probe, sessionRef, sizes, df = true)
|
||||||
|
val frag = bigSend(credential, sessionId, control, probe, sessionRef, sizes, df = false)
|
||||||
|
val train = downTrain(
|
||||||
|
credential, sessionId, control, probe, sessionRef, trainCount, trainSizeBytes, trainIntervalUs,
|
||||||
|
)
|
||||||
|
|
||||||
|
tests.add(df.test); tests.add(frag.test); tests.add(train.test)
|
||||||
|
|
||||||
|
// A downstream MTU below the classic 1500-byte Ethernet payload is worth saying out loud:
|
||||||
|
// it is the usual cause of "small requests work, large responses hang".
|
||||||
|
val pathMtu = df.largestDelivered
|
||||||
|
if (pathMtu != null && pathMtu > 0) {
|
||||||
|
val ipMtu = pathMtu + IP_UDP_OVERHEAD4
|
||||||
|
if (ipMtu < 1500) {
|
||||||
|
findings.add(
|
||||||
|
finding(
|
||||||
|
"mtu.reduced_downstream", Category.MTU, Severity.LOW, df.test.id,
|
||||||
|
"Downstream path MTU is $ipMtu bytes, below 1500",
|
||||||
|
"The largest datagram that reached this device without fragmenting was " +
|
||||||
|
"$pathMtu bytes of payload ($ipMtu on the wire). Tunnels (PPPoE, VPN, " +
|
||||||
|
"IPv6-in-IPv4) commonly do this; it is only a fault when something " +
|
||||||
|
"on the path also blocks the ICMP messages that let senders discover it.",
|
||||||
|
),
|
||||||
|
)
|
||||||
|
}
|
||||||
|
// The dangerous combination: unfragmented large packets vanish AND fragments do too,
|
||||||
|
// so a sender that never gets told will retransmit into a black hole.
|
||||||
|
val fragLargest = frag.largestDelivered ?: 0
|
||||||
|
if (fragLargest <= pathMtu && sizes.any { it > pathMtu }) {
|
||||||
|
findings.add(
|
||||||
|
finding(
|
||||||
|
"mtu.downstream_blackhole", Category.MTU, Severity.MEDIUM, frag.test.id,
|
||||||
|
"Datagrams above $pathMtu bytes are dropped downstream, fragmented or not",
|
||||||
|
"Nothing larger than $pathMtu bytes arrived, even when the network was " +
|
||||||
|
"free to fragment it. Traffic that relies on large responses will " +
|
||||||
|
"stall rather than fail cleanly.",
|
||||||
|
),
|
||||||
|
)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
if (train.received == 0) {
|
||||||
|
findings.add(
|
||||||
|
finding(
|
||||||
|
"connectivity.downstream_blocked", Category.CONNECTIVITY, Severity.HIGH, train.test.id,
|
||||||
|
"No server-initiated packets arrived",
|
||||||
|
"The server sent ${train.sent} packets toward this device and none arrived, " +
|
||||||
|
"while the round-trip echo worked. Something on the path forwards replies " +
|
||||||
|
"but drops traffic the device did not individually solicit.",
|
||||||
|
),
|
||||||
|
)
|
||||||
|
} else if (train.lossPct >= 5.0) {
|
||||||
|
findings.add(
|
||||||
|
finding(
|
||||||
|
"connectivity.downstream_loss", Category.CONNECTIVITY, Severity.MEDIUM, train.test.id,
|
||||||
|
"Downstream loss of ${round1(train.lossPct)}%",
|
||||||
|
"${train.sent - train.received} of ${train.sent} packets sent toward this " +
|
||||||
|
"device were lost. Downstream loss is invisible to a round-trip test, " +
|
||||||
|
"which reports only that *something* was lost somewhere.",
|
||||||
|
),
|
||||||
|
)
|
||||||
|
}
|
||||||
|
if (train.reordered > 0) {
|
||||||
|
findings.add(
|
||||||
|
finding(
|
||||||
|
"connectivity.downstream_reorder", Category.CONNECTIVITY, Severity.LOW, train.test.id,
|
||||||
|
"${train.reordered} downstream packet(s) arrived out of order",
|
||||||
|
"Packets arrived in a different order than they were sent. Usually per-packet " +
|
||||||
|
"load balancing across links; harmless for most traffic, not for all of it.",
|
||||||
|
),
|
||||||
|
)
|
||||||
|
}
|
||||||
|
return tests to findings
|
||||||
|
}
|
||||||
|
|
||||||
|
// ---- big_send ---------------------------------------------------------------------
|
||||||
|
|
||||||
|
private class SizeResult(val test: Test, val largestDelivered: Int?)
|
||||||
|
|
||||||
|
private fun bigSend(
|
||||||
|
credential: String, sessionId: String, control: ControlClient, probe: ProbeSession,
|
||||||
|
sessionRef: String, sizes: List<Int>, df: Boolean,
|
||||||
|
): SizeResult {
|
||||||
|
val testId = ids.uuid()
|
||||||
|
val started = ids.monoNs()
|
||||||
|
val requested = sizes.joinToString(",")
|
||||||
|
|
||||||
|
val reply = runCatching {
|
||||||
|
control.action(
|
||||||
|
credential, sessionId,
|
||||||
|
"""{"action":"big_send","df":$df,"sizes_bytes":[$requested]}""",
|
||||||
|
)
|
||||||
|
}
|
||||||
|
if (reply.isFailure) {
|
||||||
|
return SizeResult(
|
||||||
|
Test(
|
||||||
|
id = testId, type = if (df) TestType.MTU_PMTUD_DOWN else TestType.MTU_FRAG_DELIVERY,
|
||||||
|
sessionRef = sessionRef, tier = Tier.APP,
|
||||||
|
startedMonoNs = started, endedMonoNs = ids.monoNs(),
|
||||||
|
status = TestStatus.UNSUPPORTED,
|
||||||
|
error = TestError("action_refused", reply.exceptionOrNull()?.message ?: "big_send refused"),
|
||||||
|
),
|
||||||
|
null,
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
// The server tells us which sizes it actually put on the wire. With DF it refuses
|
||||||
|
// anything above its own egress MTU, and treating those as "lost downstream" would
|
||||||
|
// blame the client's network for our own limit.
|
||||||
|
val accepted = parseIntArray(reply.getOrNull(), "sizes_bytes").ifEmpty { sizes }
|
||||||
|
val serverMaxDf = parseInt(reply.getOrNull(), "max_df_bytes")
|
||||||
|
|
||||||
|
val arrived = probe.collectGranted(collectWindowMs)
|
||||||
|
.filter { it.type == Wire.TYPE_BIG_SEND }
|
||||||
|
.map { it.sizeBytes }
|
||||||
|
.distinct()
|
||||||
|
.sorted()
|
||||||
|
val largest = arrived.maxOrNull()
|
||||||
|
|
||||||
|
val metrics = json.encodeToJsonElement(
|
||||||
|
BigSendMetrics(
|
||||||
|
requestedBytes = sizes,
|
||||||
|
sentBytes = accepted,
|
||||||
|
deliveredBytes = arrived,
|
||||||
|
largestDeliveredBytes = largest,
|
||||||
|
dontFragment = df,
|
||||||
|
serverMaxDfBytes = serverMaxDf,
|
||||||
|
// Only meaningful for the DF run; the IP-level MTU is the payload plus headers.
|
||||||
|
pathMtuBytes = if (df && largest != null) largest + IP_UDP_OVERHEAD4 else null,
|
||||||
|
),
|
||||||
|
) as JsonObject
|
||||||
|
|
||||||
|
val status = when {
|
||||||
|
arrived.isEmpty() -> TestStatus.FAILED
|
||||||
|
arrived.size < accepted.size -> TestStatus.PARTIAL
|
||||||
|
else -> TestStatus.OK
|
||||||
|
}
|
||||||
|
return SizeResult(
|
||||||
|
Test(
|
||||||
|
id = testId, type = if (df) TestType.MTU_PMTUD_DOWN else TestType.MTU_FRAG_DELIVERY,
|
||||||
|
sessionRef = sessionRef, tier = Tier.APP,
|
||||||
|
startedMonoNs = started, endedMonoNs = ids.monoNs(),
|
||||||
|
status = status, metrics = metrics,
|
||||||
|
),
|
||||||
|
largest,
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
// ---- downtrain --------------------------------------------------------------------
|
||||||
|
|
||||||
|
private class TrainResult(
|
||||||
|
val test: Test, val sent: Int, val received: Int, val lossPct: Double, val reordered: Int,
|
||||||
|
)
|
||||||
|
|
||||||
|
private fun downTrain(
|
||||||
|
credential: String, sessionId: String, control: ControlClient, probe: ProbeSession,
|
||||||
|
sessionRef: String, count: Int, sizeBytes: Int, intervalUs: Int,
|
||||||
|
): TrainResult {
|
||||||
|
val testId = ids.uuid()
|
||||||
|
val started = ids.monoNs()
|
||||||
|
|
||||||
|
val reply = runCatching {
|
||||||
|
control.action(
|
||||||
|
credential, sessionId,
|
||||||
|
"""{"action":"downtrain","count":$count,"size_bytes":$sizeBytes,"interval_us":$intervalUs}""",
|
||||||
|
)
|
||||||
|
}
|
||||||
|
if (reply.isFailure) {
|
||||||
|
return TrainResult(
|
||||||
|
Test(
|
||||||
|
id = testId, type = TestType.TRAIN_UDP_DOWNSTREAM, sessionRef = sessionRef,
|
||||||
|
tier = Tier.APP, startedMonoNs = started, endedMonoNs = ids.monoNs(),
|
||||||
|
status = TestStatus.UNSUPPORTED,
|
||||||
|
error = TestError("action_refused", reply.exceptionOrNull()?.message ?: "downtrain refused"),
|
||||||
|
),
|
||||||
|
0, 0, 0.0, 0,
|
||||||
|
)
|
||||||
|
}
|
||||||
|
val sent = parseInt(reply.getOrNull(), "count") ?: count
|
||||||
|
|
||||||
|
val got = probe.collectGranted(collectWindowMs).filter { it.type == Wire.TYPE_DOWNTRAIN_DATA }
|
||||||
|
val received = got.size
|
||||||
|
val lossPct = if (sent == 0) 0.0 else (sent - received) * 100.0 / sent
|
||||||
|
|
||||||
|
// Reordering: a packet whose sequence is below the highest already seen. Counting
|
||||||
|
// inversions rather than "not sorted" keeps one late packet from being reported as
|
||||||
|
// dozens of reorder events.
|
||||||
|
var highest = -1
|
||||||
|
var reordered = 0
|
||||||
|
for (p in got) {
|
||||||
|
if (p.seq < highest) reordered++ else highest = p.seq
|
||||||
|
}
|
||||||
|
|
||||||
|
// Columnar evidence per the schema: what arrived, when, and how big — so every metric
|
||||||
|
// above is recomputable by a reader who does not trust our arithmetic.
|
||||||
|
val evidence = TrainEvidence(
|
||||||
|
epochMonoNs = started,
|
||||||
|
seq = got.map { it.seq },
|
||||||
|
tTxNs = got.map { null },
|
||||||
|
tRxNs = got.map { it.tRxNs },
|
||||||
|
sizeBytes = got.map { it.sizeBytes },
|
||||||
|
).toEvidence()
|
||||||
|
|
||||||
|
val interArrival = got.zipWithNext { a, b -> (b.tRxNs - a.tRxNs) / 1_000_000.0 }
|
||||||
|
val metrics = json.encodeToJsonElement(
|
||||||
|
DownTrainMetrics(
|
||||||
|
sent = sent, received = received, lossPct = round1(lossPct),
|
||||||
|
reorderedPackets = reordered,
|
||||||
|
sizeBytes = sizeBytes,
|
||||||
|
interArrivalMsAvg = interArrival.average().takeIf { interArrival.isNotEmpty() }?.let(::round1),
|
||||||
|
interArrivalMsMax = interArrival.maxOrNull()?.let(::round1),
|
||||||
|
sendIntervalUs = intervalUs,
|
||||||
|
),
|
||||||
|
) as JsonObject
|
||||||
|
|
||||||
|
val status = when {
|
||||||
|
received == 0 -> TestStatus.FAILED
|
||||||
|
received < sent -> TestStatus.PARTIAL
|
||||||
|
else -> TestStatus.OK
|
||||||
|
}
|
||||||
|
return TrainResult(
|
||||||
|
Test(
|
||||||
|
id = testId, type = TestType.TRAIN_UDP_DOWNSTREAM, sessionRef = sessionRef, tier = Tier.APP,
|
||||||
|
startedMonoNs = started, endedMonoNs = ids.monoNs(),
|
||||||
|
status = status, evidence = evidence, metrics = metrics,
|
||||||
|
),
|
||||||
|
sent, received, lossPct, reordered,
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
// ---- helpers ----------------------------------------------------------------------
|
||||||
|
|
||||||
|
private fun finding(code: String, cat: Category, sev: Severity, testId: String, title: String, desc: String) =
|
||||||
|
Finding(
|
||||||
|
id = ids.uuid(), code = code, category = cat, severity = sev, confidence = Confidence.HIGH,
|
||||||
|
title = title, description = desc, evidenceRefs = listOf(EvidenceRef(testId)),
|
||||||
|
)
|
||||||
|
|
||||||
|
/** Minimal scalar extraction from the action reply; the shape is small and server-owned. */
|
||||||
|
private fun parseInt(body: String?, key: String): Int? =
|
||||||
|
body?.let { Regex("\"$key\"\\s*:\\s*(-?\\d+)").find(it)?.groupValues?.get(1)?.toIntOrNull() }
|
||||||
|
|
||||||
|
private fun parseIntArray(body: String?, key: String): List<Int> =
|
||||||
|
body?.let { b ->
|
||||||
|
Regex("\"$key\"\\s*:\\s*\\[([^\\]]*)\\]").find(b)?.groupValues?.get(1)
|
||||||
|
?.split(",")?.mapNotNull { it.trim().toIntOrNull() }
|
||||||
|
} ?: emptyList()
|
||||||
|
|
||||||
|
private companion object {
|
||||||
|
/** IPv4 (20) + UDP (8). The v6 case is 48; reported per-family once v6 sessions land. */
|
||||||
|
const val IP_UDP_OVERHEAD4 = 28
|
||||||
|
|
||||||
|
/** Straddles the usual suspects: 1500 Ethernet, 1492 PPPoE, 1400-ish tunnels. */
|
||||||
|
val DEFAULT_SIZES = listOf(600, 1200, 1372, 1400, 1450, 1472, 1500, 2000, 4000)
|
||||||
|
|
||||||
|
fun round1(v: Double) = Math.round(v * 10.0) / 10.0
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Metrics for mtu.pmtud_down / mtu.frag_delivery. */
|
||||||
|
@Serializable
|
||||||
|
data class BigSendMetrics(
|
||||||
|
@SerialName("requested_bytes") val requestedBytes: List<Int>,
|
||||||
|
@SerialName("sent_bytes") val sentBytes: List<Int>,
|
||||||
|
@SerialName("delivered_bytes") val deliveredBytes: List<Int>,
|
||||||
|
@SerialName("largest_delivered_bytes") val largestDeliveredBytes: Int? = null,
|
||||||
|
@SerialName("dont_fragment") val dontFragment: Boolean,
|
||||||
|
/** The server's own DF ceiling; sizes above it were never sent and are not path evidence. */
|
||||||
|
@SerialName("server_max_df_bytes") val serverMaxDfBytes: Int? = null,
|
||||||
|
@SerialName("path_mtu_bytes") val pathMtuBytes: Int? = null,
|
||||||
|
)
|
||||||
|
|
||||||
|
/** Metrics for train.udp_downstream. */
|
||||||
|
@Serializable
|
||||||
|
data class DownTrainMetrics(
|
||||||
|
val sent: Int,
|
||||||
|
val received: Int,
|
||||||
|
@SerialName("loss_pct") val lossPct: Double,
|
||||||
|
@SerialName("reordered_packets") val reorderedPackets: Int,
|
||||||
|
@SerialName("size_bytes") val sizeBytes: Int,
|
||||||
|
@SerialName("inter_arrival_ms_avg") val interArrivalMsAvg: Double? = null,
|
||||||
|
@SerialName("inter_arrival_ms_max") val interArrivalMsMax: Double? = null,
|
||||||
|
@SerialName("send_interval_us") val sendIntervalUs: Int,
|
||||||
|
)
|
||||||
@@ -36,6 +36,12 @@ class ServerMeasurement(
|
|||||||
val udpPort: Int,
|
val udpPort: Int,
|
||||||
val echoCount: Int = 20,
|
val echoCount: Int = 20,
|
||||||
val echoPaddingBytes: Int = 64,
|
val echoPaddingBytes: Int = 64,
|
||||||
|
/**
|
||||||
|
* Whether to ask the server to push traffic back (downstream MTU and downstream train).
|
||||||
|
* Costs a few hundred kB of download and needs a server that advertises the grants, so
|
||||||
|
* it is a flag rather than an assumption.
|
||||||
|
*/
|
||||||
|
val downstream: Boolean = true,
|
||||||
)
|
)
|
||||||
|
|
||||||
fun run(cfg: Config): MeasurementDocument {
|
fun run(cfg: Config): MeasurementDocument {
|
||||||
@@ -57,11 +63,32 @@ class ServerMeasurement(
|
|||||||
target = SessionTarget(ip4 = cfg.udpHost, udpPort = cfg.udpPort),
|
target = SessionTarget(ip4 = cfg.udpHost, udpPort = cfg.udpPort),
|
||||||
)
|
)
|
||||||
|
|
||||||
val (test, findings) = echoTrain(cfg, control, session, startMono)
|
val tests = ArrayList<Test>()
|
||||||
|
val allFindings = ArrayList<Finding>()
|
||||||
|
|
||||||
|
// One ProbeSession for the whole run. A second one would open a new socket and restart
|
||||||
|
// the sequence counter, which the server's anti-replay window correctly rejects — so the
|
||||||
|
// re-primed source is never recorded and every granted send goes to the old, closed port.
|
||||||
|
// Session identity lives on the server; the socket must live as long as it does.
|
||||||
|
ProbeSession(cfg.credential, session, cfg.udpHost, cfg.udpPort).use { ps ->
|
||||||
|
val (test, findings) = echoTrain(cfg, ps, startMono)
|
||||||
|
tests.add(test)
|
||||||
|
allFindings.addAll(findings)
|
||||||
|
|
||||||
|
// Downstream needs a session the server has already seen traffic from — the echo
|
||||||
|
// train just provided that — and a server that advertises the grants. Skipped
|
||||||
|
// quietly against an older server rather than reported as a failure of the network.
|
||||||
|
if (cfg.downstream && profile.supports("downtrain") && profile.supports("big-send")) {
|
||||||
|
val (dsTests, dsFindings) = DownstreamMeasurement(ids)
|
||||||
|
.run(cfg.credential, session.sessionId, control, ps, sessionRef = "sess-1")
|
||||||
|
tests.addAll(dsTests)
|
||||||
|
allFindings.addAll(dsFindings)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
control.deleteSession(cfg.credential, session.sessionId)
|
control.deleteSession(cfg.credential, session.sessionId)
|
||||||
|
|
||||||
val summary = Verdicts.derive(listOf(test), findings)
|
val summary = Verdicts.derive(tests, allFindings)
|
||||||
return MeasurementDocument(
|
return MeasurementDocument(
|
||||||
run = Run(
|
run = Run(
|
||||||
id = runId, trigger = Trigger.MANUAL, startedAt = startWall, endedAt = ids.nowWall(),
|
id = runId, trigger = Trigger.MANUAL, startedAt = startWall, endedAt = ids.nowWall(),
|
||||||
@@ -70,14 +97,14 @@ class ServerMeasurement(
|
|||||||
tiers = Tiers(app = true),
|
tiers = Tiers(app = true),
|
||||||
),
|
),
|
||||||
serverSessions = listOf(serverSession),
|
serverSessions = listOf(serverSession),
|
||||||
tests = listOf(test),
|
tests = tests,
|
||||||
findings = findings,
|
findings = allFindings,
|
||||||
summary = summary,
|
summary = summary,
|
||||||
)
|
)
|
||||||
}
|
}
|
||||||
|
|
||||||
private fun echoTrain(
|
private fun echoTrain(
|
||||||
cfg: Config, control: ControlClient, session: app.echo_lot.protocol.SessionResponse, startMono: Long,
|
cfg: Config, ps: ProbeSession, startMono: Long,
|
||||||
): Pair<Test, List<Finding>> {
|
): Pair<Test, List<Finding>> {
|
||||||
val testId = ids.uuid()
|
val testId = ids.uuid()
|
||||||
val seqs = ArrayList<Int>()
|
val seqs = ArrayList<Int>()
|
||||||
@@ -87,7 +114,6 @@ class ServerMeasurement(
|
|||||||
val rtts = ArrayList<Double>()
|
val rtts = ArrayList<Double>()
|
||||||
val observedPorts = LinkedHashSet<Int>()
|
val observedPorts = LinkedHashSet<Int>()
|
||||||
|
|
||||||
ProbeSession(cfg.credential, session, cfg.udpHost, cfg.udpPort).use { ps ->
|
|
||||||
for (i in 0 until cfg.echoCount) {
|
for (i in 0 until cfg.echoCount) {
|
||||||
val txMono = ids.monoNs() - startMono
|
val txMono = ids.monoNs() - startMono
|
||||||
val r = ps.echo(cfg.echoPaddingBytes)
|
val r = ps.echo(cfg.echoPaddingBytes)
|
||||||
@@ -102,7 +128,6 @@ class ServerMeasurement(
|
|||||||
tRx.add(null)
|
tRx.add(null)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
|
||||||
|
|
||||||
val sent = cfg.echoCount
|
val sent = cfg.echoCount
|
||||||
val received = rtts.size
|
val received = rtts.size
|
||||||
|
|||||||
@@ -0,0 +1,68 @@
|
|||||||
|
// SPDX-FileCopyrightText: 2026 Echolot contributors
|
||||||
|
// SPDX-License-Identifier: GPL-3.0-or-later
|
||||||
|
|
||||||
|
package app.echo_lot.engine
|
||||||
|
|
||||||
|
import app.echo_lot.measurement.TestStatus
|
||||||
|
import app.echo_lot.measurement.TestType
|
||||||
|
import app.echo_lot.protocol.ControlClient
|
||||||
|
import app.echo_lot.protocol.ProbeSession
|
||||||
|
import kotlin.test.Test as JTest
|
||||||
|
import kotlin.test.assertEquals
|
||||||
|
import kotlin.test.assertNotNull
|
||||||
|
import kotlin.test.assertTrue
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Runs DownstreamMeasurement against a LIVE server and checks the *documents* it produces, not
|
||||||
|
* just that packets moved: the tests must carry recomputable metrics and land on the right test
|
||||||
|
* types, because that is what an archived run is read back as. Self-skips without ECHOLOT_LIVE_*.
|
||||||
|
*/
|
||||||
|
class LiveDownstreamTest {
|
||||||
|
|
||||||
|
private val url = System.getenv("ECHOLOT_LIVE_URL")
|
||||||
|
private val pin = System.getenv("ECHOLOT_LIVE_PIN")
|
||||||
|
private val cred = System.getenv("ECHOLOT_LIVE_CRED")
|
||||||
|
private val udp = System.getenv("ECHOLOT_LIVE_UDP")
|
||||||
|
private val target = System.getenv("ECHOLOT_LIVE_TARGET") ?: "fmr"
|
||||||
|
|
||||||
|
@JTest
|
||||||
|
fun producesDownstreamTestsAndFindings() {
|
||||||
|
if (url == null || pin == null || cred == null || udp == null) {
|
||||||
|
println("LiveDownstreamTest skipped (no ECHOLOT_LIVE_* env)"); return
|
||||||
|
}
|
||||||
|
val control = ControlClient(url, setOf(pin))
|
||||||
|
val session = control.createSession(cred, target)
|
||||||
|
val (host, port) = udp.split(":").let { it[0] to it[1].toInt() }
|
||||||
|
|
||||||
|
val (tests, findings) = ProbeSession(cred, session, host, port).use { ps ->
|
||||||
|
ps.echo() // prime: the grant binds to the source the server has actually observed
|
||||||
|
DownstreamMeasurement(SystemIdSource())
|
||||||
|
.run(cred, session.sessionId, control, ps, sessionRef = "sess-1")
|
||||||
|
}
|
||||||
|
control.deleteSession(cred, session.sessionId)
|
||||||
|
|
||||||
|
for (t in tests) println("${t.type} status=${t.status} metrics=${t.metrics}")
|
||||||
|
for (f in findings) println("finding ${f.code} [${f.severity}] ${f.title}")
|
||||||
|
|
||||||
|
assertEquals(3, tests.size, "expected pmtud_down, frag_delivery and a downstream train")
|
||||||
|
val byType = tests.associateBy { it.type }
|
||||||
|
|
||||||
|
val pmtud = assertNotNull(byType[TestType.MTU_PMTUD_DOWN], "no mtu.pmtud_down test")
|
||||||
|
assertTrue(pmtud.status == TestStatus.OK || pmtud.status == TestStatus.PARTIAL,
|
||||||
|
"DF probe did not deliver anything: ${pmtud.status}")
|
||||||
|
val pathMtu = pmtud.metrics?.get("path_mtu_bytes")?.toString()?.toIntOrNull()
|
||||||
|
assertNotNull(pathMtu, "pmtud_down must report a path MTU")
|
||||||
|
assertTrue(pathMtu in 576..9000, "implausible downstream path MTU: $pathMtu")
|
||||||
|
println("downstream path MTU = $pathMtu bytes")
|
||||||
|
|
||||||
|
val frag = assertNotNull(byType[TestType.MTU_FRAG_DELIVERY], "no mtu.frag_delivery test")
|
||||||
|
assertNotNull(frag.metrics?.get("largest_delivered_bytes"))
|
||||||
|
|
||||||
|
val train = assertNotNull(byType[TestType.TRAIN_UDP_DOWNSTREAM], "no downstream train")
|
||||||
|
assertNotNull(train.evidence, "a train without columnar evidence is not recomputable")
|
||||||
|
val received = train.metrics?.get("received")?.toString()?.toIntOrNull() ?: 0
|
||||||
|
assertTrue(received > 0, "no downstream train packets arrived")
|
||||||
|
println("downstream train: $received received, loss=${train.metrics?.get("loss_pct")}, " +
|
||||||
|
"reordered=${train.metrics?.get("reordered_packets")}")
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -54,19 +54,35 @@ class LiveGrantedTest {
|
|||||||
"sizes=${down.map { it.sizeBytes }.distinct()}")
|
"sizes=${down.map { it.sizeBytes }.distinct()}")
|
||||||
assertTrue(down.isNotEmpty(), "no DOWNTRAIN_DATA arrived — granted send path is broken")
|
assertTrue(down.isNotEmpty(), "no DOWNTRAIN_DATA arrived — granted send path is broken")
|
||||||
|
|
||||||
// --- big_send: which downstream sizes survive? ---
|
// --- big_send with DF: the largest size that arrives is the downstream path MTU ---
|
||||||
val sizes = listOf(600, 1200, 1400, 1472, 1500, 2000, 4000)
|
val sizes = listOf(600, 1200, 1400, 1472, 1500, 2000, 4000)
|
||||||
val bsResp = control.action(
|
val dfResp = control.action(
|
||||||
cred, session.sessionId,
|
cred, session.sessionId,
|
||||||
"""{"action":"big_send","sizes_bytes":${sizes}}""",
|
"""{"action":"big_send","df":true,"sizes_bytes":${sizes}}""",
|
||||||
)
|
)
|
||||||
println("big_send accepted: ${bsResp.take(160)}")
|
println("big_send(df) accepted: ${dfResp.take(200)}")
|
||||||
val big = ps.collectGranted(windowMs = 4000)
|
val dfArrived = ps.collectGranted(windowMs = 4000)
|
||||||
.filter { it.type == Wire.TYPE_BIG_SEND }
|
.filter { it.type == Wire.TYPE_BIG_SEND }.map { it.sizeBytes }.sorted()
|
||||||
val arrived = big.map { it.sizeBytes }.sorted()
|
println("big_send(df) arrived: $dfArrived")
|
||||||
println("big_send arrived sizes: $arrived (requested $sizes)")
|
assertTrue(dfArrived.isNotEmpty(), "no unfragmented BIG_SEND packets arrived")
|
||||||
assertTrue(big.isNotEmpty(), "no BIG_SEND packets arrived")
|
val pathMtu = dfArrived.max()
|
||||||
println("largest downstream datagram delivered: ${arrived.maxOrNull()}")
|
|
||||||
|
// --- and without DF, to see whether fragments get through above that ---
|
||||||
|
val fragResp = control.action(
|
||||||
|
cred, session.sessionId,
|
||||||
|
"""{"action":"big_send","df":false,"sizes_bytes":${sizes}}""",
|
||||||
|
)
|
||||||
|
println("big_send(frag) accepted: ${fragResp.take(200)}")
|
||||||
|
val fragArrived = ps.collectGranted(windowMs = 4000)
|
||||||
|
.filter { it.type == Wire.TYPE_BIG_SEND }.map { it.sizeBytes }.sorted()
|
||||||
|
println("big_send(frag) arrived: $fragArrived")
|
||||||
|
|
||||||
|
// The distinction the DF flag exists for: fragmented delivery may exceed the
|
||||||
|
// unfragmented path MTU, and reporting the former as the latter would be a lie.
|
||||||
|
println("downstream path MTU (payload bytes) = $pathMtu; " +
|
||||||
|
"largest fragmented delivery = ${fragArrived.maxOrNull()}")
|
||||||
|
assertTrue((fragArrived.maxOrNull() ?: 0) >= pathMtu,
|
||||||
|
"fragmented delivery should reach at least as far as unfragmented")
|
||||||
}
|
}
|
||||||
control.deleteSession(cred, session.sessionId)
|
control.deleteSession(cred, session.sessionId)
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -47,8 +47,11 @@ class LiveMeasurementTest {
|
|||||||
|
|
||||||
assertEquals(1, doc.serverSessions.size)
|
assertEquals(1, doc.serverSessions.size)
|
||||||
assertTrue(doc.serverSessions[0].capabilities.contains("udp-probe"))
|
assertTrue(doc.serverSessions[0].capabilities.contains("udp-probe"))
|
||||||
val test = doc.tests.single()
|
// A full run is the echo train plus the three downstream tests; assert on the one this
|
||||||
assertEquals(TestType.TRAIN_UDP_UPDOWN, test.type)
|
// test is about rather than on the count, so adding a measurement is not a test edit.
|
||||||
|
for (t in doc.tests) println(" ${t.type} → ${t.status}")
|
||||||
|
for (f in doc.findings) println(" finding ${f.code} [${f.severity}] ${f.title}")
|
||||||
|
val test = doc.tests.first { it.type == TestType.TRAIN_UDP_UPDOWN }
|
||||||
assertTrue(test.status == TestStatus.OK || test.status == TestStatus.PARTIAL,
|
assertTrue(test.status == TestStatus.OK || test.status == TestStatus.PARTIAL,
|
||||||
"expected replies from live server, got ${test.status}")
|
"expected replies from live server, got ${test.status}")
|
||||||
|
|
||||||
|
|||||||
@@ -0,0 +1,102 @@
|
|||||||
|
// SPDX-FileCopyrightText: 2026 Echolot contributors
|
||||||
|
// SPDX-License-Identifier: GPL-3.0-or-later
|
||||||
|
|
||||||
|
package app.echo_lot.engine
|
||||||
|
|
||||||
|
import app.echo_lot.privacy.Anonymizer
|
||||||
|
import app.echo_lot.privacy.PrivacyLevel
|
||||||
|
import app.echo_lot.privacy.Salt
|
||||||
|
import app.echo_lot.protocol.ControlClient
|
||||||
|
import app.echo_lot.protocol.UploadRefused
|
||||||
|
import kotlinx.serialization.json.Json
|
||||||
|
import kotlinx.serialization.json.JsonObject
|
||||||
|
import kotlinx.serialization.json.jsonObject
|
||||||
|
import kotlin.test.Test
|
||||||
|
import kotlin.test.assertFalse
|
||||||
|
import kotlin.test.assertTrue
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Drives the upload path against a LIVE server: anonymize, upload, list, fetch back, delete.
|
||||||
|
*
|
||||||
|
* The point is not that the HTTP works — it is that what comes *back off the server* has been
|
||||||
|
* stripped. Uploading and then re-reading the stored document is the only check that proves the
|
||||||
|
* anonymizer ran on the bytes that actually left, rather than on a copy. Self-skips without
|
||||||
|
* ECHOLOT_LIVE_*.
|
||||||
|
*/
|
||||||
|
class LiveUploadTest {
|
||||||
|
|
||||||
|
private val url = System.getenv("ECHOLOT_LIVE_URL")
|
||||||
|
private val pin = System.getenv("ECHOLOT_LIVE_PIN")
|
||||||
|
private val cred = System.getenv("ECHOLOT_LIVE_CRED")
|
||||||
|
private val json = Json { prettyPrint = false }
|
||||||
|
|
||||||
|
private fun sampleRun(id: String) = """
|
||||||
|
{
|
||||||
|
"schema": "echolot/measurement",
|
||||||
|
"run": {
|
||||||
|
"id": "$id", "trigger": "manual", "started_at": "2026-08-01T10:00:00Z",
|
||||||
|
"notes": "kitchen table",
|
||||||
|
"device": {"manufacturer": "OnePlus", "model": "CPH2747"}
|
||||||
|
},
|
||||||
|
"networks": [{
|
||||||
|
"id": "net-1", "ssid": "Rambossek WLAN", "bssid": "78:9a:18:aa:bb:cc",
|
||||||
|
"gateway_ip4": "192.168.1.1", "public_ip4": "203.0.113.77",
|
||||||
|
"ssdp_responders": [{"friendly_name": "Living Room TV"}]
|
||||||
|
}],
|
||||||
|
"tests": [{"id": "t1", "type": "train.udp_updown", "status": "ok",
|
||||||
|
"metrics": {"rtt_ms_avg": 12.4, "loss_pct": 0.0}}],
|
||||||
|
"findings": [{"id": "f1", "code": "nat.udp_rebinding", "severity": "medium"}],
|
||||||
|
"summary": {"verdict": "warn"}
|
||||||
|
}
|
||||||
|
""".trimIndent()
|
||||||
|
|
||||||
|
@Test
|
||||||
|
fun uploadRoundTrip() {
|
||||||
|
if (url == null || pin == null || cred == null) {
|
||||||
|
println("LiveUploadTest skipped (no ECHOLOT_LIVE_* env)"); return
|
||||||
|
}
|
||||||
|
val control = ControlClient(url, setOf(pin))
|
||||||
|
val profile = control.profile(cred)
|
||||||
|
val policy = profile.uploads
|
||||||
|
println("upload policy: mode=${policy.mode} min_anon=${policy.minAnonymization} " +
|
||||||
|
"max_bytes=${policy.maxBytes} retention_days=${policy.retentionDays}")
|
||||||
|
|
||||||
|
val runId = "livetest-" + System.nanoTime().toString().takeLast(10)
|
||||||
|
val level = PrivacyLevel.max(PrivacyLevel.BALANCED, PrivacyLevel.fromWire(policy.minAnonymization))
|
||||||
|
val redacted = json.encodeToString(
|
||||||
|
JsonObject.serializer(),
|
||||||
|
Anonymizer(level, Salt.perRun(ByteArray(32) { 9 }))
|
||||||
|
.anonymize(json.parseToJsonElement(sampleRun(runId)).jsonObject),
|
||||||
|
)
|
||||||
|
assertFalse(redacted.contains("Rambossek"), "the anonymizer did not strip the SSID before upload")
|
||||||
|
|
||||||
|
if (!policy.accepted) {
|
||||||
|
// A server configured to refuse must refuse — that is the behaviour worth asserting.
|
||||||
|
try {
|
||||||
|
control.uploadRun(cred, redacted)
|
||||||
|
throw AssertionError("server advertises mode=${policy.mode} but accepted an upload")
|
||||||
|
} catch (e: UploadRefused) {
|
||||||
|
println("upload correctly refused: ${e.message?.take(140)}")
|
||||||
|
return
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
val created = control.uploadRun(cred, redacted)
|
||||||
|
println("stored: ${created.take(200)}")
|
||||||
|
|
||||||
|
val listed = control.listRuns(cred)
|
||||||
|
assertTrue(listed.contains(runId), "uploaded run is missing from the server's list")
|
||||||
|
|
||||||
|
val fetched = control.getRun(cred, runId)
|
||||||
|
assertFalse(fetched.contains("Rambossek"), "the SSID is sitting on the server")
|
||||||
|
assertFalse(fetched.contains("Living Room TV"), "an SSDP neighbour name is sitting on the server")
|
||||||
|
assertFalse(fetched.contains("kitchen table"), "a free-text note is sitting on the server")
|
||||||
|
assertTrue(fetched.contains("nat.udp_rebinding"), "the finding code should survive — it is the point")
|
||||||
|
assertTrue(fetched.contains("12.4"), "metrics should survive anonymization")
|
||||||
|
println("round trip verified: identifiers stripped, measurements intact")
|
||||||
|
|
||||||
|
control.deleteRun(cred, runId)
|
||||||
|
assertFalse(control.listRuns(cred).contains(runId), "delete did not remove the run")
|
||||||
|
println("deleted")
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -1,100 +0,0 @@
|
|||||||
// SPDX-FileCopyrightText: 2026 Echolot contributors
|
|
||||||
// SPDX-License-Identifier: GPL-3.0-or-later
|
|
||||||
|
|
||||||
// Package measurement models one measurement run (measurement-schema.md) — the archived,
|
|
||||||
// diffable, exportable unit. Design rules honored in the types: observation/interpretation
|
|
||||||
// separated (tests[] vs findings[]), two clocks (wall RFC3339 for humans, *_mono_ns for math),
|
|
||||||
// units in field names, columnar trains. params/evidence/metrics are per-test-type, so they are
|
|
||||||
// carried as JsonObject (the probe engine fills them; consumers ignore unknown fields).
|
|
||||||
package app.echo_lot.measurement
|
|
||||||
|
|
||||||
import kotlinx.serialization.SerialName
|
|
||||||
import kotlinx.serialization.Serializable
|
|
||||||
import kotlinx.serialization.json.JsonObject
|
|
||||||
|
|
||||||
@Serializable
|
|
||||||
data class MeasurementDocument(
|
|
||||||
val schema: String = "echolot/measurement",
|
|
||||||
@SerialName("schema_version") val schemaVersion: String = "1.0.0",
|
|
||||||
val run: Run,
|
|
||||||
val networks: List<Network> = emptyList(),
|
|
||||||
@SerialName("server_sessions") val serverSessions: List<ServerSession> = emptyList(),
|
|
||||||
val tests: List<Test> = emptyList(),
|
|
||||||
val findings: List<Finding> = emptyList(),
|
|
||||||
val summary: Summary? = null,
|
|
||||||
)
|
|
||||||
|
|
||||||
@Serializable
|
|
||||||
data class Run(
|
|
||||||
val id: String, // UUIDv7
|
|
||||||
val trigger: Trigger,
|
|
||||||
@SerialName("started_at") val startedAt: String, // RFC3339 UTC, human correlation only
|
|
||||||
@SerialName("ended_at") val endedAt: String? = null,
|
|
||||||
val clock: Clock,
|
|
||||||
val app: AppInfo,
|
|
||||||
val device: DeviceInfo,
|
|
||||||
val tiers: Tiers,
|
|
||||||
@SerialName("profiles_used") val profilesUsed: List<String> = emptyList(),
|
|
||||||
val notes: String? = null,
|
|
||||||
)
|
|
||||||
|
|
||||||
@Serializable
|
|
||||||
enum class Trigger {
|
|
||||||
@SerialName("manual") MANUAL,
|
|
||||||
@SerialName("scheduled") SCHEDULED,
|
|
||||||
@SerialName("monitor") MONITOR,
|
|
||||||
@SerialName("peer") PEER,
|
|
||||||
}
|
|
||||||
|
|
||||||
/** The two-clock anchor: mono_origin_wall maps the monotonic epoch to a wall time for humans;
|
|
||||||
* all math uses *_mono_ns relative to that monotonic origin. */
|
|
||||||
@Serializable
|
|
||||||
data class Clock(
|
|
||||||
@SerialName("mono_origin_wall") val monoOriginWall: String,
|
|
||||||
@SerialName("ntp_offset_ms") val ntpOffsetMs: Double? = null,
|
|
||||||
@SerialName("ntp_offset_source") val ntpOffsetSource: String? = null,
|
|
||||||
)
|
|
||||||
|
|
||||||
@Serializable
|
|
||||||
data class AppInfo(
|
|
||||||
val version: String,
|
|
||||||
val build: Int,
|
|
||||||
val git: String? = null,
|
|
||||||
val flavor: String? = null,
|
|
||||||
)
|
|
||||||
|
|
||||||
@Serializable
|
|
||||||
data class DeviceInfo(
|
|
||||||
val manufacturer: String,
|
|
||||||
val model: String,
|
|
||||||
@SerialName("android_sdk") val androidSdk: Int,
|
|
||||||
@SerialName("android_release") val androidRelease: String,
|
|
||||||
@SerialName("security_patch") val securityPatch: String? = null,
|
|
||||||
)
|
|
||||||
|
|
||||||
/** What each tier was *available*; each test records what it *used*. */
|
|
||||||
@Serializable
|
|
||||||
data class Tiers(
|
|
||||||
val app: Boolean = true,
|
|
||||||
val shizuku: Boolean = false,
|
|
||||||
val root: Boolean = false,
|
|
||||||
)
|
|
||||||
|
|
||||||
@Serializable
|
|
||||||
data class ServerSession(
|
|
||||||
val id: String,
|
|
||||||
@SerialName("profile_id") val profileId: String? = null,
|
|
||||||
@SerialName("profile_name") val profileName: String? = null,
|
|
||||||
@SerialName("control_url") val controlUrl: String,
|
|
||||||
@SerialName("server_version") val serverVersion: String? = null,
|
|
||||||
val capabilities: List<String> = emptyList(),
|
|
||||||
@SerialName("session_id") val sessionId: String,
|
|
||||||
val target: SessionTarget,
|
|
||||||
)
|
|
||||||
|
|
||||||
@Serializable
|
|
||||||
data class SessionTarget(
|
|
||||||
val ip4: String? = null,
|
|
||||||
val ip6: String? = null,
|
|
||||||
@SerialName("udp_port") val udpPort: Int = 0,
|
|
||||||
)
|
|
||||||
@@ -1,85 +0,0 @@
|
|||||||
// SPDX-FileCopyrightText: 2026 Echolot contributors
|
|
||||||
// SPDX-License-Identifier: GPL-3.0-or-later
|
|
||||||
|
|
||||||
package app.echo_lot.measurement
|
|
||||||
|
|
||||||
import kotlinx.serialization.SerialName
|
|
||||||
import kotlinx.serialization.Serializable
|
|
||||||
import kotlinx.serialization.json.Json
|
|
||||||
import kotlinx.serialization.json.JsonObject
|
|
||||||
import kotlinx.serialization.json.encodeToJsonElement
|
|
||||||
|
|
||||||
/**
|
|
||||||
* Typed builders for the per-test-type evidence shapes the schema fixes (§6.2/§6.3/§6.4). Probe
|
|
||||||
* code fills these and folds them into [Test.evidence] via [toEvidence]; keeping them typed here
|
|
||||||
* means the columnar/traceroute/resolver contracts live in one place.
|
|
||||||
*/
|
|
||||||
|
|
||||||
@PublishedApi
|
|
||||||
internal val evidenceJson = Json { encodeDefaults = true; explicitNulls = true }
|
|
||||||
|
|
||||||
/** Serialize any typed evidence object into the JsonObject the Test envelope carries. */
|
|
||||||
inline fun <reified T> T.toEvidence(): JsonObject =
|
|
||||||
evidenceJson.encodeToJsonElement(this) as JsonObject
|
|
||||||
|
|
||||||
/**
|
|
||||||
* Packet-train evidence (§6.2): columnar parallel arrays, one index per probe packet. Missing
|
|
||||||
* observations are null at that index — a 10k-packet train stays in the hundreds of kB. Server
|
|
||||||
* columns use the server session epoch; only differences within one clock are meaningful unless a
|
|
||||||
* time.server_offset test maps them.
|
|
||||||
*/
|
|
||||||
@Serializable
|
|
||||||
data class TrainEvidence(
|
|
||||||
@SerialName("epoch_mono_ns") val epochMonoNs: Long,
|
|
||||||
val seq: List<Int>,
|
|
||||||
@SerialName("t_tx_ns") val tTxNs: List<Long?>,
|
|
||||||
@SerialName("t_srv_rx_ns") val tSrvRxNs: List<Long?> = emptyList(),
|
|
||||||
@SerialName("t_srv_tx_ns") val tSrvTxNs: List<Long?> = emptyList(),
|
|
||||||
@SerialName("t_rx_ns") val tRxNs: List<Long?>,
|
|
||||||
@SerialName("size_bytes") val sizeBytes: List<Int>,
|
|
||||||
@SerialName("dscp_sent") val dscpSent: Int? = null,
|
|
||||||
@SerialName("dscp_seen_by_server") val dscpSeenByServer: List<Int?> = emptyList(),
|
|
||||||
@SerialName("ecn_sent") val ecnSent: Int? = null,
|
|
||||||
@SerialName("ecn_seen_by_server") val ecnSeenByServer: List<Int?> = emptyList(),
|
|
||||||
@SerialName("ttl_seen_by_server") val ttlSeenByServer: List<Int?> = emptyList(),
|
|
||||||
@SerialName("evidence_truncated") val evidenceTruncated: Boolean = false,
|
|
||||||
)
|
|
||||||
|
|
||||||
/** Traceroute evidence (§6.3): fixed-tuple flow + per-TTL probe replies. */
|
|
||||||
@Serializable
|
|
||||||
data class TracerouteEvidence(val flow: Flow, val hops: List<Hop>)
|
|
||||||
|
|
||||||
@Serializable
|
|
||||||
data class Flow(
|
|
||||||
@SerialName("src_port") val srcPort: Int,
|
|
||||||
@SerialName("dst_port") val dstPort: Int,
|
|
||||||
@SerialName("fixed_tuple") val fixedTuple: Boolean = true,
|
|
||||||
)
|
|
||||||
|
|
||||||
@Serializable
|
|
||||||
data class Hop(val ttl: Int, val probes: List<HopProbe>)
|
|
||||||
|
|
||||||
@Serializable
|
|
||||||
data class HopProbe(
|
|
||||||
@SerialName("reply_from") val replyFrom: String? = null,
|
|
||||||
@SerialName("rtt_ns") val rttNs: Long? = null,
|
|
||||||
val icmp: String? = null,
|
|
||||||
@SerialName("reply_ttl") val replyTtl: Int? = null,
|
|
||||||
)
|
|
||||||
|
|
||||||
/** Resolver under test (§6.4); every dns.* test carries this in params. */
|
|
||||||
@Serializable
|
|
||||||
data class ResolverSpec(
|
|
||||||
val source: ResolverSource,
|
|
||||||
val address: String? = null,
|
|
||||||
val port: Int = 53,
|
|
||||||
val transport: String, // do53-udp | do53-tcp | dot | doh
|
|
||||||
@SerialName("doh_url") val dohUrl: String? = null,
|
|
||||||
)
|
|
||||||
|
|
||||||
@Serializable
|
|
||||||
enum class ResolverSource {
|
|
||||||
@SerialName("system") SYSTEM,
|
|
||||||
@SerialName("manual") MANUAL,
|
|
||||||
@SerialName("server-recursive") SERVER_RECURSIVE,
|
|
||||||
}
|
|
||||||
@@ -1,68 +0,0 @@
|
|||||||
// SPDX-FileCopyrightText: 2026 Echolot contributors
|
|
||||||
// SPDX-License-Identifier: GPL-3.0-or-later
|
|
||||||
|
|
||||||
package app.echo_lot.measurement
|
|
||||||
|
|
||||||
import kotlinx.serialization.SerialName
|
|
||||||
import kotlinx.serialization.Serializable
|
|
||||||
|
|
||||||
/** Interpretation with references back to evidence (measurement-schema.md §7.1). A finding with
|
|
||||||
* no evidence_refs is invalid — every finding must be re-derivable from the evidence alone. */
|
|
||||||
@Serializable
|
|
||||||
data class Finding(
|
|
||||||
val id: String, // UUIDv7
|
|
||||||
val code: String, // stable registry (findings-registry.md), lint-rule style
|
|
||||||
val category: Category,
|
|
||||||
val severity: Severity,
|
|
||||||
val confidence: Confidence,
|
|
||||||
@SerialName("network_ref") val networkRef: String? = null,
|
|
||||||
val title: String,
|
|
||||||
val description: String,
|
|
||||||
@SerialName("evidence_refs") val evidenceRefs: List<EvidenceRef>,
|
|
||||||
val recommendation: String? = null,
|
|
||||||
) {
|
|
||||||
init {
|
|
||||||
require(evidenceRefs.isNotEmpty()) { "a finding must reference at least one piece of evidence" }
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
@Serializable
|
|
||||||
data class EvidenceRef(val test: String, val pointer: String? = null)
|
|
||||||
|
|
||||||
/** Fixed §7.2 categories; each maps to one traffic light. */
|
|
||||||
@Serializable
|
|
||||||
enum class Category {
|
|
||||||
@SerialName("connectivity") CONNECTIVITY,
|
|
||||||
@SerialName("dns") DNS,
|
|
||||||
@SerialName("nat") NAT,
|
|
||||||
@SerialName("mtu") MTU,
|
|
||||||
@SerialName("ipv6") IPV6,
|
|
||||||
@SerialName("security") SECURITY,
|
|
||||||
@SerialName("performance") PERFORMANCE,
|
|
||||||
@SerialName("local") LOCAL,
|
|
||||||
@SerialName("wifi") WIFI,
|
|
||||||
}
|
|
||||||
|
|
||||||
/** Ordered worst→best via [rank]; drives the §7.3 light mapping. */
|
|
||||||
@Serializable
|
|
||||||
enum class Severity(val rank: Int) {
|
|
||||||
@SerialName("critical") CRITICAL(4),
|
|
||||||
@SerialName("high") HIGH(3),
|
|
||||||
@SerialName("medium") MEDIUM(2),
|
|
||||||
@SerialName("low") LOW(1),
|
|
||||||
@SerialName("info") INFO(0);
|
|
||||||
|
|
||||||
/** §7.3: critical|high → red, medium|low → yellow, info → green. */
|
|
||||||
fun toLight(): Verdict = when (this) {
|
|
||||||
CRITICAL, HIGH -> Verdict.RED
|
|
||||||
MEDIUM, LOW -> Verdict.YELLOW
|
|
||||||
INFO -> Verdict.GREEN
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
@Serializable
|
|
||||||
enum class Confidence {
|
|
||||||
@SerialName("high") HIGH,
|
|
||||||
@SerialName("medium") MEDIUM,
|
|
||||||
@SerialName("low") LOW,
|
|
||||||
}
|
|
||||||
@@ -1,113 +0,0 @@
|
|||||||
// SPDX-FileCopyrightText: 2026 Echolot contributors
|
|
||||||
// SPDX-License-Identifier: GPL-3.0-or-later
|
|
||||||
|
|
||||||
package app.echo_lot.measurement
|
|
||||||
|
|
||||||
import kotlinx.serialization.SerialName
|
|
||||||
import kotlinx.serialization.Serializable
|
|
||||||
import kotlinx.serialization.json.JsonObject
|
|
||||||
|
|
||||||
/** One Android Network in play (measurement-schema.md §4). Shizuku-tier fields (route proto,
|
|
||||||
* lifetimes) are absent at app tier — absence means "not observed", never "not present". */
|
|
||||||
@Serializable
|
|
||||||
data class Network(
|
|
||||||
val id: String,
|
|
||||||
val transport: Transport,
|
|
||||||
@SerialName("interface") val iface: String? = null,
|
|
||||||
val link: Link,
|
|
||||||
val wifi: Wifi? = null,
|
|
||||||
val cellular: Cellular? = null,
|
|
||||||
val changes: List<NetworkChange> = emptyList(),
|
|
||||||
)
|
|
||||||
|
|
||||||
@Serializable
|
|
||||||
enum class Transport {
|
|
||||||
@SerialName("wifi") WIFI,
|
|
||||||
@SerialName("cellular") CELLULAR,
|
|
||||||
@SerialName("ethernet") ETHERNET,
|
|
||||||
@SerialName("vpn") VPN,
|
|
||||||
@SerialName("other") OTHER,
|
|
||||||
}
|
|
||||||
|
|
||||||
@Serializable
|
|
||||||
data class Link(
|
|
||||||
val mtu: Int? = null,
|
|
||||||
val addresses: List<Address> = emptyList(),
|
|
||||||
val routes: List<Route> = emptyList(),
|
|
||||||
val dns: DnsConfig? = null,
|
|
||||||
val dhcp: Dhcp? = null,
|
|
||||||
@SerialName("captive_portal") val captivePortal: CaptivePortal? = null,
|
|
||||||
)
|
|
||||||
|
|
||||||
@Serializable
|
|
||||||
data class Address(
|
|
||||||
val addr: String, // ip4 | ip6 (logical type, §8)
|
|
||||||
@SerialName("prefix_len") val prefixLen: Int,
|
|
||||||
val scope: String? = null,
|
|
||||||
val flags: List<String> = emptyList(),
|
|
||||||
@SerialName("valid_lft_s") val validLftS: Long? = null,
|
|
||||||
@SerialName("pref_lft_s") val prefLftS: Long? = null,
|
|
||||||
)
|
|
||||||
|
|
||||||
@Serializable
|
|
||||||
data class Route(
|
|
||||||
val dst: String,
|
|
||||||
val gateway: String? = null,
|
|
||||||
val iface: String? = null,
|
|
||||||
val proto: RouteProto? = null, // shizuku tier; null = not observed
|
|
||||||
@SerialName("expires_s") val expiresS: Long? = null,
|
|
||||||
)
|
|
||||||
|
|
||||||
@Serializable
|
|
||||||
enum class RouteProto {
|
|
||||||
@SerialName("dhcp") DHCP,
|
|
||||||
@SerialName("ra") RA,
|
|
||||||
@SerialName("static") STATIC,
|
|
||||||
@SerialName("unknown") UNKNOWN,
|
|
||||||
}
|
|
||||||
|
|
||||||
@Serializable
|
|
||||||
data class DnsConfig(
|
|
||||||
val servers: List<String> = emptyList(),
|
|
||||||
@SerialName("private_dns_mode") val privateDnsMode: String? = null,
|
|
||||||
@SerialName("private_dns_hostname") val privateDnsHostname: String? = null,
|
|
||||||
@SerialName("search_domains") val searchDomains: List<String> = emptyList(),
|
|
||||||
@SerialName("nat64_prefix") val nat64Prefix: String? = null,
|
|
||||||
)
|
|
||||||
|
|
||||||
@Serializable
|
|
||||||
data class Dhcp(val server: String? = null, @SerialName("lease_s") val leaseS: Long? = null)
|
|
||||||
|
|
||||||
@Serializable
|
|
||||||
data class CaptivePortal(
|
|
||||||
val detected: Boolean = false,
|
|
||||||
@SerialName("api_url") val apiUrl: String? = null,
|
|
||||||
@SerialName("venue_url") val venueUrl: String? = null,
|
|
||||||
)
|
|
||||||
|
|
||||||
@Serializable
|
|
||||||
data class Wifi(
|
|
||||||
val ssid: String? = null, // ssid (logical type)
|
|
||||||
val bssid: String? = null, // bssid (logical type)
|
|
||||||
@SerialName("rssi_dbm") val rssiDbm: Int? = null,
|
|
||||||
@SerialName("link_speed_mbps") val linkSpeedMbps: Int? = null,
|
|
||||||
@SerialName("frequency_mhz") val frequencyMhz: Int? = null,
|
|
||||||
@SerialName("channel_width_mhz") val channelWidthMhz: Int? = null,
|
|
||||||
val standard: String? = null,
|
|
||||||
val security: String? = null,
|
|
||||||
@SerialName("mac_randomization") val macRandomization: Boolean? = null,
|
|
||||||
)
|
|
||||||
|
|
||||||
@Serializable
|
|
||||||
data class Cellular(
|
|
||||||
val rat: String? = null,
|
|
||||||
val operator: String? = null,
|
|
||||||
val band: String? = null,
|
|
||||||
)
|
|
||||||
|
|
||||||
@Serializable
|
|
||||||
data class NetworkChange(
|
|
||||||
@SerialName("at_mono_ns") val atMonoNs: Long,
|
|
||||||
val kind: String, // lost | gained | link_changed
|
|
||||||
val detail: JsonObject? = null,
|
|
||||||
)
|
|
||||||
@@ -1,90 +0,0 @@
|
|||||||
// SPDX-FileCopyrightText: 2026 Echolot contributors
|
|
||||||
// SPDX-License-Identifier: GPL-3.0-or-later
|
|
||||||
|
|
||||||
package app.echo_lot.measurement
|
|
||||||
|
|
||||||
import kotlinx.serialization.SerialName
|
|
||||||
import kotlinx.serialization.Serializable
|
|
||||||
|
|
||||||
@Serializable
|
|
||||||
enum class Verdict {
|
|
||||||
@SerialName("green") GREEN,
|
|
||||||
@SerialName("yellow") YELLOW,
|
|
||||||
@SerialName("red") RED,
|
|
||||||
@SerialName("inconclusive") INCONCLUSIVE,
|
|
||||||
}
|
|
||||||
|
|
||||||
@Serializable
|
|
||||||
data class Summary(
|
|
||||||
val overall: Verdict,
|
|
||||||
val categories: Map<String, CategorySummary>,
|
|
||||||
)
|
|
||||||
|
|
||||||
@Serializable
|
|
||||||
data class CategorySummary(
|
|
||||||
val verdict: Verdict,
|
|
||||||
@SerialName("worst_finding") val worstFinding: String? = null,
|
|
||||||
@SerialName("tests_run") val testsRun: Int,
|
|
||||||
@SerialName("tests_failed") val testsFailed: Int,
|
|
||||||
)
|
|
||||||
|
|
||||||
/**
|
|
||||||
* Deterministic verdict derivation, fixed by measurement-schema.md §7.3:
|
|
||||||
*
|
|
||||||
* - A category's verdict = the light of its worst-severity finding
|
|
||||||
* (critical|high → red, medium|low → yellow, info/none → green).
|
|
||||||
* - A category is `inconclusive` when > 50% of its tests are failed/unsupported.
|
|
||||||
* - Overall = the worst category light; `inconclusive` only when ALL categories are.
|
|
||||||
*
|
|
||||||
* The mapping test-type → category comes from [TestType.category]. Only categories that have
|
|
||||||
* findings or tests appear in the summary.
|
|
||||||
*/
|
|
||||||
object Verdicts {
|
|
||||||
|
|
||||||
private fun isInconclusiveTest(s: TestStatus) =
|
|
||||||
s == TestStatus.FAILED || s == TestStatus.UNSUPPORTED
|
|
||||||
|
|
||||||
fun derive(tests: List<Test>, findings: List<Finding>): Summary {
|
|
||||||
val testsByCat = tests.groupBy { TestType.category(it.type) }
|
|
||||||
val findingsByCat = findings.groupBy { it.category }
|
|
||||||
val categories = (testsByCat.keys + findingsByCat.keys)
|
|
||||||
|
|
||||||
val perCat = LinkedHashMap<String, CategorySummary>()
|
|
||||||
for (cat in Category.entries) {
|
|
||||||
if (cat !in categories) continue
|
|
||||||
val catTests = testsByCat[cat].orEmpty()
|
|
||||||
val catFindings = findingsByCat[cat].orEmpty()
|
|
||||||
|
|
||||||
val failed = catTests.count { isInconclusiveTest(it.status) }
|
|
||||||
val inconclusive = catTests.isNotEmpty() && failed * 2 > catTests.size
|
|
||||||
|
|
||||||
val worst = catFindings.maxByOrNull { it.severity.rank }
|
|
||||||
val verdict = when {
|
|
||||||
inconclusive -> Verdict.INCONCLUSIVE
|
|
||||||
worst == null -> Verdict.GREEN
|
|
||||||
else -> worst.severity.toLight()
|
|
||||||
}
|
|
||||||
perCat[serialName(cat)] = CategorySummary(
|
|
||||||
verdict = verdict,
|
|
||||||
worstFinding = worst?.id,
|
|
||||||
testsRun = catTests.size,
|
|
||||||
testsFailed = failed,
|
|
||||||
)
|
|
||||||
}
|
|
||||||
|
|
||||||
val overall = deriveOverall(perCat.values)
|
|
||||||
return Summary(overall = overall, categories = perCat)
|
|
||||||
}
|
|
||||||
|
|
||||||
/** Overall = worst light; inconclusive only if every category is inconclusive. */
|
|
||||||
private fun deriveOverall(cats: Collection<CategorySummary>): Verdict {
|
|
||||||
if (cats.isEmpty()) return Verdict.INCONCLUSIVE
|
|
||||||
if (cats.all { it.verdict == Verdict.INCONCLUSIVE }) return Verdict.INCONCLUSIVE
|
|
||||||
val rank = mapOf(Verdict.GREEN to 0, Verdict.YELLOW to 1, Verdict.RED to 2)
|
|
||||||
// Non-inconclusive categories decide the overall light.
|
|
||||||
return cats.filter { it.verdict != Verdict.INCONCLUSIVE }
|
|
||||||
.maxByOrNull { rank.getValue(it.verdict) }!!.verdict
|
|
||||||
}
|
|
||||||
|
|
||||||
private fun serialName(cat: Category): String = cat.name.lowercase()
|
|
||||||
}
|
|
||||||
@@ -1,149 +0,0 @@
|
|||||||
// SPDX-FileCopyrightText: 2026 Echolot contributors
|
|
||||||
// SPDX-License-Identifier: GPL-3.0-or-later
|
|
||||||
|
|
||||||
package app.echo_lot.measurement
|
|
||||||
|
|
||||||
import kotlinx.serialization.SerialName
|
|
||||||
import kotlinx.serialization.Serializable
|
|
||||||
import kotlinx.serialization.json.JsonObject
|
|
||||||
|
|
||||||
/** The generic test envelope (measurement-schema.md §6). params must fully reproduce the test;
|
|
||||||
* evidence is append-only raw truth; metrics must be recomputable from evidence. All three are
|
|
||||||
* per-test-type JSON, so they are carried as JsonObject. */
|
|
||||||
@Serializable
|
|
||||||
data class Test(
|
|
||||||
val id: String, // UUIDv7
|
|
||||||
val type: String, // TestType registry (§6.1)
|
|
||||||
@SerialName("network_ref") val networkRef: String? = null,
|
|
||||||
@SerialName("session_ref") val sessionRef: String? = null, // null for local-only tests
|
|
||||||
val tier: Tier,
|
|
||||||
@SerialName("started_mono_ns") val startedMonoNs: Long,
|
|
||||||
@SerialName("ended_mono_ns") val endedMonoNs: Long,
|
|
||||||
val status: TestStatus,
|
|
||||||
val error: TestError? = null,
|
|
||||||
val params: JsonObject? = null,
|
|
||||||
val evidence: JsonObject? = null,
|
|
||||||
val metrics: JsonObject? = null,
|
|
||||||
)
|
|
||||||
|
|
||||||
@Serializable
|
|
||||||
enum class Tier {
|
|
||||||
@SerialName("app") APP,
|
|
||||||
@SerialName("shizuku") SHIZUKU,
|
|
||||||
@SerialName("root") ROOT,
|
|
||||||
}
|
|
||||||
|
|
||||||
@Serializable
|
|
||||||
enum class TestStatus {
|
|
||||||
@SerialName("ok") OK,
|
|
||||||
@SerialName("failed") FAILED,
|
|
||||||
@SerialName("unsupported") UNSUPPORTED,
|
|
||||||
@SerialName("skipped") SKIPPED,
|
|
||||||
@SerialName("partial") PARTIAL,
|
|
||||||
}
|
|
||||||
|
|
||||||
@Serializable
|
|
||||||
data class TestError(val code: String, val detail: String? = null)
|
|
||||||
|
|
||||||
/**
|
|
||||||
* The v1 test-type registry (§6.1). String constants (dotted, family-first) so probe code and the
|
|
||||||
* server's measurement-schema test-type registry stay aligned. [category] maps a type to one of
|
|
||||||
* the fixed §7.2 categories for verdict rollup.
|
|
||||||
*/
|
|
||||||
object TestType {
|
|
||||||
// link
|
|
||||||
const val LINK_SNAPSHOT = "link.snapshot"
|
|
||||||
const val LINK_DHCP_RENEWAL_WATCH = "link.dhcp_renewal_watch"
|
|
||||||
const val LINK_IP_MONITOR = "link.ip_monitor"
|
|
||||||
/** Who advertises IPv6 on this link (+ gateway identity). Registry addition, v1.1. */
|
|
||||||
const val LINK_RA_SOURCE = "link.ra_source"
|
|
||||||
// net — connectivity validation (reproduces Android's NetworkMonitor generate_204 checks)
|
|
||||||
const val NET_CAPTIVE_PORTAL = "net.captive_portal"
|
|
||||||
// icmp
|
|
||||||
const val ICMP_PING4 = "icmp.ping4"
|
|
||||||
const val ICMP_PING6 = "icmp.ping6"
|
|
||||||
// trace
|
|
||||||
const val TRACEROUTE_UDP4 = "traceroute.udp4"
|
|
||||||
const val TRACEROUTE_UDP6 = "traceroute.udp6"
|
|
||||||
const val TRACEROUTE_ICMP4 = "traceroute.icmp4"
|
|
||||||
const val TRACEROUTE_ICMP6 = "traceroute.icmp6"
|
|
||||||
// train
|
|
||||||
const val TRAIN_UDP_UPDOWN = "train.udp_updown"
|
|
||||||
// mtu
|
|
||||||
const val MTU_PMTUD_UP = "mtu.pmtud_up"
|
|
||||||
const val MTU_PMTUD_DOWN = "mtu.pmtud_down"
|
|
||||||
const val MTU_BLACKHOLE = "mtu.blackhole"
|
|
||||||
const val MTU_MSS_OBSERVED = "mtu.mss_observed"
|
|
||||||
const val MTU_FRAG_DELIVERY = "mtu.frag_delivery"
|
|
||||||
// nat
|
|
||||||
const val NAT_STUN_5780 = "nat.stun_5780"
|
|
||||||
const val NAT_MAPPING_LIFETIME_UDP = "nat.mapping_lifetime_udp"
|
|
||||||
const val NAT_MAPPING_LIFETIME_TCP = "nat.mapping_lifetime_tcp"
|
|
||||||
const val NAT_HAIRPIN = "nat.hairpin"
|
|
||||||
const val NAT_CONNECT_BACK = "nat.connect_back"
|
|
||||||
const val NAT_CGNAT_DETECT = "nat.cgnat_detect"
|
|
||||||
// dns
|
|
||||||
const val DNS_RESOLVER_INVENTORY = "dns.resolver_inventory"
|
|
||||||
const val DNS_CANARY = "dns.canary"
|
|
||||||
const val DNS_INTERCEPTION = "dns.interception"
|
|
||||||
const val DNS_TTL_INTEGRITY = "dns.ttl_integrity"
|
|
||||||
const val DNS_ANSWER_INTEGRITY = "dns.answer_integrity"
|
|
||||||
const val DNS_DNSSEC = "dns.dnssec"
|
|
||||||
const val DNS_NXDOMAIN_WILDCARD = "dns.nxdomain_wildcard"
|
|
||||||
const val DNS_REBIND_FILTER = "dns.rebind_filter"
|
|
||||||
const val DNS_AAAA_FILTER = "dns.aaaa_filter"
|
|
||||||
const val DNS_DNS64 = "dns.dns64"
|
|
||||||
const val DNS_COMPARE = "dns.compare"
|
|
||||||
// sec
|
|
||||||
const val SEC_TLS_REFERENCE = "sec.tls_reference"
|
|
||||||
const val SEC_CLIENTHELLO_ECHO = "sec.clienthello_echo"
|
|
||||||
const val SEC_HTTP_ECHO = "sec.http_echo"
|
|
||||||
const val SEC_SNI_FILTER = "sec.sni_filter"
|
|
||||||
const val SEC_DSCP_ECN_SURVIVAL = "sec.dscp_ecn_survival"
|
|
||||||
const val SEC_ARP_WATCH = "sec.arp_watch"
|
|
||||||
// port
|
|
||||||
const val PORT_REACH_SWEEP = "port.reach_sweep"
|
|
||||||
const val PORT_UDP_USABILITY = "port.udp_usability"
|
|
||||||
// perf
|
|
||||||
const val PERF_THROUGHPUT_TCP = "perf.throughput_tcp"
|
|
||||||
const val PERF_THROUGHPUT_UDP = "perf.throughput_udp"
|
|
||||||
const val PERF_BUFFERBLOAT = "perf.bufferbloat"
|
|
||||||
const val PERF_RRC_LATENCY = "perf.rrc_latency"
|
|
||||||
// v6
|
|
||||||
const val V6_DUALSTACK_COMPARE = "v6.dualstack_compare"
|
|
||||||
const val V6_HAPPY_EYEBALLS = "v6.happy_eyeballs"
|
|
||||||
const val V6_BROKENNESS = "v6.brokenness"
|
|
||||||
const val V6_NAT64_CLAT = "v6.nat64_clat"
|
|
||||||
// wifi
|
|
||||||
const val WIFI_ENVIRONMENT_SCAN = "wifi.environment_scan"
|
|
||||||
const val WIFI_ROAM_LOG = "wifi.roam_log"
|
|
||||||
const val WIFI_SIGNAL_LOG = "wifi.signal_log"
|
|
||||||
// local
|
|
||||||
const val LOCAL_MDNS_INVENTORY = "local.mdns_inventory"
|
|
||||||
const val LOCAL_SSDP_INVENTORY = "local.ssdp_inventory"
|
|
||||||
const val LOCAL_LLMNR_INVENTORY = "local.llmnr_inventory"
|
|
||||||
const val LOCAL_GATEWAY_SERVICES = "local.gateway_services"
|
|
||||||
const val LOCAL_NTP = "local.ntp"
|
|
||||||
// peer
|
|
||||||
const val PEER_REACHABILITY = "peer.reachability"
|
|
||||||
const val PEER_ISOLATION = "peer.isolation"
|
|
||||||
const val PEER_MULTICAST = "peer.multicast"
|
|
||||||
const val PEER_LAN_TRAIN = "peer.lan_train"
|
|
||||||
const val PEER_LEASE_DIFF = "peer.lease_diff"
|
|
||||||
// time
|
|
||||||
const val TIME_SERVER_OFFSET = "time.server_offset"
|
|
||||||
|
|
||||||
/** Maps a dotted test type to its §7.2 category for verdict rollup. */
|
|
||||||
fun category(type: String): Category = when (type.substringBefore('.')) {
|
|
||||||
"link", "icmp", "trace", "traceroute", "train", "port", "time", "net" -> Category.CONNECTIVITY
|
|
||||||
"dns" -> Category.DNS
|
|
||||||
"nat" -> Category.NAT
|
|
||||||
"mtu" -> Category.MTU
|
|
||||||
"v6" -> Category.IPV6
|
|
||||||
"sec" -> Category.SECURITY
|
|
||||||
"perf" -> Category.PERFORMANCE
|
|
||||||
"local", "peer" -> Category.LOCAL
|
|
||||||
"wifi" -> Category.WIFI
|
|
||||||
else -> Category.CONNECTIVITY
|
|
||||||
}
|
|
||||||
}
|
|
||||||
@@ -1,71 +0,0 @@
|
|||||||
// SPDX-FileCopyrightText: 2026 Echolot contributors
|
|
||||||
// SPDX-License-Identifier: GPL-3.0-or-later
|
|
||||||
|
|
||||||
package app.echo_lot.measurement
|
|
||||||
|
|
||||||
import kotlinx.serialization.json.Json
|
|
||||||
import kotlin.test.Test as JTest
|
|
||||||
import kotlin.test.assertEquals
|
|
||||||
import kotlin.test.assertTrue
|
|
||||||
|
|
||||||
class SerializationTest {
|
|
||||||
|
|
||||||
private val json = Json { ignoreUnknownKeys = true; encodeDefaults = true }
|
|
||||||
|
|
||||||
@JTest
|
|
||||||
fun documentRoundTrips() {
|
|
||||||
val doc = MeasurementDocument(
|
|
||||||
run = Run(
|
|
||||||
id = "0198c5f2-0000-7000-8000-000000000000",
|
|
||||||
trigger = Trigger.MANUAL,
|
|
||||||
startedAt = "2026-07-31T14:03:21.114Z",
|
|
||||||
clock = Clock(monoOriginWall = "2026-07-31T14:03:21.114Z"),
|
|
||||||
app = AppInfo(version = "0.1.0", build = 1),
|
|
||||||
device = DeviceInfo("OnePlus", "CPH2747", 36, "16"),
|
|
||||||
tiers = Tiers(app = true, shizuku = true),
|
|
||||||
),
|
|
||||||
networks = listOf(
|
|
||||||
Network(
|
|
||||||
id = "net-1", transport = Transport.WIFI, iface = "wlan0",
|
|
||||||
link = Link(mtu = 1500, addresses = listOf(Address("192.0.2.23", 24, "global"))),
|
|
||||||
wifi = Wifi(ssid = "example", rssiDbm = -54),
|
|
||||||
),
|
|
||||||
),
|
|
||||||
tests = listOf(
|
|
||||||
Test(
|
|
||||||
id = "t-1", type = TestType.ICMP_PING4, networkRef = "net-1", tier = Tier.APP,
|
|
||||||
startedMonoNs = 0, endedMonoNs = 38_000_000, status = TestStatus.OK,
|
|
||||||
evidence = TrainEvidence(
|
|
||||||
epochMonoNs = 0, seq = listOf(0, 1), tTxNs = listOf(0L, 20_000_000L),
|
|
||||||
tRxNs = listOf(16_500_000L, null), sizeBytes = listOf(64, 64),
|
|
||||||
).toEvidence(),
|
|
||||||
),
|
|
||||||
),
|
|
||||||
)
|
|
||||||
|
|
||||||
val encoded = json.encodeToString(MeasurementDocument.serializer(), doc)
|
|
||||||
val decoded = json.decodeFromString(MeasurementDocument.serializer(), encoded)
|
|
||||||
assertEquals(doc.run.id, decoded.run.id)
|
|
||||||
assertEquals(Transport.WIFI, decoded.networks[0].transport)
|
|
||||||
assertEquals(TestType.ICMP_PING4, decoded.tests[0].type)
|
|
||||||
// snake_case field names on the wire
|
|
||||||
assertTrue(encoded.contains("\"schema_version\""))
|
|
||||||
assertTrue(encoded.contains("\"mono_origin_wall\""))
|
|
||||||
assertTrue(encoded.contains("\"t_tx_ns\""))
|
|
||||||
// null preserved at train index 1
|
|
||||||
assertTrue(encoded.contains("[16500000,null]"))
|
|
||||||
}
|
|
||||||
|
|
||||||
@JTest
|
|
||||||
fun findingRequiresEvidence() {
|
|
||||||
try {
|
|
||||||
Finding(
|
|
||||||
id = "f-1", code = "x", category = Category.DNS, severity = Severity.INFO,
|
|
||||||
confidence = Confidence.LOW, title = "t", description = "d", evidenceRefs = emptyList(),
|
|
||||||
)
|
|
||||||
throw AssertionError("expected IllegalArgumentException for empty evidence_refs")
|
|
||||||
} catch (e: IllegalArgumentException) {
|
|
||||||
// expected — a finding with no evidence is invalid (§7.1)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
@@ -1,109 +0,0 @@
|
|||||||
// SPDX-FileCopyrightText: 2026 Echolot contributors
|
|
||||||
// SPDX-License-Identifier: GPL-3.0-or-later
|
|
||||||
|
|
||||||
package app.echo_lot.measurement
|
|
||||||
|
|
||||||
import kotlin.test.Test as JTest
|
|
||||||
import kotlin.test.assertEquals
|
|
||||||
|
|
||||||
class VerdictsTest {
|
|
||||||
|
|
||||||
private fun test(type: String, status: TestStatus, id: String = type): Test =
|
|
||||||
Test(id = id, type = type, tier = Tier.APP, startedMonoNs = 0, endedMonoNs = 1, status = status)
|
|
||||||
|
|
||||||
private fun finding(cat: Category, sev: Severity, id: String = "f-$cat-$sev"): Finding =
|
|
||||||
Finding(
|
|
||||||
id = id, code = "x.$cat", category = cat, severity = sev, confidence = Confidence.HIGH,
|
|
||||||
title = "t", description = "d", evidenceRefs = listOf(EvidenceRef("some-test")),
|
|
||||||
)
|
|
||||||
|
|
||||||
@JTest
|
|
||||||
fun categoryLightFromWorstSeverity() {
|
|
||||||
val tests = listOf(test(TestType.DNS_CANARY, TestStatus.OK))
|
|
||||||
val findings = listOf(
|
|
||||||
finding(Category.DNS, Severity.LOW),
|
|
||||||
finding(Category.DNS, Severity.HIGH), // worst → red
|
|
||||||
finding(Category.DNS, Severity.INFO),
|
|
||||||
)
|
|
||||||
val s = Verdicts.derive(tests, findings)
|
|
||||||
assertEquals(Verdict.RED, s.categories["dns"]!!.verdict)
|
|
||||||
assertEquals("f-DNS-HIGH", s.categories["dns"]!!.worstFinding)
|
|
||||||
assertEquals(Verdict.RED, s.overall)
|
|
||||||
}
|
|
||||||
|
|
||||||
@JTest
|
|
||||||
fun noFindingsIsGreen() {
|
|
||||||
val s = Verdicts.derive(listOf(test(TestType.MTU_BLACKHOLE, TestStatus.OK)), emptyList())
|
|
||||||
assertEquals(Verdict.GREEN, s.categories["mtu"]!!.verdict)
|
|
||||||
assertEquals(Verdict.GREEN, s.overall)
|
|
||||||
}
|
|
||||||
|
|
||||||
@JTest
|
|
||||||
fun mediumAndLowAreYellow() {
|
|
||||||
val s = Verdicts.derive(
|
|
||||||
listOf(test(TestType.SEC_HTTP_ECHO, TestStatus.OK)),
|
|
||||||
listOf(finding(Category.SECURITY, Severity.MEDIUM)),
|
|
||||||
)
|
|
||||||
assertEquals(Verdict.YELLOW, s.categories["security"]!!.verdict)
|
|
||||||
}
|
|
||||||
|
|
||||||
@JTest
|
|
||||||
fun majorityFailedIsInconclusive() {
|
|
||||||
// 2 of 3 dns tests failed → > 50% → inconclusive, even with a finding present.
|
|
||||||
val tests = listOf(
|
|
||||||
test(TestType.DNS_CANARY, TestStatus.FAILED, "a"),
|
|
||||||
test(TestType.DNS_TTL_INTEGRITY, TestStatus.UNSUPPORTED, "b"),
|
|
||||||
test(TestType.DNS_COMPARE, TestStatus.OK, "c"),
|
|
||||||
)
|
|
||||||
val s = Verdicts.derive(tests, listOf(finding(Category.DNS, Severity.HIGH)))
|
|
||||||
assertEquals(Verdict.INCONCLUSIVE, s.categories["dns"]!!.verdict)
|
|
||||||
assertEquals(2, s.categories["dns"]!!.testsFailed)
|
|
||||||
assertEquals(3, s.categories["dns"]!!.testsRun)
|
|
||||||
}
|
|
||||||
|
|
||||||
@JTest
|
|
||||||
fun exactlyHalfFailedIsNotInconclusive() {
|
|
||||||
// 1 of 2 failed → not > 50% → the finding decides.
|
|
||||||
val tests = listOf(
|
|
||||||
test(TestType.NAT_HAIRPIN, TestStatus.FAILED, "a"),
|
|
||||||
test(TestType.NAT_CONNECT_BACK, TestStatus.OK, "b"),
|
|
||||||
)
|
|
||||||
val s = Verdicts.derive(tests, listOf(finding(Category.NAT, Severity.CRITICAL)))
|
|
||||||
assertEquals(Verdict.RED, s.categories["nat"]!!.verdict)
|
|
||||||
}
|
|
||||||
|
|
||||||
@JTest
|
|
||||||
fun overallIsWorstCategory() {
|
|
||||||
val tests = listOf(
|
|
||||||
test(TestType.DNS_CANARY, TestStatus.OK, "d"),
|
|
||||||
test(TestType.MTU_BLACKHOLE, TestStatus.OK, "m"),
|
|
||||||
)
|
|
||||||
val findings = listOf(
|
|
||||||
finding(Category.DNS, Severity.MEDIUM), // yellow
|
|
||||||
finding(Category.MTU, Severity.CRITICAL), // red
|
|
||||||
)
|
|
||||||
val s = Verdicts.derive(tests, findings)
|
|
||||||
assertEquals(Verdict.RED, s.overall)
|
|
||||||
}
|
|
||||||
|
|
||||||
@JTest
|
|
||||||
fun overallInconclusiveOnlyWhenAllAre() {
|
|
||||||
val tests = listOf(
|
|
||||||
test(TestType.DNS_CANARY, TestStatus.FAILED, "d"), // dns inconclusive
|
|
||||||
test(TestType.MTU_BLACKHOLE, TestStatus.OK, "m"), // mtu green
|
|
||||||
)
|
|
||||||
val s = Verdicts.derive(tests, emptyList())
|
|
||||||
assertEquals(Verdict.INCONCLUSIVE, s.categories["dns"]!!.verdict)
|
|
||||||
assertEquals(Verdict.GREEN, s.categories["mtu"]!!.verdict)
|
|
||||||
assertEquals(Verdict.GREEN, s.overall) // not all inconclusive → mtu decides
|
|
||||||
}
|
|
||||||
|
|
||||||
@JTest
|
|
||||||
fun categoryMappingCoversFamilies() {
|
|
||||||
assertEquals(Category.CONNECTIVITY, TestType.category(TestType.TRACEROUTE_UDP4))
|
|
||||||
assertEquals(Category.IPV6, TestType.category(TestType.V6_BROKENNESS))
|
|
||||||
assertEquals(Category.LOCAL, TestType.category(TestType.PEER_MULTICAST))
|
|
||||||
assertEquals(Category.PERFORMANCE, TestType.category(TestType.PERF_BUFFERBLOAT))
|
|
||||||
assertEquals(Category.CONNECTIVITY, TestType.category(TestType.TIME_SERVER_OFFSET))
|
|
||||||
}
|
|
||||||
}
|
|
||||||
@@ -69,6 +69,8 @@ object TestType {
|
|||||||
const val TRACEROUTE_ICMP6 = "traceroute.icmp6"
|
const val TRACEROUTE_ICMP6 = "traceroute.icmp6"
|
||||||
// train
|
// train
|
||||||
const val TRAIN_UDP_UPDOWN = "train.udp_updown"
|
const val TRAIN_UDP_UPDOWN = "train.udp_updown"
|
||||||
|
/** Server-to-client train under a §3.4 grant: the direction a round trip cannot separate. */
|
||||||
|
const val TRAIN_UDP_DOWNSTREAM = "train.udp_downstream"
|
||||||
// mtu
|
// mtu
|
||||||
const val MTU_PMTUD_UP = "mtu.pmtud_up"
|
const val MTU_PMTUD_UP = "mtu.pmtud_up"
|
||||||
const val MTU_PMTUD_DOWN = "mtu.pmtud_down"
|
const val MTU_PMTUD_DOWN = "mtu.pmtud_down"
|
||||||
|
|||||||
@@ -0,0 +1,23 @@
|
|||||||
|
// SPDX-FileCopyrightText: 2026 Echolot contributors
|
||||||
|
// SPDX-License-Identifier: GPL-3.0-or-later
|
||||||
|
|
||||||
|
plugins {
|
||||||
|
alias(libs.plugins.kotlin.jvm)
|
||||||
|
alias(libs.plugins.kotlin.serialization)
|
||||||
|
}
|
||||||
|
|
||||||
|
// The anonymizer (measurement-schema.md §8). Pure Kotlin/JVM and deliberately
|
||||||
|
// dependency-free beyond JSON: it must be trivially auditable, because a bug
|
||||||
|
// here leaks a user's network onto someone else's server.
|
||||||
|
dependencies {
|
||||||
|
implementation(libs.kotlinx.serialization.json)
|
||||||
|
testImplementation(kotlin("test"))
|
||||||
|
}
|
||||||
|
|
||||||
|
kotlin {
|
||||||
|
jvmToolchain(21)
|
||||||
|
compilerOptions { jvmTarget.set(org.jetbrains.kotlin.gradle.dsl.JvmTarget.JVM_17) }
|
||||||
|
}
|
||||||
|
java { sourceCompatibility = JavaVersion.VERSION_17; targetCompatibility = JavaVersion.VERSION_17 }
|
||||||
|
|
||||||
|
tasks.test { useJUnitPlatform() }
|
||||||
@@ -0,0 +1,237 @@
|
|||||||
|
// SPDX-FileCopyrightText: 2026 Echolot contributors
|
||||||
|
// SPDX-License-Identifier: GPL-3.0-or-later
|
||||||
|
|
||||||
|
// Package privacy implements the anonymization contract of measurement-schema.md §8.
|
||||||
|
//
|
||||||
|
// The threat model is specific. An engineer running their own server wants the full document —
|
||||||
|
// SSIDs and MACs are what make a run useful a week later. Someone measuring against a stranger's
|
||||||
|
// server wants the numbers to survive and the identifiers not to. So this is a *transform*, not a
|
||||||
|
// filter: the output is still a valid measurement document with the same tests, metrics and
|
||||||
|
// findings; only the identifying scalars change, and consistently, so "same SSID as last run" is
|
||||||
|
// still answerable from pseudonyms alone.
|
||||||
|
//
|
||||||
|
// Two properties are load-bearing and are what the tests pin:
|
||||||
|
// - Consistency within a document: one input value always maps to one pseudonym, so
|
||||||
|
// correlations inside a run survive.
|
||||||
|
// - No consistency *across* documents unless the user asks for it: the salt is per-run by
|
||||||
|
// default, so pseudonyms cannot be used to track a device between uploads. A stable salt is
|
||||||
|
// opt-in (`Salt.stable`) for people diffing their own history on their own server.
|
||||||
|
package app.echo_lot.privacy
|
||||||
|
|
||||||
|
import kotlinx.serialization.json.*
|
||||||
|
import java.security.MessageDigest
|
||||||
|
import java.util.Locale
|
||||||
|
|
||||||
|
/** How much to strip. Ordered: FULL < BALANCED < STRICT. Wire values match the server's. */
|
||||||
|
enum class PrivacyLevel(val wire: String) {
|
||||||
|
/** Nothing removed. The right choice for your own server. */
|
||||||
|
FULL("full"),
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Identifiers pseudonymized, neighbour inventory dropped. Topology and timing survive:
|
||||||
|
* you can still see that the gateway is a MikroTik at a /24 boundary with 3 % loss, but not
|
||||||
|
* which MikroTik, on which SSID, next to whose Chromecast.
|
||||||
|
*/
|
||||||
|
BALANCED("balanced"),
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Numbers only: tests keep their metrics and status, evidence is dropped, findings keep their
|
||||||
|
* codes and severities but lose descriptions (which quote real names). What is left cannot
|
||||||
|
* identify a network, and is still enough for aggregate "how common is this fault" work.
|
||||||
|
*/
|
||||||
|
STRICT("strict");
|
||||||
|
|
||||||
|
companion object {
|
||||||
|
fun fromWire(s: String?): PrivacyLevel =
|
||||||
|
entries.firstOrNull { it.wire == s?.lowercase(Locale.ROOT) } ?: FULL
|
||||||
|
|
||||||
|
/** The stricter of two levels — used to honour a server's minimum. */
|
||||||
|
fun max(a: PrivacyLevel, b: PrivacyLevel): PrivacyLevel = if (a.ordinal >= b.ordinal) a else b
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* The pseudonymization salt. Per-run by default: a fresh random salt means the same SSID uploaded
|
||||||
|
* twice yields two different pseudonyms, so an upload endpoint cannot link runs to a device.
|
||||||
|
* A stable salt trades that away for cross-run diffing and is only appropriate on a server you
|
||||||
|
* own — the app makes that an explicit choice, not a default.
|
||||||
|
*/
|
||||||
|
class Salt private constructor(internal val bytes: ByteArray, val stable: Boolean) {
|
||||||
|
companion object {
|
||||||
|
fun perRun(random: ByteArray): Salt = Salt(random.copyOf(), stable = false)
|
||||||
|
fun stable(secret: ByteArray): Salt = Salt(secret.copyOf(), stable = true)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Transforms a measurement document to [level].
|
||||||
|
*
|
||||||
|
* Field classification is by JSON key name, because the schema names things consistently
|
||||||
|
* (`ssid`, `bssid`, `mac`, `ip4`, `ip6`, `fqdn`, …) and a name-driven pass is auditable by
|
||||||
|
* reading one table. Anything unrecognized is treated as identifying when it is a string inside
|
||||||
|
* a known-sensitive container, and left alone otherwise — see [Classification].
|
||||||
|
*/
|
||||||
|
class Anonymizer(private val level: PrivacyLevel, private val salt: Salt) {
|
||||||
|
|
||||||
|
private val cache = HashMap<String, String>()
|
||||||
|
|
||||||
|
fun anonymize(doc: JsonObject): JsonObject {
|
||||||
|
if (level == PrivacyLevel.FULL) return stamp(doc)
|
||||||
|
val walked = walkObject(doc, path = emptyList())
|
||||||
|
val out = if (level == PrivacyLevel.STRICT) strip(walked) else walked
|
||||||
|
return stamp(out)
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Records what was done, so a reader of the archived/uploaded document is never guessing. */
|
||||||
|
private fun stamp(doc: JsonObject): JsonObject {
|
||||||
|
val run = doc["run"]?.jsonObject ?: return doc
|
||||||
|
val privacy = buildJsonObject {
|
||||||
|
put("anonymization", level.wire)
|
||||||
|
put("salt", if (salt.stable) "stable" else "per_run")
|
||||||
|
}
|
||||||
|
return JsonObject(doc + ("run" to JsonObject(run + ("privacy" to privacy))))
|
||||||
|
}
|
||||||
|
|
||||||
|
// ---- the tree walk -------------------------------------------------------------------
|
||||||
|
|
||||||
|
private fun walkObject(obj: JsonObject, path: List<String>): JsonObject = buildJsonObject {
|
||||||
|
for ((k, v) in obj) {
|
||||||
|
val childPath = path + k
|
||||||
|
when {
|
||||||
|
Classification.dropAtBalanced(childPath) -> Unit // omit entirely
|
||||||
|
else -> put(k, walk(k, v, childPath))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
private fun walk(key: String, v: JsonElement, path: List<String>): JsonElement = when (v) {
|
||||||
|
is JsonObject -> walkObject(v, path)
|
||||||
|
is JsonArray -> JsonArray(v.map { walk(key, it, path) })
|
||||||
|
is JsonPrimitive ->
|
||||||
|
if (v.isString) transform(Classification.typeOf(key, path), v.content).let(::JsonPrimitive)
|
||||||
|
else v
|
||||||
|
}
|
||||||
|
|
||||||
|
private fun transform(type: LogicalType?, value: String): String = when (type) {
|
||||||
|
null -> value
|
||||||
|
LogicalType.SSID -> pseudo("ssid", value) { "net-" + it.take(6) }
|
||||||
|
LogicalType.MAC, LogicalType.BSSID -> macPreservingOui(value)
|
||||||
|
LogicalType.IP4 -> ip4(value)
|
||||||
|
LogicalType.IP6 -> ip6(value)
|
||||||
|
LogicalType.FQDN -> fqdn(value)
|
||||||
|
LogicalType.OPAQUE_ID -> "redacted"
|
||||||
|
LogicalType.FREETEXT -> "[removed: may contain identifying text]"
|
||||||
|
}
|
||||||
|
|
||||||
|
// ---- per-type transforms -------------------------------------------------------------
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Keeps the OUI (which vendor) and pseudonymizes the NIC part (which unit). Vendor is the
|
||||||
|
* diagnostically valuable half — "the RA comes from a MikroTik" survives, "…from THAT
|
||||||
|
* MikroTik" does not.
|
||||||
|
*/
|
||||||
|
private fun macPreservingOui(value: String): String {
|
||||||
|
val sep = if (value.contains('-')) '-' else ':'
|
||||||
|
val parts = value.split(sep)
|
||||||
|
if (parts.size != 6 || parts.any { it.length != 2 }) return pseudo("mac", value) { "mac-" + it.take(8) }
|
||||||
|
val nic = pseudo("mac", value) { it }
|
||||||
|
return (parts.take(3) + listOf(nic.substring(0, 2), nic.substring(2, 4), nic.substring(4, 6)))
|
||||||
|
.joinToString(sep.toString())
|
||||||
|
.lowercase(Locale.ROOT)
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Prefix-preserving within the same class, with reserved ranges kept verbatim: RFC1918 and
|
||||||
|
* CGNAT addresses say something about the topology and nothing about the person, and a run
|
||||||
|
* where 192.168.1.1 became a random public address would be actively misleading to read.
|
||||||
|
* Public addresses keep only their /16 so the network is still locatable at ISP granularity.
|
||||||
|
*/
|
||||||
|
private fun ip4(value: String): String {
|
||||||
|
val o = value.split(".")
|
||||||
|
if (o.size != 4 || o.any { it.toIntOrNull() == null }) return value
|
||||||
|
val n = o.map { it.toInt() }
|
||||||
|
val reserved = n[0] == 10 ||
|
||||||
|
(n[0] == 172 && n[1] in 16..31) ||
|
||||||
|
(n[0] == 192 && n[1] == 168) ||
|
||||||
|
(n[0] == 169 && n[1] == 254) ||
|
||||||
|
(n[0] == 100 && n[1] in 64..127) ||
|
||||||
|
n[0] == 127 || n[0] == 0 || n[0] >= 224
|
||||||
|
if (reserved) return value
|
||||||
|
val h = pseudo("ip4", value) { it }
|
||||||
|
return "${n[0]}.${n[1]}.${h.substring(0, 2).toInt(16)}.${h.substring(2, 4).toInt(16)}"
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* IPv6 keeps the scope and the first 32 bits (so 2001:db8:… still reads as global unicast in
|
||||||
|
* the same allocation) and pseudonymizes the rest — the interface identifier is the part that
|
||||||
|
* is a device fingerprint, especially with EUI-64.
|
||||||
|
*/
|
||||||
|
private fun ip6(value: String): String {
|
||||||
|
val v = value.lowercase(Locale.ROOT)
|
||||||
|
if (v == "::1" || v == "::" || v.startsWith("fe80:") || v.startsWith("ff")) return v
|
||||||
|
val groups = v.substringBefore('%').split(":")
|
||||||
|
if (groups.size < 3) return v
|
||||||
|
val h = pseudo("ip6", value) { it }
|
||||||
|
return "${groups[0]}:${groups[1]}:${h.substring(0, 4)}:${h.substring(4, 8)}::${h.substring(8, 12)}"
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Per-label pseudonyms with the public suffix kept, so "it resolved somewhere under .local"
|
||||||
|
* or "…under example.com" survives without naming the host. The suffix list is deliberately
|
||||||
|
* short: guessing wrong keeps *more* pseudonymized, never less.
|
||||||
|
*/
|
||||||
|
private fun fqdn(value: String): String {
|
||||||
|
if (value.isEmpty()) return value
|
||||||
|
val trailing = value.endsWith(".")
|
||||||
|
val labels = value.trimEnd('.').split(".")
|
||||||
|
if (labels.size == 1) return pseudo("fqdn", value) { "host-" + it.take(6) }
|
||||||
|
val keep = if (labels.last() in publicSuffixes) 1 else 0
|
||||||
|
val head = labels.dropLast(keep).map { l -> pseudo("label", l) { "l-" + it.take(6) } }
|
||||||
|
return (head + labels.takeLast(keep)).joinToString(".") + if (trailing) "." else ""
|
||||||
|
}
|
||||||
|
|
||||||
|
// ---- STRICT ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
/**
|
||||||
|
* STRICT keeps the shape of the document and the numbers, and nothing that quotes the
|
||||||
|
* network back. Evidence goes (trains carry addresses and hostnames), finding prose goes
|
||||||
|
* (it interpolates real names), networks go entirely.
|
||||||
|
*/
|
||||||
|
private fun strip(doc: JsonObject): JsonObject = buildJsonObject {
|
||||||
|
for ((k, v) in doc) {
|
||||||
|
when (k) {
|
||||||
|
"networks", "server_sessions" -> Unit
|
||||||
|
"tests" -> put(k, JsonArray((v as? JsonArray ?: JsonArray(emptyList())).map { t ->
|
||||||
|
val o = t.jsonObject
|
||||||
|
JsonObject(o.filterKeys { it != "evidence" && it != "params" })
|
||||||
|
}))
|
||||||
|
"findings" -> put(k, JsonArray((v as? JsonArray ?: JsonArray(emptyList())).map { f ->
|
||||||
|
val o = f.jsonObject
|
||||||
|
JsonObject(o.filterKeys { it != "description" && it != "title" && it != "evidence_refs" })
|
||||||
|
}))
|
||||||
|
"run" -> put(k, JsonObject(v.jsonObject.filterKeys { it != "notes" }))
|
||||||
|
else -> put(k, v)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// ---- pseudonym machinery ---------------------------------------------------------------
|
||||||
|
|
||||||
|
/** Deterministic per (domain, value, salt); memoized so one value maps to one pseudonym. */
|
||||||
|
private fun pseudo(domain: String, value: String, shape: (String) -> String): String =
|
||||||
|
cache.getOrPut("$domain | ||||||