Compare commits
8
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
3a4cb1c327 | ||
|
|
3cdbccee18 | ||
|
|
80d2092f1b | ||
|
|
89a5ff9139 | ||
|
|
ce6d0c2f64 | ||
|
|
57a5ef8796 | ||
|
|
c19f382640 | ||
|
|
d04babff51 |
@@ -941,3 +941,64 @@ Also: a `BackHandler` now returns from Settings/History to the run screen. The s
|
|||||||
state variable with nothing connecting it to the back stack, so the system Back gesture left the
|
state variable with nothing connecting it to the back stack, so the system Back gesture left the
|
||||||
app entirely. Enabled only when there is somewhere to go back to, so Back still exits from the run
|
app entirely. Enabled only when there is somewhere to go back to, so Back still exits from the run
|
||||||
screen.
|
screen.
|
||||||
|
|
||||||
|
### Upstream throughput (server-v0.6.3, 2026-08-01)
|
||||||
|
The mirror of the downstream case: the client generates the traffic and the server counts it. No
|
||||||
|
grant is involved — the client is sending its own packets, so there is nothing to amplify — but it
|
||||||
|
does need the server's tally, because **only the far end knows how much arrived**. Without that
|
||||||
|
number a sender measures how fast it can *transmit*, which is usually just the speed of the local
|
||||||
|
NIC and is a different question from the one being asked.
|
||||||
|
|
||||||
|
`TYPE_THROUGHPUT_UP` (0x0F) is counted and deliberately **never answered**: a reply would double
|
||||||
|
the traffic and drag the return path into a measurement that is specifically about the outbound
|
||||||
|
one.
|
||||||
|
|
||||||
|
The tally is a counter, not a list, and short-circuits **before** the observation log. A
|
||||||
|
five-second run at 20 Mbps is around ten thousand packets; one struct each would turn a
|
||||||
|
measurement into an allocation storm on a shared server, and nothing needs the per-packet detail
|
||||||
|
since the client holds the send-side record. The gap between the two counts is the loss.
|
||||||
|
|
||||||
|
`direction=up` on the throughput action sends nothing — it zeroes the counter, so a second run in
|
||||||
|
one session measures itself rather than inheriting the first one's packets. The live test asserts
|
||||||
|
`received <= sent`, which is what catches a counter that was never reset.
|
||||||
|
|
||||||
|
Live against fmr: **3125 sent, 3125 counted, 0 % loss, 10.0 Mbit/s** at a 10 Mbit/s request, with
|
||||||
|
`measures_network: false` — correct, since what arrived matched what was offered, so the path was
|
||||||
|
never the constraint.
|
||||||
|
|
||||||
|
### Raw shell dumps leaked the whole LAN (2026-08-01)
|
||||||
|
Found by running the Shizuku shell tier for the first time. The tier works — `tiers.shizuku: true`,
|
||||||
|
`exec_path: UserService` (so the UserService binds on the OnePlus, as recorded), `runs_as
|
||||||
|
shell(2000)`, 7/7 commands — and the run promptly uploaded **every MAC address on the local
|
||||||
|
network** to fmr at the `balanced` level: router, phones, whatever else was on the wifi. Fourteen
|
||||||
|
of them.
|
||||||
|
|
||||||
|
The probes embed raw command output verbatim (`ip neigh`, `ip route`, `id`), which is genuinely
|
||||||
|
good evidence and also a complete household device inventory. The anonymizer could not see it:
|
||||||
|
classification is by field name and by whole-value shape, and `ip_neigh` is one long string that is
|
||||||
|
itself neither a MAC nor an address. measurement-schema.md §9 item 2 had flagged raw dumps as "hard
|
||||||
|
to anonymize" and proposed dropping them from exports; nothing enforced either.
|
||||||
|
|
||||||
|
**Scrubbing beats dropping.** Identifiers inside any unclassified string are now replaced in place,
|
||||||
|
using the same pseudonyms as everywhere else — so a MAC that appears both in a parsed field and in
|
||||||
|
a raw dump still reads as one device. The dump stays readable and auditable: you can still see the
|
||||||
|
neighbour table's shape, the host count, RFC1918 addresses and vendor prefixes. Dropping the
|
||||||
|
evidence would have protected the same data while destroying the reason for collecting it.
|
||||||
|
|
||||||
|
Two implementation notes worth keeping:
|
||||||
|
- **One pass, not three.** Sequential passes re-process their own output: once a MAC became
|
||||||
|
`78:9a:18:xx:yy:zz`, the IPv6 pattern matched it — six hex groups separated by colons *is* an
|
||||||
|
address — and destroyed the vendor prefix the MAC rule had just preserved. Ordered alternation
|
||||||
|
resolves each position once, MAC first.
|
||||||
|
- The patterns are conservative on purpose. A missed address gets caught by another rule or not at
|
||||||
|
all; an over-eager one mangles timestamps and version strings, corrupting evidence to protect
|
||||||
|
nothing.
|
||||||
|
|
||||||
|
`RealDocumentTest` runs the anonymizer over a captured run when `ECHOLOT_REAL_RUN` points at one,
|
||||||
|
and fails on any MAC that survives. It self-skips otherwise, so no one's network is committed to the
|
||||||
|
repo. Against the actual leaked document: **14 MACs in, 0 surviving.**
|
||||||
|
|
||||||
|
Also fixed: the Settings *Preview what an upload would send* button did nothing. It read
|
||||||
|
`UiState.history`, which is empty until the History screen has been opened — the same root cause as
|
||||||
|
the "0 run(s)" count. It now reads the archive directly, and says so when there is nothing to
|
||||||
|
preview rather than silently ignoring the tap.
|
||||||
|
|||||||
@@ -101,11 +101,10 @@ class MainActivity : ComponentActivity() {
|
|||||||
onApplyRetention = vm::applyRetention,
|
onApplyRetention = vm::applyRetention,
|
||||||
onDeleteAll = vm::deleteAllRuns,
|
onDeleteAll = vm::deleteAllRuns,
|
||||||
onPreviewUpload = {
|
onPreviewUpload = {
|
||||||
// Preview the newest run, since that is the one the user just made
|
// Straight from the archive: the newest run is the one the user just
|
||||||
// and the one they are deciding about.
|
// made and the one they are deciding about. Always shows something,
|
||||||
vm.state.history.firstOrNull()?.let { r ->
|
// even when there is nothing to preview yet.
|
||||||
lifecycleScope.launch { preview = vm.uploadPreview(r.id) }
|
lifecycleScope.launch { preview = vm.previewNewestRun() }
|
||||||
}
|
|
||||||
},
|
},
|
||||||
onCheckServer = vm::checkServer,
|
onCheckServer = vm::checkServer,
|
||||||
onEnroll = vm::enroll,
|
onEnroll = vm::enroll,
|
||||||
|
|||||||
@@ -194,6 +194,21 @@ class RunViewModel(app: Application) : AndroidViewModel(app) {
|
|||||||
store.read(id)?.let { store.redactedForUpload(it) }
|
store.read(id)?.let { store.redactedForUpload(it) }
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Preview of the most recent run, read from the archive rather than from [UiState.history].
|
||||||
|
*
|
||||||
|
* The history list is only populated once the History screen has been opened, so a preview
|
||||||
|
* driven from it did nothing at all on a freshly-opened Settings screen — a button that
|
||||||
|
* silently does nothing is worse than one that says why.
|
||||||
|
*/
|
||||||
|
suspend fun previewNewestRun(): String = withContext(Dispatchers.IO) {
|
||||||
|
val newest = store.list().firstOrNull()
|
||||||
|
?: return@withContext "No archived runs yet. Run a measurement first, then this will " +
|
||||||
|
"show exactly what an upload would send."
|
||||||
|
store.read(newest.id)?.let { store.redactedForUpload(it) }
|
||||||
|
?: "That run could not be read back from the archive."
|
||||||
|
}
|
||||||
|
|
||||||
fun archivedBytes(): Long = store.totalBytes()
|
fun archivedBytes(): Long = store.totalBytes()
|
||||||
|
|
||||||
/** Counted from the archive itself, not from [UiState.history], which is empty until the
|
/** Counted from the archive itself, not from [UiState.history], which is empty until the
|
||||||
|
|||||||
@@ -16,6 +16,7 @@ import androidx.compose.foundation.verticalScroll
|
|||||||
import androidx.compose.material3.Button
|
import androidx.compose.material3.Button
|
||||||
import androidx.compose.material3.Card
|
import androidx.compose.material3.Card
|
||||||
import androidx.compose.material3.FilterChip
|
import androidx.compose.material3.FilterChip
|
||||||
|
import androidx.compose.material3.LocalContentColor
|
||||||
import androidx.compose.material3.MaterialTheme
|
import androidx.compose.material3.MaterialTheme
|
||||||
import androidx.compose.material3.OutlinedTextField
|
import androidx.compose.material3.OutlinedTextField
|
||||||
import androidx.compose.material3.Switch
|
import androidx.compose.material3.Switch
|
||||||
@@ -130,12 +131,21 @@ fun SettingsScreen(
|
|||||||
}
|
}
|
||||||
Text(privacyExplanation(privacy), style = MaterialTheme.typography.bodySmall)
|
Text(privacyExplanation(privacy), style = MaterialTheme.typography.bodySmall)
|
||||||
|
|
||||||
|
// At FULL nothing is pseudonymized, so a salt has nothing to act on. Shown
|
||||||
|
// disabled rather than hidden: the setting is still stored and still applies the
|
||||||
|
// moment the level changes, and a control that vanishes hides that fact.
|
||||||
Toggle(
|
Toggle(
|
||||||
label = "Stable pseudonyms across runs",
|
label = "Stable pseudonyms across runs",
|
||||||
detail = "Lets you compare uploaded runs over time (same SSID reads the same " +
|
detail = if (privacy == PrivacyLevel.FULL) {
|
||||||
"each time). It also links your uploads together, so leave it off on a " +
|
"Not used at this level — nothing is pseudonymized, so there is nothing " +
|
||||||
"server you don't run yourself.",
|
"to keep stable. Choose balanced or strict to use this."
|
||||||
checked = stableSalt,
|
} else {
|
||||||
|
"Lets you compare uploaded runs over time (same SSID reads the same " +
|
||||||
|
"each time). It also links your uploads together, so leave it off on " +
|
||||||
|
"a server you don't run yourself."
|
||||||
|
},
|
||||||
|
checked = stableSalt && privacy != PrivacyLevel.FULL,
|
||||||
|
enabled = privacy != PrivacyLevel.FULL,
|
||||||
) { stableSalt = it; settings.stableSalt = it }
|
) { stableSalt = it; settings.stableSalt = it }
|
||||||
|
|
||||||
TextButton(onClick = onPreviewUpload) { Text("Preview what an upload would send") }
|
TextButton(onClick = onPreviewUpload) { Text("Preview what an upload would send") }
|
||||||
@@ -237,13 +247,22 @@ private fun privacyExplanation(level: PrivacyLevel): String = when (level) {
|
|||||||
}
|
}
|
||||||
|
|
||||||
@Composable
|
@Composable
|
||||||
private fun Toggle(label: String, detail: String, checked: Boolean, onChange: (Boolean) -> Unit) {
|
private fun Toggle(
|
||||||
|
label: String,
|
||||||
|
detail: String,
|
||||||
|
checked: Boolean,
|
||||||
|
enabled: Boolean = true,
|
||||||
|
onChange: (Boolean) -> Unit,
|
||||||
|
) {
|
||||||
Row(Modifier.fillMaxWidth(), verticalAlignment = Alignment.Top) {
|
Row(Modifier.fillMaxWidth(), verticalAlignment = Alignment.Top) {
|
||||||
Column(Modifier.weight(1f)) {
|
Column(Modifier.weight(1f)) {
|
||||||
Text(label, style = MaterialTheme.typography.bodyMedium)
|
// Dimmed together with the switch, so "this does nothing right now" reads at a glance
|
||||||
Text(detail, style = MaterialTheme.typography.bodySmall)
|
// instead of only on close inspection.
|
||||||
|
val alpha = if (enabled) 1f else 0.5f
|
||||||
|
Text(label, style = MaterialTheme.typography.bodyMedium, color = LocalContentColor.current.copy(alpha = alpha))
|
||||||
|
Text(detail, style = MaterialTheme.typography.bodySmall, color = LocalContentColor.current.copy(alpha = alpha))
|
||||||
}
|
}
|
||||||
Switch(checked = checked, onCheckedChange = onChange)
|
Switch(checked = checked, onCheckedChange = onChange, enabled = enabled)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
@@ -65,4 +65,41 @@ class LiveThroughputTest {
|
|||||||
assertTrue(m.contains(""""limited_by":"duration""""),
|
assertTrue(m.contains(""""limited_by":"duration""""),
|
||||||
"the run did not end on the clock, so the rate measures the server, not the path: $m")
|
"the run did not end on the clock, so the rate measures the server, not the path: $m")
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// Upstream is the direction only the far end can measure. The assertion that matters is that
|
||||||
|
// the server's count is present and plausible against what we sent — a test that only checked
|
||||||
|
// "we transmitted some Mbps" would pass against a server that counted nothing at all.
|
||||||
|
@Test
|
||||||
|
fun measuresUpstreamAgainstTheServersCount() {
|
||||||
|
if (url == null || pin == null || cred == null || udp == null) {
|
||||||
|
println("LiveThroughputTest(up) skipped"); return
|
||||||
|
}
|
||||||
|
val control = ControlClient(url, setOf(pin), "0.2.0")
|
||||||
|
val session = control.createSession(cred, target)
|
||||||
|
val (host, port) = udp.split(":").let { it[0] to it[1].toInt() }
|
||||||
|
|
||||||
|
val (test, findings) = ProbeSession(cred, session, host, port).use { ps ->
|
||||||
|
ps.echo()
|
||||||
|
ThroughputMeasurement(SystemIdSource()).runUpstream(
|
||||||
|
cred, session.sessionId, control, ps, sessionRef = "sess-1",
|
||||||
|
durationS = 3, kbps = 10_000,
|
||||||
|
)
|
||||||
|
}
|
||||||
|
control.deleteSession(cred, session.sessionId)
|
||||||
|
|
||||||
|
val m = assertNotNull(test.metrics).toString()
|
||||||
|
println("upstream: ${test.status} $m")
|
||||||
|
for (f in findings) println("finding ${f.code} [${f.severity}] ${f.title}")
|
||||||
|
|
||||||
|
assertEquals(TestStatus.OK, test.status, "the server counted nothing: $m")
|
||||||
|
val recv = Regex(""""received_packets":(\d+)""").find(m)?.groupValues?.get(1)?.toInt()
|
||||||
|
val sent = Regex(""""sent_packets":(\d+)""").find(m)?.groupValues?.get(1)?.toInt()
|
||||||
|
assertNotNull(recv); assertNotNull(sent)
|
||||||
|
assertTrue(sent > 100, "barely anything was sent, so the rate means nothing: $m")
|
||||||
|
assertTrue(recv > 0, "the server received none of $sent packets: $m")
|
||||||
|
// The counts should be close on a healthy path; wildly different means the two sides are
|
||||||
|
// counting different things rather than the network losing packets.
|
||||||
|
assertTrue(recv <= sent, "the server counted MORE than we sent — the counter is not being reset")
|
||||||
|
println("sent $sent, server saw $recv")
|
||||||
|
}
|
||||||
}
|
}
|
||||||
@@ -20,4 +20,8 @@ kotlin {
|
|||||||
}
|
}
|
||||||
java { sourceCompatibility = JavaVersion.VERSION_17; targetCompatibility = JavaVersion.VERSION_17 }
|
java { sourceCompatibility = JavaVersion.VERSION_17; targetCompatibility = JavaVersion.VERSION_17 }
|
||||||
|
|
||||||
tasks.test { useJUnitPlatform() }
|
tasks.test {
|
||||||
|
useJUnitPlatform()
|
||||||
|
// Opt-in: point this at a captured run to check the anonymizer against real data.
|
||||||
|
System.getenv("ECHOLOT_REAL_RUN")?.let { environment("ECHOLOT_REAL_RUN", it) }
|
||||||
|
}
|
||||||
|
|||||||
@@ -119,7 +119,11 @@ class Anonymizer(private val level: PrivacyLevel, private val salt: Salt) {
|
|||||||
}
|
}
|
||||||
|
|
||||||
private fun transform(type: LogicalType?, value: String): String = when (type) {
|
private fun transform(type: LogicalType?, value: String): String = when (type) {
|
||||||
null -> value
|
// Unclassified strings still get their *embedded* identifiers scrubbed. A whole-value
|
||||||
|
// check cannot see them: raw shell output is one long string that is neither a MAC nor an
|
||||||
|
// address, so it sailed through both the name table and the shape check carrying every
|
||||||
|
// MAC on the user's LAN.
|
||||||
|
null -> scrubEmbedded(value)
|
||||||
LogicalType.SSID -> pseudo("ssid", value) { "net-" + it.take(6) }
|
LogicalType.SSID -> pseudo("ssid", value) { "net-" + it.take(6) }
|
||||||
LogicalType.MAC, LogicalType.BSSID -> macPreservingOui(value)
|
LogicalType.MAC, LogicalType.BSSID -> macPreservingOui(value)
|
||||||
LogicalType.IP4 -> ip4(value)
|
LogicalType.IP4 -> ip4(value)
|
||||||
@@ -129,6 +133,38 @@ class Anonymizer(private val level: PrivacyLevel, private val salt: Salt) {
|
|||||||
LogicalType.FREETEXT -> "[removed: may contain identifying text]"
|
LogicalType.FREETEXT -> "[removed: may contain identifying text]"
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Replaces addresses and MACs found *inside* a longer string.
|
||||||
|
*
|
||||||
|
* Shizuku probes embed raw command output verbatim — `ip neigh`, `ip route`, `dumpsys` — which
|
||||||
|
* is genuinely valuable evidence and also a complete inventory of every device on the user's
|
||||||
|
* network, with hardware addresses. measurement-schema.md §9 flagged these as "hard to
|
||||||
|
* anonymize" and proposed dropping them from exports.
|
||||||
|
*
|
||||||
|
* Scrubbing beats dropping: the output stays readable and auditable — you can still see the
|
||||||
|
* shape of the neighbour table and how many hosts there were — while the identifiers become
|
||||||
|
* the same pseudonyms used everywhere else in the document. So a MAC appearing both in a
|
||||||
|
* parsed field and in a raw dump still reads as one device.
|
||||||
|
*
|
||||||
|
* Only addresses and MACs are touched, for the same reason as [Classification.inferFromValue]:
|
||||||
|
* they are the patterns that cannot be mistaken for something else in free text.
|
||||||
|
*/
|
||||||
|
private fun scrubEmbedded(value: String): String {
|
||||||
|
// Cheap bail-out: the overwhelming majority of strings are short and contain neither.
|
||||||
|
if (value.length < 7 || (!value.contains(':') && !value.contains('.'))) return value
|
||||||
|
// One pass, not three. Sequential passes re-process their own output: after a MAC became
|
||||||
|
// 78:9a:18:xx:yy:zz the IPv6 pattern matched it — six hex groups separated by colons is
|
||||||
|
// exactly an address — and mangled the vendor prefix that the MAC rule had just taken
|
||||||
|
// care to preserve. Ordered alternation resolves each position once, MAC first.
|
||||||
|
return EMBEDDED.replace(value) { m ->
|
||||||
|
when {
|
||||||
|
m.groups[1] != null -> macPreservingOui(m.value)
|
||||||
|
m.groups[2] != null -> ip6(m.value)
|
||||||
|
else -> ip4(m.value)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
// ---- per-type transforms -------------------------------------------------------------
|
// ---- per-type transforms -------------------------------------------------------------
|
||||||
|
|
||||||
/**
|
/**
|
||||||
@@ -258,7 +294,7 @@ class Anonymizer(private val level: PrivacyLevel, private val salt: Salt) {
|
|||||||
|
|
||||||
/** Deterministic per (domain, value, salt); memoized so one value maps to one pseudonym. */
|
/** Deterministic per (domain, value, salt); memoized so one value maps to one pseudonym. */
|
||||||
private fun pseudo(domain: String, value: String, shape: (String) -> String): String =
|
private fun pseudo(domain: String, value: String, shape: (String) -> String): String =
|
||||||
cache.getOrPut("$domain | |||||||