Files

T

mrambossekandClaude Fable 5 e7afc2210f findings: a registry, because the codes had already drifted

A finding code is the stable half of a result - what a dashboard groups by and
what someone greps a year of archived runs for. That only holds if a code means
exactly one thing forever, which fifteen ad-hoc string literals cannot promise.

By the time this was written the failure had happened twice:

  - Two emitters independently produced connectivity.downstream_loss and
    connectivity.loss_downstream for the same claim. Nothing objected. Anyone
    aggregating either would have silently seen half their data.
  - Two codes sat under nat.* while being declared Category.CONNECTIVITY.
    nat.udp_unreachable is not about NAT, and the prefix decides the category,
    which decides which verdict light the finding rolls up into. Renamed while
    that is still cheap.

Codes are now typed FindingSpecs carrying category and default severity;
emitters reference the spec rather than retyping the string, so a typo is a
compile error and two call sites cannot disagree about a finding's category.

docs/findings-registry.md is the contract and a test reads it, failing when the
document and the code disagree on which codes exist or how severe they are.
Documentation that drifts from its implementation is worse than none, because it
still looks authoritative. The check reads table rows only, so the prose can go
on explaining which codes were retired and why.

Closes open item 1 of measurement-schema.md section 9.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

2026-08-01 14:25:54 +02:00

5.0 KiB

Raw Blame History

Echolot findings registry

Closes open item 1 of measurement-schema.md §9.

A finding code is the stable, machine-readable half of a result. The prose around it changes freely; the code is what a dashboard groups by, what a diff between two runs keys on, and what someone greps a year of archived runs for. That only works if a code means exactly one thing, forever.

This document is the contract. It is kept in step with echolot-app/core-measurement/.../FindingRegistry.kt by a test that fails when either side has a code the other does not — a registry that drifts from its documentation is worse than none, because it looks authoritative.

Rules

The prefix determines the category, and the category determines which verdict light the finding rolls up into (§7.3). A nat.* code appearing under connectivity is not a naming quibble; it changes which light turns red. Two codes were renamed from nat.* to connectivity.* for exactly this reason.
One code per concept. Two emitters independently produced connectivity.downstream_loss and connectivity.loss_downstream for the same claim before this registry existed. Anyone aggregating either would have silently seen half their data.
Codes are declared, not typed. Emitters reference a FindingSpec, so a typo is a compile error and no two call sites can disagree about a finding's category or default severity.
Severity in the registry is the default. An emitter may escalate for a specific run; it may not quietly reclassify the finding in general.
Say what is ruled out, where that is the useful half. "Loss upstream" is worth far more when it also states that the return path is clean, because that halves where to look next.
Renaming a code is a breaking change once runs are archived at scale. Before 1.0 it is cheap; after, it needs an alias and a deprecation window.

Registry

connectivity

code	severity	means	rules out
`connectivity.udp_unreachable`	high	No UDP echo replies came back from the server at all.	—
`connectivity.udp_unreachable_upstream`	high	The server received none of the probes, so traffic is dropped on the way out.	The return path: nothing arrived to be replied to.
`connectivity.udp_loss`	medium	A large fraction of round-trip probes were lost, direction unknown.	—
`connectivity.loss_upstream`	medium	Probes were lost on the way to the server.	The return path: replies came back for everything that arrived.
`connectivity.loss_downstream`	medium	Packets were lost on the way back from the server.	The outbound path: the server received what it was answering.
`connectivity.downstream_blocked`	high	Server-initiated packets never arrive, although round trips work.	Basic reachability: the path forwards replies, just not unsolicited traffic.
`connectivity.downstream_reorder`	low	Downstream packets arrive in a different order than they were sent.	—
`connectivity.captive_portal`	high	A captive portal is intercepting connectivity checks.	—
`connectivity.no_internet`	high	Android's own connectivity checks fail on this network.	—

mtu

code	severity	means	rules out
`mtu.reduced_downstream`	low	The downstream path MTU is below the usual 1500 bytes.	—
`mtu.downstream_blackhole`	medium	Datagrams above the path MTU are dropped downstream, fragmented or not.	—
`mtu.fragments_blocked`	medium	IP fragments do not reach this device even when sent in order.	—
`mtu.fragment_reorder_sensitive`	low	Fragments are delivered in order but dropped when reordered or delayed.	Fragmentation itself: in-order fragments arrive fine.

nat

code	severity	means	rules out
`nat.udp_rebinding`	medium	A NAT remapped the UDP source port mid-flow.	—
`nat.symmetric`	medium	The NAT assigns a different external port per destination.	—

perf

code	severity	means	rules out
`perf.throughput_no_delivery`	high	No throughput traffic arrived, although the server sent it.	—
`perf.throughput_below_offered`	low	Less throughput arrived than the server sent for the whole run.	—

dns

code	severity	means	rules out
`dns.answer_rewritten`	high	A resolver returned an answer that differs from the authoritative record.	—
`dns.authoritative_unreachable`	medium	The canary zone's authoritative server could not be reached.	—

Adding a finding

Add a FindingSpec to FindingRegistry, and to its all list.
Add the row here, under the section its prefix names.
Emit it with finding(FindingRegistry.YOUR_CODE, …).

The registry test checks 1 and 2 agree, that every prefix maps to the category it claims, and that no two entries share a code.

5.0 KiB Raw Blame History