echolot/docs/build-status.md

Echolot — build status & next steps

Last updated: 2026-07-29.

Decided

• Name Echolot; domain echo-lot.app; scheme echolot://; namespace app.echo_lot.* (hyphen→underscore; appIds/packages can't contain hyphens). Prober appId app.echo_lot.prober.
• Stack: native Kotlin + Jetpack Compose, no Flutter.
• Tiers: app (no root), shizuku in v1 (wireless-ADB pairing), root future.
• License (decided 2026-07-30): all code GPL-3.0-or-later; the specs in docs/ CC-BY-4.0. Rationale: the moat is the no-root platform research, which is trivially liftable into a proprietary repackage — copyleft is the only option that prevents that, and the F-Droid / network-engineer audience reads GPL as a trust signal rather than friction. The specs go permissive on purpose: a wire protocol only becomes a standard if anyone can implement it. Not AGPL for the server — the ASP loophole it closes is speculative here, while blanket corporate AGPL bans would hit exactly the enterprise network teams most likely to self-host. Sole copyright holder, so relicensing the server to AGPL later stays possible. Open: substitute a real legal copyright holder for "Echolot contributors" in the SPDX headers.

Specs (in docs/, alongside this file)

• feature-catalog-and-feasibility.md, measurement-schema.md, probe-protocol.md. Considered draft-complete and reviewed by the user.

Capability prober — DELIVERED (as zip, 2026-07-29)

Full Kotlin/Compose project scaffolded: app.echo_lot.prober, minSdk 26 / target+compile 35, AGP 8.7.3, Kotlin 2.0.21, Compose BOM 2024.10, Shizuku api+provider 13.1.5, kotlinx-serialization. Probes implemented: link.snapshot, icmp.ping4/6 (unprivileged ICMP datagram), sockopt.matrix (TTL/TOS/RECVERR/MTU_DISCOVER), trace.errqueue_reachable, multinetwork.request_and_bind, local.mdns_discover, peer.ble_advertise, shizuku.command_battery (ip neigh / ip -6 route / ip addr / ip monitor / dumpsys network_stack DHCP+IpClient / dumpsys wifi). JSON export via share intent; results carry verdict + raw evidence.

Could NOT be compiled in the cloud sandbox: dl.google.com (Google Maven) and services.gradle.org are proxy-blocked, and no device is reachable for on-device runs. Build + iterate locally (Android Studio / Claude Code). Wrapper is pinned to Gradle 8.14.3.

Website — web/ (2026-07-30, LIVE at https://echo-lot.app)

Single-page site for echo-lot.app on Cloudflare Workers: static public/ from the edge, a small Worker for the stable short URLs (/apk, /apk.sha256, /fdroid, /source) and /api/latest. Version info is resolved from the Gitea releases API at request time (edge-cached 5 min), so tagging a release is the only publish step — the homepage shows the latest version/date/size via progressive-enhancement JS and falls back to pre-release copy otherwise. Auto light/dark via prefers-color-scheme: dark = sonar display, light = chart-recorder printout, same amber-accent token system. Deploy: npx wrangler deploy from web/, or CI via .gitea/workflows/deploy-site.yml (needs CLOUDFLARE_API_TOKEN + CLOUDFLARE_ACCOUNT_ID secrets); full story in web/README.md. Branding applied (2026-07-30): real favicon/wordmark copied into web/public/assets/ (copies — assets/branding/ is the source of truth), palette switched to abyss/teal/amber with teal-as-instrument semantics, sonar-sweep hero replaced by a Focus-lattice panel (rogue-DHCP finding vignette), tagline "measure, don't guess".

Branding — assets/branding/ (2026-07-30)

Identity chosen: the Focus mark — a quiet lattice of nodes with exactly one under examination (amber) inside teal viewfinder brackets. Tone is surgical measurement, not scanning (sonar-sweep visuals and the "sound out your network" tagline were explicitly rejected); tagline is "measure, don't guess". Palette: abyss #071522, tile #0E2433, instrument teal #35E0C4, finding amber #FFB454, foam #E8F4F2 — teal is always the instrument, the single amber point is the finding. The wordmark is hand-drawn monoline SVG paths (zero font dependency); the second "o" is the signature teal ring + amber ping. Assets: icon.svg (+ icon.png 512² — Gitea avatars reject SVG), icon-adaptive-{foreground,background}.svg (108dp adaptive-icon layers, art inside the 66dp safe circle), wordmark-on-{dark,light}.svg, banner.svg (1200×300, wired into the repo README), social-preview.svg/.png (1280×640, for the Gitea/GitHub social-preview slot). The website's placeholder favicon/wordmark can now be replaced from these.

First device report — OnePlus 15, Android 16/SDK 36 (2026-07-30)

Archived at echolot-prober/reports/CPH2747-android16-sdk36.json. Collection workflow that works: deploy over (wireless) adb, but the user starts the run and exports the JSON manually — probe runs reliably kill the adb-over-wifi session (multinetwork/multicast churn the link adb rides on).

OS findings:

• link.snapshot, icmp.ping4 (38 ms), sockopt.matrix 4/4, peer.ble_advertise: SUPPORTED.
• trace.errqueue_reachable SUPPORTED — full Os-API errqueue path exists on Android 16; the C-over-JNI shim (next-steps #2) may be unnecessary on modern devices. Still needs a real errqueue round-trip test, and older devices may differ.
• icmp.ping6 EAGAIN — topology, not capability: wifi was v4-only, v6 only on cellular, socket used the default network. Probe should bind per-network (also covers the ping4 case properly).
• shizuku.command_battery SUPPORTED, shell(2000): real ip -6 route RA data (lifetimes, per-table), ip monitor NEIGH events, wifi dump with full DhcpResults. Vendor formats look parseable.
• local.mdns_discover: lock acquired, discovery ran, 0 services in the 4 s window — needs a longer window / known-good reference network before calling it more than formally SUPPORTED.

Prober bugs the run exposed (all fixed same day):

• rotation wiped results + cancelled runs → state moved to a ViewModel;
• multinetwork.request_and_bind ERROR → missing CHANGE_NETWORK_STATE in the manifest;
• rtt_ms: "38,1" → device locale leaked into report numbers, now Locale.ROOT;
• 3/7 Shizuku commands SHIZUKU_BIND_TIMEOUT → bind/unbind-per-command raced Shizuku; now one bind per battery (execBatch). Run 2 (same device, fixed APK — …-run2.json) confirmed all three fixes:
• multinetwork.request_and_bind SUPPORTED — wifi + cellular acquired and bound concurrently (downKbps 39094 / 13231); requesting cellular even brought up a second, dual-stack PDN (rmnet_data2, private v4 /30 + carrier-NAT DNS) next to the IPv6-only default PDN — exactly the per-network behavior the production app needs.
• Shizuku battery 7/7 in 2.9 s (was 4/7 in 34 s): full ARP/NDP neighbor table, ip addr, IpClient ProvisioningConfiguration incl. SSID/BSSID and APF caps (v6000, 4096 B).
• rtt_ms: "32.0" — locale fix holds. Still open (probe improvements, not blockers): ping6 should bind per-network (still EAGAIN via v4-only default wifi); mDNS window too short to see real services (0 found on a network that has them); ip monitor 2 s window caught nothing on a quiet net — consider longer/event-triggered capture; IP_MTU read via getsockoptInt reflection unavailable — reader belongs in the errqueue shim if that lands.

Go server skeleton — server/ (2026-07-30)

Pure-Go stdlib implementation of the spec's core: control plane (enroll with single-use tokens, profile with SPKI pin, sessions with the §2.4 HKDF key schedule) + UDP data plane (ELT1 header, HMAC gate, 1024-wide anti-replay, ECHO_RESP with observation block, TIMESYNC, §3.4 anti-amplification). Wire format covered by go test (roundtrip, replay/bad-HMAC/unknown-prefix silent drops); full enroll→profile→session flow smoke-tested live. Two run modes: container (autodetected via /.dockerenv etc. or --docker; config from ECHOLOT_* env; distroless image, network_mode: host required — Docker NAT would falsify the observed sources the protocol measures) and native (--install-systemd/--uninstall-systemd, opt-in --self-update against a Gitea releases API). CI: .gitea/workflows/build-server.yml — tests on push; server-v* tags push the image to the Gitea registry + attach linux amd64/arm64 binaries that self-update consumes. Not yet: TCP/TLS echo, STUN, canary DNS, actions, observations API, real admin UI, self-update signature verification.

Prober build numbering: ascending versionCode shown on screen + as proberBuild in the JSON report; current deployed build = 3. Bump on every deployed change.

Build-2 reports: OnePlus 15 (A16) + Lenovo TB330FU (A15) — 2026-07-30

Archived as …-build2.json in echolot-prober/reports/. The findings:

• traceroute.udp4 SUPPORTED on BOTH devices (SDK 35 + 36): 6 real hops via pure-Kotlin Os.recvmsg(MSG_ERRQUEUE) in ~250 ms, identical paths (A1 → Cloudflare). The C-over-JNI errqueue shim is dead for Android 15+; keep it in mind only if pre-15 devices matter.

• Per-network ICMP works as designed: phone ping6 = "cellular only, default has no v6 path" (SUPPORTED with topology evidence instead of the old bare ERROR); per-network RTTs (wifi 20 ms vs cellular 201 ms on v4).

• Network finding (dogfood!): hudeWLAN advertises a v6 default route via RA but hands out no global v6 address — both devices have only link-local + a ::/0 route on wifi. That's a half-broken IPv6 config on the LAN, exactly the class of issue the product should flag.

• mDNS: meta-query returns 0 on BOTH devices (NsdManager limitation, now confirmed twice); concrete _http._tcp finds 4 real services (3× "Magic 2 LAN", EPSON WF-7840) on both. Production inventory must enumerate concrete types, not trust the meta-query.

• Tablet Shizuku: binder alive, permission granted, but UserService bind timed out (0/7) — first-spawn dex extraction on slow storage suspected; build 3 raises the bind window to 25 s + one retry. Phone stays 7/7 with rich neighbor/RA/DHCP evidence. Build-3 verdict: not timing. Both 25 s attempts timed out (50 s total) — the UserService spawn genuinely fails on the Lenovo/A15. Build 4 adds a reflection fallback to the legacy Shizuku.newProcess remote-process API when the bind fails; exec_path in the evidence says which path ran. Whatever the outcome, core-shizuku must not assume UserService works everywhere. (Phone build 3: still 7/7; ip_monitor now catches provoked NEIGH PROBE→REACHABLE transitions, and a mid-path router dropping one TTL round showed the "*" hop path works in traceroute.udp4.)

  Build-4 verdict (…-build4.json): the fallback WORKS. Tablet Shizuku now SUPPORTED via exec_path: newProcess fallback — 6/7 commands with real data (full neighbor table, per-netId v6 route tables, Lenovo's IpClient dump — a distinctly different format from OnePlus's, i.e. the per-vendor parser sample we wanted). core-shizuku design consequence: dual-path executor is mandatory — UserService where it binds, newProcess where it doesn't; both confirmed live on real hardware. Known rough edges: ip monitor times out under newProcess (subshell trick doesn't survive that path); the 2×25 s bind wait before falling back is worth shortening once a device is known-bad. Tablet UserService root cause still open — multi-user (3 users) is the prime suspect; Shizuku app is current (13.6.0), so the stale-app theory is dead.

• ip_monitor returned no events this run even with the provoked gateway ping (gateway was already REACHABLE, so no NEIGH transition happened). Evidence-dependent, not a bug.

Pipeline shakeout — 2026-07-30, partially done

Repo pushed to git.rambossek.at/EchoLot/echolot (ssh :2222). Actions runner works: server-test green on push. server-v0.1.0 release shipped with all assets (echolot-server_linux_{amd64,arm64} + SHA256SUMS) after two workflow fixes: mixed paths+tags triggers never fire (split into test-server.yml / build-server.yml), and a greedy sed grabbed a nested user id instead of the release id → uploads 404ed (first-match grep now; release creation also falls back to GET-by-tag on re-runs). Binaries verified downloadable → --self-update has its artifact source. Image job green too on the dedicated compilesau-echolot runner (label echolot): needed a PAT with package read/write as the REGISTRY_TOKEN repo secret — the built-in Actions token is rejected by the container registry. git.rambossek.at/echolot/echolot-server:{v0.1.0,latest} pulls anonymously (3.9 MB distroless, verified). Full chain proven: tag → tests → binaries → release assets → registry image → anonymous pull. Note the tag convention: image tags carry the bare version (v0.1.0), release tags the namespaced one (server-v0.1.0); selfupdate normalizes.

Toolchain upgrade + build-5 tablet re-run (2026-07-30)

Moved to the laptop (USB adb — far more stable than wireless, as expected). Upgraded the build off the JDK-21 ceiling: AGP 8.7.3 → 9.2.0, Gradle 8.14.3 → 9.6.0, Kotlin 2.0.21 → 2.2.10, which lets the toolchain run on JDK 25. Two AGP-9 migration edits were required:

• AGP 9 has built-in Kotlin support, so applying org.jetbrains.kotlin.android alongside it fails with "extension with name 'kotlin' already registered" — the alias must be dropped (kotlin.compose / kotlin.serialization stay; they are separate compiler plugins).
• The kotlinOptions { jvmTarget } block came from that plugin and no longer resolves; AGP derives jvmTarget from compileOptions instead.

compileSdk/targetSdk 35 → 36, because Android Studio only ships API 36.1 by default and installing 35 was avoidable churn. Build-5 re-run on the TB330FU is verdict-identical to build-4 across all 10 probes (…-build5.json) — the toolchain jump and targetSdk 36 are behaviour-neutral on this device, including the newProcess Shizuku fallback (still 7/7, exec_path: newProcess fallback, UserService still never binds).

Also of note: traceroute.udp4 is SUPPORTED here (6 hops via errqueue, "no native shim needed"), which is what retires next-step 2 below — the JNI shim is not required on either known device.

OnePlus 15 build-5 re-run (CPH2747-android16-sdk36-build5.json): all 10 SUPPORTED, identical to build-3. exec_path: UserService on the phone vs newProcess fallback on the tablet, both 7/7 — the dual-path executor requirement is now confirmed on the current toolchain on both devices. Phone is on Android 16 / SDK 36, i.e. targetSdk = 36 is also exercised natively there.

Two collection-loop gotchas found while driving the phone over USB:

• icmp.ping6's verdict is topology-dependent, not build-dependent. A first run with Shizuku stopped and only 2 networks up reported UNSUPPORTED; with cellular+wifi present it is the expected SUPPORTED / "echo reply on cellular only — default network has no v6 path". Do not read a ping6 verdict without checking link.snapshot's network_count in the same report.
• The Shizuku permission dialog times out after 30 s (shizuku.command_battery logs Shizuku permission not granted at ~30027 ms). Screenshot→read→tap round-trips can exceed that. Grant the permission before starting a run, or accept that the first run after a fresh install burns the Shizuku probe; the grant is persistent, so the next run is clean.
• The phone auto-rotates; tap coordinates must be recomputed per orientation, and uiautomator dump can report a stale idle state right after a tap — trust the exported JSON's durationMs values as ground truth for whether a run really completed, not the button label.

Next steps

1. Build locally, run on several physical devices (varied Android versions/vendors), collect the JSON reports — especially the real per-device Shizuku dump formats.
2. If trace.errqueue_reachable = PARTIAL, add a C-over-JNI errqueue shim. Retired — SUPPORTED on both known devices; traceroute.udp4 reads real hops via Os.recvmsg + StructMsghdr reflection, so no :native module is needed.
3. Start the Go server skeleton (enrollment + profile + sessions + UDP echo with observation blocks + canary-DNS reference records) per probe-protocol.md.
4. Fold confirmed capabilities into the production core-probe / core-shizuku modules.

Production probe server — LIVE on dedicated VM "fmr" (2026-07-31)

echolot-server v0.2.0 runs natively (systemd, no docker) on a dedicated VM: 2×IPv4 + 2×IPv6 service addresses (fmr-1/fmr-2.echo-lot.app, dual-stack DNS), a third IPv6 (::2) reserved for SSH only — verified untouched by the daemon (explicit multi-address binds, no wildcard). Control: fmr-1:8443 (SPKI pin zRV9qkiLnRexAeh4RrSfJzbPWO+U/2Oj2/NVM/KfXlg=, verified externally over v4+v6). UDP data plane on all four service addresses :8442 — the second IP is the stun-5780 substrate. Daily randomized self-update timer installed (checksum-verified against SHA256SUMS; signature verification still TODO before treating the source as untrusted). Host config in /etc/echolot-server.env. SSH access for sessions: ssh claude-echolot.

Server v0.3.0 — STUN + TCP echo + observations + actions (2026-07-31)

Shipped and deployed to fmr via the server's own --self-update (first real exercise: checksum-verified download v0.2.0→v0.3.0, atomic replace, restart — worked). Added over v0.2.0:

• STUN (RFC 5389 + 5780): 4 service addrs × primary/alt-port grid. Externally verified on v4 AND v6 — binding success with XOR-MAPPED, RESPONSE-ORIGIN, OTHER-ADDRESS present, so the profile now advertises stun-5780 (the second IP earns its keep).
• TCP echo (:8441): JSON greeting with observed src + real Linux TCP_INFO — verified externally mss:1440 (v6, 1500−60), options [sack,wscale], then byte-echo.
• Observations API GET /v1/sessions/{id}/observations (per-packet UDP view, connect-back results, TCP records correlated by source IP).
• Actions POST /v1/sessions/{id}/actions: delayed_echo (DELAYED_ECHO at the observed data-plane source — NAT-lifetime primitive) and connect_back (dials the control-plane source, records connected/refused/timeout+rtt).
• Capabilities computed from what's actually wired: udp-probe, delayed-echo, connect-back, tcp-echo, stun-5780. Still not implemented: TLS-echo/JA4, HTTP echo, tls-reference, canary DNS (§6.1 reference records), and the train/big-send/frag/throughput actions. Admin UI still token-mint + health only.

Canary DNS live — server v0.3.1 on fmr (2026-07-31)

Zone c.echo-lot.app delegated (NS → fmr-1/fmr-2) and authoritative on all 4 service IPs udp+tcp/53. Verified through full public recursion: ttl-5 A→192.0.2.5 (Cloudflare), ttl-3600 AAAA→2001:db8::3600 (Google), big-txt TXT returned (TCP fallback, truncated over UDP as designed). End-to-end session attribution works: a <nonce>.<session-prefix>.c.echo-lot.app query resolved via a public resolver shows up in GET /v1/sessions/{id}/observations → dns_canary with the resolver's real egress IP, transport, and EDNS. First real test already caught a finding: Google applies 0x20 case randomization (mixed-case qname), Cloudflare does not — captured via case_preserved. Capabilities now: udp-probe, delayed-echo, connect-back, tcp-echo, stun-5780, canary-dns. Kept the hand-rolled stdlib DNS (no miekg/dns) — validated against independent clients. Deployed via --self-update (v0.3.0→v0.3.1, checksum-verified).

Server v0.3.2 + v0.3.3 (2026-07-31)

• v0.3.2 — control-plane security (live on fmr, externally verified): POST /v1/echo reflects the received request head+body (b64) and observed TLS (version/cipher/SNI/ALPN) — captured real SNI fmr-1.echo-lot.app and an injected header over public TLS1.3; GET /v1/tls-reference returns the served DER chain + pin (cross-checked against the openssl-derived pin). Optional cleartext echo listener (default off). Capability http-echo.
• v0.3.3 — MTU probe (data plane): MTU_PROBE (0x09) → small MTU_ACK (0x0A) carrying the received datagram size; client DF-probes increasing sizes to find path MTU / black holes. ACK is tiny → never amplifies. Tested.
• Note on trains: upstream trains (TRAIN_DATA 0x03) are already observable — every HMAC-valid packet is recorded (seq/t_rx/size/type) with no per-packet response, so loss/reordering/inter- arrival are visible via GET observations. The dedicated data-plane TRAIN_REPORT (0x05) is deferred: §3.4 anti-amplification means it needs an asymmetric grant + columnar multi-datagram encoding — a focused batch, not a corner to rush. Remaining spec: tls-echo (ClientHello+JA4), TRAIN_REPORT, big/frag-send, throughput, downtrain; real admin UI.

Server self-test + host tuning — v0.3.4/v0.3.5, fmr proven good (2026-07-31)

The daemon now proves its own host is a clean measurement target:

• sysctl audit (GET /admin/selftest, startup warnings): on first run it flagged exactly 4 real issues on fmr — accept_ra=1 on a static-v6 host, accept_redirects=1, send_redirects=1, icmp_ratelimit=1000. Recommended server/deploy/99-echolot-sysctl.conf applied (v6 default route/addrs are proto static with 0 RA-derived routes, so disabling accept_ra is safe — verified v6 egress intact after). Now sysctl_ok=true, 0 warnings.
• egress-MTU self-proof: DF PMTUD via IP_MTU_DISCOVER + getsockopt IP_MTU (v0.3.4 had a bug — read IP_MTU without connecting → ENOTCONN; v0.3.5 connects first). fmr reports 1500 on both v4 and v6 → mtu_ok=true, so client MTU tests are trustworthy.
• Both signals ride in the profile as server_selftest{mtu_ok,sysctl_ok} so a client can skip MTU testing when the server can't support it honestly. fmr profile now: {mtu_ok: true, sysctl_ok: true}.

Server v0.3.6 — tls-echo / JA4: spec §4 COMPLETE (2026-07-31)

The elt-echo TLS variant runs on the TCP-echo port (8441), multiplexed by a timed 0x16 peek (plain echo stays server-speaks-first; a TLS ClientHello routes to the TLS path). It captures the full ClientHello, returns it raw (b64) + as a JA4 fingerprint (FoxIO), then TLS byte-echoes — the sec.clienthello_echo evidence. Hand-rolled ClientHello parser (ciphers/exts/ALPN/ supported_versions/sig-algs, GREASE-excluded), unit-tested. Cross-client verified on fmr: openssl → t13d3013eo (30 ciphers), python ssl → t13d1712eo (17) — different stacks, different fingerprints, correct _a structure both. Capability tls-echo.

Spec §4 (TCP/TLS/HTTP/STUN) is now fully implemented. Server capabilities: udp-probe, delayed-echo, connect-back, http-echo, tcp-echo, tls-echo, stun-5780, canary-dns. Remaining spec: §5 heavy actions (downtrain/big_send/frag_send/throughput) + the TRAIN_REPORT retrieval path — all gated on the anti-amplification grant machinery (§3.4) — and a real admin UI.

Production app — echolot-app/, core-protocol proven live (2026-07-31)

Started the Android client, bottom-up from the verifiable spine. echolot-app/ is a multi-module Gradle build; core-protocol is a pure Kotlin/JVM module (no Android SDK) implementing the client half of probe-protocol.md: SPKI-pinned control plane (enroll/profile/session via HttpsURLConnection — Android-API-1 compatible, hostname verification off since trust is the pin), HKDF-SHA256 session keys, and the ELT1 UDP data plane (HMAC gate, ECHO+observation, MTU probe) — byte-compatible with the Go server. Unit tests pass incl. the RFC 5869 HKDF vector (so key derivation provably matches the server). Verified END-TO-END against live fmr via scripts/test-fmr.sh (mint token over SSH → enroll on public control plane → run LiveServerTest): profile (8 caps), session, ECHO rtt ~11ms with the observation block round-tripping the client's observed NAT port, MTU probe 1400→1400, observations 298B. Two client bugs found+fixed doing it: java.net.http did hostname verification (switched to HttpsURLConnection) and ECHO needed ≥72-byte requests for the full 40-byte observation to survive §3.4 anti-amplification. Next: core-measurement (schema types), core-probe (port prober probes), core-shizuku (dual-path), Compose UI.