echolot/docs/feature-catalog-and-feasibility.md

# Echolot (Android Network Debug App) — Feature Catalog & Feasibility (v0.1 draft)

Status: brainstorm/design input, 2026-07-29. Premises: F/OSS, no root in v1, Wi-Fi + cellular + USB ethernet, own server infrastructure available, audience = network engineers, detailed archived/exportable measurements with a summary score.

Decisions so far: name = **Echolot** (domain echo-lot.app, scheme echolot://, suggested appId at.rambossek.echolot); **Shizuku support in v1** (not v1.x); **native Kotlin + Jetpack Compose** (no Flutter — all value is in platform-level APIs; Flutter would be a Dart skin over a full Kotlin app and complicates F-Droid reproducible builds); **peer mode with BLE out-of-band control channel** is a headline feature.

## 1. What no-root Android allows and forbids

The whole design hangs on this. Verified against Linux/Android capability rules; re-check details against the current Android release before implementation.

**Allowed without root:**

- UDP/TCP sockets on ephemeral ports, all the usual `setsockopt`s: `IP_TTL`, `IP_RECVERR` (+ `MSG_ERRQUEUE`), `IP_MTU_DISCOVER` (DF bit / probe mode), `IP_TOS`/DSCP, ECN bits via `IP_TOS`, `SO_TIMESTAMP(NS)`.
- ICMP echo: `socket(AF_INET, SOCK_DGRAM, IPPROTO_ICMP)` and the ICMPv6 equivalent work for any UID on Android (`ping_group_range` is opened system-wide). Echo only — the kernel filters other ICMP types.
- Traceroute: UDP (or ICMP-echo) probes with increasing TTL + `IP_RECVERR`; time-exceeded senders are delivered on the error queue. No raw socket needed.
- Multicast send/receive with `CHANGE_WIFI_MULTICAST_STATE` + `MulticastLock` → mDNS, SSDP, LLMNR.
- Platform info: `ConnectivityManager`/`LinkProperties` (addresses, prefixes, routes, gateways, DNS servers, MTU, NAT64 prefix, private-DNS state, captive-portal state incl. RFC 8908 API), `WifiManager`/`WifiInfo` (RSSI, link speed, channel/band, Wi-Fi standard, security type; SSID/BSSID require fine-location permission), `TelephonyManager` (cell type, signal, APN-ish info), deprecated-but-working `DhcpInfo`.
- Multi-network: `requestNetwork` for `TRANSPORT_CELLULAR`, `TRANSPORT_WIFI`, `TRANSPORT_ETHERNET` concurrently; `Network.bindSocket()` per probe. USB ethernet adapters supported by the kernel appear as ethernet networks.
- `VpnService`: capture/inspect the device's own traffic without root (PCAPdroid model).

**Shizuku tier (v1, no root — ADB-shell privileges via wireless-ADB pairing):**

The `shell` user's SELinux domain is far less restricted than an app's. Verified capabilities to build on (each needs per-release verification; build as version-tagged parsers with graceful degradation, and record in every report which tier produced which evidence):

- `ip neigh` → ARP/ND table: gateway MAC monitoring, ARP-spoof heuristics (moves from root-only to Shizuku tier).
- `ip -6 route` / `ip addr` with lifetimes → RA-derived routes (`proto ra`, expiry), address valid/preferred lifetimes.
- `ip monitor` → live netlink stream of addr/route/neigh changes (watch DHCP renewals and RA churn in real time).
- `svc wifi disable|enable`, `cmd wifi` → trigger reconnect/re-lease programmatically, observe via `ip monitor` + dumpsys.
- `dumpsys network_stack` / `dumpsys wifi` → IpClient logs: DHCP OFFER/ACK details, lease params, provisioning events.
- `logcat` → network-stack log lines.

**Not possible without root (→ future root-only module):**

- Raw/packet sockets: own DHCP packets, ARP, RA sniffing, promiscuous/monitor capture, ESP/GRE probes.
- Binding ports < 1024 (this alone kills a real DHCP client — replies go to UDP 68).
- `/proc/net/*` (blocked since Android 10), netlink `RTM_GETLINK` (blocked since 11) → no ARP/neighbor table, no MAC addresses (own interface reports `02:00:00:00:00:00`).
- Wi-Fi monitor mode, beacon-level analysis.

**Platform gotchas to design around:** Wi-Fi scan throttling (≈4 scans/2 min foreground); fine-location permission (+ location services ON) required for SSID/BSSID/scan results; per-SSID MAC randomization; Doze/background limits → long tests need a foreground service, monitor mode uses WorkManager with coarse intervals; `NetworkInterface` enumeration is partially restricted for modern target SDKs → treat `LinkProperties` per `Network` as the source of truth.

## 2. The original wishlist, item by item

| Item | Feasibility (no root) | Approach |
|---|---|---|
| v4/v6 config readout | Full | `LinkProperties` per network: addresses, prefixes, routes, DNS, MTU, NAT64 prefix. Analyze: address scope/type, privacy extensions in use, prefix sanity, gateway in subnet, DNS reachability. |
| Own DHCP requests + option analysis | **Blocked** | Needs port 68 or raw socket (server replies always go to UDP 68, never the requester's ephemeral port — so send-only broadcast is useless). VpnService does NOT help: TUN is a virtual L3 interface; link-scoped traffic (DHCP/ARP/ND/RA) is handled by IpClient directly on wlan0 and never routed through the VPN, and writing into the TUN only feeds your own stack. v1 fallback: read `DhcpInfo` (server, lease) + diff `LinkProperties` across lease renewals; flag server-outside-subnet, weird lease times, gateway/DNS flapping. Middle tier (v1.x): Shizuku/ADB-shell → parse `dumpsys network_stack` / `dumpsys wifi` IpClient logs (real observed DHCP OFFER/ACK details, lease params, RA provisioning events) — unstable format, needs per-release parser + graceful degradation; verify with a prototype spike. Full crafted-packet prober → root module. |
| IPv6 RA inspection | **Blocked** (raw ICMPv6) | Indirect: multiple v6 default routes with different link-local next-hops, prefixes from unrelated aggregates, default-route churn → "multiple RA sources suspected". M/O flags and lifetimes not readable. |
| Multiple DHCP servers / RAs | Partial | Above heuristics only; reliable detection needs sniffing (root module). Peer mode (see 3.7) can help: a second device may hold a lease from the other server — diff their views. |
| Packet loss with patterns | Full | UDP trains to own server, sequence numbers + timestamps both directions; report per-direction loss, burstiness (Gilbert-Elliott fit), reordering, duplication, jitter distribution — not just a percentage. |
| Multiple IP networks on one segment (VLAN leakage) | Good | mDNS/SSDP/LLMNR inventory: announcements revealing addresses in foreign prefixes; own subnet vs observed peers. Plus DHCP-flap heuristics. |
| NAT analysis | Full | RFC 5780 STUN (needs 2 IPs on server): mapping + filtering behavior, hairpinning, port preservation/allocation pattern. Mapping lifetime measurement (idle-timeout binary search, UDP and TCP separately). CGNAT: traceroute into 100.64.0.0/10, STUN-mapped vs interface addr. Inbound: server connects back to mapped port. |
| MTU measurement | Full (with server) | Both directions with DF via `IP_PMTUDISC_PROBE` + binary search; MSS clamping (server reports SYN MSS seen); PMTU blackhole detection (large-DF timeout while small passes); v6 fragmentation behavior; DNS large-response/TCP-fallback test. |
| IDS/MITM detection | Good (with server) | Echo-comparison principle: server returns exactly what it received — HTTP request bytes, TLS ClientHello (JA4), DSCP/ECN, remaining TTL, source port. App diffs against what it sent. Catches transparent proxies, header injection/stripping, TLS interception (also: cert-chain pinning against out-of-band-fetched chain), DPI resets (SNI-based RST vs timeout), DSCP bleaching. ARP-spoof detection: not possible app-only, but feasible in the Shizuku tier via `ip neigh` polling (gateway MAC change detection). |
| DNS analysis + canary zone | Full | Own resolver implementation (UDP/TCP/DoT/DoH). User-configurable **override resolvers**: full dns.* battery runs once per resolver, in parallel, incl. TTL-integrity (clamping/freezing/serve-stale vs spec-defined reference records) and answer-integrity (RDATA/flags/order/ECS vs ground truth); `dns.compare` diffs all resolvers. Canary zone with unique per-run labels → authoritative server logs actual querying resolver, source AS, EDNS/0x20/DNSSEC behavior, minimization. Client side: provider vs own-server resolver diff, interception test (DNS query to a known non-DNS IP:53 — any answer = transparent redirect), NXDOMAIN wildcarding, DNSSEC validation (known-bad zone must SERVFAIL), rebind protection (does resolver strip answers pointing to RFC1918?), AAAA filtering, DNS64 synthesis correctness, latency/cache behavior. |

## 3. Additional test catalog (suggestions)

### 3.1 IPv6 & dual-stack
- Broken-v6 detection: v6 configured but blackholed (very common). Compare v4 vs v6 for every test; measure real Happy-Eyeballs behavior and fallback delay.
- NAT64/DNS64/464XLAT/CLAT detection and correctness.
- v6 privacy/temporary address usage; source-address selection sanity.

### 3.2 Path & routing
- Paris-style traceroute (fixed flow tuple) v4+v6, UDP and ICMP modes; ECMP path variance by varying flow label/ports.
- Remaining-TTL analysis at server → hop-count asymmetry, transparent-proxy hint (TTL reset).
- Egress identity: public IP, reverse DNS, ASN/geo (server-side lookup), consistency across protocols.

### 3.3 Transport & middleboxes
- ECN negotiation and survival; DSCP bleaching/remarking (server echo).
- Outbound port/protocol sweep: 25, 465, 587, 110/143/993, 22, 53-TCP/UDP, 123, 445, 3389, 1194, 500/4500, 51820, 8080/8443, QUIC 443/UDP. Distinguish RST vs timeout vs proxy-intercepted.
- "Is UDP usable at all" (hotel/guest networks); QUIC vs TCP-443 performance delta.
- Rate-limit/shaping detection: ramp test, token-bucket signature; per-port throttling comparison.
- IPv4 fragmentation and v6 fragment-header survival (server sends fragmented; client reports).

### 3.4 Performance
- Throughput (multi-stream TCP + paced UDP), both directions.
- Bufferbloat: latency under load, RPM-style / Waveform-style grade.
- Idle→active latency step on cellular (RRC state transitions).
- Gateway RTT vs first-external-hop RTT vs server RTT → segment isolation ("is it my Wi-Fi, my router, or my ISP").

### 3.5 Wi-Fi layer (within API limits)
- RSSI/link-speed/channel/band/standard logging over time; roaming log (BSSID transitions, gap duration) in a walk-around mode.
- Same-SSID environment scan: BSSIDs, bands, security types; security-downgrade and evil-twin heuristics (same SSID, different security).
- Channel congestion snapshot from scan results (co-channel BSS count).

### 3.6 Gateway & LAN services
- UPnP-IGD / NAT-PMP / PCP probe: discover, query external IP, attempt (and immediately release) a mapping — capability report + security finding.
- mDNS/SSDP/LLMNR device inventory; duplicate names, foreign-subnet announcements.
- Captive-portal analysis: RFC 8908/8910 API data + multi-URL probe matrix (HTTP/HTTPS/DNS behavior while captive).
- NTP: offset/latency to pool + own server; is 123/UDP intercepted?

### 3.7 Peer mode (two or more devices running the app)
Covers what no server can see: AP client isolation, LAN loss/jitter/throughput, multicast/IGMP delivery, cross-VLAN reachability, band-steering comparison (one device per band), roaming behavior diffing, and "do we even see the same network" (diff both devices' LinkProperties and leases — helps with the multi-DHCP question).

Design: **control plane over BLE GATT** (out-of-band — coordinating over the medium under test is self-defeating; when Wi-Fi is broken is when coordination matters most). Small CBOR messages: orchestration, result summaries. Data plane over the network under test; bulk result exchange syncs later over the network or the server. Third rendezvous path via the server when both devices have cellular. Clock sync for one-way delay: offset handshake over the LAN when usable (BLE jitter too high for precise sync — coarse coordination only). >2 devices: star topology, one coordinator. Permissions: BLUETOOTH_SCAN/CONNECT/ADVERTISE (Android 12+) with `neverForLocation`.

### 3.8 Modes & UX features
- Continuous monitor mode: lightweight periodic checks, outage/change timeline ("what broke at 03:12").
- Diff view: compare any two archived measurements (before/after, Wi-Fi vs cellular vs USB-eth, site A vs site B).
- Simultaneous multi-path run: same battery over Wi-Fi + cellular + ethernet at once via per-network socket binding.
- Scheduled runs via WorkManager (battery-honest).

### 3.9 Later (v1.5/v2)
- VpnService capture module: own-traffic pcap, per-app flows, DNS-leak analysis. Sees only routed L3 app traffic — no DHCP/ARP/ND/RA. Conflicts with real VPNs → separate opt-in.
- Root module: real DHCP prober (all options, multiple-offer capture), RA sniffer (flags, lifetimes, RDNSS), ARP/ND table + spoof detection, monitor-mode Wi-Fi where hardware allows, ESP/GRE reachability.
- Anonymized export (strip/pseudonymize addresses, SSIDs, hostnames consistently within a report).

## 4. Server-side design sketch

One self-contained daemon (Go or Rust; open-source alongside the app), deployable on multiple VMs/locations. Needs at least one host with **two public IPv4 addresses** (RFC 5780 STUN) and native v6.

Components: control API (TLS, versioned, session tokens; sessions correlate all probes of one run); UDP echo/pattern responder with receive-timestamps and observed-header report (TTL, DSCP, ECN, source port, size); TCP endpoints (MSS/window report, connect-back tester); STUN (RFC 5780); authoritative DNS for the canary zone with query logging exposed through the control API; recursive resolver for comparison; HTTPS echo endpoint (returns exact request bytes + ClientHello JA4 + observed TLS params) and reference cert-chain endpoint; large/fragmented-packet senders; NTP responder optional.

Abuse controls from day one (it is an F/OSS-published traffic reflector): session-token gating, per-IP rate limits, no amplification (responses ≤ requests unless session-authenticated), connect-back only to the session's source address.

### 4.1 Self-hosting, enrollment & profiles (decided direction)

Anyone can deploy the server in their own infrastructure. Configuration of the app happens via a **bootstrap QR / paste-string** — same payload, two transports:

```
echolot://enroll?v=1&u=<control-URL>&p=pin-sha256:<SPKI-hash>&t=<enrollment-token>
```

- QR carries only: control endpoint, server SPKI pin, one-time enrollment token. App verifies TLS against the **pin, not CAs** (self-signed homelab certs work), presents token, downloads the full profile: probe targets per location, STUN addresses, canary zone, long-lived credential, `capabilities` list.
- Trust anchor is transferred out-of-band (camera↔screen) → the reference channel is immune to CA-level/on-path interference — exactly the property the MITM/tampering tests need. Enrollment doubles as trust bootstrap.
- Profile is refreshed from the control server on each run → server-side updates without re-scanning; multi-location = one control server handing out all targets, or multiple independent profiles aggregated in the app.
- Capability negotiation via the profile's `capabilities` list (`stun-5780`, `canary-dns`, `connect-back`, `fragmented-send`, …) — third parties will run mismatched server versions forever; degrade gracefully per capability, not per version number.
- App side: CameraX + **ZXing** for scanning (NOT ML Kit — proprietary, would disqualify from F-Droid main repo). Custom URI scheme registered so any scanner app opens the enrollment.
- Server side: same single daemon serves a small admin web UI (localhost or separate admin port, admin password set at install): health/self-test, enrollment-token management (expiry, scopes enroll vs run-tests, per-token rate limits), QR rendered client-side in JS. Reinforces Go single-static-binary + `embed` for the web UI; cross-compile for ARM (VPSes, RPis).

## 5. App architecture notes

- **Decided: Kotlin + coroutines, Jetpack Compose UI. No Flutter** — all differentiating functionality is platform-level (Os sockets, Network callbacks, VpnService, NsdManager, Shizuku Binder, foreground services); Flutter would mean platform channels around a full Kotlin app plus F-Droid build friction.
- Module layout: `core-probe` (measurement engine, no UI deps → JUnit/CLI-testable, desktop-portable), `core-shizuku` (privileged tier, version-tagged parsers), `core-server` (protocol client), `core-peer` (BLE control plane + peer tests), `feature-*` per test family, thin `app` (Compose).
- Sockets via `android.system.Os` for ~95%; plan a small C-over-JNI shim (a few hundred lines, deliberately not Rust — lower contributor bar) for the corners with patchy `Os` coverage: `MSG_ERRQUEUE` reads, cmsg parsing, `SO_TIMESTAMPING`.
- Storage: Room; measurement = schema-versioned JSON (or CBOR) containing raw evidence (every packet summary, every reply, timestamps) + derived findings + per-category verdicts. Export via `ACTION_SEND`/SAF: full JSON + rendered human-readable report (HTML/PDF).
- Scoring: per-category traffic lights that expand into findings, each with severity, plain-language explanation, and raw evidence links. Engineers distrust opaque scores — always show the reasoning chain.
- Permissions: INTERNET, ACCESS_NETWORK_STATE, ACCESS_WIFI_STATE, ACCESS_FINE_LOCATION (Wi-Fi identifiers/scans), CHANGE_WIFI_MULTICAST_STATE, POST_NOTIFICATIONS, FOREGROUND_SERVICE(+type). Explain the location permission prominently — it's the one users will question.

## 6. Open decisions

(Decided: app framework = Kotlin/Compose; Shizuku tier in v1; peer mode with BLE control plane.)

1. Server language/stack: leaning Go (single static binary, embedded admin UI, ARM cross-compile) — confirm vs Rust or composing existing components (e.g. coturn for STUN).
2. minSdk: 26 keeps old spare phones usable; 29+ simplifies (cleaner APIs, but scan throttling and /proc/net loss apply anyway). Suggest 26–28 floor, decide per API audit.
3. License: GPLv3 vs Apache-2.0 (server and app can differ).
4. Measurement format: JSON vs CBOR; whether to define the schema first (recommended — it drives both app and server).
5. Canary-zone domain + how much server-side logging to retain (privacy statement needed even for a debug tool).