server: reserve measurement addresses; serve the UI on both families

fmr keeps .151/::151 for measurement. Their diagnostic value is entirely
in their listening state being known: a TLS handshake that completes on a
port nothing listens on proves interception, with no competing
explanation. One stray bind turns that proof into a shrug, and nothing
about the failure is visible — the run still says the network is clean.

CheckReserved refuses to start when a listener would take one. Wildcards
are refused outright, because that is how this actually happens: every
listener defaults to ":port" and the next one added gets copied from an
existing default, claiming every address without anyone deciding to.

Reserved is not silent, though. The first version of the guard would have
refused the live config's UDP and canary-DNS binds on .151, which are
deliberate — as is STUN's RFC 5780 alternate. Reserving an address and
then forbidding the measurements that need it defeats the purpose. The
rule is narrower: no services, and never ports 80 or 443.

The adb-beacon receiver was wildcard-bound to 0.0.0.0:443, holding port
443 on every IPv4 address including the reserved one, so the IPv4
interception test had been compromised for as long as it had run. It is
disabled; restore with systemctl enable --now echolot-adb-beacon. This
also marks the guard's limit: it governs this server's listeners, and a
process outside its config can still pollute a reserved address.

The admin UI and ACME responder were single-address, which is why the UI
could only live on ::2 and why the server was reachable over IPv6 alone —
the thing that made it look nonexistent from a phone without working
IPv6. Both now take address lists like every other listener.

Verified from outside: .150/::150/::2 answer on 443 with a valid cert for
fmr.echo-lot.app, .151/::151 are closed on 80 and 443, and canary DNS is
still up on .151.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
mrambossek
2026-08-01 22:17:29 +02:00
co-authored by Claude Opus 5
parent d9ee8bc2ae
commit 5b5a54db7d
5 changed files with 392 additions and 15 deletions
+36
View File
@@ -1127,3 +1127,39 @@ active"), to measure the tunnel itself as the network under test, or both. Not y
Related: `icmp.ping6` now records `attempted` alongside `ok` per network, because collapsing them
made the app report "IPv6 is configured, but ICMPv6 gets no reply" about an interface it had never
succeeded in sending on — a claim about the user's carrier with no evidence behind it.
## Reserved measurement addresses, and the web UI on both families (2026-08-01)
fmr has two IPv4 (.150/.151) and three IPv6 (::150/::151/::2) addresses. `.150`/`::150` now carry
the services; `.151`/`::151` are reserved for measurement, declared in `ECHOLOT_RESERVED_ADDRS`.
Reserved does **not** mean silent. The UDP data plane, the canary DNS and STUN's RFC 5780 alternate
all belong there — reserving an address and then forbidding the measurements that need it would
defeat the purpose. What must never appear is a service, and above all not ports 80 or 443: a
handshake completing on a port known not to be listening is what proves interception, and that
proof survives exactly as long as nothing binds those ports. `config.CheckReserved` enforces it at
startup, refusing wildcard binds outright (every listener defaults to `:port`, so the next one added
will claim reserved addresses without anyone deciding to).
The first version of the guard was too strict and the live config caught it: it would have refused
the existing UDP and DNS binds on `.151`. The rule is about services and web ports, not about
listening at all.
**The adb-beacon receiver was wildcard-bound to `0.0.0.0:443`**, occupying port 443 on every IPv4
address including the reserved one — so the IPv4 interception test had been compromised for as long
as it had been running, silently. It is now `systemctl disable --now echolot-adb-beacon`; restore
with `systemctl enable --now`. Note what this implies: the guard covers this server's own listeners,
and a stray process outside its config can still pollute a reserved address. A startup probe that
*verifies* 80/443 are actually free on the reserved addresses would be a stronger guarantee than
checking our own configuration, and is not yet built.
The admin UI and the ACME responder now take comma-separated addresses like every other listener;
they were single-address, which is why the UI could only ever live on `::2`. It serves on
`.150:443`, `[::150]:443` and `[::2]:443` — `::2` retained until `fmr.echo-lot.app` becomes a CNAME
to `fmr-1` (`.150`/`::150`), since dropping it first would break both the UI and ACME renewal for
the very name the certificate is issued to. sshd likewise now listens on `.150`, `::150` **and**
`::2`, added rather than moved for the same reason.
The point of all this: `fmr.echo-lot.app` gains an A record, so the server stops being reachable
only over IPv6 — which is what made it unreachable from a phone with no working IPv6, presenting as
"this host does not exist" in two different browsers.
+51 -15
View File
@@ -325,7 +325,17 @@ func serve(cfg *config.Config) error {
}
admin := ui.Handler()
adminSrv := &http.Server{Addr: cfg.AdminListen, Handler: admin, ReadHeaderTimeout: 10 * time.Second}
// One listener per configured address, all serving the same handler.
//
// Multi-address rather than a wildcard because this host reserves addresses for measurement:
// binding 0.0.0.0 would put the admin UI on port 443 of the reserved pair, and their value
// comes precisely from nothing answering there. Explicit addresses are also what let the
// service and management addresses differ without a second process.
adminAddrs := config.Addrs(cfg.AdminListen)
if len(adminAddrs) == 0 {
return fmt.Errorf("admin: no listen address configured")
}
var adminTLS *tls.Config
if cfg.AdminTLSCert != "" {
// Terminated here rather than behind a reverse proxy: this binary already serves TLS for
// the control plane, so it is reuse rather than new machinery, and one process with one
@@ -335,16 +345,33 @@ func serve(cfg *config.Config) error {
if err != nil {
return fmt.Errorf("admin TLS: %w", err)
}
adminSrv.TLSConfig = reloader.TLSConfig()
adminTLS = reloader.TLSConfig()
if exp := reloader.NotAfter(); !exp.IsZero() {
slog.Info("admin UI TLS", "listen", cfg.AdminListen, "cert_expires", exp.Format(time.RFC3339))
slog.Info("admin UI TLS", "listen", adminAddrs, "cert_expires", exp.Format(time.RFC3339))
if time.Until(exp) < 14*24*time.Hour {
slog.Warn("admin certificate expires soon", "expires", exp.Format(time.RFC3339))
}
}
go func() { errCh <- fmt.Errorf("admin: %w", adminSrv.ListenAndServeTLS("", "")) }()
} else {
go func() { errCh <- fmt.Errorf("admin: %w", adminSrv.ListenAndServe()) }()
}
// Kept for shutdown: each listener gets its own server, and a graceful stop has to reach all
// of them or an in-flight admin request is cut off mid-response on every address but one.
var adminSrvs []*http.Server
for _, addr := range adminAddrs {
// Bound before the goroutine starts, so a bad address fails startup rather than being
// reported asynchronously after the process has already declared itself healthy.
ln, err := net.Listen("tcp", addr)
if err != nil {
return fmt.Errorf("admin listen %s: %w", addr, err)
}
srv := &http.Server{Handler: admin, ReadHeaderTimeout: 10 * time.Second, TLSConfig: adminTLS}
adminSrvs = append(adminSrvs, srv)
go func(ln net.Listener, addr string) {
if adminTLS != nil {
errCh <- fmt.Errorf("admin %s: %w", addr, srv.ServeTLS(ln, "", ""))
return
}
errCh <- fmt.Errorf("admin %s: %w", addr, srv.Serve(ln))
}(ln, addr)
}
// ACME HTTP-01 responder. Permanent rather than started per renewal: nothing binds and
@@ -358,14 +385,21 @@ func serve(cfg *config.Config) error {
if err := acmehttp.EnsureWebroot(webroot); err != nil {
return fmt.Errorf("acme webroot: %w", err)
}
acmeSrv := &http.Server{
Addr: cfg.ACMEHTTPListen,
Handler: acmehttp.Handler(webroot, cfg.AdminBaseURL),
ReadHeaderTimeout: 10 * time.Second,
acmeHandler := acmehttp.Handler(webroot, cfg.AdminBaseURL)
for _, addr := range config.Addrs(cfg.ACMEHTTPListen) {
ln, err := net.Listen("tcp", addr)
if err != nil {
return fmt.Errorf("acme-http listen %s: %w", addr, err)
}
srv := &http.Server{Handler: acmeHandler, ReadHeaderTimeout: 10 * time.Second}
go func(ln net.Listener, addr string) {
errCh <- fmt.Errorf("acme-http %s: %w", addr, srv.Serve(ln))
}(ln, addr)
}
slog.Info("acme http-01 responder", "listen", cfg.ACMEHTTPListen, "webroot", webroot,
"redirects_to", cfg.AdminBaseURL)
go func() { errCh <- fmt.Errorf("acme-http: %w", acmeSrv.ListenAndServe()) }()
// Every address the name may resolve to needs the responder: the CA picks one, and a
// challenge that lands on an unbound address fails a renewal rather than a request.
slog.Info("acme http-01 responder", "listen", config.Addrs(cfg.ACMEHTTPListen),
"webroot", webroot, "redirects_to", cfg.AdminBaseURL)
}
// UDP data plane — one socket per configured address. Distinct sockets
@@ -457,7 +491,7 @@ func serve(cfg *config.Config) error {
}
slog.Info("listening",
"control", ctlAddrs, "admin", cfg.AdminListen, "udp", udpAddrs,
"control", ctlAddrs, "admin", adminAddrs, "udp", udpAddrs,
"tcp", config.Addrs(cfg.TCPListen), "stun", config.Addrs(cfg.StunListen),
"dns", config.Addrs(cfg.DNSListen), "capabilities", ctl.Capabilities)
@@ -467,7 +501,9 @@ func serve(cfg *config.Config) error {
shutCtx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
defer cancel()
_ = ctlSrv.Shutdown(shutCtx)
_ = adminSrv.Shutdown(shutCtx)
for _, srv := range adminSrvs {
_ = srv.Shutdown(shutCtx)
}
for _, c := range udpConns {
_ = c.Close()
}
+31
View File
@@ -41,6 +41,11 @@ type Config struct {
// Admin UI / health listener (spec §7: localhost-only by default)
AdminListen string // ECHOLOT_ADMIN_LISTEN / --admin-listen
// ReservedAddrs are IPs reserved for measurement: addresses whose listening state must stay
// known, so that "nothing answered on port 443" is a fact about the network rather than a
// fact about this server's configuration. Enforced by CheckReserved.
ReservedAddrs string // ECHOLOT_RESERVED_ADDRS / --reserved-addrs
// State directory: device store, generated TLS material.
StateDir string // ECHOLOT_STATE_DIR / --state-dir
@@ -162,6 +167,7 @@ func Load(args []string) (*Config, *Actions, error) {
fs.StringVar(&c.HTTPEchoListen, "http-echo-listen", envOr("HTTP_ECHO_LISTEN", ""), "optional CLEARTEXT http-echo listen address(es); empty disables (spec §4)")
fs.StringVar(&c.MTUProbeTargets, "mtu-probe-targets", envOr("MTU_PROBE_TARGETS", "1.1.1.1,2606:4700:4700::1111"), "egress-MTU self-proof anchors, comma-separated")
fs.StringVar(&c.AdminListen, "admin-listen", envOr("ADMIN_LISTEN", "127.0.0.1:8444"), "admin/health listen address (keep localhost)")
fs.StringVar(&c.ReservedAddrs, "reserved-addrs", envOr("RESERVED_ADDRS", ""), "comma-separated IPs reserved for measurement; no listener but the STUN alternate may bind them")
fs.StringVar(&c.StateDir, "state-dir", envOr("STATE_DIR", defaultStateDir()), "state directory (device store, generated TLS)")
fs.StringVar(&c.Name, "name", envOr("NAME", "echolot"), "server profile name")
fs.StringVar(&c.SelfUpdateAPI, "self-update-api", envOr("SELF_UPDATE_API", ""), "Gitea repo API base for self-update; empty disables")
@@ -216,6 +222,9 @@ func Load(args []string) (*Config, *Actions, error) {
if err := c.checkAdminExposure(); err != nil {
return nil, nil, err
}
if err := c.CheckReserved(c.Listeners()); err != nil {
return nil, nil, err
}
}
if c.Docker && (a.InstallSystemd || a.UninstallSystemd || a.SelfUpdate) {
return nil, nil, fmt.Errorf("systemd/self-update actions are native-mode only (container detected; override with ECHOLOT_DOCKER=0 if this is wrong)")
@@ -276,6 +285,28 @@ func (c *Config) checkAdminExposure() error {
return nil
}
// Listeners enumerates every configured listen spec, for CheckReserved.
//
// Kept as one list here rather than checked at each call site, so a listener added later is
// caught by the compiler when this function is updated — and, more to the point, so that the
// person adding one sees the reserved-address rule exists at all.
func (c *Config) Listeners() []Listener {
return []Listener{
// The instrument: these belong on the reserved addresses as much as anywhere.
{Name: "control-listen", Spec: c.ControlListen, Measurement: true},
{Name: "udp-listen", Spec: c.UDPListen, Measurement: true},
{Name: "tcp-listen", Spec: c.TCPListen, Measurement: true},
{Name: "dns-listen", Spec: c.DNSListen, Measurement: true},
{Name: "stun-listen", Spec: c.StunListen, Measurement: true},
// http-echo is deliberately not marked as measurement: it is cleartext HTTP, so on a
// reserved address it would be the very listener that ruins the port-80 test.
{Name: "http-echo-listen", Spec: c.HTTPEchoListen},
// Services. These have no business on an address kept for measuring.
{Name: "admin-listen", Spec: c.AdminListen},
{Name: "acme-http-listen", Spec: c.ACMEHTTPListen},
}
}
// Addrs splits a comma-separated listen spec into individual addresses.
// Explicit per-address binds matter on multi-IP hosts: a wildcard bind
// (":8443") would also claim addresses reserved for other purposes (e.g. an
+123
View File
@@ -0,0 +1,123 @@
// SPDX-FileCopyrightText: 2026 Echolot contributors
// SPDX-License-Identifier: GPL-3.0-or-later
package config
import (
"fmt"
"net"
"sort"
"strings"
)
// Listener is one configured listen spec, named for the error message.
type Listener struct {
Name string // the flag/env this came from, e.g. "control-listen"
Spec string // comma-separated listen addresses
// Measurement marks a listener that is part of the instrument rather than a service on the
// host. Those belong on the reserved addresses — STUN's RFC 5780 alternate, the UDP data
// plane, the canary DNS — and reserving an address only to forbid the measurements that need
// it would defeat the purpose.
Measurement bool
}
// webPorts are the ports whose closed state on a reserved address is itself the measurement.
//
// A TLS handshake that completes on a port known not to be listening proves interception, with no
// competing explanation. That proof is the whole reason for reserving an address, and it survives
// exactly as long as nothing binds these two ports there.
var webPorts = map[string]bool{"80": true, "443": true}
// CheckReserved refuses to start when a listener would occupy an address reserved for measurement.
//
// The reserved addresses are the instrument, not the service. Their diagnostic value comes from
// their listening state being *known*: if nothing listens on port 443 there, then a TLS handshake
// that completes proves something on the path intercepted it, with no other explanation available.
// One stray listener silently converts that proof into an ambiguity.
//
// This is a hard stop rather than a warning for the same reason as [Config.checkAdminExposure]: the
// failure is invisible. A polluted reserved address does not crash, log, or behave oddly — it just
// quietly turns a conclusive test into an inconclusive one, and the first symptom is a measurement
// that says the network is clean when it is not. Nobody reads a warning for that.
//
// Wildcard binds are the realistic way this happens. Every listener defaults to ":port", and the
// next one added will be copied from an existing default; that binds every address on the host,
// reserved ones included, without anyone deciding to.
func (c *Config) CheckReserved(listeners []Listener) error {
reserved := c.ReservedIPs()
if len(reserved) == 0 {
return nil
}
var problems []string
for _, l := range listeners {
for _, addr := range Addrs(l.Spec) {
host, port, err := net.SplitHostPort(addr)
if err != nil {
// Not host:port — a bare port or something malformed. Leave it to the listener
// itself to complain; guessing here would produce a confusing error about the
// wrong problem.
continue
}
host = strings.Trim(host, "[]")
if host == "" || host == "0.0.0.0" || host == "::" {
problems = append(problems, fmt.Sprintf(
" --%s=%q binds every address on this host, including the reserved ones",
l.Name, addr))
continue
}
ip := net.ParseIP(host)
if ip == nil {
continue // a hostname; cannot resolve it here without lying about what we checked
}
for _, r := range reserved {
if !ip.Equal(r) {
continue
}
switch {
case webPorts[port]:
problems = append(problems, fmt.Sprintf(
" --%s=%q puts port %s on reserved address %s, which is the one thing "+
"that address exists to keep closed", l.Name, addr, port, r))
case !l.Measurement:
problems = append(problems, fmt.Sprintf(
" --%s=%q binds reserved address %s; only measurement listeners belong there",
l.Name, addr, r))
}
}
}
}
if len(problems) == 0 {
return nil
}
sort.Strings(problems)
return fmt.Errorf(
"refusing to start: these listeners would occupy addresses reserved for measurement\n%s\n"+
"\nReserved: %s\n"+
"Those addresses are the instrument. A test can only prove interception on a port that\n"+
"is known not to be listening, so anything bound there destroys the conclusion rather\n"+
"than merely sharing the address.\n"+
" Fix it one of three ways:\n"+
" - bind each listener to explicit service addresses instead of a wildcard\n"+
" - remove the address from ECHOLOT_RESERVED_ADDRS if it is no longer reserved\n"+
" - unset ECHOLOT_RESERVED_ADDRS if this host has no reserved addresses",
strings.Join(problems, "\n"), joinIPs(reserved))
}
// ReservedIPs parses the configured reserved addresses, ignoring anything unparseable.
func (c *Config) ReservedIPs() []net.IP {
var out []net.IP
for _, s := range Addrs(c.ReservedAddrs) {
if ip := net.ParseIP(strings.Trim(s, "[]")); ip != nil {
out = append(out, ip)
}
}
return out
}
func joinIPs(ips []net.IP) string {
s := make([]string, 0, len(ips))
for _, ip := range ips {
s = append(s, ip.String())
}
return strings.Join(s, ", ")
}
+151
View File
@@ -0,0 +1,151 @@
// SPDX-FileCopyrightText: 2026 Echolot contributors
// SPDX-License-Identifier: GPL-3.0-or-later
package config
import (
"strings"
"testing"
)
const (
svc4 = "89.185.109.150"
res4 = "89.185.109.151"
res6 = "2001:1ad0:c4fe:6767::151"
)
func withReserved(l ...Listener) error {
c := &Config{ReservedAddrs: res4 + "," + res6}
return c.CheckReserved(l)
}
func TestWildcardBindIsRefused(t *testing.T) {
// The realistic failure: every listener defaults to ":port", and the next one added gets
// copied from an existing default. Nobody decides to claim the reserved address; it just
// happens, and nothing looks wrong afterwards.
err := withReserved(Listener{Name: "control-listen", Spec: ":8443"})
if err == nil {
t.Fatal("a wildcard bind was allowed while addresses were reserved")
}
if !strings.Contains(err.Error(), "binds every address") {
t.Fatalf("the error should say why a wildcard is the problem, got: %v", err)
}
}
func TestWebPortsOnReservedAreRefusedEvenForMeasurement(t *testing.T) {
// The strictest rule, and the one carrying the diagnostic value: 80 and 443 must stay closed
// on a reserved address whatever wants them, because their closed state *is* the measurement.
for _, spec := range []string{res4 + ":443", "[" + res6 + "]:80"} {
err := withReserved(Listener{Name: "control-listen", Spec: spec, Measurement: true})
if err == nil {
t.Fatalf("port 80/443 on a reserved address was allowed: %q", spec)
}
if !strings.Contains(err.Error(), "keep closed") {
t.Errorf("the error should explain what is lost, got: %v", err)
}
}
}
func TestServiceOnReservedIsRefused(t *testing.T) {
if err := withReserved(Listener{Name: "admin-listen", Spec: res4 + ":8444"}); err == nil {
t.Fatal("a service was allowed onto a reserved address")
}
}
func TestMeasurementListenersBelongOnReserved(t *testing.T) {
// The live fmr config: the UDP data plane, canary DNS and STUN all bind the reserved pair on
// purpose. A guard that refused this would be describing a rule nobody wants.
err := withReserved(
Listener{Name: "udp-listen", Spec: res4 + ":8442,[" + res6 + "]:8442", Measurement: true},
Listener{Name: "dns-listen", Spec: res4 + ":53,[" + res6 + "]:53", Measurement: true},
Listener{Name: "stun-listen", Spec: res4 + ":3478", Measurement: true},
)
if err != nil {
t.Fatalf("measurement listeners must be allowed on reserved addresses: %v", err)
}
}
func TestServiceAddressesAreFine(t *testing.T) {
err := withReserved(
Listener{Name: "control-listen", Spec: svc4 + ":8443,[2001:1ad0:c4fe:6767::150]:8443"},
Listener{Name: "admin-listen", Spec: "127.0.0.1:8444"},
)
if err != nil {
t.Fatalf("service addresses should be allowed: %v", err)
}
}
func TestHttpEchoIsNotTreatedAsMeasurement(t *testing.T) {
// http-echo is cleartext HTTP. On a reserved address it is precisely the listener that would
// ruin the port-80 test, so it does not get the measurement exemption.
if err := withReserved(Listener{Name: "http-echo-listen", Spec: res4 + ":8080"}); err == nil {
t.Fatal("http-echo was allowed onto a reserved address")
}
}
func TestNoReservationMeansNoOpinion(t *testing.T) {
// A host with nothing reserved must keep working exactly as before, wildcards included.
c := &Config{}
if err := c.CheckReserved([]Listener{{Name: "control-listen", Spec: ":8443"}}); err != nil {
t.Fatalf("with no reserved addresses this must not interfere: %v", err)
}
}
func TestEveryOffenderIsNamed(t *testing.T) {
// Reporting one problem at a time turns a config fix into several restart cycles, and on a
// remote host each cycle is a chance to lock yourself out.
err := withReserved(
Listener{Name: "control-listen", Spec: ":8443"},
Listener{Name: "tcp-listen", Spec: res4 + ":8441"},
)
if err == nil {
t.Fatal("expected a refusal")
}
for _, want := range []string{"control-listen", "tcp-listen"} {
if !strings.Contains(err.Error(), want) {
t.Errorf("the error should name %s; got: %v", want, err)
}
}
}
func TestHostnamesAreNotGuessedAt(t *testing.T) {
// Resolving here would check a name against whatever DNS says at startup, which is not
// necessarily what it will say later — and a guard that is sometimes right is worse than one
// with a stated limit.
if err := withReserved(Listener{Name: "control-listen", Spec: "fmr-1.echo-lot.app:8443"}); err != nil {
t.Fatalf("a hostname must be left alone, not resolved: %v", err)
}
}
func TestListenersCoversEverySpec(t *testing.T) {
// A listener missing from Listeners() is invisible to the guard, which is the one way this
// protection fails silently. Fill every spec with the reserved address: each one that is
// actually enumerated produces a complaint naming it.
// Every spec on port 443 of the reserved address: the web-port rule applies to measurement
// listeners too, so each one that is genuinely enumerated must produce a complaint.
c := &Config{
ReservedAddrs: res4,
ControlListen: res4 + ":443",
UDPListen: res4 + ":443",
TCPListen: res4 + ":443",
DNSListen: res4 + ":443",
HTTPEchoListen: res4 + ":443",
AdminListen: res4 + ":443",
ACMEHTTPListen: res4 + ":443",
StunListen: res4 + ":443",
}
err := c.CheckReserved(c.Listeners())
if err == nil {
t.Fatal("expected a refusal")
}
// Every spec is on the reserved address; the web-port rule catches even the measurement ones,
// so anything missing from Listeners() is invisible here and that is what this asserts.
for _, want := range []string{
"control-listen", "udp-listen", "tcp-listen", "dns-listen",
"http-echo-listen", "admin-listen", "acme-http-listen", "stun-listen",
} {
if !strings.Contains(err.Error(), want) {
t.Errorf("%s is not enumerated in Listeners(), so the guard cannot see it", want)
}
}
}