Phase 5j: Prometheus /metrics endpoint

Adds a tiny axum-served HTTP endpoint that exposes the same
CacheMetrics counters that back GetMetrics + gossip, in Prometheus
text exposition format (v0.0.4). Enable per node by setting
`cluster.prom_bind` in the daemon config.

* metrics.rs: MetricsReply::to_prometheus() emits one HELP + TYPE +
  sample line per counter. started_unix is a gauge; everything else is
  a counter. Preallocates ~1 KiB so no reallocs mid-format.
* prom.rs (new): PromServer::bind spins up axum on a TcpListener,
  serves GET /metrics, returns 404 elsewhere. Graceful shutdown via
  oneshot channel; abort() variant for the sync-drop path in
  ClusterServices. Snapshots on every scrape (no cache) — Relaxed
  atomic loads are cheap enough that even 1 Hz is sub-microsecond.
* config.rs: new optional `cluster.prom_bind: SocketAddr` field.
  Default None means no server; typical value is 127.0.0.1:7702.
* services.rs: wires PromServer into ClusterServices when both a
  router and prom_bind exist. Warns (doesn't fail) if prom_bind is set
  without RPC — nothing would ever change on a scrape.

+8 tests:
- metrics: to_prometheus emits every counter with correct type; zeros
  still produce valid exposition (fresh daemon scrape)
- prom: content-type is text/plain; version=0.0.4; live updates
  between requests (no cache); unknown paths 404; shutdown stops
  serving
- services: end-to-end scrape returns router-driven counters;
  prom_addr() is None when unconfigured (no accidentally-leaked port)

Raw-TCP HTTP client in tests instead of pulling in reqwest — 30 lines
of tokio::net + string split for GET / read-to-close is small enough
to justify not adding a dep.

249 tests pass (+8 from Phase 5i). Pre-existing macOS `du -sb` failure
unchanged.
This commit is contained in:
Omar Sobh
2026-07-12 04:36:34 -07:00
parent 29be728089
commit 523b22f148
6 changed files with 665 additions and 108 deletions
+18 -10
View File
@@ -6,7 +6,10 @@ use std::path::{Path, PathBuf};
#[derive(Debug, Clone, Deserialize, Serialize, PartialEq)]
#[serde(rename_all = "lowercase")]
pub enum NodeRole { Primary, Secondary }
pub enum NodeRole {
Primary,
Secondary,
}
#[derive(Debug, Clone, Deserialize, Serialize)]
pub struct NodeConfig {
@@ -143,6 +146,14 @@ pub struct ClusterConfig {
/// gossip + peer-status but returns `NotConfigured` for Blob RPCs.
#[serde(default)]
pub blob_store_root: Option<PathBuf>,
/// Phase 5j: optional bind address for the Prometheus `/metrics`
/// endpoint. When set, the daemon spins up a tiny HTTP server on
/// this address serving the same counters exposed via `GetMetrics`
/// and gossip. Typical value is `127.0.0.1:7702` (Prometheus scrapes
/// via the LAN listener the node advertises to its scrape target
/// group). Absent means "no scrape endpoint".
#[serde(default)]
pub prom_bind: Option<SocketAddr>,
}
/// Compute the default RPC address for a gossip address: same IP, port + 1.
@@ -182,7 +193,8 @@ impl ClusterConfig {
/// explicit `bind_rpc_lan` override; otherwise derives from `bind_lan`
/// with port + 1.
pub fn rpc_lan(&self) -> Option<SocketAddr> {
self.bind_rpc_lan.or_else(|| self.bind_lan.and_then(default_rpc_addr))
self.bind_rpc_lan
.or_else(|| self.bind_lan.and_then(default_rpc_addr))
}
/// The Tailscale address this node listens on for RPC (QUIC).
@@ -367,10 +379,7 @@ tailscale_addr = "100.64.1.5:7701"
);
let laptop = cluster.peer("laptop").expect("laptop peer");
assert!(
laptop.lan_addr.is_none(),
"roaming peer has no LAN address"
);
assert!(laptop.lan_addr.is_none(), "roaming peer has no LAN address");
assert!(laptop.tailscale_addr.is_some());
cluster.validate().expect("valid cluster config");
@@ -402,12 +411,10 @@ tailscale_addr = "100.64.1.5:7701"
bind_rpc_tailscale: None,
tls: None,
blob_store_root: None,
prom_bind: None,
};
let err = cluster.validate().unwrap_err().to_string();
assert!(
err.contains("no bind address"),
"unexpected error: {err}"
);
assert!(err.contains("no bind address"), "unexpected error: {err}");
}
#[test]
@@ -434,6 +441,7 @@ tailscale_addr = "100.64.1.5:7701"
bind_rpc_tailscale: None,
tls: None,
blob_store_root: None,
prom_bind: None,
};
let err = cluster.validate().unwrap_err().to_string();
assert!(