Phase 1c: QUIC transport with fleet-CA mTLS + cluster-ping
Wraps quinn 0.11 in cluster/transport.rs with rustls 0.23 + rcgen 0.13 for identity management. Every peer connection is mTLS: both sides must present certs signed by the shared fleet CA, and rustls verifies the peer's cert SAN matches the requested server name. Public API on `cluster::transport`: - `NodeIdentity` (cert chain + private key + trusted CA) - `NodeIdentity::generate_test_pair(a, b)` — ephemeral CA + two signed leaves; shape matches the production PEM-file loader (Phase 1d) - `QuicServer::bind(addr, identity)` — bind mTLS-enforced listener - `QuicServer::accept()` — accept one connection (Option<Result<_>>) - `QuicClient::new(local_addr, identity)` — build client endpoint - `QuicClient::connect(peer_addr, expected_name)` — outbound with SAN check - `ping(&conn, payload)` — bidi stream RPC; server echoes as `pong:<payload>` - `ping_handler_loop(conn)` — server-side accept/echo forever - `CLAWSTOR_RPC_ALPN` constant, single ALPN "clawstor-rpc/1" Config extension: - `bind_rpc_lan`, `bind_rpc_tailscale` on `ClusterConfig` (both optional). - Defaults: gossip port + 1 (so gossip UDP and QUIC UDP don't collide). - Helper: `ClusterConfig::rpc_lan()`, `rpc_tailscale()`, `PeerEntry::rpc_lan()`, `rpc_tailscale()`. Gossip.rs now advertises the RPC address, not the gossip address, on the well-known `clawstor.rpc.lan` / `clawstor.rpc.tailscale` keys. CLI: - `cluster-ping --name <me> --peer <you> --rpc-addr <addr> [--payload X]` Runs a full mTLS handshake and single ping. For dev/loopback use today (both sides need to share a CA); persistent-identity ping lands in Phase 1d. Tests (6 new, real UDP + TLS handshake, no mocks): - generate_test_pair produces two distinct leaves that share a CA - ping_pong_between_two_mtls_peers: real 2-node QUIC round trip with full mTLS chain verification, `open_bi()`/`accept_bi()`, byte-exact response check - client_rejects_peer_with_wrong_ca: TLS chain verification failure when the server presents a cert signed by a different CA - client_rejects_wrong_server_name: SAN mismatch is enforced - server_binds_wildcard_and_reports_concrete_local_addr: port 0 → real - ping_rejects_oversize_payload: MAX_MESSAGE_BYTES cap enforced client-side Fixed-port allocator range 42000+ so transport tests don't conflict with gossip tests (41000+). 72 tests pass. Pre-existing `hot::test_project_target_size_bytes` macOS-only failure unchanged. File sizes (all under 1300-line ceiling): - cluster/transport.rs: 485 - cluster/gossip.rs: 570 - cluster.rs: 275 - config.rs: 480 - main.rs: 564 Follow-on Phase 1 cut (1d): - Load NodeIdentity from persistent PEM files at /etc/claw-store/tls/ - Fleet CA bootstrap ceremony (rcgen → write CA cert; per-node leaf CSR) - Daemon-level RPC server that runs alongside gossip + serves real operations (blob get/put, metadata sync) - cluster-status reads from live daemon via API instead of standalone
This commit is contained in:
Omar Sobh
@@ -93,22 +93,39 @@ impl PeerEntry {
|
||||
/// Cluster membership configuration. Optional at the top level so existing
|
||||
/// single-node deployments (pre-v2) keep loading. Once present, describes the
|
||||
/// local node's zone + bind addresses, and enumerates known peers.
|
||||
#[derive(Debug, Clone, Deserialize, Serialize, PartialEq)]
|
||||
#[derive(Debug, Clone, Deserialize, Serialize, PartialEq, Default)]
|
||||
pub struct ClusterConfig {
|
||||
/// This node's zone tag.
|
||||
pub zone: String,
|
||||
/// LAN listen socket (typically `0.0.0.0:7701`). Omit on roaming nodes.
|
||||
/// LAN listen socket for gossip (typically `0.0.0.0:7701`). Chitchat runs
|
||||
/// on this UDP port. Omit on roaming nodes.
|
||||
#[serde(default)]
|
||||
pub bind_lan: Option<SocketAddr>,
|
||||
/// Tailscale listen socket (Tailscale IP + port). Omit on strictly-LAN nodes.
|
||||
/// Tailscale listen socket for gossip. Omit on strictly-LAN nodes.
|
||||
#[serde(default)]
|
||||
pub bind_tailscale: Option<SocketAddr>,
|
||||
/// LAN listen socket for RPC (QUIC). Defaults to `bind_lan.port + 1`
|
||||
/// so gossip and RPC don't collide on the same UDP endpoint.
|
||||
#[serde(default)]
|
||||
pub bind_rpc_lan: Option<SocketAddr>,
|
||||
/// Tailscale listen socket for RPC (QUIC). Defaults to
|
||||
/// `bind_tailscale.port + 1`.
|
||||
#[serde(default)]
|
||||
pub bind_rpc_tailscale: Option<SocketAddr>,
|
||||
/// Static seed list of peers. Runtime membership (Phase 1b) will extend this
|
||||
/// via gossip; the config list bootstraps discovery.
|
||||
#[serde(default)]
|
||||
pub peers: Vec<PeerEntry>,
|
||||
}
|
||||
|
||||
/// Compute the default RPC address for a gossip address: same IP, port + 1.
|
||||
/// Used to derive `bind_rpc_lan` / `bind_rpc_tailscale` when the operator
|
||||
/// hasn't set them explicitly.
|
||||
fn default_rpc_addr(gossip: SocketAddr) -> Option<SocketAddr> {
|
||||
let port = gossip.port().checked_add(1)?;
|
||||
Some(SocketAddr::new(gossip.ip(), port))
|
||||
}
|
||||
|
||||
impl ClusterConfig {
|
||||
/// Sanity check the cluster config: at least one bind address, no duplicate
|
||||
/// peer names, every peer has at least one address.
|
||||
@@ -133,6 +150,37 @@ impl ClusterConfig {
|
||||
pub fn peer(&self, name: &str) -> Option<&PeerEntry> {
|
||||
self.peers.iter().find(|p| p.name == name)
|
||||
}
|
||||
|
||||
/// The LAN address this node listens on for RPC (QUIC). Prefers the
|
||||
/// explicit `bind_rpc_lan` override; otherwise derives from `bind_lan`
|
||||
/// with port + 1.
|
||||
pub fn rpc_lan(&self) -> Option<SocketAddr> {
|
||||
self.bind_rpc_lan.or_else(|| self.bind_lan.and_then(default_rpc_addr))
|
||||
}
|
||||
|
||||
/// The Tailscale address this node listens on for RPC (QUIC).
|
||||
pub fn rpc_tailscale(&self) -> Option<SocketAddr> {
|
||||
self.bind_rpc_tailscale
|
||||
.or_else(|| self.bind_tailscale.and_then(default_rpc_addr))
|
||||
}
|
||||
}
|
||||
|
||||
impl PeerEntry {
|
||||
/// The peer's LAN RPC address, if it has one.
|
||||
///
|
||||
/// This is derived from the peer's `lan_addr` (which is their gossip
|
||||
/// address in the config, matching how they identify themselves). RPC
|
||||
/// runs on gossip port + 1 by convention. Gossip-based discovery in
|
||||
/// Phase 1b will supplant this once a peer has broadcast its own
|
||||
/// [`gossip::keys::RPC_ADDR_LAN`](crate::cluster::gossip::keys).
|
||||
pub fn rpc_lan(&self) -> Option<SocketAddr> {
|
||||
self.lan_addr.and_then(default_rpc_addr)
|
||||
}
|
||||
|
||||
/// The peer's Tailscale RPC address.
|
||||
pub fn rpc_tailscale(&self) -> Option<SocketAddr> {
|
||||
self.tailscale_addr.and_then(default_rpc_addr)
|
||||
}
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone, Deserialize, Serialize)]
|
||||
@@ -323,6 +371,8 @@ tailscale_addr = "100.64.1.5:7701"
|
||||
bind_lan: None,
|
||||
bind_tailscale: None,
|
||||
peers: vec![],
|
||||
bind_rpc_lan: None,
|
||||
bind_rpc_tailscale: None,
|
||||
};
|
||||
let err = cluster.validate().unwrap_err().to_string();
|
||||
assert!(
|
||||
@@ -351,6 +401,8 @@ tailscale_addr = "100.64.1.5:7701"
|
||||
tailscale_addr: None,
|
||||
},
|
||||
],
|
||||
bind_rpc_lan: None,
|
||||
bind_rpc_tailscale: None,
|
||||
};
|
||||
let err = cluster.validate().unwrap_err().to_string();
|
||||
assert!(
|
||||
|
||||
Reference in New Issue
Block a user