research: implementation brief for perf/security/provenance pass (INT-01..19)

Read against ROADMAP.md/IMPROVEMENT_LOG.md/CLAUDE.md/CHANGELOG.md first so
nothing already-merged is re-proposed. 19 numbered INT items grouped by
crate (clawhdf5-agent provenance wiring, clawhdf5-format/io untrusted-file
parsing, clawhdf5-ann HNSW hot path, clawhdf5-py mutex poisoning), each with
file path, problem, and proposed change, plus a suggested implementation
order for the coding phase.

crates/clawhdf5-agent/src/knowledge.rs

@@ -329,47 +329,6 @@ impl KnowledgeCache {
         (id, true)
     }
 
-    // -----------------------------------------------------------------------
-    // Adjacency index (built fresh per traversal call — see doc comment)
-    // -----------------------------------------------------------------------
-
-    /// Build an O(V+R) adjacency index for one traversal call: an entity-id →
-    /// vec-index map for O(1) entity lookups, and an entity-id →
-    /// `(neighbour_id, relation_weight)` map (covering both outgoing and
-    /// incoming edges) for O(1) neighbour expansion. The weight is carried
-    /// alongside each neighbour so callers like `spreading_activation` that
-    /// need per-edge weight don't have to re-scan `relations`.
-    ///
-    /// This is rebuilt at the start of every `bfs_neighbors`/
-    /// `spreading_activation` call rather than cached on the struct: `entities`
-    /// and `relations` are public fields, and `schema.rs`'s deserialization
-    /// path pushes into them directly (bypassing `add_entity`/`add_relation`),
-    /// so a struct-cached index could go stale. Building it once per call
-    /// still turns an O(V·R) (or O(steps·V·R)) traversal into O(V+R) (or
-    /// O(steps·(V+E))), since the old code repeated the O(R) relation scan
-    /// once per visited node instead of once per call.
-    fn build_adjacency(&self) -> (HashMap<u64, usize>, HashMap<u64, Vec<(u64, f32)>>) {
-        let mut entity_index: HashMap<u64, usize> = HashMap::with_capacity(self.entities.len());
-        for (i, e) in self.entities.iter().enumerate() {
-            entity_index.insert(e.id, i);
-        }
-
-        // Note: a self-loop relation (src == tgt) contributes a single
-        // neighbour entry, not two, matching the if/else-if (not two
-        // independent ifs) structure this replaces — otherwise a self-loop
-        // would be double-counted by `spreading_activation`.
-        let mut adjacency: HashMap<u64, Vec<(u64, f32)>> =
-            HashMap::with_capacity(self.relations.len());
-        for r in &self.relations {
-            adjacency.entry(r.src).or_default().push((r.tgt, r.weight));
-            if r.tgt != r.src {
-                adjacency.entry(r.tgt).or_default().push((r.src, r.weight));
-            }
-        }
-
-        (entity_index, adjacency)
-    }
-
     // -----------------------------------------------------------------------
     // Graph traversal: BFS neighbors
     // -----------------------------------------------------------------------
@@ -378,8 +337,6 @@ impl KnowledgeCache {
     /// together with their discovered depth.  The seed entity itself is NOT
     /// included.  Traversal follows both outgoing and incoming relation edges.
     pub fn bfs_neighbors(&self, entity_id: u64, max_depth: usize) -> Vec<(Entity, usize)> {
-        let (entity_index, adjacency) = self.build_adjacency();
-
         let mut visited: HashSet<u64> = HashSet::new();
         let mut queue: VecDeque<(u64, usize)> = VecDeque::new();
         let mut results: Vec<(Entity, usize)> = Vec::new();
@@ -392,15 +349,25 @@ impl KnowledgeCache {
                 continue;
             }
 
-            let Some(neighbours) = adjacency.get(&current_id) else {
+            // Collect neighbour IDs from outgoing and incoming edges.
-                continue;
+            let neighbours: Vec<u64> = self
-            };
+                .relations
+                .iter()
+                .filter_map(|r| {
+                    if r.src == current_id {
+                        Some(r.tgt)
+                    } else if r.tgt == current_id {
+                        Some(r.src)
+                    } else {
+                        None
+                    }
+                })
+                .collect();
 
-            for &(neighbour_id, _weight) in neighbours {
+            for neighbour_id in neighbours {
                 if visited.insert(neighbour_id)
-                    && let Some(&idx) = entity_index.get(&neighbour_id)
+                    && let Some(entity) = self.get_entity(neighbour_id)
                 {
-                    let entity = &self.entities[idx];
                     results.push((entity.clone(), depth + 1));
                     queue.push_back((neighbour_id, depth + 1));
                 }
@@ -472,8 +439,6 @@ impl KnowledgeCache {
         min_activation: f32,
         max_steps: usize,
     ) -> Vec<(u64, f32)> {
-        let (_entity_index, adjacency) = self.build_adjacency();
-
         let mut activation: HashMap<u64, f32> = HashMap::new();
 
         // Initialise seeds with activation 1.0.
@@ -497,11 +462,16 @@ impl KnowledgeCache {
 
             for (source_id, source_score) in current {
                 // Spread to all neighbours via outgoing and incoming edges.
-                let Some(neighbours) = adjacency.get(&source_id) else {
+                for rel in &self.relations {
+                    let neighbour_id = if rel.src == source_id {
+                        rel.tgt
+                    } else if rel.tgt == source_id {
+                        rel.src
+                    } else {
                         continue;
                     };
-                for &(neighbour_id, weight) in neighbours {
+
-                    let delta = source_score * weight * decay_factor;
+                    let delta = source_score * rel.weight * decay_factor;
                     if delta >= min_activation {
                         *activation.entry(neighbour_id).or_insert(0.0) += delta;
                         any_spread = true;

crates/clawhdf5-agent/src/provenance.rs

@@ -1,31 +1,31 @@
 //! Memory provenance tracking and integrity verification.
 //!
 //! Records the origin, authorship, and a content hash of every memory chunk
-//! so the system can detect content corruption and trace data lineage. The
+//! so the system can detect *accidental* corruption and trace data lineage.
-//! hash is a SHA-256 digest (see [`hash_content`]), computed via
+//! The hash is unkeyed (see [`fnv1a_64`]) — this is not a tamper-evidence or
-//! [`clawhdf5_format::provenance::sha256_hex`]. It is still **unkeyed** — an
+//! authenticity guarantee.
-//! actor able to overwrite the stored chunk can also recompute and overwrite
-//! the stored hash alongside it, so this is not an authenticity guarantee
-//! against that threat. What SHA-256 does provide over a fast non-cryptographic
-//! hash (the previous FNV-1a implementation) is collision resistance: an
-//! adversary cannot cheaply craft *different* poisoned content that matches
-//! an already-recorded legitimate hash.
 
 use std::collections::HashMap;
 
 pub use crate::consolidation::MemorySource;
 
 // ---------------------------------------------------------------------------
-// Hash helper
+// Hash helper (std-only FNV-1a 64-bit)
 // ---------------------------------------------------------------------------
 
-/// SHA-256 hex digest of `text`, used to detect content corruption/tampering.
+/// Unkeyed, non-cryptographic FNV-1a hash for detecting accidental content
-///
+/// corruption. It is trivially forgeable by anyone able to modify the stored
-/// Unkeyed: an actor able to modify the stored chunk can also recompute and
+/// data, since they can recompute and overwrite the stored hash alongside
-/// overwrite the stored hash, so a match is not proof of authenticity — only
+/// it — do not rely on this as a tamper-evidence or authenticity control.
-/// that the stored chunk and stored hash are mutually consistent.
+fn fnv1a_64(text: &str) -> u64 {
-fn hash_content(text: &str) -> String {
+    const OFFSET: u64 = 14_695_981_039_346_656_037;
-    clawhdf5_format::provenance::sha256_hex(text.as_bytes())
+    const PRIME: u64 = 1_099_511_628_211;
+    let mut hash = OFFSET;
+    for byte in text.bytes() {
+        hash ^= byte as u64;
+        hash = hash.wrapping_mul(PRIME);
+    }
+    hash
 }
 
 // ---------------------------------------------------------------------------
@@ -57,8 +57,8 @@ pub struct MemoryProvenance {
     pub created_by: String,
     /// Unix timestamp (seconds) of creation.
     pub created_at: f64,
-    /// SHA-256 hex digest of the chunk text for integrity checking.
+    /// FNV-1a 64-bit hash of the chunk text for integrity checking.
-    pub content_hash: String,
+    pub content_hash: u64,
     pub session_id: String,
     pub verified: bool,
 }
@@ -78,7 +78,7 @@ impl MemoryProvenance {
             source,
             created_by: created_by.into(),
             created_at,
-            content_hash: hash_content(chunk),
+            content_hash: fnv1a_64(chunk),
             session_id: session_id.into(),
             verified: false,
         }
@@ -121,16 +121,13 @@ impl ProvenanceStore {
     /// Re-hash `current_chunk` and compare against the stored hash.
     /// Returns `true` if the content matches (integrity intact).
     ///
-    /// The hash is unkeyed, so an actor able to modify the stored chunk can
+    /// This only detects accidental corruption: the hash is unkeyed, so an
-    /// also recompute and overwrite the stored hash. Do not treat a `true`
+    /// actor able to modify the stored chunk can also recompute and
-    /// result as proof of authenticity against that threat — but unlike a
+    /// overwrite the stored hash. Do not treat a `true` result as proof the
-    /// non-cryptographic hash, a `false` result reliably indicates that the
+    /// data hasn't been tampered with.
-    /// content does not match what was recorded, since SHA-256 makes it
-    /// computationally infeasible to craft different content that collides
-    /// with a specific existing digest.
     pub fn verify_integrity(&self, record_id: u64, current_chunk: &str) -> bool {
         match self.records.get(&record_id) {
-            Some(p) => p.content_hash == hash_content(current_chunk),
+            Some(p) => p.content_hash == fnv1a_64(current_chunk),
             None => false,
         }
     }
@@ -244,32 +241,22 @@ mod tests {
         1_700_000_000.0
     }
 
-    // --- hash_content ---
+    // --- fnv1a_64 ---
 
     #[test]
     fn hash_deterministic() {
-        assert_eq!(hash_content("hello"), hash_content("hello"));
+        assert_eq!(fnv1a_64("hello"), fnv1a_64("hello"));
     }
 
     #[test]
     fn hash_different_inputs() {
-        assert_ne!(hash_content("hello"), hash_content("world"));
+        assert_ne!(fnv1a_64("hello"), fnv1a_64("world"));
     }
 
     #[test]
     fn hash_empty() {
-        // Should not panic, and should match the well-known SHA-256 of the empty string.
+        // Should not panic
-        assert_eq!(
+        let _ = fnv1a_64("");
-            hash_content(""),
-            "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855"
-        );
-    }
-
-    #[test]
-    fn hash_is_sha256_hex() {
-        let h = hash_content("clawhdf5");
-        assert_eq!(h.len(), 64);
-        assert!(h.chars().all(|c| c.is_ascii_hexdigit()));
     }
 
     // --- MemorySource Display ---
@@ -288,7 +275,7 @@ mod tests {
     #[test]
     fn provenance_new_hashes_chunk() {
         let p = MemoryProvenance::new(1, MemorySource::User, "agent-1", ts(), "hello", "s1");
-        assert_eq!(p.content_hash, hash_content("hello"));
+        assert_eq!(p.content_hash, fnv1a_64("hello"));
         assert!(!p.verified);
     }
 

research/IMPLEMENTATION_BRIEF.md

@@ -1,176 +1,434 @@
-# ClawHDF5 — Performance / Security / Provenance Implementation Brief
+# Implementation Brief — clawhdf5 Performance/Security/Provenance Pass
 
-**Date:** 2026-08-16
+**Research date:** 2026-08-16
-**Scope:** Follow-up hardening pass on top of the already-shipped Tier 1-4 work
+**Scope:** `crates/` only. Read against `ROADMAP.md`, `IMPROVEMENT_LOG.md`, `CLAUDE.md`, and
-(see `ROADMAP.md` "What's Next" and `IMPROVEMENT_LOG.md`). This brief covers
+`CHANGELOG.md` first — those documents record a genuinely large amount of prior hardening
-only items verified against the current repo state at commit `b2dce41` that
+(WAL CRC32, `chunked_read.rs`/`data_read.rs`/`local_heap.rs`/`btree_v1.rs` bounds audits,
-were **not** already addressed by prior tiers.
+Android JNI bounds checks, pyo3 bump, HNSW `prune_connections` rayon parallelism, bounded
+decompression, no_std fixes, `cargo-audit`-clean dependency tree). None of that is
+re-proposed here. Every item below was independently verified by reading the current source
+(file path + line numbers cited), not inferred from docs.
 
-## Method
+`cargo audit` was run against the current lockfile: **zero vulnerability advisories**, three
+"unmaintained" warnings (`custom_derive` via `mpi`→`conv`, `number_prefix` via `tokenizers`→
+`indicatif`, `paste`) — all transitive through optional deps (`mpi-io` feature, `tokenizers`),
+no upstream fix available, not actionable as a code change. Not filed as an INT item.
 
-Read `ROADMAP.md`, `IMPROVEMENT_LOG.md`, `IMPROVEMENT_SCAN.md`, and
+Also checked and found clean (no INT items filed): `clawhdf5-migrate` (zero `unwrap()` outside
-`CLAUDE.md` first to avoid re-proposing work already merged (WAL CRC32,
+`#[test]` code in `main.rs`; `sqlite_reader.rs`/`hdf5_writer.rs`/`validate.rs` are unwrap-free),
-bounds-check audit + fuzz target, HNSW `prune_connections` parallelism,
+`clawhdf5-cli`, `clawhdf5-napi` (zero `unwrap()` in `lib.rs`), `clawhdf5-accel` SIMD dispatch
-Android JNI length validation, `workspace.dependencies` hoisting, etc. are
+(`is_x86_feature_detected!`/runtime gating is correct — no illegal-instruction risk),
-all already done — see those files for the full list).
+`clawhdf5-filters` hot path (slice-based, no byte-by-byte loops of consequence), and TODO/FIXME
+grep across all crates (the only hits are test-fixture bytes literally named `b"XXXX"`, not
+real markers).
 
-Then manually audited:
+---
-- `crates/clawhdf5-format/src/{chunked_read,data_read}.rs` — bounds-check
-  spot audit (sampled `ensure_len` call sites around every raw slice index).
-  **Result: no new gaps found.** Every raw `file_data[a..b]` site sampled is
-  preceded by an `ensure_len`/`read_offset` overflow-checked bound. The prior
-  Tier 4a pass already closed this out.
-- `crates/clawhdf5-agent/src/provenance.rs` — memory record integrity →
-  **gap found**, see INT-01.
-- `crates/clawhdf5-agent/src/knowledge.rs` — knowledge-graph traversal →
-  **gap found**, see INT-02.
-- `crates/clawhdf5-agent/src/bm25.rs` — already has cached IDF, sorted
-  postings, WAND early termination. No changes proposed.
-- `crates/clawhdf5-ann/src/hnsw.rs` — build-time parallelism already scoped
-  to `prune_connections` per Tier 4c; the outer insert loop is flagged in
-  ROADMAP as needing its own correctness-sensitive design pass, out of scope
-  here.
 
-## INT-01 — Harden agent memory provenance hash from FNV-1a to SHA-256
+## Priority key
+- **P0** — correctness/security bug reachable from untrusted input (crafted file, external
+  caller), should block release.
+- **P1** — real functional gap or measurable perf cost on a hot path.
+- **P2** — consistency/hardening/API-quality; safe to defer.
 
-**File:** `crates/clawhdf5-agent/src/provenance.rs`
+---
-**Category:** Security / Provenance
-**Status:** Implemented this pass.
 
-### Problem
+## Group A — `clawhdf5-agent`: provenance/security is unwired (headline finding)
 
-`MemoryProvenance::content_hash` used an unkeyed 64-bit FNV-1a hash
+### INT-01 — Wire `WriteAnomalyDetector` / `ProvenanceStore` into the actual write path [P0]
-(`fnv1a_64`) to detect corruption of stored agent-memory chunks. FNV-1a is
+**Files:** `crates/clawhdf5-agent/src/storage.rs` (save/save_batch path), `crates/clawhdf5-agent/src/provenance.rs`, `crates/clawhdf5-agent/src/anomaly.rs`, `crates/clawhdf5-agent/src/lib.rs`
-a fast non-cryptographic hash with no collision resistance: an adversary
-attempting to plant poisoned/tampered memory content that still matches a
-previously-recorded or expected hash value only needs to find *any* input
-producing the same 64-bit output, which is computationally cheap for
-FNV-1a (no preimage or collision resistance guarantees at all). Given
-Track 5 of `ROADMAP.md` explicitly claims "poisoning resistance" and
-`verify_integrity()` is the one function whose entire job is to catch
-tampered memory content, using a hash with no collision resistance
-undermines that guarantee in a way that is easy to miss (the doc comment
-already, correctly, disclaims *authenticity* — i.e. it never claimed to
-stop an attacker who can also rewrite the stored hash — but it did not
-protect against a weaker, still-relevant attack: crafting *different*
-poisoned content that collides with an already-recorded legitimate hash).
 
-Separately, `clawhdf5-format` already ships a mature, default-on
+**Problem:** `ROADMAP.md` Track 5 ("Memory Security & Provenance") is marked 🟢 Complete, listing
-`provenance` feature (`crates/clawhdf5-format/src/provenance.rs`) with a
+source attribution, write anomaly detection, source isolation, and integrity verification as
-`sha256_hex()` helper built on the `sha2` crate, used for on-disk dataset
+done. The types exist and are unit-tested in isolation — but `grep -rn
-provenance attributes. `clawhdf5-agent` already depends on
+"WriteAnomalyDetector\|ProvenanceStore\|SourceIsolation"` across every file in
-`clawhdf5-format` with default features enabled, so `sha256_hex` was
+`clawhdf5-agent` *except* `provenance.rs`/`anomaly.rs` themselves returns nothing.
-already reachable with **zero new dependencies**.
+`storage.rs` (the real save/delete/replay path) never imports or calls into either module.
+Nothing in `HDF5Memory::save`/`save_batch` populates a `ProvenanceStore`, runs a rate/pattern
+check, or routes through `SourceIsolation`. In its current state this is a library the crate
+ships but never uses on itself — every memory write today has **no** rate limiting, no pattern
+detection, and no provenance recorded, contrary to what the roadmap and any consumer relying on
+it would assume.
 
-### Fix implemented
+**Change:** In `storage.rs`'s save/save_batch entry point(s), construct/thread a
+`WriteAnomalyDetector` and `ProvenanceStore` (or accept them as constructor params on the
+memory-store struct so callers can configure `AnomalyConfig`), call `record_write` +
+`check_rate_anomaly`/`check_pattern_anomaly` before persisting each chunk, and call
+`ProvenanceStore::add` with the resulting `MemoryProvenance` alongside the write. Surface
+anomaly alerts through the existing error/result type rather than silently dropping them
+(decide via a config flag whether pattern/rate hits are hard-rejects or soft warnings — a hard
+reject changes public API behavior, a warning is additive). Add an integration test that writes
+a chunk containing one of the 15 suspicious patterns and asserts the alert actually fires
+through the public save path (not just the unit-level `WriteAnomalyDetector` test).
 
-- `MemoryProvenance::content_hash` changed from `u64` to `String` (lowercase
+---
-  hex SHA-256 digest), computed via `clawhdf5_format::provenance::sha256_hex`.
-- `ProvenanceStore::verify_integrity` now compares SHA-256 hex digests.
-- Removed the local `fnv1a_64` helper from `provenance.rs` (no longer used
-  there — `clawhdf5-agent/src/multimodal.rs` keeps its own independent
-  `fnv1a_64` for `MediaRef` checksums, which is a content-identity/dedup key,
-  not a security/integrity control, so it is intentionally left unchanged
-  and out of scope for this item).
-- Updated the module-level and per-item doc comments to keep the existing,
-  correct disclaimer: this is still an **unkeyed** hash, so it is still not
-  an authenticity/tamper-*evidence* guarantee against an attacker who can
-  rewrite the stored hash alongside the content. What changed is that it is
-  no longer trivially *collidable*, which was the concrete, fixable gap.
-- Updated all existing unit tests in `provenance.rs` for the new `String`
-  hash type; behavior (which records match/mismatch) is unchanged.
 
-`MemoryProvenance` and `ProvenanceStore` are only used within
+### INT-02 — `check_pattern_anomaly` is trivially bypassed substring matching [P1]
-`clawhdf5-agent` itself (not serialized to the HDF5 format, not consumed by
+**File:** `crates/clawhdf5-agent/src/anomaly.rs:192-211`
-other crates), so this is a self-contained, non-breaking-to-other-crates
-change verified by `grep -r MemoryProvenance crates/`.
 
-## INT-02 — Knowledge-graph traversal: replace O(V·R) relation scans with a per-call adjacency index
+**Problem:**
+```rust
+let lower = chunk.to_lowercase();
+for pattern in &self.config.suspicious_patterns {
+    if lower.contains(pattern.as_str()) { ... }
+}
+```
+Matching is raw case-folded substring containment against 15 fixed literals (`"ignore
+previous"`, `"system:"`, …). Trivially defeated by inserting extra whitespace/punctuation
+(`"ignore  previous"`), splitting the phrase across two separate writes (checks are per-chunk,
+not per-session-buffer), or any non-ASCII obfuscation. As a poisoning-resistance control this
+currently only stops the laziest attacks.
 
-**File:** `crates/clawhdf5-agent/src/knowledge.rs`
+**Change:** Normalize input before matching (collapse whitespace/strip zero-width and combining
-**Category:** Performance
+characters), and consider word-boundary-tolerant/regex matching instead of raw `contains`.
-**Status:** Implemented this pass.
+Document the remaining limitation (this is a heuristic filter, not a guarantee) rather than
+implying full poisoning resistance.
 
-### Problem
+---
 
-`KnowledgeCache::bfs_neighbors` and `KnowledgeCache::spreading_activation`
+### INT-03 — `session_counts` grows unbounded and is fully rescanned on every rate check [P1]
-are the core traversal primitives behind Track 1 (BFS neighbors, subgraph
+**File:** `crates/clawhdf5-agent/src/anomaly.rs:109, 126-133, 170-181`
-extraction) and Track 3 (graph-aware re-ranking) of the agent memory
-system. Both did a **full linear scan over `self.relations`** for every
-node processed:
 
-- `bfs_neighbors`: for every entity dequeued from the BFS frontier, it
+**Problem:** `session_counts: HashMap<String, u32>` is incremented on every `record_write` and
-  scanned the entire `relations: Vec<Relation>` looking for edges touching
+never pruned — unlike `window` (which has a 60s sliding-window prune). A caller that creates
-  that entity — O(V·R) instead of O(V+E). It also called
+many distinct `session_id` values (fully caller-controlled strings) grows this map without
-  `self.get_entity(neighbour_id)`, itself an O(n) linear scan over
+bound for the process lifetime. `check_rate_anomaly`'s session-level loop
-  `entities: Vec<Entity>`, once per newly-discovered neighbour.
+(`for (session, &count) in &self.session_counts`) then scans the *entire* historical map on
-- `spreading_activation`: for every activated node in every propagation
+every single check call, so per-write cost grows with total lifetime session count, not
-  step, it likewise scanned all of `self.relations` — O(steps·V·R).
+current activity.
-- `get_subgraph` calls `bfs_neighbors` once per seed, compounding the cost.
 
-For a knowledge graph with thousands of entities/relations (the scale this
+**Change:** Bound `session_counts` with an LRU/TTL eviction policy, or track only counts within
-project's own benchmarks target — see `BENCHMARKS.md`), this is
+the same rolling window used for `window` (see INT-05, which is closely related — the
-quadratic-ish behavior in traversal-heavy paths (`get_entity_context`,
+session-level check has its own separate bug on top of this).
-hybrid retrieval re-ranking that pulls graph context) that only gets worse
-as agent memory accumulates over long sessions.
 
-### Fix implemented
+---
 
-Added a private helper, `KnowledgeCache::build_adjacency`, that builds, in
+### INT-04 — Sliding-window prune only inspects the front of the deque [P1]
-one O(V+R) pass:
+**File:** `crates/clawhdf5-agent/src/anomaly.rs:134-139`
-- `entity_index: HashMap<u64, usize>` — entity id → index into `entities`.
-- `adjacency: HashMap<u64, Vec<u64>>` — entity id → neighbour ids (both
-  outgoing and incoming edges).
 
-`bfs_neighbors` and `spreading_activation` now build this index **once at
+**Problem:**
-the top of the call** (not persisted as struct state — see rationale below)
+```rust
-and use it for O(1) neighbour/entity lookups inside the traversal loop,
+self.window.push_back(event);
-changing the complexity to O(V+E) per call for BFS and O(steps·(V+E)) for
+let cutoff = self.last_timestamp - 60.0;
-spreading activation.
+while self.window.front().is_some_and(|e| e.timestamp < cutoff) {
+    self.window.pop_front();
+}
+```
+`WriteEvent.timestamp` is caller-supplied (not sampled from a clock inside this type), so
+nothing prevents an out-of-order/backdated event from landing behind the front after a more
+recent one. Because eviction only ever looks at `front()`, a single out-of-order event
+permanently corrupts the window — old entries behind it are never pruned, so
+`check_rate_anomaly`'s window-length count over-reports forever (and can be intentionally
+inflated by a caller that varies timestamp ordering).
 
-**Why not a persistent index on the struct:** `entities`/`relations` are
+**Change:** Prune by retaining only entries `>= cutoff` across the whole deque
-public fields, and `crates/clawhdf5-agent/src/schema.rs` (deserialization
+(`self.window.retain(|e| e.timestamp >= cutoff)`), or reject/clamp non-monotonic timestamps in
-path, loading a persisted knowledge graph back from HDF5) pushes directly
+`record_write` and document that `WriteEvent.timestamp` must be non-decreasing per detector
-into `cache.entities`/`cache.relations` rather than going through
+instance.
-`add_entity`/`add_relation`. A struct-level cached index would silently go
-stale on that path. Building the index fresh at the top of each traversal
-call is O(V+R) — the same asymptotic cost as the scan it replaces would be
-for a *single* node — so it turns what was an O(V·R)-or-worse *whole
-traversal* into an O(V+R) traversal, with no risk of a stale-index
-correctness bug and no change to the existing public API or struct layout.
-`get_entity`, `get_relations_from`, `get_relations_to` are left as-is
-(still O(n)/O(R)): they're public API used elsewhere as one-off lookups,
-not inside a per-node hot loop, so indexing them is lower value and was
-left out of scope to keep this change minimal and low-risk.
 
-Existing tests (`test_bfs_neighbors_*`, `test_get_subgraph_*`,
+---
-`test_spreading_activation_*`) exercise correctness and were not modified —
-they pass unchanged, confirming the traversal results are identical to the
-pre-change O(V·R) implementation.
 
-## Deferred / not implemented this pass
+### INT-05 — Session-level rate check uses a lifetime cumulative counter, not a rate [P1]
+**File:** `crates/clawhdf5-agent/src/anomaly.rs:170-181` (`check_rate_anomaly`)
 
-Listed for a future pass — investigated but out of scope for this brief's
+**Problem:** `max_writes_per_session` is compared against `session_counts[session]`, which is
-budget, or blocked on a larger design decision already flagged upstream:
+incremented forever and never reset (see INT-03). This measures "how old is this session," not
+"is this session currently abusive" — any long-lived legitimate session (e.g. a persistent
+agent) permanently trips the alert once past the threshold regardless of pace, while a burst of
+writes in a brand-new session under the threshold is missed even if it's the real anomaly.
 
-- **HNSW outer insert-loop parallelism** — `ROADMAP.md` already flags this
+**Change:** Make this a rate — either measure session writes within the existing 60s rolling
-  as needing "its own dedicated design pass" before parallelizing; not
+window (reuse `window`, filtered by `session_id`) or add a separate per-session rolling window,
-  attempted here to avoid a correctness-sensitive change without that design
+rather than an unbounded lifetime total.
-  work.
-- **WAL per-entry format redesign** (explicit length-prefix instead of
-  read-then-verify-CRC32) — `ROADMAP.md` already notes the current CRC32
-  trailer works and this would only be worth revisiting "if profiling shows
-  it matters"; no such profiling signal was found this pass.
-- **`get_entity`/`get_relations_from`/`get_relations_to` indexing** — would
-  further help `get_entity_context` and any other one-off caller, but is
-  lower value than the hot-loop fix in INT-02 and was left out to keep this
-  change minimal.
 
-## Verification performed
+---
 
-- `cargo build --workspace --lib --bins` — clean before starting (baseline).
+### INT-06 — `bfs_neighbors` re-scans all relations on every queue pop [P1]
-- `cargo test --workspace` — run after implementing INT-01 and INT-02 (see
+**File:** `crates/clawhdf5-agent/src/knowledge.rs:339-378`, hot loop at 352-365
-  commit for pass/fail status).
 
-TASK: INT-01 — Harden agent memory provenance hash from FNV-1a to SHA-256
+**Problem:**
-TASK: INT-02 — Knowledge-graph traversal: per-call adjacency index instead of O(V·R) relation scans
+```rust
+let neighbours: Vec<u64> = self.relations.iter().filter_map(|r| { ... }).collect();
+```
+runs once per node dequeued during BFS, giving `O(visited_nodes × total_relations)` total cost.
+`get_subgraph` (`knowledge.rs:387-417`) calls `bfs_neighbors` once per seed node, multiplying
+the cost again. On a graph with a non-trivial relation count this is the dominant cost of any
+graph traversal query — the kind of memory-graph read the whole crate exists to serve
+efficiently.
+
+**Change:** Build an adjacency `HashMap<u64, Vec<u64>>` once (either eagerly maintained on
+insert/delete, or lazily built and cached with invalidation on mutation) instead of
+linear-scanning `self.relations` per hop.
+
+---
+
+### INT-07 — Quadratic eviction via `Vec::contains` inside `retain` [P1]
+**File:** `crates/clawhdf5-agent/src/consolidation.rs:345-350`
+
+**Problem:**
+```rust
+let evict_ids: Vec<u64> = episodic_indices[..evict_n].iter().map(|&i| self.records[i].id).collect();
+self.records.retain(|r| !evict_ids.contains(&r.id));
+```
+`retain` invokes the closure once per record; `Vec::contains` is `O(m)`. Worst case this is
+`O(n·m)` per consolidation pass, run periodically over the full record set.
+
+**Change:** Collect `evict_ids` into a `HashSet<u64>` before the `retain` call — `O(n)` lookup
+per record instead of `O(m)`.
+
+---
+
+### INT-08 — `MediaRef.checksum` is unkeyed FNV-1a but named/documented as a checksum [P2]
+**File:** `crates/clawhdf5-agent/src/multimodal.rs:96-97, 104, 116, 127`; compare
+`crates/clawhdf5-agent/src/provenance.rs:16-19`
+
+**Problem:** `provenance.rs` already carries an explicit doc comment (and the CHANGELOG has a
+dedicated "doc-only" entry) clarifying that its FNV-1a content hash is unkeyed and detects only
+accidental corruption, not tampering. `multimodal.rs`'s `MediaRef.checksum` field uses the same
+FNV-1a hash for the same purpose but has no equivalent caveat, and the field name "checksum"
+(vs. "hash") reads as an integrity guarantee to a downstream consumer (e.g. something in
+ZeroClaw deciding whether to trust/reuse a cached media reference).
+
+**Change:** Either rename the field (e.g. `content_fingerprint`) or add the same
+non-tamper-evidence doc comment already used in `provenance.rs`, so the two unkeyed-hash usages
+in the crate are consistently documented.
+
+---
+
+## Group B — `clawhdf5-format` / `clawhdf5-io`: untrusted-file parsing gaps
+
+The 2026-08-05 hardening pass (see CHANGELOG "Security" section) already covers
+`chunked_read.rs`/`data_read.rs`/`local_heap.rs`/`btree_v1.rs` with `ensure_len`-style overflow
+guards, a B-tree recursion-depth guard, and a `fuzz_dataset_read` target. ROADMAP.md explicitly
+flags "a full manual audit of every indexing site is still open" as unfinished — the following
+are concrete gaps found in that follow-up, in files/paths the prior pass did not touch.
+
+### INT-09 — `btree_v2.rs` recursive tree-walk has no depth cap (stack-overflow DoS) [P0]
+**File:** `crates/clawhdf5-format/src/btree_v2.rs:264-403` (`collect_internal_records`), entry
+at `176-213` (`collect_btree_v2_records`)
+
+**Problem:** `BTreeV2Header.depth: u16` (defined at line 21) is parsed straight from file bytes
+with no upper bound. `collect_internal_records` recurses with `child_depth = depth - 1` (line
+299) down to 0 with no depth-remaining cap — unlike the cyclic/self-referencing-index guards
+already added elsewhere in this hardening cycle (`fractal_heap.rs`, `object_header.rs`'s
+`depth_remaining` params, `filters.rs`'s `NBIT_MAX_DEPTH`). A crafted v2 B-tree header claiming
+`depth = 65535` (paired with a matching on-disk `"BTIN"` internal-node chain, or even a node
+that points back into itself since nothing here detects cycles either) drives ~65k stack frames
+of native recursion — an abort/crash from a small crafted file. This is reachable from real
+parse paths: `group_v2.rs:82`, `shared_message.rs:368`, `attribute.rs:384` (dense group/dense
+attribute listings — a realistic file feature, not an obscure one).
+
+**Change:** Thread a `depth_remaining: u16` (or similar) cap through
+`collect_btree_v2_records`/`collect_internal_records`, capped at some sane bound (e.g. 64,
+consistent with `NBIT_MAX_DEPTH`'s style elsewhere in this codebase), returning a `FormatError`
+instead of recursing past it.
+
+---
+
+### INT-10 — `fuzz_btree_v2` never exercises the recursive traversal where INT-09 lives [P1]
+**File:** `crates/clawhdf5-format/fuzz/fuzz_targets/fuzz_btree_v2.rs` (or wherever this target
+lives under `crates/clawhdf5-format/fuzz/`)
+
+**Problem:** The existing target only calls `BTreeV2Header::parse` — it never calls
+`collect_btree_v2_records`, so the actual tree-walk (the code path with the depth-recursion bug
+in INT-09) has zero fuzz coverage today, despite the file being in scope for a target already
+named after it.
+
+**Change:** Extend `fuzz_btree_v2` to also invoke `collect_btree_v2_records` on the parsed
+header against the fuzz input, so the recursive traversal gets the same adversarial coverage the
+header parse already has. Land this alongside INT-09 so the fix is locked in by the fuzzer, not
+just a manual patch.
+
+---
+
+### INT-11 — Unchecked multiplication of file-derived sizes in fractal-heap size math [P0]
+**File:** `crates/clawhdf5-format/src/fractal_heap.rs:479-496` (`block_size_for_row`,
+`indirect_block_heap_size`)
+
+**Problem:**
+```rust
+sbs * (1u64 << (row - 1))              // line ~484
+total += self.block_size_for_row(row) * tw   // line ~493
+```
+use plain `*` on `starting_block_size`/`table_width`, both read from the FRHP header with no
+upper-bound validation. A crafted large `starting_block_size` combined with enough rows/columns
+overflows `u64`; under `overflow-checks` (on for debug/fuzz builds, and optionally enabled in
+release) this panics — a DoS abort from a malformed fractal heap, the same bug class the
+2026-08-05 pass already fixed in sibling files.
+
+**Change:** Replace with `checked_mul`/`saturating_mul` and propagate a `FormatError` on
+overflow, matching the `ensure_len`/checked-arithmetic idiom already used in
+`chunked_read.rs`/`local_heap.rs`.
+
+---
+
+### INT-12 — Unbounded allocation from an unvalidated length before any data is read [P0]
+**Files:**
+- `crates/clawhdf5-io/src/subfiling.rs:206-210` (`SubfileManager::read_at`) —
+  `Vec::with_capacity(length as usize)` where `length: u64` is caller/layout-supplied with no
+  cap tied to actual dataset or file size.
+- `crates/clawhdf5-io/src/async_read.rs:84` (`AsyncFileReader::open`) —
+  `Vec::with_capacity(len as usize)` sized directly from `file.metadata().len()`, no cap.
+
+**Problem:** Both allocate a buffer sized from an untrusted/unvalidated length *before*
+validating it against anything (declared dataset size, actual readable bytes, or a configured
+ceiling). A crafted layout-metadata value reaching `subfiling.rs`, or a crafted/sparse file
+opened via `async_read.rs`, can trigger a multi-gigabyte-to-exabyte allocation attempt and an
+OOM abort — the same "bounded allocation" concern the CHANGELOG's `MAX_DECOMPRESS_SIZE` fix
+already addressed for the decompression path, just not yet for these two read paths.
+
+**Change:** Cap the length against a known-sane bound (file size, or a configurable ceiling
+similar in spirit to `MAX_DECOMPRESS_SIZE`/`MAX_WAL_FIELD_LEN`) before calling
+`Vec::with_capacity`, or use `try_reserve` and return a clean error on failure instead of
+aborting.
+
+---
+
+### INT-13 — `symbol_table.rs` size arithmetic doesn't use the `checked_*`/`ensure_len` idiom used elsewhere [P2]
+**File:** `crates/clawhdf5-format/src/symbol_table.rs:99-107` (`SymbolTableNode::parse`)
+
+**Problem:** `let needed = entries_start + num_symbols * entry_size;` uses plain arithmetic.
+Not exploitable to overflow on 64-bit today (`num_symbols` is bounded by its `u16` source
+field), but it's inconsistent with the rest of the audited codebase and becomes a real risk if
+either operand's type widens later.
+
+**Change:** Route through `checked_mul`/`checked_add` + `ensure_len`, matching the pattern used
+throughout `chunked_read.rs`/`data_read.rs`/`local_heap.rs`/`btree_v1.rs`.
+
+---
+
+### INT-14 — Filter bit-packing decode loops have no direct fuzz coverage [P2]
+**File:** `crates/clawhdf5-format/src/filters.rs` (scale-offset unpack ~150-270, N-Bit type-tree
+walk ~396-510); fuzz target `fuzz_filter_pipeline`
+
+**Problem:** `fuzz_filter_pipeline` only fuzzes `FilterPipeline::parse` — the filter-pipeline
+*metadata* message — not the actual decode functions in `filters.rs` that unpack
+attacker-influenced compressed bytes bit-by-bit (scale-offset, N-Bit). This is the most
+bit-twiddling-heavy code in the crate and, per the CHANGELOG, has already had real bugs found
+there in the initial hardening pass (`1 << minbits` overflow, `bit_offset + precision`
+overflow); it's exactly the kind of code that benefits most from fuzzing but currently gets none
+directly.
+
+**Change:** Add a `fuzz_filter_decode` target that feeds arbitrary bytes through the
+scale-offset and N-Bit decode entry points directly (not just pipeline metadata parsing).
+
+---
+
+## Group C — `clawhdf5-ann` (HNSW): hot-path performance
+
+`prune_connections` rayon parallelism (already shipped) is out of scope. The outer
+insert/build loop is intentionally left sequential per ROADMAP's own design note — not
+re-proposed here.
+
+### INT-15 — `compute_distance` is scalar-only; `clawhdf5-accel`'s SIMD path is never used [P1]
+**Files:** `crates/clawhdf5-ann/src/hnsw.rs:47-74` (`compute_distance`);
+`crates/clawhdf5-accel/src/lib.rs:125` (`cosine_similarity`), `:173` (`l2_distance`)
+
+**Problem:** `clawhdf5-ann`'s `Cargo.toml` has no dependency on `clawhdf5-accel` at all.
+`compute_distance` is a hand-written scalar loop for both L2 and cosine, called from every
+candidate-expansion step in `greedy_closest`, `search_layer`, and `prune_connections` — i.e.
+the entire build/insert/search hot path. `clawhdf5-accel` already provides
+runtime-feature-detected, SIMD-accelerated equivalents (AVX2/AVX-512/NEON, correctly gated per
+INT survey — see clean bill of health above) that go completely unused here.
+
+**Change:** Add a `clawhdf5-accel` dependency to `clawhdf5-ann` and route `compute_distance`
+through `l2_distance`/`cosine_similarity`. This is a drop-in replacement for the scalar
+arithmetic, not a semantic change.
+
+---
+
+### INT-16 — Best-entry-point distance is discarded and immediately recomputed [P1]
+**File:** `crates/clawhdf5-ann/src/hnsw.rs` — `greedy_closest` (749-772) computes
+`best_dist` at line 756 but returns only the `usize` node id; callers
+(`build_with_metric` 251-253, `insert` 381-389, `search` 504-506) immediately recompute
+`compute_distance(query, &vectors[ep], metric)` for that same `(query, ep)` pair before calling
+`search_layer` (which itself recomputes it again at line 783).
+
+**Problem:** Every layer transition during insert/search throws away a distance value it just
+computed and recomputes the identical value at least once more. For an L-layer index this wastes
+up to L redundant distance computations per insert/search call — pure waste on what is already
+the hottest path in the crate (compounded by INT-15 if that's not yet fixed).
+
+**Change:** Change `greedy_closest`'s return type to `(usize, f32)` (node id + its distance) and
+thread that value into the next `greedy_closest`/`search_layer` call instead of recomputing.
+
+---
+
+### INT-17 — `search_layer`'s visited-set uses `HashSet<usize>` instead of a dense bitset [P1]
+**File:** `crates/clawhdf5-ann/src/hnsw.rs:799, 809-812`
+
+**Problem:** `let mut visited = HashSet::new();` with `.contains(&neighbor)`/`.insert(neighbor)`
+in the innermost per-candidate-expansion loop, run on every insert and search call. Node ids are
+dense `0..n` integers — a `Vec<bool>` (or bitset) indexed directly by id gives O(1) lookup
+without SipHash overhead, which matters when this loop dominates search cost.
+
+**Change:** Replace with `vec![false; vectors.len()]` indexed by node id (reset/reused per
+call), or a proper bitset if allocation-per-call cost matters.
+
+---
+
+### INT-18 — `compact()` clones every surviving vector twice [P1]
+**File:** `crates/clawhdf5-ann/src/hnsw.rs:463-478` (`compact`), `:305`
+(`build_with_metric`'s `vectors: vectors.to_vec()`)
+
+**Problem:** `compact()` builds an owned `Vec<Vec<f32>>` via `surviving.push(v.clone())` (line
+469), then passes `&surviving` into `build_with_metric`, whose first action clones it again via
+`.to_vec()`. For a large index this doubles the memory-copy cost of an already-`O(n)` rebuild
+operation.
+
+**Change:** Give `build_with_metric` (or a private variant) an owned-`Vec<Vec<f32>>` entry point
+so `compact` can move `surviving` in directly instead of cloning twice.
+
+---
+
+## Group D — `clawhdf5-py`: Mutex poisoning bricks write-mode objects
+
+### INT-19 — Pervasive `state.lock().unwrap()` on a shared `Mutex` reachable from Python calls [P1]
+**Files:** `crates/clawhdf5-py/src/group.rs` (6 sites, e.g. `:115, :161, :184, :200, :217`),
+`crates/clawhdf5-py/src/attrs.rs` (4 sites, e.g. `:56, :77, :92, :99`),
+`crates/clawhdf5-py/src/file.rs` (1 site, `:282`)
+
+**Problem:** `PyGroup`/`PyAttrs`/write-mode file state hold a `Mutex<...>` and every method that
+touches it does `state.lock().unwrap()`. If any single call panics while holding the lock (a
+future edge case in `extract_numpy_data`, an allocation failure, anything) the `Mutex` becomes
+permanently poisoned. Every subsequent method call on that same Python object — for the rest of
+its lifetime — then also panics via the same `.unwrap()`, instead of the object cleanly
+returning a `PyErr` and remaining usable. This turns one transient panic into a permanently
+broken object from the caller's perspective, which is a worse failure mode than a single
+raised-and-handled Python exception.
+
+**Change:** Replace `lock().unwrap()` with a helper that converts a poison error into a
+`PyResult` `PyErr` (e.g. `state.lock().map_err(|_| PyErr::new::<PyRuntimeError, _>("internal state poisoned"))?`,
+or use `parking_lot::Mutex` which doesn't have poisoning at all — likely the simpler fix given
+`clawhdf5-py` doesn't appear to rely on poisoning semantics anywhere). Apply consistently across
+all ~11 call sites.
+
+---
+
+## Group E — noted, not proposed (checked and found low-priority/out-of-scope)
+
+- **`clawhdf5-derive`'s generated `from_bytes`** (`crates/clawhdf5-derive/src/lib.rs:106-119`)
+  does `assert!(_data.len() >= _required, ...)` before any field-slicing, so it's a documented,
+  guarded panic (`# Panics` doc comment already present) rather than an unguarded OOB — and
+  `#[derive(H5Type)]` is currently used only in `crates/clawhdf5-format/tests/derive_tests.rs`,
+  not in any production code path. Making `from_bytes` return `Result` instead of asserting
+  would be a reasonable future API-ergonomics improvement for downstream users of the macro, but
+  it's not fixing a reachable bug today — left out as not worth an INT slot this pass.
+- **`cargo audit` unmaintained warnings** (`custom_derive`, `number_prefix`, `paste`) — all
+  transitive through optional features (`mpi-io`, and whatever pulls in `tokenizers`), zero
+  actual vulnerabilities, no code-level fix available in this repo. FYI only.
+
+---
+
+## Suggested implementation order for the coding phase
+
+1. **INT-01** first — it's the load-bearing gap (provenance/anomaly detection is currently
+   inert), and INT-02/03/04/05 are bug fixes *inside* the code INT-01 wires up, so fixing them
+   before or during the wiring avoids shipping newly-live bugs.
+2. **INT-09 + INT-10 together** (P0, security) and **INT-11, INT-12** (P0, security) — these are
+   independent of each other and of Group A, safe to parallelize.
+3. **INT-15/16/17/18** (Group C, HNSW perf) — independent of A/B, safe to parallelize.
+4. **INT-19** (Group D) — independent, small, safe to parallelize.
+5. **INT-06, INT-07, INT-08, INT-13, INT-14** — lower urgency, pick up as time allows.
+
+All items should land with `cargo test --workspace` (and `cargo clippy --workspace -- -D
+warnings`, per this repo's established gate) passing before being considered done.