Commit Graph
5 Commits
Author SHA1 Message Date
osobhandClaude Fable 5.1 bbe1baa208 ci: lint all targets, run interop suites for real, compile benches
- clippy --all-targets plus a clawhdf5-format feature matrix (parallel, lz4,
  zstd, pcodec, fast-checksum); fix the accumulated lint backlog in test,
  bench and feature-gated code (no behaviour changes).
- Install python3 + h5py/numpy/netCDF4/xarray in the CI container and set
  CLAWHDF5_REQUIRE_INTEROP=1, which makes a missing interop dependency a test
  failure. Every h5py/netCDF4 interop test used to skip silently in CI. Run
  the #[ignore]d writer_h5py_tests suite explicitly.
- cargo bench --no-run so benches can't rot; fix bench.rs and memory_bench.rs,
  which no longer compiled against the current strategy/consolidation APIs.
- Optional fuzz smoke run via CLAWHDF5_FUZZ_SECONDS.
- CHANGELOG and docs/known-issues.md updated.

Co-Authored-By: Claude Fable 5.1 <[email protected]>
2026-09-19 05:36:22 -07:00
ClawHDF5 Coding Agent 3c7c229e20 security(agent): gate elevated MemorySource construction behind a distinct API
Two related trust-boundary gaps, both closed:

1. ConsolidationEngine::add_memory took a plain `source: MemorySource`
   parameter, so any caller could claim MemorySource::System/Correction —
   which get elevated importance weighting in score_correction — for
   content whose actual origin the caller doesn't control or hasn't
   verified. Split into add_memory(UntrustedSource) for ordinary
   caller-supplied content (User/Tool/Retrieval only, no elevated variant
   exists to claim) and add_trusted_memory(TrustedSource) for content whose
   elevated trust the caller has independently verified (System/
   Correction). Updated the one production consumer outside this crate
   (clawhdf5-bench's consolidation_efficiency benchmark) and all tests.

2. The provenance/anomaly wiring added in the previous commit introduced
   the same pattern: infer_memory_source mapped source_channel == "system"
   or "correction" straight to the elevated MemorySource variants. Since
   MemoryEntry.source_channel is unvalidated caller-supplied text, this let
   a write dodge check_source_anomaly's User-flood detection by simply
   self-labeling source_channel = "system". infer_memory_source now never
   returns System/Correction — only Tool/Retrieval (recognized channel
   names) or User (everything else, the conservative default).

INT-05
2026-08-17 00:49:34 +00:00
ClawHDF5 Coding Agent 603fcf8757 perf(agent): use HashSet for eviction ID membership checks in consolidation
records.retain(|r| !evict_ids.contains(&r.id)) called Vec::contains
(linear scan) for every record against evict_ids, giving O(n·m) cost
on both Working- and Episodic-tier eviction every consolidation tick.
Build evict_ids as a HashSet for O(1) membership checks.

INT-15
2026-08-17 00:29:03 +00:00
ClawHDF5 Coding Agent d787ac04c8 perf(agent): avoid cloning working-tier records in consolidation add_memory
score_surprise only reads r.embedding by reference, so cloning every
Working-tier record's full chunk text + embedding Vec<f32> on every
add_memory call was wasted work, discarded immediately after use.
Collect Vec<&MemoryRecord> instead and change score_surprise's
signature to take &[&MemoryRecord].

INT-14
2026-08-17 00:28:55 +00:00
redclawsystems 3f222f6956 Merge pull request 'docs(clawhdf5): document DType variants, fix unresolved doc links' (#17) from sdlc-docs/clawhdf5-types-20260514-165210 into main 2026-05-14 23:54:48 +00:00