feat(agent): HDF5Memory::search with source filters, re-ranking, confidence

`HDF5Memory::search(query_embedding, query_text, &SearchOptions)` is the
store's full search path. `SearchOptions::new(k)` is plain hybrid search
with the tuned default fusion; each further stage is opt-in:

- `with_sources([..])`: only records from these source channels. The
  filter applies before ranking, so a filtered search still returns up
  to k results, normalised over what it can return. The HNSW pool is
  over-fetched in proportion to what the filter removes, and the allowed
  records are scanned exactly whenever that costs fewer distance
  evaluations than the index would (~pool x M) — and as the fallback if
  the pool comes back short. Keyword matches are filtered too.
- `with_rerank(ReRankConfig)` re-ranks a max(3k, 10) candidate pool by
  relevance, recency, source authority and activation;
  `with_confidence(ConfidenceConfig)` drops low-confidence results;
  `at_time(now)` pins the recency clock.

These were reachable only through the OpenClaw backend, which is now
`search` with both on. Its Hebbian boost now goes to the k results it
returns rather than the whole 3k candidate pool. `hybrid_search` and
`hybrid_search_with` are wrappers and unchanged (tested bit for bit).

Measured on tank (search_harness --options-study --full, 3 runs): at
100K every filter — 50%, 10%, 1% of the store, and records far from the
query — returns the exact filtered top 10, and none is slower than an
unfiltered search (1%: 2.3 ms vs 4.6 ms). Re-rank + confidence costs
about 3%. A first version decided between index and exact scan by pool
size vs store size; it measured 0.976 recall at 12.3 ms on the
far-from-query filter, which is why the rule compares costs instead.

Tests: tests/search_options.rs (filter correctness and full pages via
both paths, far-from-query fallback, edge cases, equality with
hybrid_search_with, re-rank recency, confidence, boost scope).

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>

crates/clawhdf5-agent/src/openclaw.rs

@@ -13,9 +13,8 @@ use std::path::{Path, PathBuf};
 use std::time::{SystemTime, UNIX_EPOCH};
 
 use crate::{
-    AgentMemory, HDF5Memory, MemoryConfig, MemoryEntry,
-    confidence::{ConfidenceConfig, ScoredResult, reject_low_confidence},
-    reranker::{ReRankConfig, RerankInput, rerank},
+    AgentMemory, HDF5Memory, MemoryConfig, MemoryEntry, SearchOptions,
+    confidence::ConfidenceConfig, reranker::ReRankConfig,
 };
 
 // ─────────────────────────────────────────────────────────────────────────────
@@ -524,71 +523,27 @@ impl ClawhdfBackend {
 
 impl MemoryBackend for ClawhdfBackend {
     /// Search using hybrid vector + BM25 retrieval, then re-rank and
-    /// confidence-filter.
+    /// confidence-filter — [`HDF5Memory::search`] with both stages on.
     fn search(
         &mut self,
         query_text: &str,
         query_embedding: &[f32],
         k: usize,
     ) -> Vec<MemorySearchResult> {
-        // 1. Hybrid retrieval (vector + BM25, fused by score).
-        let candidates = k.saturating_mul(3).max(10);
-        let raw = self.memory.hybrid_search_with(
-            query_embedding,
-            query_text,
-            crate::hybrid::DEFAULT_FUSION,
-            candidates,
-        );
-
-        if raw.is_empty() {
-            return Vec::new();
-        }
-
-        let now = Self::now_secs();
-
-        // 2. Re-rank using temporal recency, source authority, Hebbian weight.
-        let rerank_inputs: Vec<RerankInput> = raw
-            .iter()
-            .map(|r| RerankInput {
-                index: r.index,
-                timestamp: r.timestamp,
-                source_channel: r.source_channel.clone(),
-                raw_activation: r.activation,
-                relevance: r.score,
-            })
-            .collect();
-
-        let reranked = rerank(&rerank_inputs, &self.rerank_config, now);
-
-        // 3. Confidence rejection.
-        let scored: Vec<ScoredResult> = reranked
-            .iter()
-            .map(|r| ScoredResult {
-                index: r.index,
-                score: r.combined_score,
-            })
-            .collect();
-
-        let confident = reject_low_confidence(&scored, &self.confidence_config);
-
-        // 4. Map back to MemorySearchResult; preserve raw text via index lookup.
-        let raw_by_idx: HashMap<usize, &crate::SearchResult> =
-            raw.iter().map(|r| (r.index, r)).collect();
-
-        confident
+        let options = SearchOptions::new(k)
+            .with_rerank(self.rerank_config)
+            .with_confidence(self.confidence_config.clone())
+            .at_time(Self::now_secs());
+        self.memory
+            .search(query_embedding, query_text, &options)
             .into_iter()
-            .take(k)
-            .filter_map(|sr| {
-                let r = raw_by_idx.get(&sr.index)?;
-                let path = r.source_channel.clone();
-                Some(MemorySearchResult {
-                    text: r.chunk.clone(),
-                    score: sr.score,
-                    path: path.clone(),
-                    line_range: None,
-                    timestamp: Some(r.timestamp),
-                    source: path,
-                })
+            .map(|r| MemorySearchResult {
+                text: r.chunk,
+                score: r.score,
+                path: r.source_channel.clone(),
+                line_range: None,
+                timestamp: Some(r.timestamp),
+                source: r.source_channel,
             })
             .collect()
     }