Commit Graph
6 Commits
Author SHA1 Message Date
Omar Sobh d3c65ccb58 Merge remote-tracking branch 'origin/clawmates/mission-01a00c41-421200ee' into verify/v3-plus-v6
# Conflicts:
#	crates/clawhdf5/Cargo.toml
2026-08-17 06:15:19 -07:00
ClawHDF5 Coding Agent 55c3737130 fix(migrate): truncate on a char boundary in validate::truncate
truncate() sliced source.chunk (arbitrary UTF-8 from the source SQLite
database) at a raw byte offset. A multi-byte character straddling byte
40 panics with "byte index 40 is not a char boundary" instead of
producing the mismatch diagnostic the code exists to report — and this
is the default validate_hdf5 path, not test-only. Cut on the nearest
char boundary at or before 40 instead.

INT-10
2026-08-17 00:27:48 +00:00
Omar Sobh b08df7b628 clawmates: phase work
Mission: 01a00c41-bac0-7eb3-a8c8-8b7044f3086d
Phase: 01a00c41-bac2-71e3-a58b-c473421200ee

Committed by the ClawMates delivery pipeline from the agents' working tree. Authored by agents, not by the named committer.
2026-08-16 20:44:28 +00:00
Omar Sobh 55959b4920 ci: wire up CI, fix no_std build, fix stale package names in scripts
CI / test (push) Failing after 15s
- Add .gitea/workflows/ci.yml running scripts/ci-test.sh (fmt, clippy,
  test, no_std check) on push/PR to main.
- Fix stale rustyhdf5-py/rustyhdf5-format package names in
  ci-test.sh/check-nostd.sh, which had been silently no-op'ing those
  checks (cargo warns but doesn't fail on an unknown --exclude/-p
  target).
- With those checks actually running, fix the real issues they surface:
  - clippy: useless_conversion in chunked_write.rs, byte_char_slices in
    global_heap.rs/object_header.rs.
  - cargo fmt: apply formatting across the workspace (whitespace only).
  - no_std (thumbv7em-none-eabihf) build errors in clawhdf5-format:
    core::sync::atomic::AtomicU64 doesn't exist on that target (no
    native 64-bit atomics) — switch profiling.rs's counters to
    portable-atomic, which falls back to a CAS-based emulation there
    and is a no-op wrapper elsewhere. Add missing alloc imports for
    Box (filters.rs), Vec (filters_szip.rs), and format! (dict_encoding.rs)
    on no_std paths. Replace f64::powi (std/libm-only) with a small
    local exponentiation-by-squaring helper in the scale-offset filter.
2026-08-05 10:50:13 -07:00
osobhandClaude Opus 4.8 8534c7d204 feat: migrate-engine improvements (content validation, schema, streaming, incremental)
clawhdf5-migrate:
- Real content validation: the post-migration check reads the written HDF5
  back (new hdf5_reader) and compares actual content — chunk text, embeddings,
  and every session/entity/relation field — to the source, not just row counts.
  A representative sample of chunk rows is verified by default; --validate-full
  checks every row. A count-preserving corruption no longer passes.
- Configurable schema: SQL is built from a SchemaConfig (table + ordered column
  names, defaulting to the ZeroClaw layout) instead of hardcoded queries, with
  --chunks-table / --sessions-table / --entities-table / --relations-table.
- Streaming count pass: --dry-run does a COUNT(*)-only pass per table instead
  of loading every row.
- Incremental migration: --incremental reads the existing output, reads only
  source chunks with id greater than the last migrated id, and appends them
  (metadata groups refreshed from source) rather than re-migrating everything.

clawhdf5-format:
- read_as_f32 / read_as_f64 now decode IEEE-754 half-precision (2-byte) floats
  via a no_std-safe bit conversion — needed to read float16-stored embeddings
  back (e.g. for migrate's content validation), previously a TypeMismatch.

Tests: f16 read unit test; migrate tests for content-corruption detection,
custom table names, and incremental append; CLI smoke-tested end-to-end and the
dense/incremental output verified with h5py.

Co-Authored-By: Claude Opus 4.8 <[email protected]>
2026-06-04 02:19:31 +00:00
redclawsystems 3f222f6956 Merge pull request 'docs(clawhdf5): document DType variants, fix unresolved doc links' (#17) from sdlc-docs/clawhdf5-types-20260514-165210 into main 2026-05-14 23:54:48 +00:00