7 Commits

Include renames

Author	SHA1	Message	Date
Omar SobhandClaude Opus 5	9575b84803	style: clear the fmt gate and two lib clippy warnings ... Deferred deliberately while the TWIN-2B/2C campaign had live marches: each march is a fresh `cargo test` invocation, so reformatting `turek_hron_fsi2.rs` mid-campaign would have forced a test-binary rebuild and cost comparability for a cosmetic gate. The family closed, so this is now free. - `cargo fmt --all` across 8 files that had drifted (including the FSI2/FSI3 harnesses touched by the UMEAN/ES override commits). - `rtx-feature-store/tests/integration_tests.rs` had trailing whitespace rustfmt refused to format around ("left behind trailing whitespace" internal error), so the whole file was being skipped; stripped it and the file formats now. - Two `unnecessary_parentheses` warnings in the rtx-transformers lib (`continual/progressive.rs`, `curriculum/mod.rs`) — these were the only rustytorch warnings surfacing through omni-cortex's workspace clippy gate, which is how they were found. No behaviour change. rtx-fsi test binaries still build. Co-Authored-By: Claude Opus 5 (1M context) <[email protected]> Claude-Session: https://claude.ai/code/session_01B1feFAQxjbCRHePUdxuNra	2026-09-02 19:15:53 -07:00
Omar SobhandClaude Fable 5	38645c7c74	fix(autograd): AbsBackward produced NaN for exactly-zero inputs ... sign was computed as x/|x|, which is 0/0 = NaN at x = 0; one zero element in an |pred - target| loss poisoned every upstream gradient (hit deterministically by dg-gnn HetGAT training). Compute x/(|x| + tiny) instead so sign(0) = 0 (the subgradient convention). Regression test included. Co-Authored-By: Claude Fable 5 <[email protected]>	2026-08-20 11:53:30 -07:00
Omar SobhandClaude Fable 5	9969d8a661	feat(backend): add differentiable index_select / index_add row ops ... Add two row-indexing ops along dim 0 to the `Backend` trait so gather / scatter-add message passing (GNNs, segment softmax, bias tiling) can be trained through `Autodiff<B>`: - `index_select(tensor, indices)` — out[i, ..] = tensor[indices[i], ..] - `index_add(tensor, indices, num_rows)` — out = zeros; out[idx[i], ..] += tensor[i, ..] They are each other's adjoint, which is what the backward passes use. Both trait methods have default bodies (host round-trip via to_data / from_data) so every existing backend keeps compiling and is correct; backends override with native kernels: - rtx-backend-cpu: new ops/index.rs (rayon-parallel gather over output rows above a size threshold, sequential deterministic scatter-add), wired into CpuBackend and CpuBackendF64, with unit tests for D=1/2/3, duplicates, untouched rows, empty inputs, bounds panics and adjointness. - rtx-autograd: Autodiff<B> overrides both ops and records IndexSelectBackward / IndexAddBackward (new ops/index.rs); finite- difference gradchecks on the real CpuBackend cover repeated-index accumulation, untouched-row zero grads, bias tiling via index_select of a [1,F] row, and a full per-segment softmax. - rtx-fusion: forward both ops to the inner backend. Co-Authored-By: Claude Fable 5 <[email protected]>	2026-08-20 11:05:42 -07:00
osobhandClaude Opus 4.8	96809d48a1	SMT D314 (rustytorch): gradcheck tanh/sigmoid + GatedMemoryUpdater (GRU cell) ... F.1 — finite-difference gradchecks for tanh, sigmoid, and the exact gated composite (1-z)*m + z*c (z=sigmoid, c=tanh) added to tape_cpu_gradcheck.rs. All pass (rel-err < 2e-2) — the activation VJPs that already existed are now proven correct on the real CpuBackend (E0 discipline), so the gated cell can rely on them. F.2 — GatedMemoryUpdater: a GRU-style gated recurrent memory cell. z = sigmoid([M‖x]·W_z); c = tanh([M‖x]·W_c); M_t = (1-z)⊙M + z⊙c The convex update is a non-expansion (|M_t| ≤ max(|M_0|, 1)), so free rollout stays bounded with NO clamp — and the learned gate can both jump at a regime switch (z≈1) and hold+denoise in steady state (z≈0), which the residual+leaky ClonedMemoryUpdater cannot. Same method surface (new/step/predict/train_step/ train_step_memory) so the omni-think facade is cell-generic. Tests: gated cell trains (loss drops); 500-step free rollout stays in [-1,1] without a clamp. clippy(-D)/fmt clean. Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>	2026-06-16 21:09:10 -07:00
osobhandClaude Opus 4.8	9a31a07cc9	rtx-autograd: training smoke tests proving the tape learns ... End-to-end training on `Autodiff<CpuBackend>` (forward → backward → extract grads → SGD → repeat), guarding the gradient-correctness fixes: - `mlp_trains_and_loss_decreases`: 2-layer MLP (matmul + gelu), loss 8.65 → 0.39. - `attention_set_encoder_learns_window_mean`: a minimal attention set-encoder (the SMT predictive-state teacher shape) learns to predict its input window's per-dim mean, loss 0.27 → ~0.0. The embedding fans out to Q/K/V and the residual (4 uses), so this also regression-guards the fan-out gradient accumulation fix inside a real attention block. All ops used are gradient-checked in `tape_cpu_gradcheck.rs`. Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>	2026-06-15 21:24:56 -07:00
osobhandClaude Opus 4.8	fc895a2de5	rtx-autograd: make the tape correct + sound on real backends ... The decorator autograd (`Autodiff<B>`) had never been gradient-checked against a real tensor backend — the entire test suite runs on a shape-only `MockBackend` whose ops return their input, so they validate graph structure but never gradient values. Running it through `CpuBackend` for the first time (new `tests/tape_cpu_gradcheck.rs`, finite-difference checks) surfaced three bugs that made the tape unusable for training; this fixes all three. 1. Double-free / UB in the dimension-erasure cast. The backward ops cast a tensor to its runtime const-generic dimension via `mem::transmute_copy::<_, TensorPrimitive<N>>(&src)` in ~100 sites. That bit-copies the owned `Vec` without forgetting the source, so two values own one buffer → double-free on any heap-backed backend (and Stacked-Borrows UB from the typed pun). Replaced every site with a single `into_dim` helper that is now **fully safe** — it round-trips through `to_data`/`from_data` and rebuilds the shape with `array::from_fn`, no `unsafe` at all. (This is why the whole repo previously bypassed the tape with analytic backward.) 2. Fan-out gradients were silently dropped. `accumulate_gradients` was a stub that returned one path and discarded the other, and `AutodiffTensor::clone` minted a fresh `TensorId`. Together, reusing a tensor (residuals, `mul(s, s)`, shared Q/K/V — universal in transformers) split its gradient across two ids and summed neither, yielding a fraction of the true value. `accumulate_gradients` now sums via `B::add`; `clone` preserves the id so fan-out paths collide on one sink. 3. Softmax backward panicked. `SoftmaxBackward` / `stable_softmax_backward` subtracted a keep-dim row-sum from the full-shape grad, but the elementwise backends assert equal shapes (no broadcasting). Added `broadcast_along_dim` to tile the row-sum to full width first. Verified: `tape_cpu_gradcheck` (matmul, fan-out add·mul, softmax) passes with rel-err < 2e-2 vs central differences; full `rtx-autograd` suite green (263 passed, 0 failed); lib clippy `-D warnings` clean. Known follow-up (out of scope): `cargo miri test` still aborts on a Stacked-Borrows / integer-to-pointer violation inside `rtx-backend-cpu`'s buffer internals — a grad-free `from_data`+`add`+`sum` probe reproduces the identical error, so it is pre-existing backend UB, not an autograd issue. Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>	2026-06-15 21:06:32 -07:00
redclawsystems	4d88dc0584	Initial commit	2026-03-04 00:08:42 +00:00