15 Commits

This Branch

This Branch

All Branches

Author	SHA1	Message	Date
osobh	93f160b183	Merge pull request 'SMT D319: learned-[MEM]-query content-addressable teacher pool' (#11) from d319-content-addressable-teacher into main	2026-06-17 16:32:57 +00:00
osobh	117a1b6e28	Merge pull request 'SMT D316: train_rollout (truncated K-step BPTT)' (#10) from d316-rollout-training into main	2026-06-17 11:13:14 +00:00
osobh	f28a92cfa7	Merge pull request 'SMT D315: configurable recency-pool decay' (#9) from d315-sharp-oracle into main	2026-06-17 11:00:06 +00:00
osobh	567eefa96f	Merge pull request 'SMT D314: gradcheck tanh/sigmoid + GatedMemoryUpdater (GRU cell)' (#8) from d314-gated-updater into main	2026-06-17 04:28:33 +00:00
osobh	eb4f224595	Merge pull request 'SMT D312: timestamp-embedded (recency) teacher mode' (#7) from d312-timestamp-teacher into main	2026-06-17 01:35:00 +00:00
osobh	2a637d8119	Merge pull request 'SMT E4: stabilize cloned updater for DAgger' (#6) from d310-dagger-stability into main	2026-06-16 23:04:17 +00:00
osobh	6be0ea10b9	Merge pull request 'SMT E1+E3a/b: predictive-state teacher + cloned recurrent memory updater' (#5) from smt-set-encoder-teacher into main	2026-06-16 22:49:56 +00:00
osobh	c5f7d86011	Merge pull request 'rtx-autograd: training smoke tests proving the tape learns' (#4) from tape-training-smoke into main	2026-06-16 04:25:13 +00:00
osobhandClaude Opus 4.8	9a31a07cc9	rtx-autograd: training smoke tests proving the tape learns ... End-to-end training on `Autodiff<CpuBackend>` (forward → backward → extract grads → SGD → repeat), guarding the gradient-correctness fixes: - `mlp_trains_and_loss_decreases`: 2-layer MLP (matmul + gelu), loss 8.65 → 0.39. - `attention_set_encoder_learns_window_mean`: a minimal attention set-encoder (the SMT predictive-state teacher shape) learns to predict its input window's per-dim mean, loss 0.27 → ~0.0. The embedding fans out to Q/K/V and the residual (4 uses), so this also regression-guards the fan-out gradient accumulation fix inside a real attention block. All ops used are gradient-checked in `tape_cpu_gradcheck.rs`. Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>	2026-06-15 21:24:56 -07:00
osobh	a52ce0dd26	Merge pull request 'rtx-autograd: make the tape correct + sound on real backends' (#3) from autograd-trainable-real-backend into main	2026-06-16 04:07:05 +00:00
osobhandClaude Opus 4.8	fc895a2de5	rtx-autograd: make the tape correct + sound on real backends ... The decorator autograd (`Autodiff<B>`) had never been gradient-checked against a real tensor backend — the entire test suite runs on a shape-only `MockBackend` whose ops return their input, so they validate graph structure but never gradient values. Running it through `CpuBackend` for the first time (new `tests/tape_cpu_gradcheck.rs`, finite-difference checks) surfaced three bugs that made the tape unusable for training; this fixes all three. 1. Double-free / UB in the dimension-erasure cast. The backward ops cast a tensor to its runtime const-generic dimension via `mem::transmute_copy::<_, TensorPrimitive<N>>(&src)` in ~100 sites. That bit-copies the owned `Vec` without forgetting the source, so two values own one buffer → double-free on any heap-backed backend (and Stacked-Borrows UB from the typed pun). Replaced every site with a single `into_dim` helper that is now **fully safe** — it round-trips through `to_data`/`from_data` and rebuilds the shape with `array::from_fn`, no `unsafe` at all. (This is why the whole repo previously bypassed the tape with analytic backward.) 2. Fan-out gradients were silently dropped. `accumulate_gradients` was a stub that returned one path and discarded the other, and `AutodiffTensor::clone` minted a fresh `TensorId`. Together, reusing a tensor (residuals, `mul(s, s)`, shared Q/K/V — universal in transformers) split its gradient across two ids and summed neither, yielding a fraction of the true value. `accumulate_gradients` now sums via `B::add`; `clone` preserves the id so fan-out paths collide on one sink. 3. Softmax backward panicked. `SoftmaxBackward` / `stable_softmax_backward` subtracted a keep-dim row-sum from the full-shape grad, but the elementwise backends assert equal shapes (no broadcasting). Added `broadcast_along_dim` to tile the row-sum to full width first. Verified: `tape_cpu_gradcheck` (matmul, fan-out add·mul, softmax) passes with rel-err < 2e-2 vs central differences; full `rtx-autograd` suite green (263 passed, 0 failed); lib clippy `-D warnings` clean. Known follow-up (out of scope): `cargo miri test` still aborts on a Stacked-Borrows / integer-to-pointer violation inside `rtx-backend-cpu`'s buffer internals — a grad-free `from_data`+`add`+`sum` probe reproduces the identical error, so it is pre-existing backend UB, not an autograd issue. Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>	2026-06-15 21:06:32 -07:00
osobh	d960946557	Merge pull request 'GPU Mamba forward + rtx-tensor device-pointer API + matrix exponential' (#2) from gpu-mamba-forward-and-matrix-exp into main	2026-06-16 01:52:22 +00:00
osobh	5c2ee63d53	GPU Mamba forward + rtx-tensor device-pointer API + matrix exponential ... Persist accumulated WIP across rtx-tensor / rtx-transformers / rtx-runtime. Two related feature groups: GPU enablement (unblocks the Phase-3 spec §8 CUDA Mamba path): - rtx-tensor: `Tensor::cuda_device_ptr()` + storage GPU-buffer accessors (`raw_ptr.rs`) expose the raw CUdeviceptr that kernel launches need — the "rtx-tensor GPU memory access API" the Mamba CUDA kernels were blocked on. - rtx-transformers: `MambaBlock::forward_cuda` runs the four linear projections through cuBLAS on-device (in/out/x/dt_proj), keeping the selective scan + conv1d + activations on CPU; dispatched automatically from `forward` when on a CUDA device under the `cuda` feature. Updated `mamba_cuda_kernels.rs` accordingly. - supporting plumbing in rtx-runtime stream/bridge and rtx-tensor storage/conversion/concatenation/creation + rtx-flash-attention. Linear algebra (rtx-tensor): - `linalg/matrix_exp.rs`: real matrix exponential via scaling-and-squaring with a degree-13 Padé approximant (Higham 2005), f64 internally. - `complex/linalg.rs`: complex matmul/adjoint, Hermitian eigendecomposition (`ComplexEigenResult`), and the complex matrix exponential, nalgebra-backed. - tests for both. Builds verified on the CPU path (`cargo check -p rtx-tensor -p rtx-transformers -p rtx-runtime -p rtx-flash-attention` clean). The `cuda` feature and the rtx-backend-cuda NVCC build remain unbuildable on this host (CUDA/glibc header mismatch) — pre-existing and unrelated to these changes.	2026-06-15 18:51:30 -07:00
osobh	627db58d09	Merge pull request 'Real selective-scan Mamba: forward + gradient-checked analytic backward + trainable' (#1) from real-selective-scan-mamba into main	2026-06-03 04:32:36 +00:00
osobhandClaude Opus 4.7	15b62a8f6e	rtx-interpret: fix decoder transpose in SAE compute_and_apply_gradients ... The encoder-gradient path through the decoder was transposing the decoder before the matmul, producing `[batch, d_model] × [d_sae, d_model]` — a shape mismatch for every batch > 1. The decoder is stored as `[d_model, d_sae]`, so `recon_grad @ decoder` is already the right shape (and matches the comment at the call site, which reads "recon_grad @ decoder @ d_relu"). All 9 existing `sae::tests` still pass. Omni-Cortex's `LatentDictionary` now trains correctly on batches larger than 1. Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>	2026-04-24 13:58:44 -07:00