osobhandClaude Sonnet 5 5155c081ca feat(mamba): GPU-accelerated backward pass (backward_cuda) ...

MambaBlock::forward already had a working, tested CUDA dispatch
(forward_cuda: cuBLAS matmuls for projections, CPU for the scan).
backward() had none — it silently ran entirely CPU-serial on GPU
tensors via to_vec()/from_vec() D2H/H2D round-trips. This adds the
missing acceleration, mirroring forward_cuda's hybrid split: the
four large projection-parameter gradients (in_proj, x_proj, dt_proj,
out_proj) now go through batched GPU matmuls; the inherently
sequential scan reverse-pass and small per-channel grads stay CPU.

Extracted CpuWeights::pull and recompute_forward_cpu out of the old
inline per-batch forward-recompute block inside backward() (pure
refactor, gradient-checked unchanged by real_selective_scan.rs's
existing 6 tests) so CPU backward and the new CUDA backward share
identical forward math and can never numerically diverge on it.

New CUDA-vs-CPU gradient-check test (mamba_cuda_backward_matches_cpu,
#[ignore]-gated, GPU-only) caught a real bug during development:
Tensor::contiguous() is a no-op stub in this rtx-tensor version, and
cuda_matmul reads raw GPU storage by shape.dims() ignoring
strides/offset, so .transpose(..).matmul(..) on a GPU tensor silently
computed garbage (80-200x relative error on 3 of 4 accelerated
gradients). Fixed by building already-transposed [dim, b*l] buffers
on CPU before upload instead of transposing GPU-side. All 9 gradients
now match CPU backward within ~2.2e-5 max relative error (tolerance
1e-4).

Co-Authored-By: Claude Sonnet 5 <[email protected]>

2026-08-10 07:11:20 -07:00

..

bert_device_test.rs

Initial commit

2026-03-04 00:08:42 +00:00

cross_entropy_test.rs

Initial commit

2026-03-04 00:08:42 +00:00

d350_gpu_backbone_training.rs

fix(gaps): G4 — re-enable all rtx-transformers Phase 2/3 modules (222 compile errors fixed)

2026-06-26 16:08:02 +00:00

device_usage_test.rs

Initial commit

2026-03-04 00:08:42 +00:00

integration_example_test.rs

Initial commit

2026-03-04 00:08:42 +00:00

logcosh_loss_test.rs

Initial commit

2026-03-04 00:08:42 +00:00

mamba_cuda_backward_matches_cpu.rs

feat(mamba): GPU-accelerated backward pass (backward_cuda)

2026-08-10 07:11:20 -07:00

mamba_step_equivalence.rs

SMT E3a: expose Mamba's recurrent state as a single-step updater (equivalence-pinned)

2026-06-16 11:00:12 -07:00

mamba_temporal_learning.rs

fix(gaps): G4 — re-enable all rtx-transformers Phase 2/3 modules (222 compile errors fixed)

2026-06-26 16:08:02 +00:00

neural_ode_tests.rs

Initial commit

2026-03-04 00:08:42 +00:00

real_selective_scan.rs

fix(gaps): G4 — re-enable all rtx-transformers Phase 2/3 modules (222 compile errors fixed)

2026-06-26 16:08:02 +00:00