osobh and Claude Sonnet 5
4aaa36a57a
style: cargo fmt --workspace (whitespace/wrapping only, no semantic change)
...
Whole-workspace rustfmt pass picked up while iterating on Mamba GPU
backward work. Verified formatting-only via diff sampling; no logic
changed.
Co-Authored-By: Claude Sonnet 5 <[email protected]
2026-08-10 07:09:36 -07:00
Omar Sobh and Claude Sonnet 4.6
7c8e9a8a35
feat(batch8): multi-token prediction heads, sparse attention, length bucketing
...
CI / Format Check (push) Failing after 23s
Documentation / Build User Guide (push) Successful in 14s
Documentation / Build API Documentation (push) Failing after 16s
CI / Clippy Check (push) Failing after 16s
Performance Benchmarks / Run Benchmarks (push) Failing after 30s
CI / Build (ubuntu-latest) (push) Failing after 53s
CI / Build CPU-Only (Explicit) (push) Failing after 1m4s
CI / Test (ubuntu-latest) (push) Has been skipped
CI / Python Bindings (maturin) (macos-latest) (push) Has been skipped
CI / Python Bindings (maturin) (ubuntu-latest) (push) Has been skipped
CI / WASM Build + Size Check (push) Has been skipped
CI / Distributed Training Tests (push) Has been skipped
CI / Build (macos-latest) (push) Failing after 35s
CI / Test (macos-latest) (push) Has been skipped
CI / CI Success (push) Failing after 0s
Multi-Token Prediction heads (rtx-transformers/gpt):
- MtpConfig { num_future_tokens=4, loss_weight=0.3 }; MultiTokenPredictionHead
with k independent [hidden, vocab] weight matrices; forward() → Vec<Tensor>
- compute_loss(): log-softmax NLL for each offset 1..k; weighted by loss_weight;
MtpLossResult with per_head_losses + is_valid(); 12 tests
Sparse attention (rtx-transformers/layers):
- SparseAttentionMask: local window (radius w), global tokens (first g attend
all + all attend them), random long-range (r symmetric positions per token)
- LCG seeded for reproducibility; apply_to_scores() masks to -inf; to_additive_bias()
- SparseAttentionLayer::forward_cpu() skips masked pairs early; numerically-stable
softmax; sparsity 75% at n=512, 87% at n=1024, 97% at n=4096; 14 tests
Sequence length bucketing (rtx-transformers/training):
- LengthGroupedSampler: Fisher-Yates per-bucket shuffle, token-budget batching,
overflow bucket for long sequences; padding_efficiency() vs baseline_efficiency()
- pack_into_batch(): greedy first-fit packing; naive_padding_ratio() baseline metric
- Measured 2.2× padding reduction on power-law data (26%→67% efficiency); 18 tests
Co-Authored-By: Claude Sonnet 4.6 <[email protected]
2026-06-27 04:16:14 +00:00
Omar Sobh and Claude Sonnet 4.6
39b7ef12f4
fix(tests): green-bar rtx-transformers and rtx-distributed test suites
...
- Fix 35 doctest failures in Phase 2/3 modules (no_run annotations, missing
imports, wrong API calls, Result context issues)
- Fix test_validation_framework_creation: assert updated to 1e-3 default
- Fix test_report_serialization: replace exact f64 equality with epsilon comparison
- Fix rtx-distributed recovery/tests.rs: add missing ProcessGroup import,
use recovery_stats().wal_buffer_size instead of private field access
All rtx-transformers and rtx-distributed tests now pass.
Co-Authored-By: Claude Sonnet 4.6 <[email protected]
2026-06-26 17:56:39 +00:00
Omar Sobh and Claude Sonnet 4.6
228137555f
fix(gaps): G0/G2/G5/G8 — eliminate unimplemented! panics, re-enable rtx-distributed, rtx-tts, fix multimodal forward
...
G0 (Critical): Replace 45 unimplemented!() panics across three GPU backends
- rtx-backend-cuda: sin/cos/tanh via PTX, relu/sigmoid/leaky_relu/elu via activation.rs,
pow/clamp/gt_scalar via unary.rs, var/var_dim host-side, conv2d/max_pool2d/avg_pool2d
CPU fallback in new ops/conv.rs; new PTX kernels in element_wise.cu
- rtx-backend-rocm: all 15 ops via CPU round-trip (to_vec → compute → from_slice)
- rtx-backend-sycl: all 15 ops via CPU round-trip (to_host → compute → from_data)
G2 (High): Re-add rtx-distributed to workspace
- Vendor 4 minimal RNCCL stub crates at crates/vendor/rnccl/*
- Update rtx-distributed RNCCL path deps to point at stubs (../../../../RNCCL/* → ../../vendor/rnccl/*)
- Remove rtx-distributed from workspace exclude list, add to members
G5 (Medium): Re-enable rtx-tts (213 tests restored)
- Fix 15 rtx-nn API drift issues: LayerNorm::new, Conv1d::from_config, Conv1dPadding::Zeros,
Dropout::new(p, device), tensor methods (relu/tanh/sigmoid/cat/stack), squeeze(Some(n)),
to_vec() turbofish removal, Tensor::randn with &[...] slices
G8 (Low): Quantum stubs + multimodal forward bug
- rtx-timeseries: remove dead quantum/neuromorphic TODO comment blocks (no module files exist)
- rtx-multimodal/fusion/transformer.rs: wire TransformerBlock loop in forward()
- rtx-multimodal/fusion/strategies.rs: wire bottleneck_layers loop in forward()
- rtx-transformers/architectures/transformer_block.rs: add forward() method (pre-norm residuals;
full attention+FFN pending when those sub-layers are wired)
Co-Authored-By: Claude Sonnet 4.6 <[email protected]
2026-06-26 13:40:23 +00:00
osobh and Claude Opus 4.6
02d382d5f6
style: apply rustfmt across all crates and demos
...
Consistent formatting pass: line wrapping, import sorting, trailing
whitespace removal, let-chain indentation, merged derive attributes,
and unsafe block reformatting.
Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]
2026-04-12 07:01:58 -07:00
redclawsystems
4d88dc0584
Initial commit
2026-03-04 00:08:42 +00:00
