Each task owns a disjoint BLOCK_SIZE-row slice of the result; the inner
blocked kernel is unchanged. Needed for dg-gnn HetGAT training throughput
(node-level [M,64]x[64,64] matmuls dominated single-threaded step time).
Co-Authored-By: Claude Fable 5 <[email protected]>
Add two row-indexing ops along dim 0 to the `Backend` trait so gather /
scatter-add message passing (GNNs, segment softmax, bias tiling) can be
trained through `Autodiff<B>`:
- `index_select(tensor, indices)` — out[i, ..] = tensor[indices[i], ..]
- `index_add(tensor, indices, num_rows)` — out = zeros; out[idx[i], ..] += tensor[i, ..]
They are each other's adjoint, which is what the backward passes use.
Both trait methods have default bodies (host round-trip via to_data /
from_data) so every existing backend keeps compiling and is correct;
backends override with native kernels:
- rtx-backend-cpu: new ops/index.rs (rayon-parallel gather over output
rows above a size threshold, sequential deterministic scatter-add),
wired into CpuBackend and CpuBackendF64, with unit tests for D=1/2/3,
duplicates, untouched rows, empty inputs, bounds panics and adjointness.
- rtx-autograd: Autodiff<B> overrides both ops and records
IndexSelectBackward / IndexAddBackward (new ops/index.rs); finite-
difference gradchecks on the real CpuBackend cover repeated-index
accumulation, untouched-row zero grads, bias tiling via index_select
of a [1,F] row, and a full per-segment softmax.
- rtx-fusion: forward both ops to the inner backend.
Co-Authored-By: Claude Fable 5 <[email protected]>
Whole-workspace rustfmt pass picked up while iterating on Mamba GPU
backward work. Verified formatting-only via diff sampling; no logic
changed.
Co-Authored-By: Claude Sonnet 5 <[email protected]>
Phase 1 of the rustytorch f32→f64 plan, backend layer. All 11 ops modules
(basic/creation/unary/gemm/reduction/activation/shape/conv/pooling/normalization/
attention) are now generic over the element via a `CpuFloat` bound
(`num_traits::Float + Send + Sync + 'static`); f32/Vec<f32> → E/Vec<E>, literals →
E::zero()/one()/from(..). The ops were already pure scalar + rayon (no SIMD), so
the f32 path is byte-identical (E inferred as f32 under CpuBackend) — no SIMD/BLAS
dual-path needed.
Adds `CpuBackendF64` (FloatElem = f64, TensorPrimitive = CpuTensorPrimitive<D,f64>)
delegating to the same generic ops, plus the DeviceOps<CpuBackendF64> impl.
CpuBackend (f32) untouched.
Validated: 35 tests pass (33 original f32 + 2 new f64); `cpu_backend_f64_exceeds_
f32_precision` preserves 1+2^-30 (f32 rounds to 1.0) — proves genuine f64. rtx-tensor
(dependent) still builds. clippy clean.
Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
