The Backend trait's index_select and index_add have default bodies that
round-trip through host memory. That is correct everywhere and was the only
implementation CUDA had. Graph message passing is made of these two ops, so
dg-gnn's HetGAT paid a device->host->device copy per layer per pass and the
RTX 5060 Ti sat at ~10% utilisation during training.
Design follows rtx-backend-metal's ops::index: gather is one thread per output
element; scatter-add walks the CSR of the adjoint selection matrix S^T, built
host-side by counting sort, so it needs NO atomics and is deterministic with
duplicate indices — the training loss is bit-identical to the host reference.
Device index buffers are cached per thread keyed by the exact index list, so a
static graph topology uploads once. Two small NVRTC kernels; no cuSPARSE.
Measured on dg-gnn, Harris 42,955 links, v8 recipe, RTX 5060 Ti:
training batch 8 9,042 -> 3,562 ms/step (2.5x)
inference single p50 55.4 -> 12.9 ms (4.3x)
inference batch 8 257 -> 20 ms/scen (13x; batching helps again)
GPU utilisation median 10% -> 21%, p90 17% -> 43%
Tests: gather with repeats, scatter-add with duplicates and untouched rows,
the adjoint identity <S x, y> == <x, S^T y> (what autograd relies on), a
hub-heavy pattern against the host reference, and the range-check panic.
rtx-backend-cuda --features cuda: 60 + 16 passed, 0 failed.
Co-Authored-By: Claude Opus 5 <[email protected]>
Found by running dg-gnn's HetGAT training on an RTX 5060 Ti: the first backward
pass panicked in transpose_2d with CUDA_ERROR_INVALID_VALUE. The tensor was a
[4,211,136 x 1] gradient. The tiled kernel maps rows onto grid.y, and CUDA caps
grid.y at 65,535 blocks, so anything past ~2.1M rows was rejected at launch.
Inference never hit it — the forward pass has no transposes.
Three fixes, each tested on the GPU:
- A row or column vector transposes to itself in memory ([N,1] and [1,N] are
the same N floats). swap_dims now shares the buffer and swaps the shape; no
kernel. This is the case that actually failed.
- transpose_2d_gpu hands matrices with rows/32 > 65,535 to the 1-D generic
permute, whose grid.x allows 2^31-1 blocks.
- transpose_generic_gpu passed the INPUT's strides (swapped) as the kernel's
output_strides — but the kernel DECODES each flat output index with those,
so they must be the contiguous strides of the output shape. For a 2-D input
that was [1, cols], which decodes every index to itself: the "permute" was
a plain copy. Verified against the original code — a [2,3,4] swap_dims(0,2)
returned b[1][0][0] = 12.0 where 1.0 is correct. Every higher-D dim swap
went through this path; nothing had checked it numerically.
Also stamps kernel outputs with the contiguous strides of their new shape
rather than the input's strides swapped. Nothing outside ops/shape.rs reads
strides today, so that was latent, but is_contiguous() now tells the truth.
rtx-backend-cuda --features cuda: 55 + 16 passed, 0 failed (was 51 + 16; four
new tests, one of which fails on the original code for each bug above).
Co-Authored-By: Claude Opus 5 <[email protected]>
Whole-workspace rustfmt pass picked up while iterating on Mamba GPU
backward work. Verified formatting-only via diff sampling; no logic
changed.
Co-Authored-By: Claude Sonnet 5 <[email protected]>
The workspace root was upgraded to thiserror = "2" in an earlier commit,
but 56 per-crate Cargo.toml files still independently declared "1.0".
These crates do not use workspace.dependencies inheritance for thiserror.
All updated to thiserror = "2" for complete fleet alignment.
Includes: rtx-backend, rtx-tensor, rtx-losses, rtx-backend-cuda/rocm/metal,
all training crates (rtx-auto, rtx-rl, rtx-distributed, rtx-federated, etc.),
specialized crates (rtx-science, rtx-platform, rtx-nmf, rtx-neuro-*),
production crates (rtx-streaming, rtx-serving-api), and all demo crates.
cargo check --workspace: PASSES.
