Omar SobhandClaude Opus 5 796b8487ad rtx-backend-cuda: native index_select / index_add — HetGAT training 2.5x, inference 4.3x ...

The Backend trait's index_select and index_add have default bodies that
round-trip through host memory. That is correct everywhere and was the only
implementation CUDA had. Graph message passing is made of these two ops, so
dg-gnn's HetGAT paid a device->host->device copy per layer per pass and the
RTX 5060 Ti sat at ~10% utilisation during training.

Design follows rtx-backend-metal's ops::index: gather is one thread per output
element; scatter-add walks the CSR of the adjoint selection matrix S^T, built
host-side by counting sort, so it needs NO atomics and is deterministic with
duplicate indices — the training loss is bit-identical to the host reference.
Device index buffers are cached per thread keyed by the exact index list, so a
static graph topology uploads once. Two small NVRTC kernels; no cuSPARSE.

Measured on dg-gnn, Harris 42,955 links, v8 recipe, RTX 5060 Ti:

  training batch 8      9,042 -> 3,562 ms/step   (2.5x)
  inference single p50   55.4 ->  12.9 ms        (4.3x)
  inference batch 8       257 ->    20 ms/scen   (13x; batching helps again)
  GPU utilisation       median 10% -> 21%, p90 17% -> 43%

Tests: gather with repeats, scatter-add with duplicates and untouched rows,
the adjoint identity <S x, y> == <x, S^T y> (what autograd relies on), a
hub-heavy pattern against the host reference, and the range-check panic.
rtx-backend-cuda --features cuda: 60 + 16 passed, 0 failed.

Co-Authored-By: Claude Opus 5 <[email protected]>

2026-09-11 23:09:01 -05:00

..

benches

Initial commit

2026-03-04 00:08:42 +00:00

examples

Initial commit

2026-03-04 00:08:42 +00:00

src

rtx-backend-cuda: native index_select / index_add — HetGAT training 2.5x, inference 4.3x

2026-09-11 23:09:01 -05:00

tests

Initial commit

2026-03-04 00:08:42 +00:00

build.rs

feat(perf): GPU perf batch 1 — wire GPU execution paths for FP8, FA3, CUDA Graphs, SnapKV

2026-06-26 20:39:58 +00:00

Cargo_workspace.toml

Initial commit

2026-03-04 00:08:42 +00:00

Cargo.toml

fix(ci,rtx-tensor): resolve clippy --all-features intel-mkl conflict; gate MKL to x86_64-linux

2026-06-27 08:12:27 -07:00

Cargo.toml.backup

Initial commit

2026-03-04 00:08:42 +00:00

check_least_squares_api.rs

Initial commit

2026-03-04 00:08:42 +00:00

contract_path_verification.rs

Initial commit

2026-03-04 00:08:42 +00:00

einsum_fusion_comprehensive_test.rs

Initial commit

2026-03-04 00:08:42 +00:00

einsum_fusion_standalone_test.rs

Initial commit

2026-03-04 00:08:42 +00:00

ifft_verification_test.rs

Initial commit

2026-03-04 00:08:42 +00:00

manual_linalg_test.rs

Initial commit

2026-03-04 00:08:42 +00:00

manual_lu_test.rs

Initial commit

2026-03-04 00:08:42 +00:00

tensordot_smoke_test.rs

Initial commit

2026-03-04 00:08:42 +00:00

validate_phase_magnitude.rs

Initial commit

2026-03-04 00:08:42 +00:00