---
name: gpu-kernel-authoring
description: Writing the same kernel across CUDA, Metal and ROCm without three divergent implementations, and the FFI boundary back into Rust.
when_to_use: You are the kernel author on a GPU team, adding or changing a compute kernel.
tags: [gpu, kernels]
---

# One kernel, three dialects

The three targets differ less than they appear. Thread indexing, memory scopes
and barriers all map onto each other; what differs is spelling and launch
configuration. Write the algorithm once, then port the spelling.

| Concept | CUDA | Metal | ROCm/HIP |
|---|---|---|---|
| thread id | `threadIdx.x` | `thread_position_in_threadgroup` | `hipThreadIdx_x` |
| block id | `blockIdx.x` | `threadgroup_position_in_grid` | `hipBlockIdx_x` |
| shared mem | `__shared__` | `threadgroup` | `__shared__` |
| barrier | `__syncthreads()` | `threadgroup_barrier(mem_flags::mem_threadgroup)` | `__syncthreads()` |
| warp/wave | 32 | 32 (SIMD-group) | **64** on CDNA, 32 on RDNA |

**The wave-size difference is the one that silently produces wrong answers.**
Any kernel that assumes 32 lanes — a warp-shuffle reduction, a ballot, an
implicit intra-warp sync — is incorrect on CDNA. Query it (`warpSize`,
`[[threads_per_simdgroup]]`) rather than hardcoding, and never rely on implicit
lockstep: it was never guaranteed on CUDA either since Volta's independent
thread scheduling.

## Get it correct on one target first

Write the scalar CPU reference, then the first GPU version, and diff the outputs
before porting anywhere. A kernel ported three ways from an unverified original
gives you three wrong answers and no baseline to find them with.

Compare with a tolerance derived from the operation, not a guess: float addition
is not associative, so a parallel reduction legitimately differs from a
sequential one. `1e-6` on an accumulation over a million elements is a failing
test that is not a bug.

## The FFI boundary

Kernels are reached from Rust through `extern "C"`. Three rules that prevent the
majority of crashes at this seam:

- **Own the allocation on one side.** Device memory allocated in C and freed in
  Rust's `Drop` — or the reverse — is a lifetime nobody can read. Wrap the
  device pointer in a Rust type whose `Drop` calls the same allocator's free.
- **Every launch returns a status; check it.** A kernel launch failure is
  asynchronous and surfaces on the *next* synchronising call, so an unchecked
  launch reports its error somewhere unrelated. Check after launch AND after
  synchronise.
- **`#[repr(C)]` on every struct crossing the boundary.** Rust's default layout
  is unspecified and does change.

## What to write down

A kernel's launch geometry is a decision, not a constant: record why the block
size is what it is (occupancy target, shared-memory budget, register pressure)
next to it. The next person to change the shared-memory allocation needs to know
the block size was chosen against it.