---
name: roofline-model
description: The roofline model — is your kernel compute-bound or memory-bound? Estimate arithmetic intensity before optimizing.
when_to_use: You're the arch_analyst or bench_engineer role sizing up a GPU kernel before or after implementation.
tags: [gpu, performance, analysis]
---

# The roofline model

Before you optimize anything, know which wall you're hitting.

## The two rooflines

- **Peak memory bandwidth** — how many bytes/sec your GPU can pull from HBM.
- **Peak compute** — how many FLOPs/sec your GPU can execute.

Where they meet defines the **ridge point** (in FLOPs / byte). Kernels to the left of the ridge are memory-bound; to the right, compute-bound.

## Reference numbers (2026)

| Device | Peak BW (TB/s) | Peak FP32 (TFLOP/s) | FP16/BF16 tensor | Ridge (FLOP/B, FP32) |
|---|---|---|---|---|
| NVIDIA B200 (Blackwell) | 8.0 | 80 | ~2200 tensor | ~10 |
| NVIDIA H100 | 3.35 | 67 | 989 tensor | ~20 |
| AMD MI300X (CDNA3) | 5.3 | 163 | ~2600 tensor | ~31 |
| Apple M3 Max | 0.4 | 14 | — | ~35 |

*Rough figures — verify against the actual SKU. Ratios matter more than absolutes.*

## Arithmetic intensity

FLOPs performed per byte read from HBM. Compute it BEFORE writing the kernel:

```
GEMM (A: MxK, B: KxN, C: MxN, all FP32):
  FLOPs   = 2*M*N*K
  Bytes   = 4*(M*K + K*N + M*N)                  (naive, no tiling)
  AI      = 2*M*N*K / (4*(M*K + K*N + M*N))
  For M=N=K=1024: AI ≈ 170 FLOP/B  → compute-bound on H100
  For M=N=1024, K=32: AI ≈ 15 FLOP/B → memory-bound on H100
```

## What the diagnosis tells you

**Memory-bound** — DON'T optimize the math. Reduce bytes:
- Fuse kernels to keep data in registers/shared.
- Use lower precision (FP16/BF16/INT8) if numerics allow.
- Better tiling to reuse loaded data.
- Coalesce (see [[gpu-coalescing-and-occupancy]]).

**Compute-bound** — DON'T optimize the loads. Feed the ALU:
- Use tensor cores (Nvidia) / matrix cores (AMD) / AMX / metal Simd matrix ops.
- Instruction-level parallelism (multiple independent FMAs per thread).
- Higher occupancy is often COUNTERPRODUCTIVE — you want registers, not more threads.

**Balanced (near the ridge)** — hardest. Small changes tip you into one regime or the other. Profile OFTEN.

## Reporting

Every kernel benchmark report includes:
```
Kernel:                 gemm_fp32_tiled
Achieved throughput:    52 TFLOP/s  (78% of 67 TFLOP/s peak)
Achieved bandwidth:     280 GB/s    (8% of 3.35 TB/s peak)
Arithmetic intensity:   180 FLOP/B
Regime:                 compute-bound
Bottleneck to attack:   tensor core underutilization — WMMA fragment misaligned
```

## Anti-patterns

- **Optimizing math on a memory-bound kernel.** Doubling FLOPs while DRAM saturates changes nothing.
- **Micro-benchmarking without measuring HBM traffic.** Wall clock alone can't tell you which regime you're in.
- **Skipping roofline "because we already know it's slow".** You don't know the CEILING until you plot it.