Omar SobhandClaude Opus 5
4358964c05
55 of 85 role skill bindings pointed at skills that were never authored,
so 10 of 11 team templates bound a smaller context bundle than their role
prompts assumed. Three roles bound nothing at all (gpu.bench_engineer,
threejs.shader_author, threejs.perf_engineer) while their prompts described
procedures they had no way to read.
The loader comment at team_template_loader.rs:167 already diagnosed this —
snake_case slugs in TOML against kebab-case skill files — and it was
half-fixed: the kebab names were corrected, the snake_case ones left.
It was invisible because both existing tests assert authored ⊆ referenced
(30/30, green) and the second explicitly declines to check the other
direction. So the failing half was the half nobody asserted.
Resolved every name by one of three explicit choices:
- 23 skills authored where the role genuinely needed the procedure
(gpu, threejs, research, analysis, frontend, mobile, backend, platform)
- renames onto authored skills where one existed in substance, including
the four-near-duplicate cases that collapse onto one real skill
- 22 aspirational references deleted — a binding an agent cannot read is
a promise, not a capability
Two tests now hold it. The unit test checks referenced ⊆ authored against
the files. The new integration test runs both loaders in boot order and
asserts the bindings survive the trip through the database, which is a
different question: resolution goes through skills_catalog rows, so a skill
file that exists but fails to ingest still leaves the role empty.
Negative controls: the unit test failed naming all 55; the integration test
fails naming the exact role when one name is reverted.
threejs.shader_author and .perf_engineer gained a second and third skill
after the collapse — pin_in_context pins idx < 2, so a role left with one
skill silently pins less than the policy intends.
Co-Authored-By: Claude Opus 5 <[email protected]>
51 lines
2.5 KiB
Markdown
51 lines
2.5 KiB
Markdown
---
|
||
name: gpu-profiling-workflow
|
||
description: Using Nsight, rocprof and Metal frame capture to find where a kernel actually spends time, instead of guessing.
|
||
when_to_use: You are the bench engineer on a GPU team and need to explain or improve a kernel's runtime.
|
||
tags: [gpu, profiling]
|
||
---
|
||
|
||
# Measure the machine, not your model of it
|
||
|
||
GPU intuition is unusually unreliable: the bottleneck is far more often memory
|
||
movement or occupancy than arithmetic. Profile before changing anything.
|
||
|
||
## Order of questions
|
||
|
||
1. **Is the GPU busy at all?** Kernel time versus wall time. A "slow kernel"
|
||
that occupies 8% of wall time is a host-side or transfer problem, and no
|
||
amount of kernel tuning will show up.
|
||
2. **Memory or compute bound?** Achieved bandwidth against the device peak, and
|
||
achieved FLOPs against peak. See `roofline-model` — the roofline tells you
|
||
which ceiling you are under, and therefore which optimisations can possibly
|
||
help.
|
||
3. **Occupancy?** Only after 1 and 2. Occupancy is a means, not a goal: a
|
||
kernel at 40% occupancy saturating bandwidth is finished, and raising
|
||
occupancy will not make it faster.
|
||
|
||
## The tools
|
||
|
||
- **Nsight Compute** (`ncu`) — per-kernel counters. Start with
|
||
`--set full` on ONE kernel invocation, not the whole run; it serialises and
|
||
replays kernels, so a full-application profile takes minutes and changes the
|
||
timing you were trying to measure.
|
||
- **Nsight Systems** (`nsys`) — the timeline. This is where question 1 is
|
||
answered: gaps between kernels, host-device copies, stream serialisation.
|
||
Use it *first*; `ncu` optimises a kernel that `nsys` may show is irrelevant.
|
||
- **rocprof** — the ROCm equivalent. `--stats` for the summary,
|
||
`--hip-trace`/`--hsa-trace` for the timeline.
|
||
- **Metal frame capture** — Xcode's GPU capture. Per-encoder timings and the
|
||
shader profiler's per-line cost. It is a *frame* capture: for compute work,
|
||
bracket the dispatch in a capture scope explicitly or you get nothing.
|
||
|
||
## Warm up, and say what you measured
|
||
|
||
First-call timings include JIT compilation, allocator growth and page faults —
|
||
routinely 10-100× the steady state. Discard warmup iterations and report a
|
||
distribution, not a single number: a median with a spread tells the reader
|
||
whether the change is real. See `criterion-benchmarking` for the statistics.
|
||
|
||
Record the device, driver version and clock state alongside the number. GPUs
|
||
throttle; a measurement without its conditions cannot be compared to the one
|
||
you take next month.
|