Omar SobhandClaude Opus 5
4358964c05
55 of 85 role skill bindings pointed at skills that were never authored,
so 10 of 11 team templates bound a smaller context bundle than their role
prompts assumed. Three roles bound nothing at all (gpu.bench_engineer,
threejs.shader_author, threejs.perf_engineer) while their prompts described
procedures they had no way to read.
The loader comment at team_template_loader.rs:167 already diagnosed this —
snake_case slugs in TOML against kebab-case skill files — and it was
half-fixed: the kebab names were corrected, the snake_case ones left.
It was invisible because both existing tests assert authored ⊆ referenced
(30/30, green) and the second explicitly declines to check the other
direction. So the failing half was the half nobody asserted.
Resolved every name by one of three explicit choices:
- 23 skills authored where the role genuinely needed the procedure
(gpu, threejs, research, analysis, frontend, mobile, backend, platform)
- renames onto authored skills where one existed in substance, including
the four-near-duplicate cases that collapse onto one real skill
- 22 aspirational references deleted — a binding an agent cannot read is
a promise, not a capability
Two tests now hold it. The unit test checks referenced ⊆ authored against
the files. The new integration test runs both loaders in boot order and
asserts the bindings survive the trip through the database, which is a
different question: resolution goes through skills_catalog rows, so a skill
file that exists but fails to ingest still leaves the role empty.
Negative controls: the unit test failed naming all 55; the integration test
fails naming the exact role when one name is reverted.
threejs.shader_author and .perf_engineer gained a second and third skill
after the collapse — pin_in_context pins idx < 2, so a role left with one
skill silently pins less than the policy intends.
Co-Authored-By: Claude Opus 5 <[email protected]>
2.3 KiB
name, description, when_to_use, tags
| name | description | when_to_use | tags | ||
|---|---|---|---|---|---|
| mobile-e2e-and-simulators | Running mobile tests that mean something — simulator versus device, and what only fails on real hardware. | You are the tester on a mobile team setting up or debugging end-to-end runs. |
|
The simulator is not the device
Simulators are excellent for layout and flow, and actively misleading for anything touching hardware, permissions or performance. Know which question you are answering.
What the simulator answers honestly
Layout across screen sizes, navigation flows, most business logic, accessibility labels, localisation. Run these in CI on simulators — they are fast, parallel and deterministic.
What only a real device answers
- Performance. A simulator uses your host CPU and GPU. Frame rate, memory pressure and battery there are meaningless.
- Permissions and their denial paths. The interesting flow is "user said no", and simulator permission behaviour differs.
- Camera, GPS, biometrics, push. Approximated or absent.
- Network transitions. Wi-Fi to cellular, offline, captive portals. This is where the network layer's assumptions surface.
- Keyboard behaviour. Third-party keyboards and predictive input change layout in ways the simulator's keyboard does not.
Determinism in E2E
The mobile equivalents of the browser rules: select by accessibility id rather than by position, wait on conditions rather than durations, and reset app state between tests. A test depending on the previous test's leftover state is the most common cause of a suite that passes locally and fails in CI.
Cold start versus warm start matters more than on web: a test that only ever runs warm never exercises launch-path initialisation, which is where a surprising share of crashes live.
Keep the CI matrix honest
Two simulators — the smallest supported and the newest — catch most layout regressions cheaply. Add one physical device for the release candidate, not for every commit; a device farm on every push is expensive and rarely finds what the simulator missed except on the axes listed above.
Record which it was
A test result without its target is unactionable. "Passed on iPhone 15 simulator, iOS 18" and "passed on a physical Pixel 6" are different claims, and only the second one says anything about performance.