fix(skills): every team-template skill binding now resolves

55 of 85 role skill bindings pointed at skills that were never authored,
so 10 of 11 team templates bound a smaller context bundle than their role
prompts assumed. Three roles bound nothing at all (gpu.bench_engineer,
threejs.shader_author, threejs.perf_engineer) while their prompts described
procedures they had no way to read.

The loader comment at team_template_loader.rs:167 already diagnosed this —
snake_case slugs in TOML against kebab-case skill files — and it was
half-fixed: the kebab names were corrected, the snake_case ones left.

It was invisible because both existing tests assert authored ⊆ referenced
(30/30, green) and the second explicitly declines to check the other
direction. So the failing half was the half nobody asserted.

Resolved every name by one of three explicit choices:

  - 23 skills authored where the role genuinely needed the procedure
    (gpu, threejs, research, analysis, frontend, mobile, backend, platform)
  - renames onto authored skills where one existed in substance, including
    the four-near-duplicate cases that collapse onto one real skill
  - 22 aspirational references deleted — a binding an agent cannot read is
    a promise, not a capability

Two tests now hold it. The unit test checks referenced ⊆ authored against
the files. The new integration test runs both loaders in boot order and
asserts the bindings survive the trip through the database, which is a
different question: resolution goes through skills_catalog rows, so a skill
file that exists but fails to ingest still leaves the role empty.

Negative controls: the unit test failed naming all 55; the integration test
fails naming the exact role when one name is reverted.

threejs.shader_author and .perf_engineer gained a second and third skill
after the collapse — pin_in_context pins idx < 2, so a role left with one
skill silently pins less than the policy intends.

Co-Authored-By: Claude Opus 5 <[email protected]>
This commit is contained in:
Omar Sobh
2026-08-19 07:42:48 -07:00
co-authored by Claude Opus 5
parent ba98c29481
commit 4358964c05
35 changed files with 1392 additions and 28 deletions
@@ -0,0 +1,54 @@
---
name: mobile-e2e-and-simulators
description: Running mobile tests that mean something — simulator versus device, and what only fails on real hardware.
when_to_use: You are the tester on a mobile team setting up or debugging end-to-end runs.
tags: [mobile, testing]
---
# The simulator is not the device
Simulators are excellent for layout and flow, and actively misleading for
anything touching hardware, permissions or performance. Know which question you
are answering.
## What the simulator answers honestly
Layout across screen sizes, navigation flows, most business logic, accessibility
labels, localisation. Run these in CI on simulators — they are fast, parallel and
deterministic.
## What only a real device answers
- **Performance.** A simulator uses your host CPU and GPU. Frame rate, memory
pressure and battery there are meaningless.
- **Permissions and their denial paths.** The interesting flow is "user said no",
and simulator permission behaviour differs.
- **Camera, GPS, biometrics, push.** Approximated or absent.
- **Network transitions.** Wi-Fi to cellular, offline, captive portals. This is
where the network layer's assumptions surface.
- **Keyboard behaviour.** Third-party keyboards and predictive input change
layout in ways the simulator's keyboard does not.
## Determinism in E2E
The mobile equivalents of the browser rules: select by accessibility id rather
than by position, wait on conditions rather than durations, and reset app state
between tests. A test depending on the previous test's leftover state is the
most common cause of a suite that passes locally and fails in CI.
Cold start versus warm start matters more than on web: a test that only ever
runs warm never exercises launch-path initialisation, which is where a
surprising share of crashes live.
## Keep the CI matrix honest
Two simulators — the smallest supported and the newest — catch most layout
regressions cheaply. Add one physical device for the release candidate, not for
every commit; a device farm on every push is expensive and rarely finds what the
simulator missed except on the axes listed above.
## Record which it was
A test result without its target is unactionable. "Passed on iPhone 15 simulator,
iOS 18" and "passed on a physical Pixel 6" are different claims, and only the
second one says anything about performance.