Commit Graph
1 Commits
Author SHA1 Message Date
osobhandClaude Opus 4.7 15b62a8f6e rtx-interpret: fix decoder transpose in SAE compute_and_apply_gradients
The encoder-gradient path through the decoder was transposing the decoder
before the matmul, producing `[batch, d_model] × [d_sae, d_model]` — a
shape mismatch for every batch > 1. The decoder is stored as
`[d_model, d_sae]`, so `recon_grad @ decoder` is already the right shape
(and matches the comment at the call site, which reads
"recon_grad @ decoder @ d_relu").

All 9 existing `sae::tests` still pass. Omni-Cortex's `LatentDictionary`
now trains correctly on batches larger than 1.

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
2026-04-24 13:58:44 -07:00