SparseAutoencoder::new_seeded draws encoder/decoder weights via
randn_seeded with per-tensor SplitMix64-derived seeds (same
derivation as MambaBlock::new_seeded), so identical (config, seed)
gives bit-exact SAEs — without it, cross-instance loss comparisons
are noise (measured downstream: 0.09 vs 0.65 starts on identical
data).
SAETrainer::reinitialize_neuron couples the encoder-unit weight
reinit with zeroing that unit's optimizer moment rows (encoder row,
bias slot, decoder column), so external generate-and-test callers
can't reset weights while leaving optimizer state stale — previously
only the trainer's internal dead-neuron resampling did both. Note
train_step's update rule is plain SGD today, so the moment reset is
inert until the Adam path is switched on; the coupling is the
contract either way, and a doctored-checkpoint test pins the
row/column semantics.
Also drops a vacuous assert!(true) smoke test that failed clippy's
assertions_on_constants.
Co-Authored-By: Claude Fable 5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01B1feFAQxjbCRHePUdxuNra