 osobhandClaude Opus 4.7
|
a77b22d47c
|
rtx-csm: Phase 13.8 — emotion2vec_plus_base port, slice 1 (inspector)
First slice of the multi-session port that will replace the Phase 13.3
prosody-rule SER placeholder with a real classifier via the existing
EmotionDetector trait.
examples/emotion2vec_inspect.rs:
- downloads model.pt + config.yaml + tokens.txt from
emotion2vec/emotion2vec_plus_base on HF Hub
- descends fairseq-style nested checkpoint via --key model
- dumps all 185 tensors with shapes/dtypes + per-prefix summary
- uses pickle::read_pth_tensor_info, same pattern as audioseal_inspect
Architecture confirmed (full notes in docs/emotion2vec_port_notes.md):
- 93 M params, F32 (the 1.12 GB file is mostly optimizer state)
- local_encoder: 7 Conv1d layers (wav2vec2 feature extractor:
[(512,10,5)] + [(512,3,2)]×4 + [(512,2,2)]×2, T → T/320)
- project_features: Linear 512 → 768
- relative_positional_encoder: 5 Conv1d layers (kernel 19)
- context_encoder: 4-layer transformer prenet (prenet_depth=4)
- blocks.0..7: 8-layer main transformer (depth=8, embed_dim=768,
12 heads, mlp_ratio=4, fused QKV qkv.weight=[2304, 768])
- proj: Linear 768 → 9 (angry/disgusted/fearful/happy/neutral/other/
sad/surprised/<unk>)
Slicing plan (remaining):
Slice 2 (~half-day): candle module scaffolding + from_pickle loaders
Slice 3 (~half-day): forward pass + shape verification
Slice 4 (~hour): EmotionDetector impl + swap into audio_to_manifest
and converse_server
Lib suite 110/110.
Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
|
2026-04-27 23:14:06 -07:00 |
|