rustyverse/rustytorch
2
0
Fork 0
Code Issues Pull Requests Actions 5 Packages Projects Releases Wiki Activity
403 Commits 1 Branch 0 Tags
c7d28705803351e500390c841a95a7f98c1de031
Commit Graph
2 Commits
Author SHA1 Message Date
osobhandClaude Opus 4.7 a5cedfb46a rtx-csm: emotional_speech_guide — CREMA-D vs RAVDESS firdhokk verdict 
8-gen bench (4 emotions × 2 corpora) at seed=42 against firdhokk
Whisper-LV3:

  target    RAVDESS              CREMA-D
  happy     happy (0.999) ✓      happy (0.999) ✓
  angry     neutral (0.92)       sad (0.99)
  fearful   happy (0.998)        fearful (0.984) ✓
  sad       angry (0.99)         fearful (0.99)

CREMA-D 2/4 vs RAVDESS 1/4. Larger / more naturalistic corpus
produces more class-pure fearful direction. Neither corpus solves
angry or sad — recipe shifts into 'vague expressivity' rather than
class-specific corners.

Practical: prefer CREMA-D when available; A/B both per emotion if
class precision matters.

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
 2026-04-30 00:01:02 -07:00
osobhandClaude Opus 4.7 a77b22d47c rtx-csm: Phase 13.8 — emotion2vec_plus_base port, slice 1 (inspector) 
First slice of the multi-session port that will replace the Phase 13.3
prosody-rule SER placeholder with a real classifier via the existing
EmotionDetector trait.

examples/emotion2vec_inspect.rs:
  - downloads model.pt + config.yaml + tokens.txt from
    emotion2vec/emotion2vec_plus_base on HF Hub
  - descends fairseq-style nested checkpoint via --key model
  - dumps all 185 tensors with shapes/dtypes + per-prefix summary
  - uses pickle::read_pth_tensor_info, same pattern as audioseal_inspect

Architecture confirmed (full notes in docs/emotion2vec_port_notes.md):
  - 93 M params, F32 (the 1.12 GB file is mostly optimizer state)
  - local_encoder: 7 Conv1d layers (wav2vec2 feature extractor:
    [(512,10,5)] + [(512,3,2)]×4 + [(512,2,2)]×2, T → T/320)
  - project_features: Linear 512 → 768
  - relative_positional_encoder: 5 Conv1d layers (kernel 19)
  - context_encoder: 4-layer transformer prenet (prenet_depth=4)
  - blocks.0..7: 8-layer main transformer (depth=8, embed_dim=768,
    12 heads, mlp_ratio=4, fused QKV qkv.weight=[2304, 768])
  - proj: Linear 768 → 9 (angry/disgusted/fearful/happy/neutral/other/
    sad/surprised/<unk>)

Slicing plan (remaining):
  Slice 2 (~half-day): candle module scaffolding + from_pickle loaders
  Slice 3 (~half-day): forward pass + shape verification
  Slice 4 (~hour): EmotionDetector impl + swap into audio_to_manifest
                  and converse_server

Lib suite 110/110.

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
 2026-04-27 23:14:06 -07:00