rustytorch

98 Commits 1 Branch 0 Tags

2 Commits

Include renames

Author	SHA1	Message	Date
osobhandClaude Opus 4.7	ed4e5e4b85	rtx-csm: Phase 13.3 — prosody-rule SER baseline + --auto-emotion-tag Closes the third gap from the audio-ML Rust ecosystem survey: speech emotion recognition. Honest scope — this is a hand-tuned placeholder, not a real classifier. The trait makes a future emotion2vec_plus_base candle port a one-line swap. src/ser.rs (~330 LOC): - EmotionDetector trait - ProsodyDetector impl: autocorrelation F0 (65-400 Hz, voiced via autocorr peak ratio) + RMS + voiced-ratio aggregation - 5 buckets compatible with Phase 12.2 emotion-hint format: [neutral] [calm] [sad] [angry] [excited] - 6 unit tests (autocorr accuracy on a pure tone, silence handling, sad/excited/neutral edge cases, tag-format invariant) audio_to_manifest gains --auto-emotion-tag: classifies each diarized clip and writes the resolved label into the manifest row's emotion_tag. Static --emotion-tag stays as a fallback. End-to-end verified: 2-speaker concat → both clips classified [neutral] (correct — synthetic CSM samples are prosodically flat). Manifest round-trips through lora_train_emotional unchanged. Lib suite 110/110 (6 new SER tests). Pure-DSP, zero ML deps, zero runtime risk. The data-prep pipeline is now end-to-end auto-labeled in-crate: audio_to_manifest --auto-emotion-tag raw.wav → manifest.jsonl → lora_train_emotional → lora_eval → converse_server with --lora Zero Python, zero ort, zero whisper.cpp. Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>	2026-04-27 18:24:53 -07:00
osobhandClaude Opus 4.7	2feb5c9d67	rtx-csm: Phase 13.2 — audio→manifest pipeline (no Python sidecar) Single command turns raw audio (podcast/audiobook/conversation) into a training-ready JSONL manifest + per-segment clip wavs. audio_to_manifest: 1. Diarize (Phase 13.1: Silero V5 + WavLM-SV + clustering) 2. Per segment: slice audio + Moonshine encode/decode → transcript 3. Write `<stem>.spk{N}.{idx:04}.wav` + manifest.jsonl Manifest rows match ManifestRow exactly (Phase 12.3), so the output flows directly into lora_train_emotional / load_from_manifest. Knobs: --segments-json (reuse precomputed diar), --emotion-tag and --stage applied uniformly, --min-transcript-chars filters ASR failures, plus all Phase 13.1 diarization knobs. DiarizedSegment gained serde::Deserialize for the segments-json reuse path. Verified end-to-end: 2-speaker concat → 2 segments diarized in 192 ms → Moonshine transcribed → 2 manifest rows + 2 clips written → round- trips through lora_train_emotional cleanly (LoRA injected with extended coverage, adapter saved with embedded metadata, lib suite 104/104). The full no-Python data-prep loop now reads: audio_to_manifest raw.wav → manifest.jsonl lora_train_emotional manifest.jsonl → voice.safetensors lora_eval base + lora for A/B generate / converse_server with --lora voice.safetensors Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>	2026-04-27 18:09:32 -07:00

Author

SHA1

Message

Date

osobhandClaude Opus 4.7

ed4e5e4b85

rtx-csm: Phase 13.3 — prosody-rule SER baseline + --auto-emotion-tag

Closes the third gap from the audio-ML Rust ecosystem survey:
speech emotion recognition. Honest scope — this is a hand-tuned
placeholder, not a real classifier. The trait makes a future
emotion2vec_plus_base candle port a one-line swap.

src/ser.rs (~330 LOC):
  - EmotionDetector trait
  - ProsodyDetector impl: autocorrelation F0 (65-400 Hz, voiced via
    autocorr peak ratio) + RMS + voiced-ratio aggregation
  - 5 buckets compatible with Phase 12.2 emotion-hint format:
    [neutral] [calm] [sad] [angry] [excited]
  - 6 unit tests (autocorr accuracy on a pure tone, silence handling,
    sad/excited/neutral edge cases, tag-format invariant)

audio_to_manifest gains --auto-emotion-tag: classifies each diarized
clip and writes the resolved label into the manifest row's
emotion_tag. Static --emotion-tag stays as a fallback.

End-to-end verified: 2-speaker concat → both clips classified
[neutral] (correct — synthetic CSM samples are prosodically flat).
Manifest round-trips through lora_train_emotional unchanged.

Lib suite 110/110 (6 new SER tests). Pure-DSP, zero ML deps, zero
runtime risk.

The data-prep pipeline is now end-to-end auto-labeled in-crate:
  audio_to_manifest --auto-emotion-tag raw.wav → manifest.jsonl
  → lora_train_emotional → lora_eval → converse_server with --lora
Zero Python, zero ort, zero whisper.cpp.

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>

2026-04-27 18:24:53 -07:00

osobhandClaude Opus 4.7

2feb5c9d67

rtx-csm: Phase 13.2 — audio→manifest pipeline (no Python sidecar)

Single command turns raw audio (podcast/audiobook/conversation) into a
training-ready JSONL manifest + per-segment clip wavs.

audio_to_manifest:
  1. Diarize (Phase 13.1: Silero V5 + WavLM-SV + clustering)
  2. Per segment: slice audio + Moonshine encode/decode → transcript
  3. Write `<stem>.spk{N}.{idx:04}.wav` + manifest.jsonl

Manifest rows match ManifestRow exactly (Phase 12.3), so the output
flows directly into lora_train_emotional / load_from_manifest.

Knobs: --segments-json (reuse precomputed diar), --emotion-tag and
--stage applied uniformly, --min-transcript-chars filters ASR failures,
plus all Phase 13.1 diarization knobs.

DiarizedSegment gained serde::Deserialize for the segments-json
reuse path.

Verified end-to-end: 2-speaker concat → 2 segments diarized in 192 ms
→ Moonshine transcribed → 2 manifest rows + 2 clips written → round-
trips through lora_train_emotional cleanly (LoRA injected with extended
coverage, adapter saved with embedded metadata, lib suite 104/104).

The full no-Python data-prep loop now reads:
  audio_to_manifest raw.wav → manifest.jsonl
  lora_train_emotional manifest.jsonl → voice.safetensors
  lora_eval base + lora for A/B
  generate / converse_server with --lora voice.safetensors

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>

2026-04-27 18:09:32 -07:00