rustyverse/rustytorch
2
0
Fork 0
Code Issues Pull Requests Actions 46 Packages Projects Releases Wiki Activity
440 Commits 1 Branch 0 Tags
af19860c04e064dfdf86cfe92c82b96c61a2f341
Commit Graph
407 Commits
Author SHA1 Message Date
osobhandClaude Opus 4.7 15dd3575d4 Add rtx-csm: Rust-native port of Sesame CSM-1B with LoRA voice cloning 
A new model crate at crates/models/rtx-csm implementing end-to-end
inference, quantization, and fine-tuning for Sesame's Conversational
Speech Model (CSM-1B). Built on candle 0.9 + Kyutai Mimi codec.

Key capabilities:
- Inference (FP F16 on Metal, F32 on CPU, BF16 on CUDA)
- Quantized inference (Q8_0 / Q4_K_M GGUF, ~3x speedup, ~50% memory)
- Streaming Mimi decode with proper StreamTensor state machine
- In-context voice cloning via SpeakerProfile
- Classifier-Free Guidance (Koel-TTS recipe)
- Long-form chunked generation with rolling context
- Audio post-processing (HPF + declick + EBU R128 LUFS)
- Text input normalization (brackets, times, unicode, length caps)
- Frame-level repetition guard (loop-escape)
- Top-k + top-p sampling
- LoRA fine-tuning end-to-end (training + inference, on FP and Q8 bases)
- In-process Whisper ASR via whisper-rs (under --features asr)
- Standalone TTS HTTP server (Axum)
- Bench harness with manifest export + per-prompt WER

Phases delivered: quantization, ASR/WER eval, LoRA voice cloning, HTTP
service. AudioSeal/WavLM/Unmute remain as documented future work.

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
 2026-04-25 18:33:57 -07:00
osobhandClaude Opus 4.7 15b62a8f6e rtx-interpret: fix decoder transpose in SAE compute_and_apply_gradients 
The encoder-gradient path through the decoder was transposing the decoder
before the matmul, producing `[batch, d_model] × [d_sae, d_model]` — a
shape mismatch for every batch > 1. The decoder is stored as
`[d_model, d_sae]`, so `recon_grad @ decoder` is already the right shape
(and matches the comment at the call site, which reads
"recon_grad @ decoder @ d_relu").

All 9 existing `sae::tests` still pass. Omni-Cortex's `LatentDictionary`
now trains correctly on batches larger than 1.

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
 2026-04-24 13:58:44 -07:00
osobhandClaude Opus 4.6 85b77d49f2 Add audio neural layers and model architectures for ClawSample integration 
New nn layers:
- ConvTranspose1d with stride, padding, output_padding (9 tests)
- LSTM/BiLSTM with multi-layer support and hidden state (10 tests)

Audio source separation:
- Demucs ONNX inference with segmented overlap-add processing
- Native HtDemucs architecture (encoder/decoder with BiLSTM bottleneck)
- StemType enum: vocals, drums, bass, other, piano, guitar

Audio generation:
- Stable Audio Open ONNX inference scaffold
- GenerationParams (prompt, duration, steps, cfg_scale, seed)

ONNX export scripts:
- export_demucs_onnx.py — Demucs v4 to ONNX with segment chunking
- export_stable_audio_onnx.py — Stable Audio Open components
- export_mert_onnx.py — MERT music understanding transformer

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
 2026-04-17 12:27:12 -07:00
osobhandClaude Opus 4.6 5fe8c67b04 Fix rtx-serving-api binary: create main.rs entry point 
The binary path pointed to non-existent src/bin/server.rs.
Created proper src/main.rs with tokio async entry point that:
- Initializes tracing
- Loads config from RTX_HOST/RTX_PORT/RTX_TIMEOUT env vars
- Instantiates ServingServer and calls serve()

Fixed Cargo.toml: path = "src/bin/server.rs" → path = "src/main.rs"

Validated: binary builds, starts, health endpoint returns healthy,
inference endpoint returns mock completions.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
 2026-04-12 20:16:38 -07:00
osobhandClaude Opus 4.6 02d382d5f6 style: apply rustfmt across all crates and demos 
Consistent formatting pass: line wrapping, import sorting, trailing
whitespace removal, let-chain indentation, merged derive attributes,
and unsafe block reformatting.

Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
 2026-04-12 07:01:58 -07:00
osobh d52d359f52 Fix 3 compile errors: rtx-metal (Linux cfg), rtx-onnx (ort API), rtx-fusion (edition) 
- rtx-metal: fix MetalError import in sparse/conversion.rs non-macOS stub
- rtx-onnx: update session.rs and tensor_bridge.rs for ort 2.x API changes
- rtx-fusion: fix Cargo.toml package name
- rtx-hub: fix discovery.rs type mismatch
- Full workspace (80+ crates) now compiles clean on Linux
 2026-03-15 17:33:54 -07:00
redclawsystems 4d88dc0584 Initial commit 2026-03-04 00:08:42 +00:00