165 lines
3.9 KiB
Markdown
165 lines
3.9 KiB
Markdown
# rtx-tts
|
|
|
|
Text-to-Speech (TTS) synthesis for RustyTorch with comprehensive text processing capabilities.
|
|
|
|
## Features
|
|
|
|
- **Text Normalization**: Advanced text preprocessing including:
|
|
- Case normalization
|
|
- Abbreviation expansion (Dr. → Doctor, St. → Street, etc.)
|
|
- Number to word conversion (42 → "forty two")
|
|
- Special character handling ($, %, °)
|
|
- Whitespace normalization
|
|
|
|
- **Phonemization**: Grapheme-to-phoneme (G2P) conversion:
|
|
- Dictionary-based lookup for common English words
|
|
- ARPAbet phoneme representation
|
|
- Stress marker support
|
|
- Rule-based fallback for unknown words
|
|
|
|
- **Prosody Prediction**: Natural speech synthesis markers:
|
|
- Pause detection at sentence and phrase boundaries
|
|
- Emphasis prediction from capitalization
|
|
- Intonation patterns (rising/falling)
|
|
- Speaking rate estimation
|
|
|
|
## Quick Start
|
|
|
|
```rust
|
|
use rtx_tts::{TextPipeline, TextNormalizerConfig};
|
|
|
|
// Create a text processing pipeline
|
|
let pipeline = TextPipeline::default();
|
|
|
|
// Process input text
|
|
let text = "Hello, Dr. Smith! I have 42 apples.";
|
|
let processed = pipeline.process_full(text)?;
|
|
|
|
println!("Normalized: {}", processed.normalized);
|
|
println!("Phonemes: {}", processed.phoneme_string(true));
|
|
|
|
// Prosody annotations
|
|
for annotation in &processed.prosody_annotations {
|
|
println!("Segment: {} (markers: {:?})", annotation.text, annotation.markers);
|
|
}
|
|
```
|
|
|
|
## Architecture
|
|
|
|
The TTS pipeline consists of three main stages:
|
|
|
|
1. **Text Normalization** - Clean and normalize input text
|
|
2. **Phonemization** - Convert text to phoneme sequences
|
|
3. **Prosody Prediction** - Add timing and intonation markers
|
|
|
|
## Examples
|
|
|
|
### Text Normalization
|
|
|
|
```rust
|
|
use rtx_tts::TextNormalizer;
|
|
|
|
let normalizer = TextNormalizer::default();
|
|
let result = normalizer.normalize("Dr. Smith has 50 dollars and 3 cats")?;
|
|
// Output: "doctor smith has fifty dollars and three cats"
|
|
```
|
|
|
|
### Phonemization
|
|
|
|
```rust
|
|
use rtx_tts::Phonemizer;
|
|
|
|
let phonemizer = Phonemizer::default();
|
|
let phonemes = phonemizer.g2p("hello world")?;
|
|
// Returns ARPAbet phoneme sequences with stress markers
|
|
```
|
|
|
|
### Prosody Prediction
|
|
|
|
```rust
|
|
use rtx_tts::{Prosody, ProsodyMarker};
|
|
|
|
let prosody = Prosody::default();
|
|
let annotations = prosody.predict("How are you?")?;
|
|
|
|
// Check for rising intonation (questions)
|
|
let has_rising = annotations.iter().any(|a| {
|
|
a.markers.iter().any(|m| matches!(m, ProsodyMarker::Rising))
|
|
});
|
|
```
|
|
|
|
## Configuration
|
|
|
|
Each module can be configured independently:
|
|
|
|
```rust
|
|
use rtx_tts::{
|
|
TextNormalizer, TextNormalizerConfig,
|
|
Phonemizer, PhonemizerConfig, PhonemizerBackend,
|
|
Prosody, ProsodyConfig,
|
|
TextPipeline,
|
|
};
|
|
|
|
// Custom normalization
|
|
let norm_config = TextNormalizerConfig {
|
|
lowercase: true,
|
|
remove_punctuation: false,
|
|
expand_abbreviations: true,
|
|
normalize_numbers: true,
|
|
normalize_whitespace: true,
|
|
};
|
|
|
|
// Custom phonemization
|
|
let phone_config = PhonemizerConfig {
|
|
language: "en-us".to_string(),
|
|
backend: PhonemizerBackend::Dictionary,
|
|
include_stress: true,
|
|
};
|
|
|
|
// Custom prosody
|
|
let prosody_config = ProsodyConfig {
|
|
use_punctuation_pauses: true,
|
|
default_speaking_rate: 1.0,
|
|
pause_at_commas: true,
|
|
pause_at_sentences: true,
|
|
detect_emphasis: true,
|
|
};
|
|
|
|
// Build pipeline
|
|
let pipeline = TextPipeline::new(
|
|
TextNormalizer::new(norm_config),
|
|
Phonemizer::new(phone_config),
|
|
Prosody::new(prosody_config),
|
|
);
|
|
```
|
|
|
|
## Testing
|
|
|
|
Run tests with:
|
|
|
|
```bash
|
|
cargo test
|
|
```
|
|
|
|
Run with coverage:
|
|
|
|
```bash
|
|
cargo test --all-features
|
|
```
|
|
|
|
## Performance
|
|
|
|
All text processing operations are implemented in pure Rust with zero allocations where possible. The phonemizer uses efficient HashMap lookups for dictionary-based phonemization.
|
|
|
|
## Future Work
|
|
|
|
- Neural acoustic models for mel-spectrogram generation
|
|
- GPU-accelerated neural vocoders (WaveGlow, HiFi-GAN)
|
|
- Multi-language support
|
|
- SSML markup support
|
|
- Voice cloning capabilities
|
|
|
|
## License
|
|
|
|
This project is licensed under the same terms as RustyTorch (MIT OR Apache-2.0).
|