# Local Voice Pipeline (STT + TTS) for Hermes Gateway

## Overview

This reference covers setting up **fully local** Speech-to-Text (STT) and Text-to-Speech (TTS) for Telegram voice messages on a headless server — no cloud APIs, no costs.

## Architecture

```
Telegram Voice (.ogg) → Gateway → ffmpeg → faster-whisper (STT) → Text
                                                              ↓
LLM (OpenRouter/Ollama) → Response Text → Piper (TTS) → MP3 → Telegram Voice
```

All components run locally on the server.

---

## STT: faster-whisper

### Installation

```bash
# In Hermes venv
/usr/local/lib/hermes-agent/venv/bin/python -m pip install faster-whisper
```

### Model Options

| Model | Size | Speed | Quality | Languages | RAM |
|-------|------|-------|---------|-----------|-----|
| `tiny` | 39 MB | Fastest | Low | 99 | ~1 GB |
| `base` | 74 MB | Fast | Good | 99 | ~1 GB |
| `small` | 244 MB | Medium | Better | 99 | ~2 GB |
| `medium` | 769 MB | Slow | High | 99 | ~5 GB |
| `large-v3` | 1.5 GB | Slowest | Best | 99 | ~10 GB |

**Recommendation for server:** `base` or `small` — good balance.

### Configuration (`~/.hermes/config.yaml`)

```yaml
stt:
  enabled: true
  provider: local
  local:
    model: base          # tiny, base, small, medium, large-v3
    language: ''         # auto-detect, or 'de', 'en', 'fr', etc.
  use_gateway: true
```

### Compute Types (for CPU)

| Type | Speed | Quality | RAM |
|------|-------|---------|-----|
| `int8` | Fastest | Good | Lowest |
| `int8_float16` | Fast | Better | Medium |
| `float16` | Medium | Best | Higher |
| `float32` | Slow | Best | Highest |

**Default:** `int8` — best for CPU-only servers.

### Testing STT

```bash
/usr/local/lib/hermes-agent/venv/bin/python -c "
from faster_whisper import WhisperModel
model = WhisperModel('base', device='cpu', compute_type='int8')
segments, info = model.transcribe('test.ogg', language='de')
for s in segments:
    print(f'[{s.start:.1f}s -> {s.end:.1f}s] {s.text}')
print(f'Language: {info.language} (p={info.language_probability:.2f})')
"
```

---

## TTS: Piper

### Installation

```bash
# In Hermes venv
/usr/local/lib/hermes-agent/venv/bin/python -m pip install piper-tts
```

### Voice Models

Voices auto-download on first use from: https://github.com/rhasspy/piper/blob/master/VOICES.md

**Popular voices:**

| Language | Voice | Quality | Size |
|----------|-------|---------|------|
| English (US) | `en_US-lessac-medium` | Natural, clear | ~50 MB |
| English (US) | `en_US-amy-medium` | Female, warm | ~50 MB |
| English (UK) | `en_GB-northern_english_male-medium` | British accent | ~50 MB |
| **German** | `de_DE-thorsten-medium` | **Best German male** | ~50 MB |
| **German** | `de_DE-karlsson-medium` | German male, different tone | ~50 MB |
| **German** | `de_DE-eva_k-x_low` | Female, fast (low quality) | ~10 MB |
| French | `fr_FR-siwis-medium` | Good French | ~50 MB |
| Spanish | `es_ES-sharvard-medium` | Good Spanish | ~50 MB |

### Configuration (`~/.hermes/config.yaml`)

```yaml
tts:
  provider: piper
  piper:
    voice: en_US-lessac-medium  # or de_DE-thorsten-medium for German
  use_gateway: true
```

### Pre-download Voice (avoids first-request latency)

```bash
# Test download
/usr/local/lib/hermes-agent/venv/bin/python -c "
from piper.voice import PiperVoice
voice = PiperVoice.load('de_DE-thorsten-medium')  # triggers download
print(f'Voice loaded: {voice.config.num_symbols} symbols, {voice.config.sample_rate} Hz')
"

# Check downloaded files
ls -la ~/.cache/piper/
```

### Testing TTS

```bash
# Via Hermes tool
text_to_speech text="Hallo, das ist ein Test auf Deutsch."

# Or direct Python
/usr/local/lib/hermes-agent/venv/bin/python -c "
from piper.voice import PiperVoice
voice = PiperVoice.load('de_DE-thorsten-medium')
import soundfile as sf
audio = voice.synthesize('Hallo, das ist ein Test.', speaker_id=0)
sf.write('/tmp/test.wav', audio, voice.config.sample_rate)
print('Generated /tmp/test.wav')
"
```

---

## Telegram Voice Integration

### Requirements

1. **ffmpeg** for audio conversion (`.ogg` → `.wav` for STT)
2. **Gateway** configured with Telegram platform
3. **STT + TTS** configured as above

```bash
apt-get install -y ffmpeg
```

### Flow Details

1. **User sends voice** → Telegram delivers `.ogg` (Opus codec) to gateway
2. **Gateway downloads** → saves to temp file
3. **ffmpeg converts** → `.wav` (16kHz mono) for STT
4. **faster-whisper transcribes** → German/English text
5. **LLM processes** → generates response
6. **Piper synthesizes** → `.wav` → converted to `.mp3`
7. **Gateway sends** → Telegram voice message

### Latency Estimates (CPU-only, 8 cores)

| Step | Time (approx) |
|------|---------------|
| Download + ffmpeg | 0.5-1s |
| STT (base model) | 1-3s |
| LLM (Ollama deepseek-r1:7b) | 3-10s |
| TTS (Piper) | 0.5-2s |
| **Total** | **5-16s** |

### Optimization Tips

```yaml
# In config.yaml - reduce STT model for speed
stt:
  local:
    model: tiny        # fastest, lower accuracy
    # OR
    model: base        # recommended balance

# For TTS - use lower quality voice for speed
tts:
  piper:
    voice: de_DE-eva_k-x_low  # fast, smaller, lower quality
```

---

## Troubleshooting

### STT Not Working

```bash
# Check model downloads
ls -la ~/.cache/huggingface/hub/models--Systran--faster-whisper-*/

# Test manually
/usr/local/lib/hermes-agent/venv/bin/python -c "
from faster_whisper import WhisperModel
model = WhisperModel('base', device='cpu', compute_type='int8')
print('Model loaded OK')
"
```

### TTS Not Working / No Audio

```bash
# Check voice cache
ls -la ~/.cache/piper/

# Test direct synthesis
/usr/local/lib/hermes-agent/venv/bin/python -c "
from piper.voice import PiperVoice
voice = PiperVoice.load('en_US-lessac-medium')
audio = voice.synthesize('Test', speaker_id=0)
print(f'Generated {len(audio)} samples at {voice.config.sample_rate} Hz')
"
```

### Telegram Voice Not Sent

```bash
# Check gateway logs
journalctl -u hermes-gateway -n 50 | grep -i -e voice -e tts -e stt -e ogg

# Verify ffmpeg
ffmpeg -version

# Check Telegram bot has voice permissions
# BotFather → Bot Settings → Voice Messages: ENABLED
```

### Gateway Crash on Voice

```bash
# Increase memory limits
# In service file:
MemoryMax=4G
MemoryHigh=3G

# Or reduce model sizes (tiny STT, low-quality TTS)
```

---

## Language-Specific Notes

### German STT

- `base` model: Good for standard German
- `small` model: Better for dialects, technical terms
- Set `language: 'de'` in config for faster processing (skips detection)

### German TTS

- `de_DE-thorsten-medium`: **Best quality**, natural male voice
- `de_DE-karlsson-medium`: Alternative male voice
- `de_DE-eva_k-x_low`: Fast, robotic female — only for speed-critical

### Mixed Language

- STT auto-detects well with `language: ''`
- TTS: Use appropriate voice per language (config is global, so pick primary)

---

## Advanced: Custom Voice Training (Piper)

For custom voices: https://github.com/rhasspy/piper/blob/master/TRAINING.md

Requires: ~20min clean audio + transcripts, GPU for training (faster).

Not covered here — use pre-trained voices.

---

## Cost Summary

| Component | Cost |
|-----------|------|
| Ollama (LLM) | **Free** (local) |
| faster-whisper (STT) | **Free** (local) |
| Piper (TTS) | **Free** (local) |
| ffmpeg | **Free** |
| Telegram Bot API | **Free** |
| **Total** | **$0/month** |

No API keys, no cloud dependencies, works offline.