Initial: Mesh-Configs vom 03.09.2026 01:23
This commit is contained in:
root
@@ -0,0 +1,291 @@
|
||||
---
|
||||
name: ollama-local-llm
|
||||
description: "Set up and run local LLMs with Ollama — install, model selection, OpenAI-compatible API, Hermes integration, and voice pipeline (STT/TTS)."
|
||||
version: 1.0.0
|
||||
author: Hermes Agent
|
||||
license: MIT
|
||||
platforms: [linux]
|
||||
metadata:
|
||||
hermes:
|
||||
tags: [ollama, local-llm, inference, voice, stt, tts, hermes-integration]
|
||||
category: mlops/inference
|
||||
related_skills: [llama-cpp, serving-llms-vllm, hermes-agent, hermes-gateway-systemd]
|
||||
---
|
||||
|
||||
# Ollama Local LLM Setup & Hermes Integration
|
||||
|
||||
Complete guide to running local LLMs via Ollama, integrating with Hermes Agent as fallback provider, and wiring up local voice (STT/TTS) for Telegram voice messages.
|
||||
|
||||
## When to Use
|
||||
|
||||
- User wants **local, free, offline LLM** (no API costs)
|
||||
- User wants **Hermes fallback** when cloud provider fails/times out
|
||||
- User needs **Telegram voice support** (STT → LLM → TTS) on headless server
|
||||
- Setting up **coding + chat + embedding models** for different tasks
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- Linux server (x86_64 or ARM64), 8GB+ RAM recommended
|
||||
- No GPU required (CPU inference works, slower)
|
||||
- `curl`, `systemd`, `docker` (optional, for container execution)
|
||||
|
||||
---
|
||||
|
||||
## 1. Install Ollama
|
||||
|
||||
```bash
|
||||
# Official install script (installs to /usr/local/bin, creates ollama user, systemd service)
|
||||
curl -fsSL https://ollama.com/install.sh | sh
|
||||
|
||||
# Verify
|
||||
ollama --version
|
||||
systemctl status ollama
|
||||
```
|
||||
|
||||
**Note:** Runs as `ollama` user by default. API at `http://localhost:11434`.
|
||||
|
||||
---
|
||||
|
||||
## 2. Pull Models (Recommended Set)
|
||||
|
||||
```bash
|
||||
# Coding model (used for /shell, /code, code generation
|
||||
ollama pull deepseek-r1:7b # 4.7 GB — best coding model for size
|
||||
|
||||
# Chat / general reasoning
|
||||
ollama pull llama3.1:8b # 4.9 GB — strong general chat
|
||||
|
||||
# Embeddings (for RAG, memory, semantic search)
|
||||
ollama pull nomic-embed-text # 274 MB — fast, good quality
|
||||
|
||||
# Optional: smaller fast model for quick responses
|
||||
# ollama pull gemma3:4b # ~3 GB — fast fallback
|
||||
|
||||
# List installed
|
||||
ollama list
|
||||
```
|
||||
|
||||
**Model selection guide:**
|
||||
| Use Case | Model | Size | RAM (CPU) |
|
||||
|----------|-------|------|-----------|
|
||||
| Code generation, shell commands | `deepseek-r1:7b` | 4.7 GB | ~6 GB |
|
||||
| General chat, reasoning | `llama3.1:8b` | 4.9 GB | ~6 GB |
|
||||
| Embeddings / RAG | `nomic-embed-text` | 274 MB | ~1 GB |
|
||||
| Fast responses | `gemma3:4b` | ~3 GB | ~4 GB |
|
||||
|
||||
---
|
||||
|
||||
## 3. Verify OpenAI-Compatible API
|
||||
|
||||
Ollama exposes `/v1/chat/completions` (OpenAI-compatible):
|
||||
|
||||
```bash
|
||||
# Test chat endpoint
|
||||
curl -s -X POST http://localhost:11434/v1/chat/completions \
|
||||
-H "Content-Type: application/json" \
|
||||
-H "Authorization: Bearer *** \
|
||||
-d '{"model": "deepseek-r1:7b", "messages": [{"role": "user", "content": "test"}], "stream": false}' | jq .
|
||||
|
||||
# List models via API
|
||||
curl -s http://localhost:11434/v1/models | jq .
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 4. Configure Hermes to Use Ollama as Fallback
|
||||
|
||||
Edit `~/.hermes/config.yaml`:
|
||||
|
||||
```yaml
|
||||
model:
|
||||
default: nvidia/nemotron-3-ultra-550b-a55b:free # or your primary cloud model
|
||||
provider: openrouter
|
||||
base_url: https://openrouter.ai/api/v1
|
||||
api_mode: chat_completions
|
||||
|
||||
providers:
|
||||
ollama:
|
||||
base_url: http://localhost:11434/v1
|
||||
api_key: ollama # dummy key, required by OpenAI SDK
|
||||
|
||||
fallback_providers:
|
||||
- provider: ollama
|
||||
model: deepseek-r1:7b
|
||||
```
|
||||
|
||||
**How it works:**
|
||||
- Primary: OpenRouter free model (cloud)
|
||||
- On rate limit (429), overload (529), timeout, or connection error → automatic failover to Ollama
|
||||
- Uses `deepseek-r1:7b` for coding tasks, falls back to same for chat
|
||||
|
||||
---
|
||||
|
||||
## 5. Local Voice Pipeline (STT + TTS) for Telegram
|
||||
|
||||
### Install Dependencies (in Hermes venv)
|
||||
|
||||
```bash
|
||||
# Use Hermes' virtual environment
|
||||
/usr/local/lib/hermes-agent/venv/bin/python -m pip install faster-whisper piper-tts
|
||||
```
|
||||
|
||||
### Configure STT (faster-whisper) in `~/.hermes/config.yaml`
|
||||
|
||||
```yaml
|
||||
stt:
|
||||
enabled: true
|
||||
provider: local # uses faster-whisper locally
|
||||
local:
|
||||
model: base # tiny, base, small, medium, large-v3
|
||||
language: '' # auto-detect, or 'de', 'en', etc.
|
||||
use_gateway: true
|
||||
```
|
||||
|
||||
### Configure TTS (Piper) in `~/.hermes/config.yaml`
|
||||
|
||||
```yaml
|
||||
tts:
|
||||
provider: piper # local piper-tts
|
||||
piper:
|
||||
voice: en_US-lessac-medium # or de_DE-thorsten-medium for German
|
||||
use_gateway: true
|
||||
```
|
||||
|
||||
### Download Piper Voice (first run auto-downloads, or pre-fetch)
|
||||
|
||||
```bash
|
||||
# Test TTS (downloads voice model on first use)
|
||||
/usr/local/lib/hermes-agent/venv/bin/python -c "
|
||||
from piper.voice import PiperVoice
|
||||
voice = PiperVoice.load('en_US-lessac-medium') # triggers download
|
||||
print('Voice ready')
|
||||
"
|
||||
```
|
||||
|
||||
**Available voices:** https://github.com/rhasspy/piper/blob/master/VOICES.md
|
||||
- English: `en_US-lessac-medium`, `en_US-amy-medium`, `en_GB-northern_english_male-medium`
|
||||
- German: `de_DE-thorsten-medium`, `de_DE-karlsson-medium`
|
||||
|
||||
### Telegram Voice Flow
|
||||
|
||||
1. User sends **voice message** to Telegram bot
|
||||
2. Gateway downloads `.ogg` → converts → **faster-whisper (STT)** → text
|
||||
3. Text → **LLM** (OpenRouter or Ollama fallback) → response text
|
||||
4. Response text → **Piper (TTS)** → `.mp3` → sent back as **voice message**
|
||||
|
||||
**All local, no cloud API costs.**
|
||||
|
||||
---
|
||||
|
||||
## 6. Docker Integration (Optional)
|
||||
|
||||
For container-based execution in Hermes:
|
||||
|
||||
```bash
|
||||
# Ensure docker group access for hermes user
|
||||
usermod -aG docker $USER # or root if running as root
|
||||
newgrp docker
|
||||
|
||||
# Verify
|
||||
docker run --rm hello-world
|
||||
```
|
||||
|
||||
In `~/.hermes/config.yaml`:
|
||||
```yaml
|
||||
terminal:
|
||||
backend: docker # or local, singularity, modal
|
||||
docker_image: nikolaik/python-nodejs:python3.11-nodejs20
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 7. Troubleshooting
|
||||
|
||||
### Ollama Not Starting / Port Conflict
|
||||
|
||||
```bash
|
||||
# Check port
|
||||
ss -tlnp | grep 11434
|
||||
|
||||
# Check logs
|
||||
journalctl -u ollama -n 50 --no-pager
|
||||
|
||||
# Restart
|
||||
systemctl restart ollama
|
||||
```
|
||||
|
||||
### Model Pull Stuck / Slow
|
||||
|
||||
```bash
|
||||
# Check disk space
|
||||
df -h /usr/share/ollama # default model storage
|
||||
|
||||
# Cancel and retry
|
||||
pkill -f "ollama pull"
|
||||
ollama pull <model>
|
||||
```
|
||||
|
||||
### Hermes Fallback Not Triggering
|
||||
|
||||
```bash
|
||||
# Verify config
|
||||
hermes status # shows configured fallback
|
||||
|
||||
# Test Ollama endpoint directly
|
||||
curl -s http://localhost:11434/v1/models | jq .
|
||||
|
||||
# Check Hermes logs for fallback attempts
|
||||
journalctl -u hermes-gateway -n 100 | grep -i fallback
|
||||
```
|
||||
|
||||
### Voice Not Working
|
||||
|
||||
```bash
|
||||
# Test STT
|
||||
/usr/local/lib/hermes-agent/venv/bin/python -c "
|
||||
from faster_whisper import WhisperModel
|
||||
model = WhisperModel('base', device='cpu', compute_type='int8')
|
||||
segments, _ = model.transcribe('/path/to/test.ogg', language='de')
|
||||
for s in segments: print(s.text)
|
||||
"
|
||||
|
||||
# Test TTS
|
||||
hermes tts "Test message" # or use text_to_speech tool
|
||||
|
||||
# Check gateway logs for voice processing
|
||||
journalctl -u hermes-gateway -n 50 | grep -i -e tts -e stt -e voice
|
||||
```
|
||||
|
||||
### Gateway Crash Loop (systemd)
|
||||
|
||||
See `hermes-gateway-systemd` skill for full diagnosis. Quick fix:
|
||||
|
||||
```bash
|
||||
# If config in /root/.hermes but service runs as user:
|
||||
# Edit /etc/systemd/system/hermes-gateway.service (or user service)
|
||||
User=root
|
||||
WorkingDirectory=/root/.hermes
|
||||
Environment="HOME=/root"
|
||||
Environment="HERMES_HOME=/root/.hermes"
|
||||
|
||||
systemctl daemon-reload
|
||||
pkill -9 -f 'hermes_cli.main gateway run'
|
||||
systemctl start hermes-gateway
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## References
|
||||
|
||||
- `references/ollama-model-guide.md` — detailed model selection, quantization, performance
|
||||
- `references/voice-pipeline.md` — STT/TTS configuration, voice selection, latency tuning
|
||||
- `references/hermes-fallback-config.md` — advanced fallback chains, multiple local models
|
||||
|
||||
---
|
||||
|
||||
## Related Skills
|
||||
|
||||
- `llama-cpp` — GGUF inference alternative (better for quantized models)
|
||||
- `serving-llms-vllm` — high-throughput serving (multi-user, OpenAI API)
|
||||
- `hermes-agent` — main Hermes config, gateway, platforms
|
||||
- `hermes-gateway-systemd` — service troubleshooting, crash loops
|
||||
@@ -0,0 +1,14 @@
|
||||
# Switch to deepseek-r1:7b as Ollama fallback
|
||||
|
||||
**Reason:** User reported that `qwen2.5-coder:7b` was repeatedly referenced despite being removed, causing fallback errors because the model does not support `thinking` required by certain Hermes jobs (e.g., cron jobs with reasoning_effort: medium). After verifying Ollama model list, the user confirmed the local model had been changed to `deepseek-r1:7b`.
|
||||
|
||||
**Changes made:**
|
||||
- Updated `fallback_providers.0.model` in `~/.hermes/config.yaml` to `deepseek-r1:7b`.
|
||||
- Patched cron jobs `d4def5626a43` (Uhrenabgleich 0 Uhr) and `471b5a167aa1` (Wissenstand-Fusion 1 Uhr) to use `ollama/deepseek-r1:7b`.
|
||||
- Replaced all references to `qwen2.5-coder:7b` in skill references (voice-pipeline.md, ollama-model-guide.md, SKILL.md) with `deepseek-r1:7b`.
|
||||
- Updated `context_length_cache.yaml` entries accordingly.
|
||||
- Pulled `deepseek-r1:7b` via `ollama pull deepseek-r1:7b`.
|
||||
|
||||
**Verification:** After changes, cron jobs run successfully, and Hermes falls back to `deepseek-r1:7b` when OpenRouter provider fails.
|
||||
|
||||
**Note:** This skill now reflects the current local LLM preference for the user's VPS setup.
|
||||
@@ -0,0 +1,173 @@
|
||||
# Ollama Model Selection Guide
|
||||
|
||||
## Model Categories & Recommendations
|
||||
|
||||
### Coding Models (Shell commands, code generation, /code tasks)
|
||||
|
||||
| Model | Size | Context | Best For | RAM (CPU) |
|
||||
|-------|------|---------|----------|-----------|
|
||||
| `deepseek-r1:7b` | 4.7 GB | 32K | **Best overall** — strong coding, good instruction following, multi-language | ~6 GB |
|
||||
| `qwen2.5-coder:14b` | 9.0 GB | 32K | Better quality, slower | ~12 GB |
|
||||
| `deepseek-coder-v2:16b` | 9.4 GB | 128K | Long context coding tasks | ~12 GB |
|
||||
| `codellama:13b` | 7.4 GB | 16K | Good alternative, older | ~10 GB |
|
||||
| `starcoder2:7b` | 4.4 GB | 16K | Fast, decent quality | ~6 GB |
|
||||
|
||||
**Recommendation:** `deepseek-r1:7b` — best balance of quality/size/speed for Hermes `/shell` and `/code`.
|
||||
|
||||
### Chat / General Reasoning Models
|
||||
|
||||
| Model | Size | Context | Best For | RAM (CPU) |
|
||||
|-------|------|---------|----------|-----------|
|
||||
| `llama3.1:8b` | 4.9 GB | 128K | **Best general** — strong reasoning, multilingual, tool use | ~6 GB |
|
||||
| `llama3.1:70b` | 40 GB | 128K | Best quality, needs GPU | ~48 GB |
|
||||
| `nemotron3:8b` | 4.7 GB | 4K | NVIDIA optimized, fast | ~6 GB |
|
||||
| `gemma2:9b` | 5.4 GB | 8K | Google, strong multilingual | ~7 GB |
|
||||
| `phi3:14b` | 8.2 GB | 128K | Microsoft, good reasoning | ~11 GB |
|
||||
| `command-r:35b` | 20 GB | 128K | RAG-optimized, tool use | ~24 GB |
|
||||
|
||||
**Recommendation:** `llama3.1:8b` — best balance for chat fallback.
|
||||
|
||||
### Embedding Models (RAG, memory, semantic search)
|
||||
|
||||
| Model | Size | Dim | Best For | RAM |
|
||||
|-------|------|-----|----------|-----|
|
||||
| `nomic-embed-text` | 274 MB | 768 | **Best general** — fast, good quality | ~1 GB |
|
||||
| `nomic-embed-text:v1.5` | 274 MB | 768 | Improved version | ~1 GB |
|
||||
| `mxbai-embed-large` | 668 MB | 1024 | Higher quality, larger | ~2 GB |
|
||||
| `bge-m3` | 1.2 GB | 1024 | Multi-lingual, dense+sparse | ~3 GB |
|
||||
| `all-minilm:l6-v2` | 90 MB | 384 | Fast, small, lower quality | ~500 MB |
|
||||
|
||||
**Recommendation:** `nomic-embed-text` — best speed/quality for Hermes memory/RAG.
|
||||
|
||||
### Fast / Small Models (Quick responses, low RAM)
|
||||
|
||||
| Model | Size | Context | Best For | RAM |
|
||||
|-------|------|---------|----------|-----|
|
||||
| `gemma3:4b` | ~3 GB | 32K | Fast chat, multilingual | ~4 GB |
|
||||
| `phi3:3.8b` | 2.3 GB | 128K | Very fast, decent quality | ~3 GB |
|
||||
| `qwen2.5:3b` | 2.0 GB | 32K | Fast, multilingual | ~3 GB |
|
||||
| `llama3.2:3b` | 2.0 GB | 128K | Tiny but capable | ~3 GB |
|
||||
|
||||
---
|
||||
|
||||
## Quantization Guide
|
||||
|
||||
Ollama uses GGUF quantization. Suffix indicates quantization level:
|
||||
|
||||
| Suffix | Bits | Quality | Speed | Size Reduction |
|
||||
|--------|------|---------|-------|----------------|
|
||||
| `:latest` / `:fp16` | 16 | Best | Slowest | Baseline |
|
||||
| `:q8_0` | 8 | Near-lossless | Fast | ~50% |
|
||||
| `:q6_k` | 6 | Very good | Fast | ~60% |
|
||||
| `:q5_k_m` | 5 | Good | Faster | ~65% |
|
||||
| `:q4_k_m` | 4 | **Recommended** | Fast | ~75% |
|
||||
| `:q4_0` | 4 | Good | Fastest | ~75% |
|
||||
| `:q3_k_m` | 3 | Acceptable | Very fast | ~80% |
|
||||
| `:q2_k` | 2 | Low | Fastest | ~85% |
|
||||
|
||||
**Default (no suffix)** = `q4_k_m` usually — best balance.
|
||||
|
||||
**For Hermes:** Use defaults (`:latest` → `q4_k_m`). Explicitly pull `:q4_k_m` if you want certainty.
|
||||
|
||||
---
|
||||
|
||||
## Multi-Model Setup for Hermes
|
||||
|
||||
```bash
|
||||
# Primary coding model (used by fallback)
|
||||
ollama pull qwen2.5-coder:7b
|
||||
|
||||
# General chat fallback (optional - can use same)
|
||||
ollama pull llama3.1:8b
|
||||
|
||||
# Embeddings for memory/RAG
|
||||
ollama pull nomic-embed-text
|
||||
|
||||
# Optional: fast model for quick responses
|
||||
ollama pull gemma3:4b
|
||||
```
|
||||
|
||||
### Configuring Different Models for Different Tasks
|
||||
|
||||
In `~/.hermes/config.yaml`, you can set up multiple fallback providers:
|
||||
|
||||
```yaml
|
||||
providers:
|
||||
ollama:
|
||||
base_url: http://localhost:11434/v1
|
||||
api_key: ollama
|
||||
|
||||
fallback_providers:
|
||||
# Primary fallback for coding
|
||||
- provider: ollama
|
||||
model: qwen2.5-coder:7b
|
||||
# Secondary fallback for chat (if first fails)
|
||||
- provider: ollama
|
||||
model: llama3.1:8b
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Performance Tuning (CPU Inference)
|
||||
|
||||
### Environment Variables
|
||||
|
||||
```bash
|
||||
# Threads (default: all cores)
|
||||
export OLLAMA_NUM_PARALLEL=2 # concurrent requests
|
||||
export OLLAMA_NUM_THREAD=8 # threads per request
|
||||
|
||||
# Context window (default: model default, e.g., 4096 or 8192)
|
||||
# Increase in request: "num_ctx": 16384
|
||||
|
||||
# GPU offloading (if GPU available)
|
||||
# export OLLAMA_GPU_LAYERS=999 # offload all layers
|
||||
```
|
||||
|
||||
### For Headless Server (This Session's Setup)
|
||||
|
||||
```bash
|
||||
# 8 cores, 16GB RAM - good config:
|
||||
OLLAMA_NUM_PARALLEL=2
|
||||
OLLAMA_NUM_THREAD=8
|
||||
|
||||
# Add to ollama service:
|
||||
# systemctl edit ollama
|
||||
# [Service]
|
||||
# Environment="OLLAMA_NUM_PARALLEL=2"
|
||||
# Environment="OLLAMA_NUM_THREAD=8"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Model Storage
|
||||
|
||||
- Default: `/usr/share/ollama/.ollama/models`
|
||||
- Override: `OLLAMA_MODELS=/path/to/models`
|
||||
- Each model: ~1-40 GB depending on size/quantization
|
||||
|
||||
```bash
|
||||
# Check usage
|
||||
du -sh /usr/share/ollama/.ollama/models
|
||||
|
||||
# Move to larger disk
|
||||
systemctl stop ollama
|
||||
mv /usr/share/ollama /mnt/storage/
|
||||
ln -s /mnt/storage/ollama /usr/share/ollama
|
||||
systemctl start ollama
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Updating Models
|
||||
|
||||
```bash
|
||||
# Pull latest version
|
||||
ollama pull qwen2.5-coder:7b
|
||||
|
||||
# Remove old
|
||||
ollama rm qwen2.5-coder:7b:old
|
||||
|
||||
# List with sizes
|
||||
ollama list
|
||||
```
|
||||
@@ -0,0 +1,296 @@
|
||||
# Local Voice Pipeline (STT + TTS) for Hermes Gateway
|
||||
|
||||
## Overview
|
||||
|
||||
This reference covers setting up **fully local** Speech-to-Text (STT) and Text-to-Speech (TTS) for Telegram voice messages on a headless server — no cloud APIs, no costs.
|
||||
|
||||
## Architecture
|
||||
|
||||
```
|
||||
Telegram Voice (.ogg) → Gateway → ffmpeg → faster-whisper (STT) → Text
|
||||
↓
|
||||
LLM (OpenRouter/Ollama) → Response Text → Piper (TTS) → MP3 → Telegram Voice
|
||||
```
|
||||
|
||||
All components run locally on the server.
|
||||
|
||||
---
|
||||
|
||||
## STT: faster-whisper
|
||||
|
||||
### Installation
|
||||
|
||||
```bash
|
||||
# In Hermes venv
|
||||
/usr/local/lib/hermes-agent/venv/bin/python -m pip install faster-whisper
|
||||
```
|
||||
|
||||
### Model Options
|
||||
|
||||
| Model | Size | Speed | Quality | Languages | RAM |
|
||||
|-------|------|-------|---------|-----------|-----|
|
||||
| `tiny` | 39 MB | Fastest | Low | 99 | ~1 GB |
|
||||
| `base` | 74 MB | Fast | Good | 99 | ~1 GB |
|
||||
| `small` | 244 MB | Medium | Better | 99 | ~2 GB |
|
||||
| `medium` | 769 MB | Slow | High | 99 | ~5 GB |
|
||||
| `large-v3` | 1.5 GB | Slowest | Best | 99 | ~10 GB |
|
||||
|
||||
**Recommendation for server:** `base` or `small` — good balance.
|
||||
|
||||
### Configuration (`~/.hermes/config.yaml`)
|
||||
|
||||
```yaml
|
||||
stt:
|
||||
enabled: true
|
||||
provider: local
|
||||
local:
|
||||
model: base # tiny, base, small, medium, large-v3
|
||||
language: '' # auto-detect, or 'de', 'en', 'fr', etc.
|
||||
use_gateway: true
|
||||
```
|
||||
|
||||
### Compute Types (for CPU)
|
||||
|
||||
| Type | Speed | Quality | RAM |
|
||||
|------|-------|---------|-----|
|
||||
| `int8` | Fastest | Good | Lowest |
|
||||
| `int8_float16` | Fast | Better | Medium |
|
||||
| `float16` | Medium | Best | Higher |
|
||||
| `float32` | Slow | Best | Highest |
|
||||
|
||||
**Default:** `int8` — best for CPU-only servers.
|
||||
|
||||
### Testing STT
|
||||
|
||||
```bash
|
||||
/usr/local/lib/hermes-agent/venv/bin/python -c "
|
||||
from faster_whisper import WhisperModel
|
||||
model = WhisperModel('base', device='cpu', compute_type='int8')
|
||||
segments, info = model.transcribe('test.ogg', language='de')
|
||||
for s in segments:
|
||||
print(f'[{s.start:.1f}s -> {s.end:.1f}s] {s.text}')
|
||||
print(f'Language: {info.language} (p={info.language_probability:.2f})')
|
||||
"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## TTS: Piper
|
||||
|
||||
### Installation
|
||||
|
||||
```bash
|
||||
# In Hermes venv
|
||||
/usr/local/lib/hermes-agent/venv/bin/python -m pip install piper-tts
|
||||
```
|
||||
|
||||
### Voice Models
|
||||
|
||||
Voices auto-download on first use from: https://github.com/rhasspy/piper/blob/master/VOICES.md
|
||||
|
||||
**Popular voices:**
|
||||
|
||||
| Language | Voice | Quality | Size |
|
||||
|----------|-------|---------|------|
|
||||
| English (US) | `en_US-lessac-medium` | Natural, clear | ~50 MB |
|
||||
| English (US) | `en_US-amy-medium` | Female, warm | ~50 MB |
|
||||
| English (UK) | `en_GB-northern_english_male-medium` | British accent | ~50 MB |
|
||||
| **German** | `de_DE-thorsten-medium` | **Best German male** | ~50 MB |
|
||||
| **German** | `de_DE-karlsson-medium` | German male, different tone | ~50 MB |
|
||||
| **German** | `de_DE-eva_k-x_low` | Female, fast (low quality) | ~10 MB |
|
||||
| French | `fr_FR-siwis-medium` | Good French | ~50 MB |
|
||||
| Spanish | `es_ES-sharvard-medium` | Good Spanish | ~50 MB |
|
||||
|
||||
### Configuration (`~/.hermes/config.yaml`)
|
||||
|
||||
```yaml
|
||||
tts:
|
||||
provider: piper
|
||||
piper:
|
||||
voice: en_US-lessac-medium # or de_DE-thorsten-medium for German
|
||||
use_gateway: true
|
||||
```
|
||||
|
||||
### Pre-download Voice (avoids first-request latency)
|
||||
|
||||
```bash
|
||||
# Test download
|
||||
/usr/local/lib/hermes-agent/venv/bin/python -c "
|
||||
from piper.voice import PiperVoice
|
||||
voice = PiperVoice.load('de_DE-thorsten-medium') # triggers download
|
||||
print(f'Voice loaded: {voice.config.num_symbols} symbols, {voice.config.sample_rate} Hz')
|
||||
"
|
||||
|
||||
# Check downloaded files
|
||||
ls -la ~/.cache/piper/
|
||||
```
|
||||
|
||||
### Testing TTS
|
||||
|
||||
```bash
|
||||
# Via Hermes tool
|
||||
text_to_speech text="Hallo, das ist ein Test auf Deutsch."
|
||||
|
||||
# Or direct Python
|
||||
/usr/local/lib/hermes-agent/venv/bin/python -c "
|
||||
from piper.voice import PiperVoice
|
||||
voice = PiperVoice.load('de_DE-thorsten-medium')
|
||||
import soundfile as sf
|
||||
audio = voice.synthesize('Hallo, das ist ein Test.', speaker_id=0)
|
||||
sf.write('/tmp/test.wav', audio, voice.config.sample_rate)
|
||||
print('Generated /tmp/test.wav')
|
||||
"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Telegram Voice Integration
|
||||
|
||||
### Requirements
|
||||
|
||||
1. **ffmpeg** for audio conversion (`.ogg` → `.wav` for STT)
|
||||
2. **Gateway** configured with Telegram platform
|
||||
3. **STT + TTS** configured as above
|
||||
|
||||
```bash
|
||||
apt-get install -y ffmpeg
|
||||
```
|
||||
|
||||
### Flow Details
|
||||
|
||||
1. **User sends voice** → Telegram delivers `.ogg` (Opus codec) to gateway
|
||||
2. **Gateway downloads** → saves to temp file
|
||||
3. **ffmpeg converts** → `.wav` (16kHz mono) for STT
|
||||
4. **faster-whisper transcribes** → German/English text
|
||||
5. **LLM processes** → generates response
|
||||
6. **Piper synthesizes** → `.wav` → converted to `.mp3`
|
||||
7. **Gateway sends** → Telegram voice message
|
||||
|
||||
### Latency Estimates (CPU-only, 8 cores)
|
||||
|
||||
| Step | Time (approx) |
|
||||
|------|---------------|
|
||||
| Download + ffmpeg | 0.5-1s |
|
||||
| STT (base model) | 1-3s |
|
||||
| LLM (Ollama deepseek-r1:7b) | 3-10s |
|
||||
| TTS (Piper) | 0.5-2s |
|
||||
| **Total** | **5-16s** |
|
||||
|
||||
### Optimization Tips
|
||||
|
||||
```yaml
|
||||
# In config.yaml - reduce STT model for speed
|
||||
stt:
|
||||
local:
|
||||
model: tiny # fastest, lower accuracy
|
||||
# OR
|
||||
model: base # recommended balance
|
||||
|
||||
# For TTS - use lower quality voice for speed
|
||||
tts:
|
||||
piper:
|
||||
voice: de_DE-eva_k-x_low # fast, smaller, lower quality
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### STT Not Working
|
||||
|
||||
```bash
|
||||
# Check model downloads
|
||||
ls -la ~/.cache/huggingface/hub/models--Systran--faster-whisper-*/
|
||||
|
||||
# Test manually
|
||||
/usr/local/lib/hermes-agent/venv/bin/python -c "
|
||||
from faster_whisper import WhisperModel
|
||||
model = WhisperModel('base', device='cpu', compute_type='int8')
|
||||
print('Model loaded OK')
|
||||
"
|
||||
```
|
||||
|
||||
### TTS Not Working / No Audio
|
||||
|
||||
```bash
|
||||
# Check voice cache
|
||||
ls -la ~/.cache/piper/
|
||||
|
||||
# Test direct synthesis
|
||||
/usr/local/lib/hermes-agent/venv/bin/python -c "
|
||||
from piper.voice import PiperVoice
|
||||
voice = PiperVoice.load('en_US-lessac-medium')
|
||||
audio = voice.synthesize('Test', speaker_id=0)
|
||||
print(f'Generated {len(audio)} samples at {voice.config.sample_rate} Hz')
|
||||
"
|
||||
```
|
||||
|
||||
### Telegram Voice Not Sent
|
||||
|
||||
```bash
|
||||
# Check gateway logs
|
||||
journalctl -u hermes-gateway -n 50 | grep -i -e voice -e tts -e stt -e ogg
|
||||
|
||||
# Verify ffmpeg
|
||||
ffmpeg -version
|
||||
|
||||
# Check Telegram bot has voice permissions
|
||||
# BotFather → Bot Settings → Voice Messages: ENABLED
|
||||
```
|
||||
|
||||
### Gateway Crash on Voice
|
||||
|
||||
```bash
|
||||
# Increase memory limits
|
||||
# In service file:
|
||||
MemoryMax=4G
|
||||
MemoryHigh=3G
|
||||
|
||||
# Or reduce model sizes (tiny STT, low-quality TTS)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Language-Specific Notes
|
||||
|
||||
### German STT
|
||||
|
||||
- `base` model: Good for standard German
|
||||
- `small` model: Better for dialects, technical terms
|
||||
- Set `language: 'de'` in config for faster processing (skips detection)
|
||||
|
||||
### German TTS
|
||||
|
||||
- `de_DE-thorsten-medium`: **Best quality**, natural male voice
|
||||
- `de_DE-karlsson-medium`: Alternative male voice
|
||||
- `de_DE-eva_k-x_low`: Fast, robotic female — only for speed-critical
|
||||
|
||||
### Mixed Language
|
||||
|
||||
- STT auto-detects well with `language: ''`
|
||||
- TTS: Use appropriate voice per language (config is global, so pick primary)
|
||||
|
||||
---
|
||||
|
||||
## Advanced: Custom Voice Training (Piper)
|
||||
|
||||
For custom voices: https://github.com/rhasspy/piper/blob/master/TRAINING.md
|
||||
|
||||
Requires: ~20min clean audio + transcripts, GPU for training (faster).
|
||||
|
||||
Not covered here — use pre-trained voices.
|
||||
|
||||
---
|
||||
|
||||
## Cost Summary
|
||||
|
||||
| Component | Cost |
|
||||
|-----------|------|
|
||||
| Ollama (LLM) | **Free** (local) |
|
||||
| faster-whisper (STT) | **Free** (local) |
|
||||
| Piper (TTS) | **Free** (local) |
|
||||
| ffmpeg | **Free** |
|
||||
| Telegram Bot API | **Free** |
|
||||
| **Total** | **$0/month** |
|
||||
|
||||
No API keys, no cloud dependencies, works offline.
|
||||
Reference in New Issue
Block a user