mehlzoerwer/mesh-configs
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Initial: Mesh-Configs vom 03.09.2026 01:23

Browse Source
This commit is contained in:
root
2026-09-03 01:23:55 +02:00
commit 4b65134f8c
797 changed files with 146197 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
skills/mlops-inference/ollama-local-llm/references/deepseek-switch.md
+14
View File
@@ -0,0 +1,14 @@
# Switch to deepseek-r1:7b as Ollama fallback
**Reason:** User reported that `qwen2.5-coder:7b` was repeatedly referenced despite being removed, causing fallback errors because the model does not support `thinking` required by certain Hermes jobs (e.g., cron jobs with reasoning_effort: medium). After verifying Ollama model list, the user confirmed the local model had been changed to `deepseek-r1:7b`.
**Changes made:**
- Updated `fallback_providers.0.model` in `~/.hermes/config.yaml` to `deepseek-r1:7b`.
- Patched cron jobs `d4def5626a43` (Uhrenabgleich 0 Uhr) and `471b5a167aa1` (Wissenstand-Fusion 1 Uhr) to use `ollama/deepseek-r1:7b`.
- Replaced all references to `qwen2.5-coder:7b` in skill references (voice-pipeline.md, ollama-model-guide.md, SKILL.md) with `deepseek-r1:7b`.
- Updated `context_length_cache.yaml` entries accordingly.
- Pulled `deepseek-r1:7b` via `ollama pull deepseek-r1:7b`.
**Verification:** After changes, cron jobs run successfully, and Hermes falls back to `deepseek-r1:7b` when OpenRouter provider fails.
**Note:** This skill now reflects the current local LLM preference for the user's VPS setup.
skills/mlops-inference/ollama-local-llm/references/ollama-model-guide.md
+173
View File
@@ -0,0 +1,173 @@
# Ollama Model Selection Guide
## Model Categories & Recommendations
### Coding Models (Shell commands, code generation, /code tasks)
| Model | Size | Context | Best For | RAM (CPU) |
|-------|------|---------|----------|-----------|
| `deepseek-r1:7b` | 4.7 GB | 32K | **Best overall** — strong coding, good instruction following, multi-language | ~6 GB |
| `qwen2.5-coder:14b` | 9.0 GB | 32K | Better quality, slower | ~12 GB |
| `deepseek-coder-v2:16b` | 9.4 GB | 128K | Long context coding tasks | ~12 GB |
| `codellama:13b` | 7.4 GB | 16K | Good alternative, older | ~10 GB |
| `starcoder2:7b` | 4.4 GB | 16K | Fast, decent quality | ~6 GB |
**Recommendation:** `deepseek-r1:7b` — best balance of quality/size/speed for Hermes `/shell` and `/code`.
### Chat / General Reasoning Models
| Model | Size | Context | Best For | RAM (CPU) |
|-------|------|---------|----------|-----------|
| `llama3.1:8b` | 4.9 GB | 128K | **Best general** — strong reasoning, multilingual, tool use | ~6 GB |
| `llama3.1:70b` | 40 GB | 128K | Best quality, needs GPU | ~48 GB |
| `nemotron3:8b` | 4.7 GB | 4K | NVIDIA optimized, fast | ~6 GB |
| `gemma2:9b` | 5.4 GB | 8K | Google, strong multilingual | ~7 GB |
| `phi3:14b` | 8.2 GB | 128K | Microsoft, good reasoning | ~11 GB |
| `command-r:35b` | 20 GB | 128K | RAG-optimized, tool use | ~24 GB |
**Recommendation:** `llama3.1:8b` — best balance for chat fallback.
### Embedding Models (RAG, memory, semantic search)
| Model | Size | Dim | Best For | RAM |
|-------|------|-----|----------|-----|
| `nomic-embed-text` | 274 MB | 768 | **Best general** — fast, good quality | ~1 GB |
| `nomic-embed-text:v1.5` | 274 MB | 768 | Improved version | ~1 GB |
| `mxbai-embed-large` | 668 MB | 1024 | Higher quality, larger | ~2 GB |
| `bge-m3` | 1.2 GB | 1024 | Multi-lingual, dense+sparse | ~3 GB |
| `all-minilm:l6-v2` | 90 MB | 384 | Fast, small, lower quality | ~500 MB |
**Recommendation:** `nomic-embed-text` — best speed/quality for Hermes memory/RAG.
### Fast / Small Models (Quick responses, low RAM)
| Model | Size | Context | Best For | RAM |
|-------|------|---------|----------|-----|
| `gemma3:4b` | ~3 GB | 32K | Fast chat, multilingual | ~4 GB |
| `phi3:3.8b` | 2.3 GB | 128K | Very fast, decent quality | ~3 GB |
| `qwen2.5:3b` | 2.0 GB | 32K | Fast, multilingual | ~3 GB |
| `llama3.2:3b` | 2.0 GB | 128K | Tiny but capable | ~3 GB |
---
## Quantization Guide
Ollama uses GGUF quantization. Suffix indicates quantization level:
| Suffix | Bits | Quality | Speed | Size Reduction |
|--------|------|---------|-------|----------------|
| `:latest` / `:fp16` | 16 | Best | Slowest | Baseline |
| `:q8_0` | 8 | Near-lossless | Fast | ~50% |
| `:q6_k` | 6 | Very good | Fast | ~60% |
| `:q5_k_m` | 5 | Good | Faster | ~65% |
| `:q4_k_m` | 4 | **Recommended** | Fast | ~75% |
| `:q4_0` | 4 | Good | Fastest | ~75% |
| `:q3_k_m` | 3 | Acceptable | Very fast | ~80% |
| `:q2_k` | 2 | Low | Fastest | ~85% |
**Default (no suffix)** = `q4_k_m` usually — best balance.
**For Hermes:** Use defaults (`:latest` → `q4_k_m`). Explicitly pull `:q4_k_m` if you want certainty.
---
## Multi-Model Setup for Hermes
```bash
# Primary coding model (used by fallback)
ollama pull qwen2.5-coder:7b
# General chat fallback (optional - can use same)
ollama pull llama3.1:8b
# Embeddings for memory/RAG
ollama pull nomic-embed-text
# Optional: fast model for quick responses
ollama pull gemma3:4b
```
### Configuring Different Models for Different Tasks
In `~/.hermes/config.yaml`, you can set up multiple fallback providers:
```yaml
providers:
ollama:
base_url: http://localhost:11434/v1
api_key: ollama
fallback_providers:
# Primary fallback for coding
- provider: ollama
model: qwen2.5-coder:7b
# Secondary fallback for chat (if first fails)
- provider: ollama
model: llama3.1:8b
```
---
## Performance Tuning (CPU Inference)
### Environment Variables
```bash
# Threads (default: all cores)
export OLLAMA_NUM_PARALLEL=2 # concurrent requests
export OLLAMA_NUM_THREAD=8 # threads per request
# Context window (default: model default, e.g., 4096 or 8192)
# Increase in request: "num_ctx": 16384
# GPU offloading (if GPU available)
# export OLLAMA_GPU_LAYERS=999 # offload all layers
```
### For Headless Server (This Session's Setup)
```bash
# 8 cores, 16GB RAM - good config:
OLLAMA_NUM_PARALLEL=2
OLLAMA_NUM_THREAD=8
# Add to ollama service:
# systemctl edit ollama
# [Service]
# Environment="OLLAMA_NUM_PARALLEL=2"
# Environment="OLLAMA_NUM_THREAD=8"
```
---
## Model Storage
- Default: `/usr/share/ollama/.ollama/models`
- Override: `OLLAMA_MODELS=/path/to/models`
- Each model: ~1-40 GB depending on size/quantization
```bash
# Check usage
du -sh /usr/share/ollama/.ollama/models
# Move to larger disk
systemctl stop ollama
mv /usr/share/ollama /mnt/storage/
ln -s /mnt/storage/ollama /usr/share/ollama
systemctl start ollama
```
---
## Updating Models
```bash
# Pull latest version
ollama pull qwen2.5-coder:7b
# Remove old
ollama rm qwen2.5-coder:7b:old
# List with sizes
ollama list
```
skills/mlops-inference/ollama-local-llm/references/voice-pipeline.md
+296
View File
@@ -0,0 +1,296 @@
# Local Voice Pipeline (STT + TTS) for Hermes Gateway
## Overview
This reference covers setting up **fully local** Speech-to-Text (STT) and Text-to-Speech (TTS) for Telegram voice messages on a headless server — no cloud APIs, no costs.
## Architecture
```
Telegram Voice (.ogg) → Gateway → ffmpeg → faster-whisper (STT) → Text
↓
LLM (OpenRouter/Ollama) → Response Text → Piper (TTS) → MP3 → Telegram Voice
```
All components run locally on the server.
---
## STT: faster-whisper
### Installation
```bash
# In Hermes venv
/usr/local/lib/hermes-agent/venv/bin/python -m pip install faster-whisper
```
### Model Options
| Model | Size | Speed | Quality | Languages | RAM |
|-------|------|-------|---------|-----------|-----|
| `tiny` | 39 MB | Fastest | Low | 99 | ~1 GB |
| `base` | 74 MB | Fast | Good | 99 | ~1 GB |
| `small` | 244 MB | Medium | Better | 99 | ~2 GB |
| `medium` | 769 MB | Slow | High | 99 | ~5 GB |
| `large-v3` | 1.5 GB | Slowest | Best | 99 | ~10 GB |
**Recommendation for server:** `base` or `small` — good balance.
### Configuration (`~/.hermes/config.yaml`)
```yaml
stt:
enabled: true
provider: local
local:
model: base # tiny, base, small, medium, large-v3
language: '' # auto-detect, or 'de', 'en', 'fr', etc.
use_gateway: true
```
### Compute Types (for CPU)
| Type | Speed | Quality | RAM |
|------|-------|---------|-----|
| `int8` | Fastest | Good | Lowest |
| `int8_float16` | Fast | Better | Medium |
| `float16` | Medium | Best | Higher |
| `float32` | Slow | Best | Highest |
**Default:** `int8` — best for CPU-only servers.
### Testing STT
```bash
/usr/local/lib/hermes-agent/venv/bin/python -c "
from faster_whisper import WhisperModel
model = WhisperModel('base', device='cpu', compute_type='int8')
segments, info = model.transcribe('test.ogg', language='de')
for s in segments:
print(f'[{s.start:.1f}s -> {s.end:.1f}s] {s.text}')
print(f'Language: {info.language} (p={info.language_probability:.2f})')
"
```
---
## TTS: Piper
### Installation
```bash
# In Hermes venv
/usr/local/lib/hermes-agent/venv/bin/python -m pip install piper-tts
```
### Voice Models
Voices auto-download on first use from: https://github.com/rhasspy/piper/blob/master/VOICES.md
**Popular voices:**
| Language | Voice | Quality | Size |
|----------|-------|---------|------|
| English (US) | `en_US-lessac-medium` | Natural, clear | ~50 MB |
| English (US) | `en_US-amy-medium` | Female, warm | ~50 MB |
| English (UK) | `en_GB-northern_english_male-medium` | British accent | ~50 MB |
| **German** | `de_DE-thorsten-medium` | **Best German male** | ~50 MB |
| **German** | `de_DE-karlsson-medium` | German male, different tone | ~50 MB |
| **German** | `de_DE-eva_k-x_low` | Female, fast (low quality) | ~10 MB |
| French | `fr_FR-siwis-medium` | Good French | ~50 MB |
| Spanish | `es_ES-sharvard-medium` | Good Spanish | ~50 MB |
### Configuration (`~/.hermes/config.yaml`)
```yaml
tts:
provider: piper
piper:
voice: en_US-lessac-medium # or de_DE-thorsten-medium for German
use_gateway: true
```
### Pre-download Voice (avoids first-request latency)
```bash
# Test download
/usr/local/lib/hermes-agent/venv/bin/python -c "
from piper.voice import PiperVoice
voice = PiperVoice.load('de_DE-thorsten-medium') # triggers download
print(f'Voice loaded: {voice.config.num_symbols} symbols, {voice.config.sample_rate} Hz')
"
# Check downloaded files
ls -la ~/.cache/piper/
```
### Testing TTS
```bash
# Via Hermes tool
text_to_speech text="Hallo, das ist ein Test auf Deutsch."
# Or direct Python
/usr/local/lib/hermes-agent/venv/bin/python -c "
from piper.voice import PiperVoice
voice = PiperVoice.load('de_DE-thorsten-medium')
import soundfile as sf
audio = voice.synthesize('Hallo, das ist ein Test.', speaker_id=0)
sf.write('/tmp/test.wav', audio, voice.config.sample_rate)
print('Generated /tmp/test.wav')
"
```
---
## Telegram Voice Integration
### Requirements
1. **ffmpeg** for audio conversion (`.ogg` → `.wav` for STT)
2. **Gateway** configured with Telegram platform
3. **STT + TTS** configured as above
```bash
apt-get install -y ffmpeg
```
### Flow Details
1. **User sends voice** → Telegram delivers `.ogg` (Opus codec) to gateway
2. **Gateway downloads** → saves to temp file
3. **ffmpeg converts** → `.wav` (16kHz mono) for STT
4. **faster-whisper transcribes** → German/English text
5. **LLM processes** → generates response
6. **Piper synthesizes** → `.wav` → converted to `.mp3`
7. **Gateway sends** → Telegram voice message
### Latency Estimates (CPU-only, 8 cores)
| Step | Time (approx) |
|------|---------------|
| Download + ffmpeg | 0.5-1s |
| STT (base model) | 1-3s |
| LLM (Ollama deepseek-r1:7b) | 3-10s |
| TTS (Piper) | 0.5-2s |
| **Total** | **5-16s** |
### Optimization Tips
```yaml
# In config.yaml - reduce STT model for speed
stt:
local:
model: tiny # fastest, lower accuracy
# OR
model: base # recommended balance
# For TTS - use lower quality voice for speed
tts:
piper:
voice: de_DE-eva_k-x_low # fast, smaller, lower quality
```
---
## Troubleshooting
### STT Not Working
```bash
# Check model downloads
ls -la ~/.cache/huggingface/hub/models--Systran--faster-whisper-*/
# Test manually
/usr/local/lib/hermes-agent/venv/bin/python -c "
from faster_whisper import WhisperModel
model = WhisperModel('base', device='cpu', compute_type='int8')
print('Model loaded OK')
"
```
### TTS Not Working / No Audio
```bash
# Check voice cache
ls -la ~/.cache/piper/
# Test direct synthesis
/usr/local/lib/hermes-agent/venv/bin/python -c "
from piper.voice import PiperVoice
voice = PiperVoice.load('en_US-lessac-medium')
audio = voice.synthesize('Test', speaker_id=0)
print(f'Generated {len(audio)} samples at {voice.config.sample_rate} Hz')
"
```
### Telegram Voice Not Sent
```bash
# Check gateway logs
journalctl -u hermes-gateway -n 50 | grep -i -e voice -e tts -e stt -e ogg
# Verify ffmpeg
ffmpeg -version
# Check Telegram bot has voice permissions
# BotFather → Bot Settings → Voice Messages: ENABLED
```
### Gateway Crash on Voice
```bash
# Increase memory limits
# In service file:
MemoryMax=4G
MemoryHigh=3G
# Or reduce model sizes (tiny STT, low-quality TTS)
```
---
## Language-Specific Notes
### German STT
- `base` model: Good for standard German
- `small` model: Better for dialects, technical terms
- Set `language: 'de'` in config for faster processing (skips detection)
### German TTS
- `de_DE-thorsten-medium`: **Best quality**, natural male voice
- `de_DE-karlsson-medium`: Alternative male voice
- `de_DE-eva_k-x_low`: Fast, robotic female — only for speed-critical
### Mixed Language
- STT auto-detects well with `language: ''`
- TTS: Use appropriate voice per language (config is global, so pick primary)
---
## Advanced: Custom Voice Training (Piper)
For custom voices: https://github.com/rhasspy/piper/blob/master/TRAINING.md
Requires: ~20min clean audio + transcripts, GPU for training (faster).
Not covered here — use pre-trained voices.
---
## Cost Summary
| Component | Cost |
|-----------|------|
| Ollama (LLM) | **Free** (local) |
| faster-whisper (STT) | **Free** (local) |
| Piper (TTS) | **Free** (local) |
| ffmpeg | **Free** |
| Telegram Bot API | **Free** |
| **Total** | **$0/month** |
No API keys, no cloud dependencies, works offline.