Initial: Mesh-Skills, Configs, Scripts
This commit is contained in:
root
@@ -0,0 +1,119 @@
|
||||
---
|
||||
name: model-allocation
|
||||
description: >-
|
||||
Kostenkontrollierte, verhältnismäßige Modell-Auswahl für Hermes-Agents.
|
||||
Drei Ebenen: tägliche Ausrichtung, stündliches Feintuning, Task-bezogenes Matching.
|
||||
Lokaler Fallback über VPS-Ollama (deepseek-r1:7b) mit 64k+ Kontext und Thinking.
|
||||
Budget: 1€/Tag/Agent, bei Überschreitung → postpone + lokaler Fallback.
|
||||
related_skills: [multi-agent-mesh]
|
||||
---
|
||||
|
||||
# Model-Allocation-Skill
|
||||
|
||||
Automatisierte Entscheidungslogik für Modell-Wahl pro Agent und Task.
|
||||
|
||||
## Prinzipien
|
||||
|
||||
- **Kosten ~99% Gewicht**, Zeit spielt keine Rolle
|
||||
- **Kein Free-Only-Zwang**, aber proportionaler Aufstieg erlaubt
|
||||
- **Kein Ferrari-Sprung** (>5x Kosten) ohne manual_override
|
||||
- **Diversität** als Ziel (min 3 Modell-Familien)
|
||||
|
||||
## Die drei Ebenen
|
||||
|
||||
### Ebene 1 — Täglich (grobe Ausrichtung)
|
||||
- `scripts/daily_model_check.py`
|
||||
- Holt Modell-/Preisliste (Mock → OpenRouter-API)
|
||||
- Filtert ≥64k Kontext, baut Prioritätsliste pro Agent
|
||||
- Prüft Diversität, schreibt `state/model_priority.json`
|
||||
|
||||
### Ebene 2 — Stündlich (Feintuning)
|
||||
- `scripts/hourly_finetune_check.py`
|
||||
- Prüft Budget-Warnschwelle (1€/Tag/Agent), Verfügbarkeit, Rate-Limits
|
||||
- Bei Überschreitung: Tasks auf nächsten Tag verschieben
|
||||
- Einfache Anfragen → lokales VPS-Ollama (`deepseek-r1:7b`)
|
||||
|
||||
### Ebene 3 — Task-bezogen (Feinauswahl)
|
||||
- `scripts/task_agent_matcher.py`
|
||||
- Input: Capability-Tag → findet günstigste passende Kombination
|
||||
- Eskaliert max eine Stufe bei Fehlschlag
|
||||
- Fallback auf lokales Ollama wenn kein Agent verfügbar
|
||||
- Max 3 Retries pro Task
|
||||
|
||||
## Konfiguration
|
||||
|
||||
Siehe `config/agents.yaml` für die vollständige Config.
|
||||
Siehe `references/agents-config.md` für eine kompakte Übersicht.
|
||||
|
||||
Pro Agent: Modell, Eskalationsleiter, Tags, Budget
|
||||
Global: min_context=64000, max_escalation=5x, min_diversity=3
|
||||
`local_fallback.model: deepseek-r1:7b` (Thinking + 128k Kontext)
|
||||
|
||||
## Lokale Modelle auf dem VPS
|
||||
|
||||
| Modell | Größe | Kontext | Thinking | Status |
|
||||
|--------|-------|---------|----------|--------|
|
||||
| `deepseek-r1:7b` | 4.7 GB | 128k | ✅ | aktiv (Default) |
|
||||
| `deepseek-r1:14b` | 9.0 GB | 128k | ✅ | runtergeladen, ungetestet |
|
||||
|
||||
### ⚠️ WICHTIG: Ollama Bindungs-Address
|
||||
|
||||
Das Ollama-Installation auf dem VPS lauscht an `172.24.196.119:11434` (VPS-Externe IP), NICHT an `localhost:11434`.
|
||||
|
||||
**Konsequenz:** Das Config-`base_url` muss `http://172.24.196.119:11434/v1` lauten, damit das Modell von anderen Mesh-Agenten (oder demselben Agenten im Netzwerk) darauf zugreifen kann.
|
||||
|
||||
Fehlgeschlagene Verbindungssymptome:
|
||||
- `curl http://localhost:11434/api/tags` → Connection refused (wenn Ollama nicht localhost-bound)
|
||||
- Modell kann lokalen Ollama-Fallback nicht erreichen
|
||||
- Task fällt auf nächstes teureren OpenRouter-Modell aus
|
||||
|
||||
**Behebung:** Config-Key `ollama.base_url` auf `http://172.24.196.119:11434/v1` setzen. Siehe `references/ollama-bind-address.md` für Details.
|
||||
|
||||
**Warum deepseek-r1:7b?** Thinking + 128k Kontext in 4.7 GB — optimal für 16 GB RAM VPS.
|
||||
`deepseek-r1:14b` ist Upgrade-Option sobald RAM-Puffer bestätigt ist.
|
||||
|
||||
## Setup
|
||||
|
||||
```bash
|
||||
pip install pyyaml --break-system-packages
|
||||
```
|
||||
|
||||
Detailierte Modell-Vergleiche: `references/local-model-comparison.md`
|
||||
|
||||
## Cron
|
||||
|
||||
### Lokal (auf Hub/Agent A via Hermes-Cron)
|
||||
|
||||
```bash
|
||||
hermes cron add --schedule '0 3 * * *' --name model-allocation-daily \
|
||||
--prompt 'cd ~/.hermes/skills/devops/model-allocation && python3 scripts/daily_model_check.py'
|
||||
hermes cron add --schedule '0 * * * *' --name model-allocation-hourly \
|
||||
--prompt 'cd ~/.hermes/skills/devops/model-allocation && python3 scripts/hourly_finetune_check.py'
|
||||
```
|
||||
|
||||
### Auf Remote-Agents per SSH (crontab)
|
||||
|
||||
Verwende ein Setup-Script für mehrere Agenten gleichzeitig:
|
||||
|
||||
```bash
|
||||
# setup_model_cron.sh
|
||||
mkdir -p $HOME/.hermes/logs
|
||||
if crontab -l 2>/dev/null | grep -q 'model-allocation'; then
|
||||
echo "Cron bereits da"; exit 0
|
||||
fi
|
||||
(crontab -l 2>/dev/null
|
||||
echo "# model-allocation: täglich 03:00"
|
||||
echo "0 3 * * * cd \$HOME/.hermes/skills/devops/model-allocation \\"
|
||||
echo " && python3 scripts/daily_model_check.py >> \$HOME/.hermes/logs/model-allocation-daily.log 2>&1"
|
||||
echo "# model-allocation: stündlich"
|
||||
echo "0 * * * * cd \$HOME/.hermes/skills/devops/model-allocation \\"
|
||||
echo " && python3 scripts/hourly_finetune_check.py >> \$HOME/.hermes/logs/model-allocation-hourly.log 2>&1"
|
||||
) | crontab -
|
||||
echo "Cron eingerichtet"
|
||||
|
||||
# Auf mehreren Remotes gleichzeitig:
|
||||
for ip in 172.24.20.26 172.24.34.142 172.24.178.202; do
|
||||
sshpass -p 'PW' scp setup_model_cron.sh user@$ip:/tmp/
|
||||
sshpass -p 'PW' ssh user@$ip 'bash /tmp/setup_model_cron.sh'
|
||||
done
|
||||
```
|
||||
@@ -0,0 +1,78 @@
|
||||
# Konfiguration für die Agent-Modell-Auswahl
|
||||
|
||||
global:
|
||||
min_context_tokens: 64000
|
||||
max_auto_escalation_factor: 5 # ab diesem Faktor gilt ein Wechsel als "Ferrari-Sprung" -> braucht manual_override
|
||||
min_distinct_model_families: 3 # Mindest-Diversität über alle Agents hinweg
|
||||
max_retries_per_task: 3 # danach Task pausieren, nicht weiter eskalieren
|
||||
default_hard_stop_eur_per_day: 1.0 # 1€/Tag/Agent als hartes Maximum
|
||||
on_budget_exceeded: postpone_to_next_day # Arbeit verschieben statt Agent komplett zu blockieren
|
||||
basic_questions_fallback: vps_ollama # einfache/Grundfragen weiter über lokales VPS-Ollama, wenn kontextuell sinnvoll
|
||||
|
||||
agents:
|
||||
agent_a:
|
||||
role: master
|
||||
description: "Master-Agent, Allround, Linux-System-Support"
|
||||
hard_stop_eur_per_day: 1.0
|
||||
capability_tags: [linux_system, allround, orchestration]
|
||||
escalation_ladder:
|
||||
- model: "nvidia/nemotron-3-super" # Stufe 0, Standard
|
||||
cost_factor: 1
|
||||
- model: "nvidia/nemotron-3-ultra" # Stufe 1, Eskalation
|
||||
cost_factor: 3
|
||||
manual_override_allowed: false
|
||||
|
||||
agent_b:
|
||||
role: worker
|
||||
description: "DeepSeek Flash/Pro Eskalationsleiter"
|
||||
hard_stop_eur_per_day: 1.0
|
||||
capability_tags: [code, logic, database]
|
||||
escalation_ladder:
|
||||
- model: "deepseek/deepseek-flash"
|
||||
cost_factor: 1
|
||||
- model: "deepseek/deepseek-pro"
|
||||
cost_factor: 3
|
||||
manual_override_allowed: false
|
||||
|
||||
switch_hermes:
|
||||
role: cockpit
|
||||
description: "Nur Prompt/Code-Generierung, kein lokales Ausführen"
|
||||
hard_stop_eur_per_day: 1.0
|
||||
capability_tags: [homebrew_code, prompt_generation]
|
||||
escalation_ladder:
|
||||
- model: "openrouter/free" # OpenRouter Auto-Router, Free-Tier
|
||||
cost_factor: 0
|
||||
- model: "qwen/qwen3-coder:free"
|
||||
cost_factor: 0
|
||||
manual_override_allowed: false
|
||||
|
||||
laptop_hermes:
|
||||
role: worker
|
||||
description: "PDF/PowerPoint/HTML-Kompilation, gelegentlich High-End auf Wunsch"
|
||||
hard_stop_eur_per_day: 1.0
|
||||
capability_tags: [document_generation, html_css_js]
|
||||
escalation_ladder:
|
||||
- model: "google/gemini-flash"
|
||||
cost_factor: 1
|
||||
- model: "anthropic/claude-sonnet"
|
||||
cost_factor: 4
|
||||
manual_override_allowed: true # Opus/GPT-5.5/Fable nur per bewusstem manuellen Opt-in
|
||||
|
||||
agent_xyz:
|
||||
role: worker
|
||||
description: "Saubere JavaScript-Datenbank-Abfragen"
|
||||
hard_stop_eur_per_day: 1.0
|
||||
capability_tags: [javascript, database_query]
|
||||
escalation_ladder:
|
||||
- model: "qwen/qwen2.5-7b"
|
||||
cost_factor: 1
|
||||
- model: "deepseek/deepseek-flash"
|
||||
cost_factor: 2
|
||||
manual_override_allowed: false
|
||||
|
||||
local_fallback:
|
||||
description: "Kostenfreier lokaler Fallback auf dem VPS via Ollama"
|
||||
model: "deepseek-r1:7b"
|
||||
min_context_tokens: 64000
|
||||
allowed_for_task_tags: [document_generation, javascript, database_query, linux_system]
|
||||
# Bewusst NICHT erlaubt für: homebrew_code (zu fehleranfällig bei 7B-Klasse)
|
||||
@@ -0,0 +1,60 @@
|
||||
# Modell-Auswahl Konfiguration
|
||||
|
||||
## Global
|
||||
|
||||
```yaml
|
||||
min_context_tokens: 64000
|
||||
max_auto_escalation_factor: 5 # ab 5x Kosten -> "Ferrari-Sprung", braucht manual_override
|
||||
min_distinct_model_families: 3 # Mindest-Diversität
|
||||
max_retries_per_task: 3
|
||||
default_hard_stop_eur_per_day: 1.0 # 1€/Tag/Agent hartes Limit
|
||||
on_budget_exceeded: postpone_to_next_day
|
||||
basic_questions_fallback: vps_ollama
|
||||
```
|
||||
|
||||
## Agents & Eskalationsleitern
|
||||
|
||||
### agent_a (Master — Linux, Allround)
|
||||
- Standard: `nvidia/nemotron-3-super` (1x)
|
||||
- Eskalation: `nvidia/nemotron-3-ultra` (3x)
|
||||
- Tags: linux_system, allround, orchestration
|
||||
- Budget: 1€/Tag
|
||||
|
||||
### agent_b (Worker — Code, Logik, DB)
|
||||
- Standard: `deepseek/deepseek-flash` (1x)
|
||||
- Eskalation: `deepseek/deepseek-pro` (3x)
|
||||
- Tags: code, logic, database
|
||||
|
||||
### switch_hermes (Cockpit — Prompt/Code)
|
||||
- Standard: `openrouter/free` (0x)
|
||||
- Eskalation: `qwen/qwen3-coder:free` (0x)
|
||||
- Tags: homebrew_code, prompt_generation
|
||||
|
||||
### laptop_hermes (Worker — Dokumente/HTML)
|
||||
- Standard: `google/gemini-flash` (1x)
|
||||
- Eskalation: `anthropic/claude-sonnet` (4x)
|
||||
- manual_override: ✅ erlaubt
|
||||
- Tags: document_generation, html_css_js
|
||||
|
||||
### agent_xyz (Worker — JS/DB-Queries)
|
||||
- Standard: `qwen/qwen2.5-7b` (1x)
|
||||
- Eskalation: `deepseek/deepseek-flash` (2x)
|
||||
- Tags: javascript, database_query
|
||||
|
||||
## Lokaler Fallback (VPS-Ollama)
|
||||
|
||||
```yaml
|
||||
local_fallback:
|
||||
model: "deepseek-r1:7b" # 128k Kontext, Thinking
|
||||
min_context_tokens: 64000
|
||||
allowed_for_task_tags: [document_generation, javascript, database_query]
|
||||
# NICHT erlaubt für: homebrew_code (zu fehleranfällig bei 7B)
|
||||
```
|
||||
|
||||
## Prinzipien
|
||||
|
||||
- Kosten ~99% Gewicht, Zeit egal
|
||||
- Proportionaler Aufstieg erlaubt, kein Ferrari-Sprung (>5x)
|
||||
- Budget erschöpft → postpone + lokaler Ollama-Fallback
|
||||
- Diversität: min 3 verschiedene Modell-Familien
|
||||
- Max 3 Retries pro Task, dann pausieren
|
||||
@@ -0,0 +1,23 @@
|
||||
# Lokale Modelle — Vergleichstabelle für VPS (16GB RAM)
|
||||
|
||||
## Thinking-Modelle (Chain-of-Thought/Reasoning)
|
||||
|
||||
| Modell | Größe | Kontext | RAM-Bedarf | Läuft auf VPS? | Empfehlung |
|
||||
|--------|-------|---------|------------|----------------|------------|
|
||||
| **deepseek-r1:7b** | 4.7 GB | 128k | ~6-7 GB | ✅ | **Sweet Spot** — aktiv |
|
||||
| **deepseek-r1:14b** | 9.0 GB | 128k | ~11-12 GB | ⚠️ knapp | geladen, ungetestet |
|
||||
| **deepseek-r1:32b** | ~20 GB | 128k | ~24 GB | ❌ | zu groß |
|
||||
| **qwq:32b** | ~20 GB | 128k | ~24 GB | ❌ | zu groß |
|
||||
|
||||
## Ohne Thinking
|
||||
|
||||
| Modell | Größe | Kontext | RAM-Bedarf | Läuft? |
|
||||
|--------|-------|---------|------------|--------|
|
||||
| **phi-4:14b** | ~9 GB | 16k | ~11 GB | ⚠️ (nur 16k Kontext) |
|
||||
| **qwen2.5:7b** | ~4.7 GB | 32k | ~6 GB | ✅ (kein Thinking) |
|
||||
| **llama3.1:8b** | ~4.9 GB | 8k | ~6 GB | ✅ (nur 8k, kein Thinking) |
|
||||
|
||||
## Faustregel
|
||||
- **16GB RAM VPS:** max 7B-14B Modelle (14B belegt ~75% RAM)
|
||||
- **RTX 2060 6GB:** max 7B (Q4_K_M ~4.5GB VRAM)
|
||||
- **ARM64 (Switch Lite):** max 3B-8B (CPU-Inference)
|
||||
Binary file not shown.
BIN
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
@@ -0,0 +1,162 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
daily_model_check.py
|
||||
|
||||
Ebene 1 des Skills: läuft 1x täglich (Cron), baut die grobe
|
||||
Modell-Prioritätsliste pro Agent und schreibt sie nach
|
||||
state/model_priority.json.
|
||||
|
||||
Nutzt aktuell Mock-Daten (mock_data.py). Für den Wechsel auf die
|
||||
echte OpenRouter-API muss nur fetch_model_list() in mock_data.py
|
||||
ersetzt werden - dieses Script bleibt unverändert.
|
||||
|
||||
Aufruf:
|
||||
python3 daily_model_check.py
|
||||
"""
|
||||
|
||||
import json
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
import yaml
|
||||
|
||||
sys.path.insert(0, str(Path(__file__).parent))
|
||||
from mock_data import fetch_model_list, fetch_timestamp # noqa: E402
|
||||
|
||||
CONFIG_PATH = Path(__file__).parent.parent / "config" / "agents.yaml"
|
||||
STATE_DIR = Path(__file__).parent.parent / "state"
|
||||
OUTPUT_PATH = STATE_DIR / "model_priority.json"
|
||||
|
||||
|
||||
def load_config():
|
||||
with open(CONFIG_PATH, "r", encoding="utf-8") as f:
|
||||
return yaml.safe_load(f)
|
||||
|
||||
|
||||
def index_models_by_id(models):
|
||||
return {m["id"]: m for m in models}
|
||||
|
||||
|
||||
def build_agent_priority(agent_name, agent_cfg, models_by_id, min_context, max_escalation_factor):
|
||||
"""Baut die Eskalationsleiter für einen Agent, gefiltert nach
|
||||
Mindest-Kontext und verfügbaren/bekannten Modellen.
|
||||
"""
|
||||
ladder = []
|
||||
warnings = []
|
||||
|
||||
for step_idx, step in enumerate(agent_cfg["escalation_ladder"]):
|
||||
model_id = step["model"]
|
||||
cost_factor = step["cost_factor"]
|
||||
|
||||
# Free-Tier-Router-Strings wie "openrouter/free" stehen nicht in
|
||||
# der Modell-Liste selbst - die werden unverändert übernommen.
|
||||
if model_id not in models_by_id:
|
||||
if model_id.startswith("openrouter/"):
|
||||
ladder.append({"model": model_id, "cost_factor": cost_factor, "note": "auto-router, nicht einzeln geprüft"})
|
||||
continue
|
||||
warnings.append(f"{agent_name}: Modell '{model_id}' nicht in aktueller Modell-Liste gefunden - übersprungen")
|
||||
continue
|
||||
|
||||
model = models_by_id[model_id]
|
||||
|
||||
if not model.get("available", True):
|
||||
warnings.append(f"{agent_name}: Modell '{model_id}' aktuell nicht verfügbar - übersprungen")
|
||||
continue
|
||||
|
||||
if model["context_length"] < min_context:
|
||||
warnings.append(
|
||||
f"{agent_name}: Modell '{model_id}' hat nur {model['context_length']} Kontext, "
|
||||
f"unter Minimum von {min_context} - übersprungen"
|
||||
)
|
||||
continue
|
||||
|
||||
# Verhältnismäßigkeits-Check: Stufen jenseits von Stufe 0 dürfen
|
||||
# den max_auto_escalation_factor nicht überschreiten, sonst braucht's
|
||||
# manual_override.
|
||||
if step_idx > 0 and cost_factor > max_escalation_factor and not agent_cfg.get("manual_override_allowed", False):
|
||||
warnings.append(
|
||||
f"{agent_name}: Eskalationsstufe '{model_id}' hat Faktor {cost_factor}x, "
|
||||
f"über dem erlaubten Automatik-Limit von {max_escalation_factor}x - "
|
||||
f"nur per manual_override nutzbar, aus Automatik-Leiter entfernt"
|
||||
)
|
||||
continue
|
||||
|
||||
ladder.append({
|
||||
"model": model_id,
|
||||
"cost_factor": cost_factor,
|
||||
"context_length": model["context_length"],
|
||||
"price_per_1k_tokens_eur": model["price_per_1k_tokens_eur"],
|
||||
"family": model["family"],
|
||||
})
|
||||
|
||||
return ladder, warnings
|
||||
|
||||
|
||||
def check_diversity(priority_by_agent, min_distinct_families):
|
||||
families_in_use = set()
|
||||
for agent_name, ladder in priority_by_agent.items():
|
||||
for step in ladder:
|
||||
if "family" in step:
|
||||
families_in_use.add(step["family"])
|
||||
|
||||
diverse_enough = len(families_in_use) >= min_distinct_families
|
||||
return diverse_enough, sorted(families_in_use)
|
||||
|
||||
|
||||
def main():
|
||||
config = load_config()
|
||||
global_cfg = config["global"]
|
||||
min_context = global_cfg["min_context_tokens"]
|
||||
max_escalation_factor = global_cfg["max_auto_escalation_factor"]
|
||||
min_distinct_families = global_cfg["min_distinct_model_families"]
|
||||
|
||||
models = fetch_model_list()
|
||||
models_by_id = index_models_by_id(models)
|
||||
|
||||
priority_by_agent = {}
|
||||
all_warnings = []
|
||||
|
||||
for agent_name, agent_cfg in config["agents"].items():
|
||||
ladder, warnings = build_agent_priority(
|
||||
agent_name, agent_cfg, models_by_id, min_context, max_escalation_factor
|
||||
)
|
||||
priority_by_agent[agent_name] = ladder
|
||||
all_warnings.extend(warnings)
|
||||
|
||||
if not ladder:
|
||||
all_warnings.append(
|
||||
f"WARNUNG: Agent '{agent_name}' hat NACH Filterung keine nutzbaren Modelle mehr - "
|
||||
f"lokaler Fallback sollte hier greifen"
|
||||
)
|
||||
|
||||
diverse_enough, families_in_use = check_diversity(priority_by_agent, min_distinct_families)
|
||||
if not diverse_enough:
|
||||
all_warnings.append(
|
||||
f"WARNUNG: Nur {len(families_in_use)} Modell-Familien im Einsatz "
|
||||
f"({', '.join(families_in_use)}), Minimum wäre {min_distinct_families}"
|
||||
)
|
||||
|
||||
output = {
|
||||
"generated_at": fetch_timestamp(),
|
||||
"min_context_tokens": min_context,
|
||||
"max_auto_escalation_factor": max_escalation_factor,
|
||||
"diversity_ok": diverse_enough,
|
||||
"model_families_in_use": families_in_use,
|
||||
"agents": priority_by_agent,
|
||||
"local_fallback": config["local_fallback"],
|
||||
"warnings": all_warnings,
|
||||
}
|
||||
|
||||
STATE_DIR.mkdir(exist_ok=True)
|
||||
with open(OUTPUT_PATH, "w", encoding="utf-8") as f:
|
||||
json.dump(output, f, indent=2, ensure_ascii=False)
|
||||
|
||||
print(f"Prioritätsliste geschrieben nach {OUTPUT_PATH}")
|
||||
if all_warnings:
|
||||
print(f"\n{len(all_warnings)} Warnung(en):")
|
||||
for w in all_warnings:
|
||||
print(f" - {w}")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -0,0 +1,114 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
hourly_finetune_check.py
|
||||
|
||||
Ebene 2 des Skills: läuft stündlich (oder öfter), prüft NUR Deltas
|
||||
gegen die von daily_model_check.py gebaute Prioritätsliste:
|
||||
|
||||
- Ist ein Agent über sein Tagesbudget (hard_stop_eur_per_day)?
|
||||
-> Kill-Switch: Agent wird auf lokalen Fallback gezwungen.
|
||||
- Wurde ein Modell inzwischen als nicht mehr verfügbar gemeldet?
|
||||
-> Wird aus der aktiven Leiter entfernt (bis zum nächsten Tages-Lauf).
|
||||
|
||||
Erwartet, dass der jeweilige Agent selbst fortlaufend seinen
|
||||
Tagesverbrauch nach state/spend_today.json schreibt, Format:
|
||||
{ "agent_a": 0.42, "agent_b": 1.10, ... }
|
||||
|
||||
Falls diese Datei fehlt (z.B. noch nicht angebunden), läuft der Check
|
||||
trotzdem durch und markiert das als Hinweis statt Fehler.
|
||||
|
||||
Aufruf:
|
||||
python3 hourly_finetune_check.py
|
||||
"""
|
||||
|
||||
import json
|
||||
from pathlib import Path
|
||||
|
||||
import yaml
|
||||
|
||||
CONFIG_PATH = Path(__file__).parent.parent / "config" / "agents.yaml"
|
||||
STATE_DIR = Path(__file__).parent.parent / "state"
|
||||
PRIORITY_PATH = STATE_DIR / "model_priority.json"
|
||||
SPEND_PATH = STATE_DIR / "spend_today.json"
|
||||
KILL_SWITCH_PATH = STATE_DIR / "kill_switches.json"
|
||||
|
||||
|
||||
def load_config():
|
||||
with open(CONFIG_PATH, "r", encoding="utf-8") as f:
|
||||
return yaml.safe_load(f)
|
||||
|
||||
|
||||
def load_priority():
|
||||
if not PRIORITY_PATH.exists():
|
||||
raise FileNotFoundError(
|
||||
f"{PRIORITY_PATH} fehlt - bitte zuerst daily_model_check.py laufen lassen"
|
||||
)
|
||||
with open(PRIORITY_PATH, "r", encoding="utf-8") as f:
|
||||
return json.load(f)
|
||||
|
||||
|
||||
def load_spend():
|
||||
if not SPEND_PATH.exists():
|
||||
return {}, ["spend_today.json fehlt noch - Budget-Check übersprungen, bitte Anbindung nachholen"]
|
||||
with open(SPEND_PATH, "r", encoding="utf-8") as f:
|
||||
return json.load(f), []
|
||||
|
||||
|
||||
def check_budgets(config, spend):
|
||||
kill_switches = {}
|
||||
notes = []
|
||||
|
||||
for agent_name, agent_cfg in config["agents"].items():
|
||||
limit = agent_cfg.get("hard_stop_eur_per_day")
|
||||
spent = spend.get(agent_name)
|
||||
|
||||
if spent is None:
|
||||
continue
|
||||
|
||||
if limit is None or limit == "BUDGET_PLACEHOLDER":
|
||||
notes.append(
|
||||
f"{agent_name}: hard_stop_eur_per_day ist noch Platzhalter - "
|
||||
f"kein Kill-Switch aktiv, bitte echten Wert eintragen"
|
||||
)
|
||||
continue
|
||||
|
||||
if spent >= float(limit):
|
||||
kill_switches[agent_name] = {
|
||||
"reason": "daily_budget_exceeded",
|
||||
"spent_eur": spent,
|
||||
"limit_eur": limit,
|
||||
"action": "postpone_to_next_day",
|
||||
"basic_questions_fallback": "vps_ollama",
|
||||
}
|
||||
notes.append(
|
||||
f"KILL-SWITCH: {agent_name} hat {spent}€ von {limit}€ Tageslimit erreicht - "
|
||||
f"restliche Tasks werden auf morgen verschoben. Grundfragen laufen bis dahin "
|
||||
f"weiter über das lokale VPS-Ollama-Modell, sofern das inhaltlich passt."
|
||||
)
|
||||
|
||||
return kill_switches, notes
|
||||
|
||||
|
||||
def main():
|
||||
config = load_config()
|
||||
priority = load_priority()
|
||||
spend, spend_notes = load_spend()
|
||||
|
||||
kill_switches, budget_notes = check_budgets(config, spend)
|
||||
|
||||
result = {
|
||||
"checked_at": priority.get("generated_at"),
|
||||
"kill_switches": kill_switches,
|
||||
"notes": spend_notes + budget_notes,
|
||||
}
|
||||
|
||||
with open(KILL_SWITCH_PATH, "w", encoding="utf-8") as f:
|
||||
json.dump(result, f, indent=2, ensure_ascii=False)
|
||||
|
||||
print(f"Feintuning-Check abgeschlossen, Ergebnis in {KILL_SWITCH_PATH}")
|
||||
for note in result["notes"]:
|
||||
print(f" - {note}")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -0,0 +1,104 @@
|
||||
"""
|
||||
Mock-Datenquelle für die OpenRouter-Modell-Liste.
|
||||
|
||||
Diese Funktion hat exakt die Signatur, die auch der echte API-Call später
|
||||
haben wird (siehe TODO), damit daily_model_check.py unverändert bleibt,
|
||||
wenn die echte API eingehängt wird.
|
||||
|
||||
TODO (später, wenn echte API gewünscht):
|
||||
import requests
|
||||
def fetch_model_list():
|
||||
resp = requests.get(
|
||||
"https://openrouter.ai/api/v1/models",
|
||||
headers={"Authorization": f"Bearer {OPENROUTER_API_KEY}"},
|
||||
timeout=10,
|
||||
)
|
||||
resp.raise_for_status()
|
||||
return resp.json()["data"]
|
||||
"""
|
||||
|
||||
from datetime import datetime, timezone
|
||||
|
||||
|
||||
def fetch_model_list():
|
||||
"""Gibt eine Liste von Modell-Dicts zurück, im gleichen Format wie
|
||||
die echte OpenRouter-API (vereinfacht auf die Felder, die wir brauchen).
|
||||
"""
|
||||
return [
|
||||
{
|
||||
"id": "nvidia/nemotron-3-super",
|
||||
"context_length": 262000,
|
||||
"price_per_1k_tokens_eur": 0.0009,
|
||||
"family": "nemotron",
|
||||
"tags": ["linux_system", "allround"],
|
||||
"available": True,
|
||||
},
|
||||
{
|
||||
"id": "nvidia/nemotron-3-ultra",
|
||||
"context_length": 1000000,
|
||||
"price_per_1k_tokens_eur": 0.0027,
|
||||
"family": "nemotron",
|
||||
"tags": ["linux_system", "allround", "orchestration"],
|
||||
"available": True,
|
||||
},
|
||||
{
|
||||
"id": "deepseek/deepseek-flash",
|
||||
"context_length": 128000,
|
||||
"price_per_1k_tokens_eur": 0.0006,
|
||||
"family": "deepseek",
|
||||
"tags": ["code", "logic", "database"],
|
||||
"available": True,
|
||||
},
|
||||
{
|
||||
"id": "deepseek/deepseek-pro",
|
||||
"context_length": 128000,
|
||||
"price_per_1k_tokens_eur": 0.0018,
|
||||
"family": "deepseek",
|
||||
"tags": ["code", "logic", "database"],
|
||||
"available": True,
|
||||
},
|
||||
{
|
||||
"id": "qwen/qwen3-coder:free",
|
||||
"context_length": 1000000,
|
||||
"price_per_1k_tokens_eur": 0.0,
|
||||
"family": "qwen",
|
||||
"tags": ["homebrew_code", "prompt_generation", "code"],
|
||||
"available": True,
|
||||
},
|
||||
{
|
||||
"id": "qwen/qwen2.5-7b",
|
||||
"context_length": 128000,
|
||||
"price_per_1k_tokens_eur": 0.0003,
|
||||
"family": "qwen",
|
||||
"tags": ["javascript", "database_query"],
|
||||
"available": True,
|
||||
},
|
||||
{
|
||||
"id": "google/gemini-flash",
|
||||
"context_length": 1000000,
|
||||
"price_per_1k_tokens_eur": 0.0004,
|
||||
"family": "gemini",
|
||||
"tags": ["document_generation", "html_css_js"],
|
||||
"available": True,
|
||||
},
|
||||
{
|
||||
"id": "anthropic/claude-sonnet",
|
||||
"context_length": 200000,
|
||||
"price_per_1k_tokens_eur": 0.0045,
|
||||
"family": "anthropic",
|
||||
"tags": ["document_generation", "html_css_js", "allround"],
|
||||
"available": True,
|
||||
},
|
||||
{
|
||||
"id": "anthropic/claude-opus",
|
||||
"context_length": 200000,
|
||||
"price_per_1k_tokens_eur": 0.0900,
|
||||
"family": "anthropic",
|
||||
"tags": ["document_generation", "html_css_js", "allround"],
|
||||
"available": True,
|
||||
},
|
||||
]
|
||||
|
||||
|
||||
def fetch_timestamp():
|
||||
return datetime.now(timezone.utc).isoformat()
|
||||
@@ -0,0 +1,132 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
task_agent_matcher.py
|
||||
|
||||
Ebene 3 des Skills: läuft bei Agent A (Master) zur Laufzeit, sobald ein
|
||||
neuer Task ansteht. Entscheidet, welcher Agent bei welchem Standard-Modell
|
||||
am besten zum Task-Typ passt - basierend auf:
|
||||
|
||||
- der von daily_model_check.py gebauten Prioritätsliste
|
||||
(state/model_priority.json)
|
||||
- aktiven Kill-Switches (state/kill_switches.json), z.B. wegen
|
||||
Budget-Überschreitung -> Agent wird übersprungen bzw. auf
|
||||
"postpone_to_next_day" gesetzt
|
||||
- den Capability-Tags pro Agent (config/agents.yaml)
|
||||
|
||||
Jeder Agent bekommt exakt dieses Konzept (Config + Prioritätsliste)
|
||||
mitgeliefert, damit alle konsistent nach derselben Logik laufen, egal
|
||||
wer die Entscheidung am Ende ausführt.
|
||||
|
||||
Aufruf (Beispiel):
|
||||
python3 task_agent_matcher.py --task-tag javascript
|
||||
python3 task_agent_matcher.py --task-tag homebrew_code
|
||||
"""
|
||||
|
||||
import argparse
|
||||
import json
|
||||
from pathlib import Path
|
||||
|
||||
import yaml
|
||||
|
||||
CONFIG_PATH = Path(__file__).parent.parent / "config" / "agents.yaml"
|
||||
STATE_DIR = Path(__file__).parent.parent / "state"
|
||||
PRIORITY_PATH = STATE_DIR / "model_priority.json"
|
||||
KILL_SWITCH_PATH = STATE_DIR / "kill_switches.json"
|
||||
|
||||
|
||||
def load_yaml(path):
|
||||
with open(path, "r", encoding="utf-8") as f:
|
||||
return yaml.safe_load(f)
|
||||
|
||||
|
||||
def load_json(path, default=None):
|
||||
if not path.exists():
|
||||
return default if default is not None else {}
|
||||
with open(path, "r", encoding="utf-8") as f:
|
||||
return json.load(f)
|
||||
|
||||
|
||||
def find_matches(task_tag, config, priority, kill_switches):
|
||||
"""Gibt eine nach Preis sortierte Liste von (agent, model, price) für
|
||||
Agents zurück, deren Capability-Tags zum Task-Typ passen und die
|
||||
NICHT gerade per Kill-Switch auf 'postpone_to_next_day' stehen.
|
||||
"""
|
||||
candidates = []
|
||||
postponed = []
|
||||
|
||||
for agent_name, agent_cfg in config["agents"].items():
|
||||
if task_tag not in agent_cfg.get("capability_tags", []):
|
||||
continue
|
||||
|
||||
if agent_name in kill_switches.get("kill_switches", {}):
|
||||
ks = kill_switches["kill_switches"][agent_name]
|
||||
postponed.append({
|
||||
"agent": agent_name,
|
||||
"reason": ks["reason"],
|
||||
"action": ks["action"],
|
||||
"fallback": ks.get("basic_questions_fallback"),
|
||||
})
|
||||
continue
|
||||
|
||||
ladder = priority.get("agents", {}).get(agent_name, [])
|
||||
for step in ladder:
|
||||
candidates.append({
|
||||
"agent": agent_name,
|
||||
"model": step["model"],
|
||||
"cost_factor": step.get("cost_factor"),
|
||||
"price_per_1k_tokens_eur": step.get("price_per_1k_tokens_eur"),
|
||||
})
|
||||
|
||||
# Günstigste zuerst; Steps ohne Preisangabe (z.B. Free-Router) ganz vorne
|
||||
candidates.sort(key=lambda c: (c["price_per_1k_tokens_eur"] is not None, c.get("price_per_1k_tokens_eur") or 0))
|
||||
|
||||
return candidates, postponed
|
||||
|
||||
|
||||
def main():
|
||||
parser = argparse.ArgumentParser(description="Task-zu-Agent-Matching (Ebene 3)")
|
||||
parser.add_argument("--task-tag", required=True, help="Capability-Tag des Tasks, z.B. javascript, homebrew_code")
|
||||
args = parser.parse_args()
|
||||
|
||||
config = load_yaml(CONFIG_PATH)
|
||||
priority = load_json(PRIORITY_PATH)
|
||||
kill_switches = load_json(KILL_SWITCH_PATH)
|
||||
|
||||
if not priority:
|
||||
print("FEHLER: state/model_priority.json fehlt - bitte zuerst daily_model_check.py laufen lassen")
|
||||
return
|
||||
|
||||
candidates, postponed = find_matches(args.task_tag, config, priority, kill_switches)
|
||||
|
||||
if not candidates:
|
||||
local_fb = config.get("local_fallback", {})
|
||||
if args.task_tag in local_fb.get("allowed_for_task_tags", []):
|
||||
print(f"Kein Cloud-Kandidat für '{args.task_tag}' verfügbar (evtl. alle betroffenen Agents "
|
||||
f"per Kill-Switch verschoben) - lokaler Fallback erlaubt:")
|
||||
print(f" -> {local_fb['model']} (kostenfrei, VPS-Ollama)")
|
||||
else:
|
||||
print(f"Kein Kandidat für '{args.task_tag}' verfügbar und lokaler Fallback für diesen "
|
||||
f"Task-Typ NICHT erlaubt (siehe config: local_fallback.allowed_for_task_tags). "
|
||||
f"Task sollte pausiert werden.")
|
||||
for p in postponed:
|
||||
print(f" (verschoben: {p['agent']} - {p['reason']}, Fallback: {p['fallback']})")
|
||||
return
|
||||
|
||||
best = candidates[0]
|
||||
print(f"Beste Wahl für Task-Tag '{args.task_tag}':")
|
||||
print(f" Agent: {best['agent']}")
|
||||
print(f" Modell: {best['model']} (Kostenfaktor {best['cost_factor']}x)")
|
||||
|
||||
if len(candidates) > 1:
|
||||
print("\nWeitere passende Kandidaten (Eskalationsreserve):")
|
||||
for c in candidates[1:]:
|
||||
print(f" - {c['agent']} / {c['model']} (Faktor {c['cost_factor']}x)")
|
||||
|
||||
if postponed:
|
||||
print("\nHinweis, folgende Agents sind aktuell wegen Budget verschoben:")
|
||||
for p in postponed:
|
||||
print(f" - {p['agent']}: {p['reason']} -> {p['action']} (Grundfragen: {p['fallback']})")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -0,0 +1,15 @@
|
||||
{
|
||||
"checked_at": "2026-08-02T01:00:17.882979+00:00",
|
||||
"kill_switches": {
|
||||
"agent_a": {
|
||||
"reason": "daily_budget_exceeded",
|
||||
"spent_eur": 1.5,
|
||||
"limit_eur": 1.0,
|
||||
"action": "postpone_to_next_day",
|
||||
"basic_questions_fallback": "vps_ollama"
|
||||
}
|
||||
},
|
||||
"notes": [
|
||||
"KILL-SWITCH: agent_a hat 1.5€ von 1.0€ Tageslimit erreicht - restliche Tasks werden auf morgen verschoben. Grundfragen laufen bis dahin weiter über das lokale VPS-Ollama-Modell, sofern das inhaltlich passt."
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,105 @@
|
||||
{
|
||||
"generated_at": "2026-08-02T01:00:17.882979+00:00",
|
||||
"min_context_tokens": 64000,
|
||||
"max_auto_escalation_factor": 5,
|
||||
"diversity_ok": true,
|
||||
"model_families_in_use": [
|
||||
"anthropic",
|
||||
"deepseek",
|
||||
"gemini",
|
||||
"nemotron",
|
||||
"qwen"
|
||||
],
|
||||
"agents": {
|
||||
"agent_a": [
|
||||
{
|
||||
"model": "nvidia/nemotron-3-super",
|
||||
"cost_factor": 1,
|
||||
"context_length": 262000,
|
||||
"price_per_1k_tokens_eur": 0.0009,
|
||||
"family": "nemotron"
|
||||
},
|
||||
{
|
||||
"model": "nvidia/nemotron-3-ultra",
|
||||
"cost_factor": 3,
|
||||
"context_length": 1000000,
|
||||
"price_per_1k_tokens_eur": 0.0027,
|
||||
"family": "nemotron"
|
||||
}
|
||||
],
|
||||
"agent_b": [
|
||||
{
|
||||
"model": "deepseek/deepseek-flash",
|
||||
"cost_factor": 1,
|
||||
"context_length": 128000,
|
||||
"price_per_1k_tokens_eur": 0.0006,
|
||||
"family": "deepseek"
|
||||
},
|
||||
{
|
||||
"model": "deepseek/deepseek-pro",
|
||||
"cost_factor": 3,
|
||||
"context_length": 128000,
|
||||
"price_per_1k_tokens_eur": 0.0018,
|
||||
"family": "deepseek"
|
||||
}
|
||||
],
|
||||
"switch_hermes": [
|
||||
{
|
||||
"model": "openrouter/free",
|
||||
"cost_factor": 0,
|
||||
"note": "auto-router, nicht einzeln geprüft"
|
||||
},
|
||||
{
|
||||
"model": "qwen/qwen3-coder:free",
|
||||
"cost_factor": 0,
|
||||
"context_length": 1000000,
|
||||
"price_per_1k_tokens_eur": 0.0,
|
||||
"family": "qwen"
|
||||
}
|
||||
],
|
||||
"laptop_hermes": [
|
||||
{
|
||||
"model": "google/gemini-flash",
|
||||
"cost_factor": 1,
|
||||
"context_length": 1000000,
|
||||
"price_per_1k_tokens_eur": 0.0004,
|
||||
"family": "gemini"
|
||||
},
|
||||
{
|
||||
"model": "anthropic/claude-sonnet",
|
||||
"cost_factor": 4,
|
||||
"context_length": 200000,
|
||||
"price_per_1k_tokens_eur": 0.0045,
|
||||
"family": "anthropic"
|
||||
}
|
||||
],
|
||||
"agent_xyz": [
|
||||
{
|
||||
"model": "qwen/qwen2.5-7b",
|
||||
"cost_factor": 1,
|
||||
"context_length": 128000,
|
||||
"price_per_1k_tokens_eur": 0.0003,
|
||||
"family": "qwen"
|
||||
},
|
||||
{
|
||||
"model": "deepseek/deepseek-flash",
|
||||
"cost_factor": 2,
|
||||
"context_length": 128000,
|
||||
"price_per_1k_tokens_eur": 0.0006,
|
||||
"family": "deepseek"
|
||||
}
|
||||
]
|
||||
},
|
||||
"local_fallback": {
|
||||
"description": "Kostenfreier lokaler Fallback auf dem VPS via Ollama",
|
||||
"model": "deepseek-r1:7b",
|
||||
"min_context_tokens": 64000,
|
||||
"allowed_for_task_tags": [
|
||||
"document_generation",
|
||||
"javascript",
|
||||
"database_query",
|
||||
"linux_system"
|
||||
]
|
||||
},
|
||||
"warnings": []
|
||||
}
|
||||
@@ -0,0 +1 @@
|
||||
{"agent_a": 1.5, "agent_b": 0.0, "switch_hermes": 0.0, "laptop_hermes": 0.0, "agent_xyz": 0.0}
|
||||
@@ -0,0 +1,7 @@
|
||||
{
|
||||
"agent_a": 0.0,
|
||||
"agent_b": 0.0,
|
||||
"switch_hermes": 0.0,
|
||||
"laptop_hermes": 0.0,
|
||||
"agent_xyz": 0.0
|
||||
}
|
||||
Reference in New Issue
Block a user