Update 2+3
All checks were successful
release-tag / release-image (push) Successful in 1m36s

This commit is contained in:
2026-08-04 12:07:45 +02:00
parent 5cb00c45d5
commit e94fb23f0c
27 changed files with 1816 additions and 201 deletions

View File

@@ -5,7 +5,8 @@ Lokale Knowledge-JSONs ──ro──┐
GLPI Knowledge Base ─────ro──┼──► Ingest ─► In-Memory Graph + Vectors
Agent runs.jsonl ─────────ro──┘ │
├──► Ollama Pool
├──► AI-THINK / Research
├──► Relation Thinking
├──► Knowledge Synthesis / Research
├──► SSE / Hierarchical LOD UI
└──► Batched Persistence
├── staging drafts
@@ -20,6 +21,22 @@ Agent runs.jsonl ─────────ro──┘
- `related_to`, `depends_on`, `supports`, `contradicts`, `extends`, `same_topic`, `caused_by`: Qwen-Inferenz mit Confidence und Evidence.
- `rejected`: intern gespeicherte Prüfung ohne sichtbare Beziehung.
## Zweistufiges Thinking
```text
Kandidatenpaar
└── Relation Thinking
├── rejected edge
└── staging edge
└── Quellenverbund (38 Quellen)
└── Artikelplanung: create / update / merge / skip
└── Knowledge Synthesis
└── strukturierter AI-Synthesis-Entwurf
```
Relation Thinking schreibt keine Artikel. Knowledge Synthesis startet nur, wenn genügend produktive Quellen, ein ausreichender Produktionsanteil und eine zulässige Generierungstiefe vorliegen. Die Artikelerstellung verwendet vollständige lokale beziehungsweise im GLPI-Cache gehaltene Quelltexte. Der Entwurf wird sofort als In-Memory-Node sichtbar und erst über die gebündelte Persistenz auf die Festplatte geschrieben.
## Schreibmodell
Produktive Knowledge-Dateien und GLPI sind read-only. AI-THINK wird nur in das konfigurierte Staging geschrieben. Alle Schreibzugriffe laufen durch einen Prozess-weiten Coordinator, der Dateien dedupliziert und seriell vor dem Graph-Snapshot schreibt.

View File

@@ -0,0 +1,8 @@
# GPU Node Limit
- Laufzeitwert `max_display_nodes` mit gebündelter Persistenz ergänzt.
- Websteuerung unter `FILTER → GPU-Limit` mit Zahlenfeld und Presets.
- Stabile, themenbewusste Auswahl bei Überschreitung des Limits.
- Aktive, zunächst ausgeblendete Notes werden temporär eingeblendet.
- Limit gilt für Neural- und Honeycomb-Ansicht; `0` bedeutet unbegrenzt.
- Environment-Startwert: `BRAIN_MAX_DISPLAY_NODES`.

View File

@@ -0,0 +1,12 @@
# Knowledge Synthesis Upgrade
- Relationserkennung und Artikelerstellung sind getrennte Modellschritte.
- Eine positive Zweier-Relation erzeugt nur noch eine Graph-Edge, keinen technischen Meta-Artikel.
- Artikelplanung entscheidet `create`, `update`, `merge` oder `skip`.
- Artikelentwürfe verwenden mindestens drei produktive Quellen und standardmäßig höchstens 30 Prozent AI-THINK-Quellen.
- Volle lokale und GLPI-KB-Quelltexte werden für die Synthese verwendet.
- `text` enthält Problembeschreibung/Geltungsbereich; `answer` enthält die echte Lösung samt Prüfung und Fehlerbehandlung.
- Generationstiefe, Produktionsanteil, Zielartikel und Quellen werden im `ai_think`-Block protokolliert.
- Gleichwertige Staging-Entwürfe werden dedupliziert.
- Ein Qualitäts-Gate prüft Mindestlänge, Quellenanteil, Generierungstiefe und Konfidenz.
- Der neue Artikel-Node wird sofort in den In-Memory-Graph aufgenommen; die Datei bleibt gebündelt persistiert.

96
KNOWLEDGE-SYNTHESIS.md Normal file
View File

@@ -0,0 +1,96 @@
# Relation Thinking und Knowledge Synthesis
Das Brain trennt die autonome Arbeit jetzt in zwei unabhängige fachliche Schritte. Beide werden durch den vorhandenen **THINKING**-Schalter und den Thinking-Kategoriefilter gesteuert.
## 1. Relation Thinking
Relation Thinking beantwortet ausschließlich Graphfragen:
- Sind zwei Wissenseinträge belastbar verwandt?
- Welcher Relationstyp gilt (`supports`, `depends_on`, `contradicts`, `same_topic` usw.)?
- Wie hoch ist die Konfidenz?
- Fehlt Evidenz für die Relationsentscheidung?
Ein positiver Entscheid erzeugt nur eine nachvollziehbare Edge mit Evidenz, Konfidenz und Herkunft. Aus einem bloßen Zweiervergleich wird **kein** Artikel mehr geschrieben.
## 2. Knowledge Synthesis
Nach einer bestätigten Relation wird geprüft, ob um das Themenpaar ein ausreichend starker Quellenverbund existiert. Standardregeln:
- mindestens drei produktive Quellen;
- höchstens acht Quellen pro Entwurf;
- mindestens 70 Prozent Produktionswissen;
- keine reine AI-THINK-Quellenkette;
- maximale AI-Generierungstiefe zwei;
- vollständige Quelltexte werden bevorzugt, nicht nur die gekürzten Graph-Summaries;
- ein vorhandener gleichwertiger Staging-Entwurf verhindert Dubletten.
Der erste Qwen-Aufruf plant eine der Aktionen:
- `create`: neuen Wissensartikel anlegen;
- `update`: einen vorhandenen produktiven Artikel als Staging-Entwurf verbessern;
- `merge`: mehrere produktive Beiträge in einen Zielartikel konsolidieren;
- `skip`: kein belastbarer Mehrwert.
Erst danach erzeugt ein zweiter, separat strukturierter Modellaufruf den eigentlichen Helpdesk-Artikel.
## Artikelstruktur
Der Staging-Entwurf befüllt die Knowledgebase-Felder fachlich korrekt:
- `text`: Problem, Symptome, Geltungsbereich und Abgrenzung;
- `answer`: konkrete Lösungsschritte;
- zusätzliche Abschnitte: Voraussetzungen, Ergebnisprüfung und Fehlerbehandlung.
Der feste Prüfhinweis steht nicht mehr als vermeintliche Lösung im Feld `answer`. Er liegt ausschließlich in `ai_think.review_notice`.
Beispiel:
```json
{
"id": "KB-AI-THINK-ARTICLE-20260804-...",
"title": "VPN-Gateway-Störung systematisch beheben",
"text": "Dieser Artikel gilt für ...",
"answer": "1. Prüfen Sie ...\n\n## Ergebnis prüfen\n- Der VPN-Tunnel ...",
"auto_reply": false,
"categories": ["AI-THINK", "AI-Staging", "AI-Synthesis", "VPN"],
"ai_think": {
"subtype": "knowledge_synthesis",
"action": "create",
"source_node_ids": ["..."],
"productive_source_count": 4,
"ai_source_count": 1,
"production_ratio": 0.8,
"generation_depth": 1,
"confidence": 0.91,
"review_notice": "Vor produktiver Nutzung im Editor prüfen, korrigieren und freigeben."
}
}
```
## Volle Quelleninhalte
Lokale Knowledge-JSONs werden für die Artikelsynthese erneut read-only von ihrem bekannten Pfad gelesen. Für GLPI-KB-Beiträge hält der GLPI-Syncer den vollständigen bereinigten Artikeltext in seinem gebündelt gespeicherten Cache. Der Browser erhält weiterhin nur den normalen Graph-Snapshot; die Volltexte sind kein zusätzliches UI-Payload.
## Qualitätsgrenzen
```env
BRAIN_ARTICLE_SYNTHESIS_ENABLED=true
BRAIN_ARTICLE_MIN_SOURCES=3
BRAIN_ARTICLE_MAX_SOURCES=8
BRAIN_ARTICLE_MIN_PRODUCTION_RATIO=0.70
BRAIN_ARTICLE_MAX_GENERATION_DEPTH=2
BRAIN_ARTICLE_MIN_CONFIDENCE=0.74
BRAIN_ARTICLE_MIN_TEXT_CHARS=180
BRAIN_ARTICLE_MIN_ANSWER_CHARS=420
```
Ein Entwurf, der diese Regeln nicht erfüllt, wird nicht geschrieben. Die Relation bleibt trotzdem im Graphen bestehen.
## Recherche
Wenn der Plan notwendige Informationslücken erkennt und `BRAIN_RESEARCH_ENABLED=true` gesetzt ist, kann vor dem Artikelentwurf kontrolliert über die konfigurierte SearXNG-Instanz recherchiert werden. Scheitert eine als notwendig deklarierte Recherche, wird kein Artikel erfunden.
## Staging und Isolation
Neue Artikel werden weiterhin ausschließlich in den ersten Pfad aus `BRAIN_STAGING_DIRS` geschrieben, tragen `auto_reply: false` und bleiben für Agent sowie produktive Knowledgebase-Suche unsichtbar, bis sie im vorhandenen Editor freigegeben werden. Der In-Memory-Graph erhält den neuen AI-THINK-Node sofort, damit Visualisierung und weitere Analyse nicht auf den nächsten Festplatten-Flush warten müssen.

View File

@@ -13,7 +13,7 @@ Eigenständiger Go-Dienst für Agent, lokale Knowledgebase, GLPI-Knowledgebase u
- Read-only Tailing der Agent-`runs.jsonl` und optionale Suchtelemetrie aus Agent und KB.
- Ollama-Pool mit mehreren unabhängigen Instanzen, Routing, Healthchecks, Cooldown und Failover.
- Embeddings über `embeddinggemma`, Beziehungsanalyse über `qwen3:8b`.
- Sequenzieller autonomer AI-THINK-Worker mit optionaler SearXNG-Recherche.
- Zweistufiger autonomer Worker: Relation Thinking für Graph-Edges und Knowledge Synthesis für echte strukturierte KB-Entwürfe.
- Gebündelte Festplattenpersistenz: Graph, GLPI-KB-Cache und AI-THINK-Dateien werden standardmäßig nur alle fünf Minuten sequenziell geschrieben.
## Vertrauens- und Schreibgrenzen
@@ -25,7 +25,7 @@ Schreibbar sind nur:
- das eigene `BRAIN_DATA_DIR` für Graphzustand und GLPI-KB-Cache;
- der erste Pfad aus `BRAIN_STAGING_DIRS` für AI-THINK-Entwürfe.
AI-THINK bleibt `auto_reply: false`, trägt die Kategorien `AI-THINK` und `AI-Staging` und wird erst nach einer expliziten Freigabe im bestehenden Editor produktiv.
AI-THINK bleibt `auto_reply: false`, trägt die Kategorien `AI-THINK` und `AI-Staging` und wird erst nach einer expliziten Freigabe im bestehenden Editor produktiv. Artikelentwürfe erhalten zusätzlich `AI-Synthesis`; ihr Feld `answer` enthält die tatsächliche Lösung statt eines Prüfhinweises.
## Schnellstart
@@ -140,6 +140,7 @@ BRAIN_LEARNING_CATEGORIES=
BRAIN_DISPLAY_CATEGORIES=
BRAIN_THINKING_CATEGORIES=
BRAIN_DEFAULT_VIEW=neural
BRAIN_MAX_DISPLAY_NODES=0 # 0 = unbegrenzt; alternativ z. B. 5000
```
Honeycomb rendert nur `knowledge`, `ai-think` und `external`. Ein 3D-Gitter mit einheitlichem Punktabstand wird auf die Gehirngeometrie beschnitten und automatisch an die sichtbare Anzahl von Notes angepasst. Edges, LOD-Gruppen und Cortex-Flächen bleiben dort unsichtbar; bei einer Anfrage leuchten nur die referenzierten Notes.
@@ -148,22 +149,34 @@ Mehr Details: [`RUNTIME-CONTROLS-HONEYCOMB.md`](RUNTIME-CONTROLS-HONEYCOMB.md).
## Autonome Anreicherung
Der Worker arbeitet bewusst sequenziell:
Der Worker arbeitet bewusst sequenziell, trennt aber jetzt zwei Aufgaben:
1. Kandidatenpaar aus bestehenden Vektoren auswählen;
2. Beziehung durch Qwen mit festem JSON-Schema prüfen;
3. Edge als `staging` oder `rejected` im In-Memory-Graph ablegen;
4. optional kontrolliert recherchieren;
5. AI-THINK-Entwurf in die Schreibwarteschlange legen;
6. beim nächsten Persistenz-Flush atomar in Staging schreiben.
2. **Relation Thinking** bewertet ausschließlich die Beziehung und erzeugt eine Edge oder verwirft sie;
3. bei einer bestätigten Relation werden drei bis acht verwandte Quellen gesammelt;
4. ein eigener Planungsaufruf entscheidet `create`, `update`, `merge` oder `skip`;
5. nur bei echtem Mehrwert schreibt ein zweiter Modellaufruf einen strukturierten Helpdesk-Artikel;
6. der Artikel landet als `AI-THINK` / `AI-Staging` / `AI-Synthesis` mit realem `text`- und `answer`-Inhalt im Staging;
7. der In-Memory-Graph sieht den Entwurf sofort, die Datei wird beim nächsten gebündelten Persistenz-Flush atomar geschrieben.
Qualitätsregeln verhindern reine AI-THINK-Ketten: standardmäßig mindestens drei produktive Quellen, mindestens 70 Prozent Produktionswissen und maximal Generierungstiefe zwei.
```env
BRAIN_AUTO_ENRICH=true
BRAIN_ENRICH_INTERVAL=90s
BRAIN_ENRICH_BATCH_SIZE=3
BRAIN_ENRICH_STEP_DELAY=3s
BRAIN_ARTICLE_SYNTHESIS_ENABLED=true
BRAIN_ARTICLE_MIN_SOURCES=3
BRAIN_ARTICLE_MAX_SOURCES=8
BRAIN_ARTICLE_MIN_PRODUCTION_RATIO=0.70
BRAIN_ARTICLE_MAX_GENERATION_DEPTH=2
BRAIN_ARTICLE_MIN_CONFIDENCE=0.74
```
Details: [`KNOWLEDGE-SYNTHESIS.md`](KNOWLEDGE-SYNTHESIS.md).
Der manuelle AI-THINK-Button bleibt vorhanden. Alternativ:
```bash
@@ -201,7 +214,8 @@ Wichtige Bereiche:
- `ollama_pool.nodes[]`: Health, Kompatibilität, Inflight, Requests, Fehler und mittlere Laufzeit;
- `glpi_kb`: letzter Sync, Dokumentanzahl, Pfad und Fehler;
- `persistence`: ausstehende Dateien, Dirty-Status, letzter Flush und Flush-Fehler;
- `enrich_*`: Zustand des autonomen AI-THINK-Workers.
- `enrich_*`: Zustand des autonomen AI-THINK-Workers;
- `relations_created`, `articles_created`, `articles_skipped`: getrennte Relation- und Artikelergebnisse.
## Validierung

View File

@@ -25,6 +25,23 @@ Mit **HONEYCOMB** wird die semantische Graphansicht durch eine gleichmäßig bes
Die Honeycomb-Ansicht verändert weder den Graphen noch die gespeicherten Node-Positionen. Sie ist eine reine Renderprojektion und kann jederzeit zurück auf **NEURAL** geschaltet werden.
## GPU-Limit für schwächere Systeme
`max_display_nodes` begrenzt die Anzahl der Nodes, die im Browser für Neural und Honeycomb aufgebaut werden. `0` bedeutet unbegrenzt. Die Begrenzung wird nach dem Anzeige-Kategoriefilter angewendet. Das vollständige Wissen bleibt im Backend und im geladenen Graph-Snapshot erhalten.
Die Auswahl ist stabil und bevorzugt aktive Notes, AI-THINK/Staging-Inhalte, Hubs und eine Grundabdeckung der vorhandenen Themenbereiche. Wird eine aktuell nicht dargestellte Note von Agent, Knowledgebase oder AI-THINK angefordert, ersetzt sie temporär einen inaktiven Node. Nach der Aktivitätsphase greift das Limit wieder auf die stabile Grundauswahl zurück.
Im Web befindet sich die Einstellung unter **FILTER → GPU-Limit**. Voreinstellungen für 1.000, 5.000, 10.000 und 25.000 Nodes sind enthalten.
Startwert per Environment:
```env
BRAIN_MAX_DISPLAY_NODES=5000
```
Zulässig sind Werte von `0` bis `500000`. Die Einstellung reduziert die Render-, Layout- und Edge-Last im Browser. Der vollständige Graph wird weiterhin übertragen, damit bei einer Anfrage gezielt ausgeblendete Notes eingeblendet werden können.
## Kategorie-Filter
Über **FILTER** öffnet sich das Control-Panel. Eine leere Liste bedeutet jeweils „alle Kategorien“. Mehrere ausgewählte Kategorien werden als ODER-Verknüpfung behandelt.
@@ -56,6 +73,7 @@ BRAIN_LEARNING_CATEGORIES=
BRAIN_DISPLAY_CATEGORIES=
BRAIN_THINKING_CATEGORIES=
BRAIN_DEFAULT_VIEW=neural
BRAIN_MAX_DISPLAY_NODES=0
```
Zulässige Werte für `BRAIN_DEFAULT_VIEW` sind `neural` und `honeycomb`.
@@ -81,7 +99,8 @@ curl -X PUT http://localhost:8090/api/runtime-settings \
"learning_categories": [],
"display_categories": ["GLPI"],
"thinking_categories": ["GLPI", "Ollama"],
"view_mode": "honeycomb"
"view_mode": "honeycomb",
"max_display_nodes": 5000
}'
```

View File

@@ -1,39 +1,44 @@
2b1c678560f1f0e9f7e4aa9f1c89baec738f0d2e948ad22e29b85581cc7d8294 ./.env.example
97f83ffd2da3c3a89184d8bb460493f67eca5ab22d45dc7ea846fc6ea25ea5b9 ./ARCHITECTURE.md
8f3cb64f13d249e7a2a92d2879eead4f1a95e0807a3a569fe6c9022a78f29396 ./.env.example
606565a8a3b98a2985cb2bdc93f9c64d307a9288e58a8a55a041000cd68aae78 ./ARCHITECTURE.md
91f75b87bd95480fd40f191b551fe2d5f9f64e590e7c5442c19ee6166a362e6e ./CHANGELOG-GLPI-POOL-PERSISTENCE.md
e2f1f0400999cb59b8be09bc2743e06e0a0b2ecdc44a583af7c9a08b70d8509e ./CHANGELOG-GPU-NODE-LIMIT.md
7c8116836000f0ac2951afd3b5a8855380967cf3d7333b1d7608545b91b7a946 ./CHANGELOG-KNOWLEDGE-SYNTHESIS.md
87f89a81e1124b18e092a4ea946cb037295cb9e884a46b392286272dc8134dd4 ./CHANGELOG-RUNTIME-HONEYCOMB.md
99171b262752a66278da5a4a8b2b48e83fbdb1eb9cdca3efa944d260d6e6bc01 ./Dockerfile
5534536965bf0479455f97324c242160202650ca1256f1ba0420b4ad67125e49 ./GLPI-KB.md
809f8489aeabbdffbe35f3f8e93cf422439adfe93cf2fc9af2fb7c2f2d83749a ./KNOWLEDGE-SYNTHESIS.md
adcd3c9ba3bdf366afcc4e15a25423e068dd761e5d5d2d6f8cb20a3686302045 ./Makefile
381d7d6ac9e3c2e63c9ecdaa42ed4c73058f5d78e57c7532bb75a9663c919530 ./OLLAMA-POOL.md
10c14f4c08b9b84699cc4cf0dac3e6faf6c0ffd7e74b428463f14c628d0b533d ./PERSISTENCE.md
a1628299224739d81e80707ce0505313ef5d05203963ee88b0c9ac13f82332ef ./README.md
f07a308db091bd42f5e9c2d3456b1ad5058882656b00f66340910e7e379e9736 ./RUNTIME-CONTROLS-HONEYCOMB.md
3a4138ceb0c998828b1d12ba3b642768b167c998db20ccd618a1dd74d5f0df5b ./README.md
73171929c19afc9c0f280a50bee1c6becfbfe1a1a4107540ff80d5e4f885ba9a ./RUNTIME-CONTROLS-HONEYCOMB.md
cfc2393e8300b792cec30c57b9dc7670b19dcf5abb190dddaa75f32b77cb1fc7 ./cmd/brain/main.go
418310e2660f34890c4be97d6e4a0873cce57fe5b8eb6bfdff25bcd2d5faa7bf ./data/graph-state.json
21b51d0e1b7ed07c20f7f3a5da76dedab8df44a94a51724b67b0c3411599fe15 ./deployment/README.md
0f9c8802afd5ae65510d2cbe89058edb4a3b1fc514af903bb9f21694a028671e ./deployment/docker-compose.full.yml
10566f930e591293ec50067a26b3c11e89d5e5f6cc547ffe7304733f0f4c3357 ./docker-compose.yml
f9ddc9f66156d6c4895a731c76047fd8a4ed897c6a5bc34630112ed6f857a646 ./deployment/docker-compose.full.yml
73f2824a517044157fb59e8fc5854d6604511cbf6cffe810beb8830456ab89fa ./docker-compose.yml
9126f8bca1144abfc77747063b2c8f31acf12839a75e060b98369e10a00061ca ./go.mod
bf239391e61040b00d2f49d805b0d49a44bd3679f3c53849dfbc5e3b5a3fcc7a ./integrations/agent/README.md
a0105475dc054977223fac36618b8cd8137c55be1d11fddcd24e9a4d3074c170 ./integrations/agent/glpi-ai-agent-neural-brain.patch
73ab7c49600cdaa4795e76c50e66ce919dec171b3a07f2d16ff6f45ce5f73365 ./integrations/knowledgebase/README.md
36666043ebf4139e13610e5fcd0b0c6f45e4e6a53fed33ec47d03a66878e7b08 ./integrations/knowledgebase/glpi-ai-knowledgebase-neural-brain.patch
50d05fa2a183f5f3eaab0545cb48d3abb64be62eb5d84dc2c6d99c7125f7344b ./internal/activity/broker.go
d14cf93908ee9af0c75303db82e6ffeaf1fc1feaf7d35c525a9b6641a1b5f335 ./internal/config/config.go
678cc853f0905f9d9a301300872f94516814bea3cf92419f79026878532ec3e4 ./internal/config/config_test.go
0a1042eee574b1e72862da76c343b7a98c35bd87eceb09dd48abc63e64ae8bee ./internal/engine/engine.go
72fc0ea5056efbe3cec3d06f783743c0453fecb13ece9b3a47d6ebc2a921c629 ./internal/engine/engine_test.go
e507585b1606b9b54acd0a121ad7691a3c985ea9e36d9f3db282d4b1ec54211b ./internal/engine/runtime.go
ddbae88b0a810620ca950dbaef47df63c7f5e298eb983f8ddf76d9b1eaeb3716 ./internal/config/config.go
d5defa74dff6b09869f59c3e40ecc083f18bb9d81a0541956a857c4d0c26c360 ./internal/config/config_test.go
b432846e506d69990d8d8d4f711141972619a901ef57b965c3cf21f8d9279ced ./internal/engine/article.go
115ceadebc1395a0b2b8379a4eb6f5555656330b12dc9fb88c4dc1761dcf7ca6 ./internal/engine/engine.go
ee96b363b124d268216219cffcac73bbc1f393a9838dd9731ef5e56cfd4271d4 ./internal/engine/engine_test.go
872b9a5d6cc0e610a50f64ae8d66ed53bf07da718da22f91c11cf43a487fb425 ./internal/engine/runtime.go
b82980a646a92751bdd27a866ba1ffc6d34a3ba81d537f7b6e5a78e1432ee6fa ./internal/glpi/client.go
525102be56bc51ce8a08655b1b2bb53b67f4a1828903585fd664ed6a5133f617 ./internal/glpi/client_test.go
0a7fbd2d40cb70995651ef64dcc0d57ae7068aeefbd9457431f830cf799d51ce ./internal/graph/store.go
4caa47f2d337086c09fa612a5b5b1855f17d57d736531b8badd7223f0345ba6d ./internal/graph/store.go
13c38603f7ad1dcc590117d06c0663ace3e7a5971b96b2a96e8a53d9bcae87ff ./internal/graph/store_test.go
4476351d388d11c6becd78b4c918fc8d47dfd70500d8b3e2ddf81f7ff61a2cea ./internal/ingest/agent.go
5e1417ed485d472b4effd05d3a5dd74a689578b77c4aac97de14c7fe97f53436 ./internal/ingest/glpikb.go
774e9155eb7d17a208d483625f9fb6b60b9f9d44a70843c9cc51e29cf147a2b7 ./internal/ingest/glpikb_test.go
c0470d74bd3c3369cfefa6ba2343abcbdaa584a8898011704cba7d1cee620141 ./internal/ingest/knowledge.go
ebd47d134e61badf5e1eb35bdf1f45a3e4c9f99dcbf0a022850409bebc587e56 ./internal/ingest/knowledge_test.go
fb1af93afeb4ebc89cad95badeaee9a0cc3a236b9761e15a41a7b4b2c7192d00 ./internal/model/model.go
05bdcd8f82756af028807a9ba37e64232e2d93b6a803468d0a29665bbdb3067f ./internal/ingest/glpikb.go
ff44d56d56b9e301fbcf0f028d1bae6f0b851665f4fee65222097f1a2450f24a ./internal/ingest/glpikb_test.go
257a4beba480dab7d9b79c1f32496f6b4f648c4c05170f51a77220dbfd21fe96 ./internal/ingest/knowledge.go
a13910fb417484d56e78ae856b71fb66c63987d9abf3b513190bc52697321a18 ./internal/ingest/knowledge_test.go
738e9079804fd39c353db54e531cfa6ec0be0a96d9cc2a19b79f813641d90a28 ./internal/model/model.go
db7a99b832bc41c61584cd726d5fc7f4f250197d716d58bc27e715923f7f5426 ./internal/ollama/client.go
61089700aa5912b58bff526b71c2a1bda74077b802f9ff69a0d7dc89c1e3e3a1 ./internal/ollama/client_test.go
e61a9a426b96851b409ac357bdc53853324b5c5da7568ff6f89fe9cc09f83a73 ./internal/persist/coordinator.go
@@ -41,9 +46,9 @@ d33318b43388f134358cf40f5b0f130eb10edcca06011955ea0544897b4e4ad1 ./internal/per
2eb686cfc9016b9b0cf15e5c342f693ec9b60c454bc4814c1c9583f6d5b5b806 ./internal/research/searxng.go
6346f7b213aa3fb36bc9f43134bba75e0b368c9a005cc1aef8d265f553ff8cef ./internal/research/searxng_test.go
1c44338935a4faeedd9671c23df235aa99ea55ed6b2e72060a5d5ddc6c7a6464 ./internal/web/server.go
0f06a3be1885b2a11d5d335cdc5f643b1f6ef51fa191e37a0f68e2ae2e6a2ade ./internal/web/server_test.go
c6202f88840edbf266e8eb5b50fed70e28c08695d8cdfb9b67f633e5fbba6eb8 ./internal/web/static/app.css
fe529752c7a975e55f3fc4f1458170c85ca18c24f5f862471c1c0f22838d474b ./internal/web/static/app.js
b8496ecc23847a9478b4ee12cf83f79fe08f5c353e892000888a84739606cc64 ./internal/web/static/index.html
d0fe0699532da9ec43b9e357bf2b9410a8b99ea56e17555df2af560fb83db4bf ./neural-brain
4d89321b95eb2af3f594adb4d8230aa85377bebe707d232a6af7f1a1f99523dd ./internal/web/server_test.go
a9e586d77223a60bb8345c21de5381699eb8dbdfc215cbb5de975ee0d62b15fa ./internal/web/static/app.css
33f4f973878985ee086425f560a95d8269a7989da36f6c35c0ff2d8aa00c709f ./internal/web/static/app.js
7e99be904fea61b19bfd5560c93eec5dc5eee75252d1410ca1661cc4a87181dd ./internal/web/static/index.html
81c0b69517037de987b37611ea7569ee88aada911dbc3565ba0aef8e0bb83ba2 ./neural-brain
83aded814b6225395935e61fe957963c3c470f368fc9089f505b6de23e959115 ./preview.png

File diff suppressed because one or more lines are too long

View File

@@ -136,7 +136,23 @@ services:
BRAIN_DISPLAY_CATEGORIES: ${BRAIN_DISPLAY_CATEGORIES:-}
BRAIN_THINKING_CATEGORIES: ${BRAIN_THINKING_CATEGORIES:-}
BRAIN_DEFAULT_VIEW: ${BRAIN_DEFAULT_VIEW:-neural}
BRAIN_MAX_DISPLAY_NODES: ${BRAIN_MAX_DISPLAY_NODES:-0}
BRAIN_ENRICH_INTERVAL: ${BRAIN_ENRICH_INTERVAL:-90s}
BRAIN_ENRICH_BATCH_SIZE: ${BRAIN_ENRICH_BATCH_SIZE:-3}
BRAIN_ENRICH_STEP_DELAY: ${BRAIN_ENRICH_STEP_DELAY:-3s}
BRAIN_ENRICH_ANCHORS: ${BRAIN_ENRICH_ANCHORS:-48}
BRAIN_SIMILARITY_THRESHOLD: ${BRAIN_SIMILARITY_THRESHOLD:-0.68}
BRAIN_RELATION_THRESHOLD: ${BRAIN_RELATION_THRESHOLD:-0.72}
BRAIN_ARTICLE_SYNTHESIS_ENABLED: ${BRAIN_ARTICLE_SYNTHESIS_ENABLED:-true}
BRAIN_ARTICLE_MIN_SOURCES: ${BRAIN_ARTICLE_MIN_SOURCES:-3}
BRAIN_ARTICLE_MAX_SOURCES: ${BRAIN_ARTICLE_MAX_SOURCES:-8}
BRAIN_ARTICLE_MIN_PRODUCTION_RATIO: ${BRAIN_ARTICLE_MIN_PRODUCTION_RATIO:-0.70}
BRAIN_ARTICLE_MAX_GENERATION_DEPTH: ${BRAIN_ARTICLE_MAX_GENERATION_DEPTH:-2}
BRAIN_ARTICLE_MIN_CONFIDENCE: ${BRAIN_ARTICLE_MIN_CONFIDENCE:-0.74}
BRAIN_ARTICLE_MIN_TEXT_CHARS: ${BRAIN_ARTICLE_MIN_TEXT_CHARS:-180}
BRAIN_ARTICLE_MIN_ANSWER_CHARS: ${BRAIN_ARTICLE_MIN_ANSWER_CHARS:-420}
BRAIN_TOP_K: ${BRAIN_TOP_K:-8}
BRAIN_MAX_CONTEXT_CHARS: ${BRAIN_MAX_CONTEXT_CHARS:-16000}
BRAIN_RESEARCH_ENABLED: ${BRAIN_RESEARCH_ENABLED:-false}
SEARXNG_URL: ${SEARXNG_URL:-}
GLPI_KB_ENABLED: ${GLPI_KB_ENABLED:-false}

View File

@@ -35,12 +35,23 @@ services:
BRAIN_DISPLAY_CATEGORIES: ${BRAIN_DISPLAY_CATEGORIES:-}
BRAIN_THINKING_CATEGORIES: ${BRAIN_THINKING_CATEGORIES:-}
BRAIN_DEFAULT_VIEW: ${BRAIN_DEFAULT_VIEW:-neural}
BRAIN_MAX_DISPLAY_NODES: ${BRAIN_MAX_DISPLAY_NODES:-0}
BRAIN_ENRICH_INTERVAL: ${BRAIN_ENRICH_INTERVAL:-90s}
BRAIN_ENRICH_BATCH_SIZE: ${BRAIN_ENRICH_BATCH_SIZE:-3}
BRAIN_ENRICH_STEP_DELAY: ${BRAIN_ENRICH_STEP_DELAY:-3s}
BRAIN_ENRICH_ANCHORS: ${BRAIN_ENRICH_ANCHORS:-48}
BRAIN_SIMILARITY_THRESHOLD: ${BRAIN_SIMILARITY_THRESHOLD:-0.68}
BRAIN_RELATION_THRESHOLD: ${BRAIN_RELATION_THRESHOLD:-0.72}
BRAIN_ARTICLE_SYNTHESIS_ENABLED: ${BRAIN_ARTICLE_SYNTHESIS_ENABLED:-true}
BRAIN_ARTICLE_MIN_SOURCES: ${BRAIN_ARTICLE_MIN_SOURCES:-3}
BRAIN_ARTICLE_MAX_SOURCES: ${BRAIN_ARTICLE_MAX_SOURCES:-8}
BRAIN_ARTICLE_MIN_PRODUCTION_RATIO: ${BRAIN_ARTICLE_MIN_PRODUCTION_RATIO:-0.70}
BRAIN_ARTICLE_MAX_GENERATION_DEPTH: ${BRAIN_ARTICLE_MAX_GENERATION_DEPTH:-2}
BRAIN_ARTICLE_MIN_CONFIDENCE: ${BRAIN_ARTICLE_MIN_CONFIDENCE:-0.74}
BRAIN_ARTICLE_MIN_TEXT_CHARS: ${BRAIN_ARTICLE_MIN_TEXT_CHARS:-180}
BRAIN_ARTICLE_MIN_ANSWER_CHARS: ${BRAIN_ARTICLE_MIN_ANSWER_CHARS:-420}
BRAIN_TOP_K: ${BRAIN_TOP_K:-8}
BRAIN_MAX_CONTEXT_CHARS: ${BRAIN_MAX_CONTEXT_CHARS:-16000}
BRAIN_RESEARCH_ENABLED: ${BRAIN_RESEARCH_ENABLED:-false}
SEARXNG_URL: ${SEARXNG_URL:-}
BRAIN_API_KEY: ${BRAIN_API_KEY:-}

View File

@@ -40,6 +40,14 @@ type Config struct {
EnrichAnchors int
SimilarityThreshold float64
RelationThreshold float64
ArticleSynthesisEnabled bool
ArticleMinSources int
ArticleMaxSources int
ArticleMinProductionRatio float64
ArticleMaxGenerationDepth int
ArticleMinConfidence float64
ArticleMinTextChars int
ArticleMinAnswerChars int
TopK int
MaxContextChars int
AutoEnrich bool
@@ -51,6 +59,7 @@ type Config struct {
DisplayCategories []string
ThinkingCategories []string
DefaultView string
MaxDisplayNodes int
RuntimeDefaultsConfigured bool
GLPIKBEnabled bool
@@ -113,6 +122,14 @@ func Load() (Config, error) {
EnrichAnchors: integer("BRAIN_ENRICH_ANCHORS", 48),
SimilarityThreshold: number("BRAIN_SIMILARITY_THRESHOLD", 0.68),
RelationThreshold: number("BRAIN_RELATION_THRESHOLD", 0.72),
ArticleSynthesisEnabled: boolean("BRAIN_ARTICLE_SYNTHESIS_ENABLED", true),
ArticleMinSources: integer("BRAIN_ARTICLE_MIN_SOURCES", 3),
ArticleMaxSources: integer("BRAIN_ARTICLE_MAX_SOURCES", 8),
ArticleMinProductionRatio: number("BRAIN_ARTICLE_MIN_PRODUCTION_RATIO", 0.70),
ArticleMaxGenerationDepth: integer("BRAIN_ARTICLE_MAX_GENERATION_DEPTH", 2),
ArticleMinConfidence: number("BRAIN_ARTICLE_MIN_CONFIDENCE", 0.74),
ArticleMinTextChars: integer("BRAIN_ARTICLE_MIN_TEXT_CHARS", 180),
ArticleMinAnswerChars: integer("BRAIN_ARTICLE_MIN_ANSWER_CHARS", 420),
TopK: integer("BRAIN_TOP_K", 8),
MaxContextChars: integer("BRAIN_MAX_CONTEXT_CHARS", 16000),
AutoEnrich: boolean("BRAIN_AUTO_ENRICH", true),
@@ -124,6 +141,7 @@ func Load() (Config, error) {
DisplayCategories: stringList("BRAIN_DISPLAY_CATEGORIES"),
ThinkingCategories: stringList("BRAIN_THINKING_CATEGORIES"),
DefaultView: strings.ToLower(env("BRAIN_DEFAULT_VIEW", "neural")),
MaxDisplayNodes: integer("BRAIN_MAX_DISPLAY_NODES", 0),
RuntimeDefaultsConfigured: true,
GLPIKBEnabled: boolean("GLPI_KB_ENABLED", false),
GLPIURL: strings.TrimRight(strings.TrimSpace(os.Getenv("GLPI_URL")), "/"),
@@ -164,12 +182,36 @@ func Load() (Config, error) {
if cfg.RelationThreshold < 0 || cfg.RelationThreshold > 1 {
return Config{}, fmt.Errorf("invalid relation threshold")
}
if cfg.ArticleMinSources < 2 || cfg.ArticleMinSources > 20 {
return Config{}, fmt.Errorf("BRAIN_ARTICLE_MIN_SOURCES must be between 2 and 20")
}
if cfg.ArticleMaxSources < cfg.ArticleMinSources || cfg.ArticleMaxSources > 32 {
return Config{}, fmt.Errorf("BRAIN_ARTICLE_MAX_SOURCES must be between BRAIN_ARTICLE_MIN_SOURCES and 32")
}
if cfg.ArticleMinProductionRatio < 0.5 || cfg.ArticleMinProductionRatio > 1 {
return Config{}, fmt.Errorf("BRAIN_ARTICLE_MIN_PRODUCTION_RATIO must be between 0.5 and 1")
}
if cfg.ArticleMaxGenerationDepth < 1 || cfg.ArticleMaxGenerationDepth > 8 {
return Config{}, fmt.Errorf("BRAIN_ARTICLE_MAX_GENERATION_DEPTH must be between 1 and 8")
}
if cfg.ArticleMinConfidence < 0 || cfg.ArticleMinConfidence > 1 {
return Config{}, fmt.Errorf("BRAIN_ARTICLE_MIN_CONFIDENCE must be between 0 and 1")
}
if cfg.ArticleMinTextChars < 50 || cfg.ArticleMinTextChars > 10000 {
return Config{}, fmt.Errorf("BRAIN_ARTICLE_MIN_TEXT_CHARS must be between 50 and 10000")
}
if cfg.ArticleMinAnswerChars < 100 || cfg.ArticleMinAnswerChars > 50000 {
return Config{}, fmt.Errorf("BRAIN_ARTICLE_MIN_ANSWER_CHARS must be between 100 and 50000")
}
if cfg.ResearchEnabled && cfg.SearXNGURL == "" {
return Config{}, fmt.Errorf("BRAIN_RESEARCH_ENABLED requires SEARXNG_URL")
}
if cfg.DefaultView != "neural" && cfg.DefaultView != "honeycomb" {
return Config{}, fmt.Errorf("BRAIN_DEFAULT_VIEW must be neural or honeycomb")
}
if cfg.MaxDisplayNodes < 0 || cfg.MaxDisplayNodes > 500000 {
return Config{}, fmt.Errorf("BRAIN_MAX_DISPLAY_NODES must be between 0 and 500000")
}
if len(cfg.OllamaURLs) < 1 || len(cfg.OllamaURLs) > 64 {
return Config{}, fmt.Errorf("OLLAMA_URLS must contain between 1 and 64 nodes")
}

View File

@@ -46,14 +46,42 @@ func TestLoadRuntimeControlDefaultsAndFilters(t *testing.T) {
t.Setenv("BRAIN_DISPLAY_CATEGORIES", "GLPI KB")
t.Setenv("BRAIN_THINKING_CATEGORIES", "Netzwerk")
t.Setenv("BRAIN_DEFAULT_VIEW", "honeycomb")
t.Setenv("BRAIN_MAX_DISPLAY_NODES", "12000")
cfg, err := Load()
if err != nil {
t.Fatal(err)
}
if cfg.LearningEnabled || cfg.ThinkingEnabled || cfg.DefaultView != "honeycomb" {
if cfg.LearningEnabled || cfg.ThinkingEnabled || cfg.DefaultView != "honeycomb" || cfg.MaxDisplayNodes != 12000 {
t.Fatalf("unexpected runtime defaults: %+v", cfg)
}
if len(cfg.LearningCategories) != 2 || len(cfg.DisplayCategories) != 1 || len(cfg.ThinkingCategories) != 1 {
t.Fatalf("unexpected category defaults: %+v", cfg)
}
}
func TestLoadRejectsInvalidMaxDisplayNodes(t *testing.T) {
t.Setenv("BRAIN_DATA_DIR", t.TempDir())
t.Setenv("BRAIN_MAX_DISPLAY_NODES", "500001")
if _, err := Load(); err == nil {
t.Fatal("expected max display nodes validation error")
}
}
func TestLoadArticleSynthesisQualitySettings(t *testing.T) {
t.Setenv("BRAIN_DATA_DIR", t.TempDir())
t.Setenv("BRAIN_ARTICLE_SYNTHESIS_ENABLED", "true")
t.Setenv("BRAIN_ARTICLE_MIN_SOURCES", "4")
t.Setenv("BRAIN_ARTICLE_MAX_SOURCES", "7")
t.Setenv("BRAIN_ARTICLE_MIN_PRODUCTION_RATIO", "0.8")
t.Setenv("BRAIN_ARTICLE_MAX_GENERATION_DEPTH", "2")
t.Setenv("BRAIN_ARTICLE_MIN_CONFIDENCE", "0.81")
t.Setenv("BRAIN_ARTICLE_MIN_TEXT_CHARS", "220")
t.Setenv("BRAIN_ARTICLE_MIN_ANSWER_CHARS", "500")
cfg, err := Load()
if err != nil {
t.Fatal(err)
}
if !cfg.ArticleSynthesisEnabled || cfg.ArticleMinSources != 4 || cfg.ArticleMaxSources != 7 || cfg.ArticleMinProductionRatio != 0.8 || cfg.ArticleMaxGenerationDepth != 2 || cfg.ArticleMinConfidence != 0.81 || cfg.ArticleMinTextChars != 220 || cfg.ArticleMinAnswerChars != 500 {
t.Fatalf("unexpected article synthesis config: %+v", cfg)
}
}

815
internal/engine/article.go Normal file
View File

@@ -0,0 +1,815 @@
package engine
import (
"context"
"crypto/sha256"
"encoding/hex"
"encoding/json"
"fmt"
"math"
"os"
"path/filepath"
"sort"
"strings"
"time"
"github.com/local/glpi-neural-brain/internal/graph"
"github.com/local/glpi-neural-brain/internal/model"
)
type articleSource struct {
Node model.Node
Content string
Score float64
Depth int
}
type articleSynthesisOutcome struct {
Created bool
Skipped bool
Reason string
Path string
Action string
Title string
}
func (e *Engine) synthesizeKnowledgeArticle(ctx context.Context, trigger string, seeds []model.Node, relation model.RelationDecision, initialResearch []model.ResearchResult) (articleSynthesisOutcome, error) {
if !e.Cfg.ArticleSynthesisEnabled {
return articleSynthesisOutcome{Skipped: true, Reason: "article_synthesis_disabled"}, nil
}
sources := e.selectArticleSources(seeds)
productionCount, aiCount, productionRatio, maxDepth := articleSourceStats(sources)
if productionCount < e.Cfg.ArticleMinSources {
e.Broker.Publish(model.Activity{
Type: "article.skipped", Source: "brain", Phase: "source-selection", NodeIDs: nodeIDsFromArticleSources(sources),
Message: "Für einen belastbaren Wissensartikel sind noch nicht genug produktive Quellen verbunden",
Strength: .3,
Metadata: map[string]any{"trigger": trigger, "reason": "insufficient_production_sources", "productive_sources": productionCount, "ai_sources": aiCount, "required_sources": e.Cfg.ArticleMinSources, "production_ratio": productionRatio},
})
return articleSynthesisOutcome{Skipped: true, Reason: "insufficient_production_sources"}, nil
}
if productionRatio < e.Cfg.ArticleMinProductionRatio {
return articleSynthesisOutcome{Skipped: true, Reason: "production_ratio_too_low"}, nil
}
generationDepth := maxDepth + 1
if generationDepth > e.Cfg.ArticleMaxGenerationDepth {
return articleSynthesisOutcome{Skipped: true, Reason: "generation_depth_limit"}, nil
}
e.Broker.Publish(model.Activity{
Type: "article.plan.started", Source: "brain", Phase: "knowledge-planning", NodeIDs: nodeIDsFromArticleSources(sources),
Message: fmt.Sprintf("%d Quellen werden auf einen echten Wissensmehrwert geprüft", len(sources)), Strength: .84,
Metadata: map[string]any{"trigger": trigger, "productive_sources": productionCount, "ai_sources": aiCount, "production_ratio": productionRatio, "generation_depth": generationDepth, "model": e.Cfg.ChatModel},
})
var plan model.ArticlePlanDecision
if err := e.Ollama.ChatJSON(ctx, articlePlanSystemPrompt(), e.articlePlanContext(sources, relation, initialResearch), articlePlanSchema(), &plan); err != nil {
return articleSynthesisOutcome{}, fmt.Errorf("article planning failed: %w", err)
}
plan.Action = safeArticleAction(plan.Action)
allowedIDs := nodeIDsFromArticleSources(sources)
plan.SourceNodeIDs = validIDs(plan.SourceNodeIDs, allowedIDs)
if len(plan.SourceNodeIDs) < e.Cfg.ArticleMinSources {
plan.SourceNodeIDs = allowedIDs
}
selected := filterArticleSources(sources, plan.SourceNodeIDs)
productionCount, aiCount, productionRatio, maxDepth = articleSourceStats(selected)
generationDepth = maxDepth + 1
if productionCount < e.Cfg.ArticleMinSources || productionRatio < e.Cfg.ArticleMinProductionRatio || generationDepth > e.Cfg.ArticleMaxGenerationDepth {
return articleSynthesisOutcome{Skipped: true, Reason: "plan_source_policy_failed", Action: plan.Action}, nil
}
if plan.Action == "skip" {
e.Broker.Publish(model.Activity{Type: "article.plan.skipped", Source: "brain", Phase: "knowledge-planning", NodeIDs: plan.SourceNodeIDs, Message: nonempty(plan.Reason, "Der Quellenverbund erzeugt keinen zusätzlichen Wissensnutzen"), Strength: .36, Metadata: map[string]any{"trigger": trigger, "action": plan.Action, "reason": plan.Reason, "expected_value": plan.ExpectedValue, "missing_information": plan.MissingInformation, "contradictions": plan.Contradictions}})
return articleSynthesisOutcome{Skipped: true, Reason: nonempty(plan.Reason, "model_skip"), Action: plan.Action}, nil
}
if (plan.Action == "update" || plan.Action == "merge") && !validProductionTarget(plan.TargetArticleID, selected) {
e.Broker.Publish(model.Activity{Type: "article.plan.skipped", Source: "brain", Phase: "knowledge-planning", NodeIDs: plan.SourceNodeIDs, Message: "Der vorgeschlagene Zielartikel ist keine produktive Quelle des geprüften Themenverbunds", Strength: .34, Metadata: map[string]any{"trigger": trigger, "action": plan.Action, "target_article_id": plan.TargetArticleID}})
return articleSynthesisOutcome{Skipped: true, Reason: "invalid_target_article", Action: plan.Action}, nil
}
if e.hasEquivalentArticleDraft(selected, plan) {
e.Broker.Publish(model.Activity{Type: "article.plan.skipped", Source: "brain", Phase: "quality-gate", NodeIDs: plan.SourceNodeIDs, Message: "Für diesen Themenverbund liegt bereits ein gleichwertiger AI-THINK-Entwurf im Staging", Strength: .3, Metadata: map[string]any{"trigger": trigger, "action": plan.Action, "target_article_id": plan.TargetArticleID, "reason": "equivalent_staging_draft"}})
return articleSynthesisOutcome{Skipped: true, Reason: "equivalent_staging_draft", Action: plan.Action}, nil
}
researchResults := append([]model.ResearchResult(nil), initialResearch...)
if len(initialResearch) > 0 {
e.addResearchToSources(selected, initialResearch)
}
if plan.NeedsResearch {
if !e.Cfg.ResearchEnabled || e.Research == nil || strings.TrimSpace(plan.ResearchQuery) == "" {
e.Broker.Publish(model.Activity{Type: "article.plan.skipped", Source: "brain", Phase: "knowledge-research", NodeIDs: plan.SourceNodeIDs, Message: "Der Artikel benötigt zusätzliche Evidenz, aber die kontrollierte Recherche ist nicht verfügbar", Strength: .34, Metadata: map[string]any{"trigger": trigger, "reason": "required_research_unavailable", "research_enabled": e.Cfg.ResearchEnabled, "research_query": plan.ResearchQuery, "missing_information": plan.MissingInformation}})
return articleSynthesisOutcome{Skipped: true, Reason: "required_research_unavailable", Action: plan.Action}, nil
}
e.Broker.Publish(model.Activity{Type: "article.research.started", Source: "brain", Phase: "knowledge-research", NodeIDs: plan.SourceNodeIDs, Message: "Für den geplanten Wissensartikel werden offene Punkte recherchiert", Strength: .9, Metadata: map[string]any{"trigger": trigger, "research_query": plan.ResearchQuery, "missing_information": plan.MissingInformation}})
results, err := e.Research.Search(ctx, plan.ResearchQuery, 5)
if err != nil {
e.Broker.Publish(model.Activity{Type: "article.research.failed", Source: "brain", Phase: "knowledge-research", NodeIDs: plan.SourceNodeIDs, Message: "Die ergänzende Artikelrecherche ist fehlgeschlagen; es wird nichts erfunden", Strength: .35, Metadata: map[string]any{"trigger": trigger, "error": err.Error()}})
return articleSynthesisOutcome{Skipped: true, Reason: "required_research_failed", Action: plan.Action}, nil
}
if len(results) == 0 {
return articleSynthesisOutcome{Skipped: true, Reason: "required_research_empty", Action: plan.Action}, nil
}
researchResults = append(researchResults, results...)
e.addResearchToSources(selected, results)
}
e.Broker.Publish(model.Activity{Type: "article.draft.started", Source: "brain", Phase: "knowledge-synthesis", NodeIDs: plan.SourceNodeIDs, Message: fmt.Sprintf("Qwen erstellt einen strukturierten KB-Entwurf · Aktion %s", strings.ToUpper(plan.Action)), Strength: .95, Metadata: map[string]any{"trigger": trigger, "action": plan.Action, "target_article_id": plan.TargetArticleID, "source_count": len(selected), "research_result_count": len(researchResults), "generation_depth": generationDepth}})
var draft model.KnowledgeArticleDraft
if err := e.Ollama.ChatJSON(ctx, articleDraftSystemPrompt(), e.articleDraftContext(selected, plan, researchResults), articleDraftSchema(), &draft); err != nil {
return articleSynthesisOutcome{}, fmt.Errorf("article drafting failed: %w", err)
}
draft.SourceNodeIDs = validIDs(draft.SourceNodeIDs, plan.SourceNodeIDs)
if len(draft.SourceNodeIDs) < e.Cfg.ArticleMinSources {
draft.SourceNodeIDs = append([]string(nil), plan.SourceNodeIDs...)
}
selected = filterArticleSources(selected, draft.SourceNodeIDs)
productionCount, aiCount, productionRatio, maxDepth = articleSourceStats(selected)
generationDepth = maxDepth + 1
if err := e.validateArticleDraft(draft, selected, productionRatio, generationDepth); err != nil {
e.Broker.Publish(model.Activity{Type: "article.draft.rejected", Source: "brain", Phase: "quality-gate", NodeIDs: draft.SourceNodeIDs, Message: "Der erzeugte Artikelentwurf hat die Qualitätsregeln nicht erfüllt", Strength: .42, Metadata: map[string]any{"trigger": trigger, "action": plan.Action, "error": err.Error(), "confidence": draft.Confidence, "productive_sources": productionCount, "ai_sources": aiCount, "production_ratio": productionRatio, "generation_depth": generationDepth}})
return articleSynthesisOutcome{Skipped: true, Reason: "quality_gate: " + err.Error(), Action: plan.Action, Title: draft.Title}, nil
}
path, articleID, created, err := e.writeKnowledgeArticleDraft(selected, plan, draft, researchResults, productionCount, aiCount, productionRatio, generationDepth)
if err != nil {
return articleSynthesisOutcome{}, err
}
if !created {
e.Broker.Publish(model.Activity{Type: "article.duplicate", Source: "brain", Phase: "quality-gate", NodeIDs: draft.SourceNodeIDs, Message: "Ein identischer AI-THINK-Artikelentwurf ist bereits vorhanden oder vorgemerkt", Strength: .28, Metadata: map[string]any{"trigger": trigger, "action": plan.Action, "path": path, "title": draft.Title}})
return articleSynthesisOutcome{Skipped: true, Reason: "duplicate", Action: plan.Action, Path: path, Title: draft.Title}, nil
}
e.addRuntimeArticleNode(articleID, selected, plan, draft, productionCount, aiCount, productionRatio, generationDepth)
e.Broker.Publish(model.Activity{Type: "article.created", Source: "brain", Phase: "staging", NodeIDs: append([]string{graph.ID("knowledge", articleID)}, draft.SourceNodeIDs...), Message: fmt.Sprintf("Neuer strukturierter KB-Entwurf im Staging · %s", draft.Title), Strength: 1, Metadata: map[string]any{"trigger": trigger, "action": plan.Action, "target_article_id": plan.TargetArticleID, "path": path, "title": draft.Title, "confidence": draft.Confidence, "productive_sources": productionCount, "ai_sources": aiCount, "production_ratio": productionRatio, "generation_depth": generationDepth, "research_result_count": len(researchResults), "write_pending": true}})
return articleSynthesisOutcome{Created: true, Path: path, Action: plan.Action, Title: draft.Title}, nil
}
func (e *Engine) selectArticleSources(seeds []model.Node) []articleSource {
snapshot := e.Graph.Snapshot()
seedIDs := map[string]bool{}
var seedVectors [][]float64
for _, seed := range seeds {
seedIDs[seed.ID] = true
if v, ok := e.Graph.Vector(seed.ID); ok && len(v) > 0 {
seedVectors = append(seedVectors, v)
}
}
centroid := vectorCentroid(seedVectors)
direct := map[string]float64{}
for _, edge := range snapshot.Edges {
if edge.Status == "rejected" || isTaxonomyEdge(edge.Type) {
continue
}
if seedIDs[edge.Source] {
direct[edge.Target] += math.Max(.2, math.Max(edge.Confidence, edge.Weight))
}
if seedIDs[edge.Target] {
direct[edge.Source] += math.Max(.2, math.Max(edge.Confidence, edge.Weight))
}
}
filters := e.thinkingCategories()
var production, ai []articleSource
for _, node := range snapshot.Nodes {
if node.Kind != "knowledge" && node.Kind != "ai-think" {
continue
}
if !matchesCategories(node, filters) {
continue
}
depth := nodeGenerationDepth(node)
if node.Kind == "ai-think" && depth >= e.Cfg.ArticleMaxGenerationDepth {
continue
}
isProduction := node.Kind == "knowledge" && node.Status == "production"
isAI := node.Kind == "ai-think"
if !isProduction && !isAI {
continue
}
score := direct[node.ID] * 2.3
if seedIDs[node.ID] {
score += 8
}
if len(centroid) > 0 {
if v, ok := e.Graph.Vector(node.ID); ok && len(v) == len(centroid) {
sim := cosineVector(centroid, v)
if sim < e.Cfg.SimilarityThreshold*.82 && direct[node.ID] == 0 && !seedIDs[node.ID] {
continue
}
score += sim * 3
}
}
score += categoryAffinity(node, seeds) * .45
content := e.sourceContent(node)
if strings.TrimSpace(content) == "" {
content = node.Summary
}
source := articleSource{Node: node, Content: content, Score: score, Depth: depth}
if isProduction {
production = append(production, source)
} else {
ai = append(ai, source)
}
}
sortArticleSources(production)
sortArticleSources(ai)
max := e.Cfg.ArticleMaxSources
if max < 1 {
max = 8
}
out := make([]articleSource, 0, max)
for _, source := range production {
if len(out) >= max {
break
}
out = append(out, source)
}
for _, source := range ai {
if len(out) >= max {
break
}
prod, aiCount, _, _ := articleSourceStats(out)
if float64(aiCount+1)/float64(prod+aiCount+1) > 1-e.Cfg.ArticleMinProductionRatio {
continue
}
out = append(out, source)
}
return out
}
func (e *Engine) sourceContent(node model.Node) string {
if node.Origin == "glpi-kb" && e.GLPIKB != nil {
if value, ok := e.GLPIKB.Content(node.ID); ok && strings.TrimSpace(value) != "" {
return value
}
}
relPath := metadataString(node.Metadata, "path")
if relPath == "" {
return node.Summary
}
roots := e.Cfg.KnowledgeDirs
if node.Origin == "knowledge-staging" {
roots = e.Cfg.StagingDirs
}
for _, root := range roots {
candidate := filepath.Clean(filepath.Join(root, filepath.FromSlash(relPath)))
rootClean := filepath.Clean(root)
rel, err := filepath.Rel(rootClean, candidate)
if err != nil || rel == ".." || strings.HasPrefix(rel, ".."+string(filepath.Separator)) {
continue
}
data, err := os.ReadFile(candidate)
if err != nil {
continue
}
var doc map[string]any
if json.Unmarshal(data, &doc) != nil {
continue
}
problem := firstMapString(doc, "text", "problem", "description")
answer := firstMapString(doc, "answer", "solution")
var b strings.Builder
if problem != "" {
b.WriteString("PROBLEM / BESCHREIBUNG:\n")
b.WriteString(problem)
}
if answer != "" {
if b.Len() > 0 {
b.WriteString("\n\n")
}
b.WriteString("LÖSUNG / ANTWORT:\n")
b.WriteString(answer)
}
if b.Len() > 0 {
return b.String()
}
}
return node.Summary
}
func (e *Engine) articlePlanContext(sources []articleSource, relation model.RelationDecision, researchResults []model.ResearchResult) string {
var b strings.Builder
fmt.Fprintf(&b, "AUSLÖSENDE_RELATION: %s\nRELATIONSERKLÄRUNG: %s\nTHEMA: %s\n\n", safeRelation(relation.RelationType), relation.Explanation, relation.TopicLabel)
b.WriteString("ZIEL: Entscheide, ob aus den Quellen ein neuer oder verbesserter Helpdesk-Wissensartikel mit echtem Mehrwert entstehen soll.\n")
b.WriteString("PRODUKTIVE ARTIKEL dürfen als update/merge-Ziel gewählt werden. Die IDs stehen bei den Quellen.\n\nQUELLEN:\n")
appendArticleSources(&b, sources, e.Cfg.MaxContextChars)
if len(researchResults) > 0 {
b.WriteString("\nBEREITS VORHANDENE RECHERCHEHINWEISE:\n")
for i, result := range researchResults {
fmt.Fprintf(&b, "\nR%d: %s\nURL: %s\nAUSZUG: %s\n", i+1, result.Title, result.URL, clamp(result.Content, 800))
}
}
return b.String()
}
func (e *Engine) articleDraftContext(sources []articleSource, plan model.ArticlePlanDecision, researchResults []model.ResearchResult) string {
var b strings.Builder
fmt.Fprintf(&b, "AKTION: %s\nZIELARTIKEL_ID: %s\nARTIKELTYP: %s\nBEGRÜNDUNG: %s\nERWARTETER_MEHRWERT: %s\nFEHLENDE_INFORMATIONEN: %s\nWIDERSPRÜCHE: %s\n\n", plan.Action, plan.TargetArticleID, plan.ArticleType, plan.Reason, plan.ExpectedValue, strings.Join(plan.MissingInformation, " | "), strings.Join(plan.Contradictions, " | "))
b.WriteString("QUELLEN:\n")
appendArticleSources(&b, sources, e.Cfg.MaxContextChars)
if len(researchResults) > 0 {
b.WriteString("\nRECHERCHEERGEBNISSE (nur als zusätzliche Evidenz, nicht als garantierte Wahrheit):\n")
for i, result := range researchResults {
fmt.Fprintf(&b, "\nR%d: %s\nURL: %s\nAUSZUG: %s\n", i+1, result.Title, result.URL, clamp(result.Content, 1000))
}
}
return b.String()
}
func appendArticleSources(b *strings.Builder, sources []articleSource, maxChars int) {
if maxChars < 4000 {
maxChars = 16000
}
remaining := maxChars
for i, source := range sources {
if remaining <= 500 {
break
}
contentLimit := remaining / max(1, len(sources)-i)
if contentLimit > 4000 {
contentLimit = 4000
}
if contentLimit < 700 {
contentLimit = 700
}
part := fmt.Sprintf("\nSOURCE_NODE_ID: %s\nEXTERNAL_ID: %s\nKIND: %s\nSTATUS: %s\nORIGIN: %s\nGENERATION_DEPTH: %d\nTITEL: %s\nKATEGORIEN: %s\nINHALT:\n%s\n", source.Node.ID, source.Node.ExternalID, source.Node.Kind, source.Node.Status, source.Node.Origin, source.Depth, source.Node.Label, strings.Join(source.Node.Categories, ", "), clamp(source.Content, contentLimit))
b.WriteString(part)
remaining -= len(part)
}
}
func articlePlanSystemPrompt() string {
return `Du planst die Pflege einer deutschsprachigen Helpdesk-Wissensdatenbank. Du erhältst mehrere bereits verwandte Quellen. Entscheide streng zwischen create, update, merge und skip.
create: Es gibt noch keinen vollständigen Artikel und die Quellen ergeben gemeinsam einen eigenständigen, nützlichen Lösungsartikel.
update: Ein vorhandener produktiver Artikel ist das klare Ziel und kann mit belastbaren Informationen verbessert werden.
merge: Mehrere produktive Artikel überschneiden sich und sollten als Staging-Entwurf in einen angegebenen Zielartikel konsolidiert werden.
skip: Kein echter Mehrwert, bloße Dublette, zu wenig Lösungssubstanz, zu viele Widersprüche oder unzureichende Quellen.
Erfinde keine Fakten. Bevorzuge konkrete Problemlösung gegenüber technischer Meta-Analyse. target_article_id ist bei update/merge zwingend eine SOURCE_NODE_ID einer produktiven Quelle. source_node_ids dürfen nur IDs aus dem Kontext enthalten. Wenn notwendige Fakten fehlen, setze needs_research=true. Gib ausschließlich JSON nach Schema zurück.`
}
func articleDraftSystemPrompt() string {
return `Du schreibst einen hochwertigen deutschsprachigen Helpdesk-KB-Entwurf aus den bereitgestellten Quellen. Der Text ist für Anwender und Support-Mitarbeiter, nicht für Graph- oder KI-Entwickler.
Regeln:
- Erfinde keine Fakten, Befehle, Pfade, Versionen oder Ursachen.
- text beschreibt Problem, Symptome, Geltungsbereich und Abgrenzung.
- answer enthält eine konkrete, nachvollziehbare und möglichst nummerierte Lösung.
- prerequisites, validation und troubleshooting enthalten nur belegbare Punkte.
- Widersprüche werden nicht still aufgelöst; verbleibende Unsicherheiten kommen in open_questions.
- Keine Formulierungen über "semantische Nähe", "Nodes", "Edges", "KI", "Qwen" oder den Denkprozess im Artikeltext.
- source_node_ids dürfen nur IDs aus dem Kontext enthalten.
- Der Entwurf muss einen klaren Mehrwert gegenüber bloßem Zusammenfassen haben.
Gib ausschließlich JSON nach Schema zurück.`
}
func articlePlanSchema() map[string]any {
return map[string]any{"type": "object", "properties": map[string]any{
"action": map[string]any{"type": "string", "enum": []string{"create", "update", "merge", "skip"}},
"target_article_id": map[string]any{"type": "string"},
"reason": map[string]any{"type": "string"},
"expected_value": map[string]any{"type": "string"},
"article_type": map[string]any{"type": "string", "enum": []string{"troubleshooting", "how_to", "reference", "concept", "decision_guide"}},
"source_node_ids": map[string]any{"type": "array", "items": map[string]any{"type": "string"}},
"missing_information": map[string]any{"type": "array", "items": map[string]any{"type": "string"}},
"contradictions": map[string]any{"type": "array", "items": map[string]any{"type": "string"}},
"needs_research": map[string]any{"type": "boolean"},
"research_query": map[string]any{"type": "string"},
}, "required": []string{"action", "target_article_id", "reason", "expected_value", "article_type", "source_node_ids", "missing_information", "contradictions", "needs_research", "research_query"}}
}
func articleDraftSchema() map[string]any {
return map[string]any{"type": "object", "properties": map[string]any{
"title": map[string]any{"type": "string"},
"text": map[string]any{"type": "string"},
"answer": map[string]any{"type": "string"},
"prerequisites": map[string]any{"type": "array", "items": map[string]any{"type": "string"}},
"validation": map[string]any{"type": "array", "items": map[string]any{"type": "string"}},
"troubleshooting": map[string]any{"type": "array", "items": map[string]any{"type": "string"}},
"categories": map[string]any{"type": "array", "items": map[string]any{"type": "string"}},
"keywords": map[string]any{"type": "array", "items": map[string]any{"type": "string"}},
"source_node_ids": map[string]any{"type": "array", "items": map[string]any{"type": "string"}},
"confidence": map[string]any{"type": "number", "minimum": 0, "maximum": 1},
"open_questions": map[string]any{"type": "array", "items": map[string]any{"type": "string"}},
}, "required": []string{"title", "text", "answer", "prerequisites", "validation", "troubleshooting", "categories", "keywords", "source_node_ids", "confidence", "open_questions"}}
}
func (e *Engine) validateArticleDraft(draft model.KnowledgeArticleDraft, sources []articleSource, productionRatio float64, generationDepth int) error {
if len([]rune(strings.TrimSpace(draft.Title))) < 8 {
return fmt.Errorf("title is too short")
}
if len([]rune(strings.TrimSpace(draft.Text))) < e.Cfg.ArticleMinTextChars {
return fmt.Errorf("problem description is shorter than %d characters", e.Cfg.ArticleMinTextChars)
}
if len([]rune(strings.TrimSpace(draft.Answer))) < e.Cfg.ArticleMinAnswerChars {
return fmt.Errorf("solution is shorter than %d characters", e.Cfg.ArticleMinAnswerChars)
}
if draft.Confidence < e.Cfg.ArticleMinConfidence {
return fmt.Errorf("confidence %.2f is below %.2f", draft.Confidence, e.Cfg.ArticleMinConfidence)
}
production, _, _, _ := articleSourceStats(sources)
if production < e.Cfg.ArticleMinSources {
return fmt.Errorf("only %d productive sources", production)
}
if productionRatio < e.Cfg.ArticleMinProductionRatio {
return fmt.Errorf("production ratio %.2f is below %.2f", productionRatio, e.Cfg.ArticleMinProductionRatio)
}
if generationDepth > e.Cfg.ArticleMaxGenerationDepth {
return fmt.Errorf("generation depth %d exceeds %d", generationDepth, e.Cfg.ArticleMaxGenerationDepth)
}
return nil
}
func (e *Engine) writeKnowledgeArticleDraft(sources []articleSource, plan model.ArticlePlanDecision, draft model.KnowledgeArticleDraft, researchResults []model.ResearchResult, productionCount, aiCount int, productionRatio float64, generationDepth int) (string, string, bool, error) {
if len(e.Cfg.StagingDirs) == 0 {
return "", "", false, fmt.Errorf("no BRAIN_STAGING_DIRS configured")
}
sourceIDs := nodeIDsFromArticleSources(sources)
sort.Strings(sourceIDs)
fingerprint := strings.Join(sourceIDs, "\x00") + "\x00" + plan.Action + "\x00" + plan.TargetArticleID
sum := sha256.Sum256([]byte(fingerprint))
short := strings.ToUpper(hex.EncodeToString(sum[:6]))
now := time.Now().UTC()
articleID := fmt.Sprintf("KB-AI-THINK-ARTICLE-%s-%s", now.Format("20060102"), short)
path := filepath.Join(e.Cfg.StagingDirs[0], strings.ToLower(articleID)+".json")
if e.Persistence.Pending(path) {
return path, articleID, false, nil
}
if _, err := os.Stat(path); err == nil {
return path, articleID, false, nil
}
categories := []string{"AI-THINK", "AI-Staging", "AI-Synthesis"}
categories = append(categories, draft.Categories...)
for _, source := range sources {
categories = append(categories, source.Node.Categories...)
}
categories = limitStrings(unique(categories), 18)
keywords := append([]string(nil), draft.Keywords...)
for _, source := range sources {
keywords = append(keywords, source.Node.Keywords...)
}
keywords = limitStrings(unique(keywords), 30)
answer := formatArticleAnswer(draft)
var evidence []map[string]any
for _, result := range researchResults {
evidence = append(evidence, map[string]any{"title": result.Title, "url": result.URL, "excerpt": clamp(result.Content, 500)})
}
targetExternalID := ""
if plan.TargetArticleID != "" {
for _, source := range sources {
if source.Node.ID == plan.TargetArticleID {
targetExternalID = source.Node.ExternalID
break
}
}
}
doc := map[string]any{
"id": articleID, "title": strings.TrimSpace(draft.Title), "text": strings.TrimSpace(draft.Text), "answer": answer,
"auto_reply": false, "min_score": 0.82, "categories": categories, "keywords": keywords,
"source": "Neural Brain / " + e.Cfg.ChatModel + " (AI-THINK Knowledge Synthesis)",
"source_uri": "brain://article/" + short, "language": "de-DE", "communication_style": "formal",
"ai_think": map[string]any{
"status": "staging", "subtype": "knowledge_synthesis", "generated_at": now, "action": plan.Action,
"target_node_id": plan.TargetArticleID, "target_article_id": targetExternalID, "reason": plan.Reason,
"expected_value": plan.ExpectedValue, "article_type": plan.ArticleType,
"source_nodes": externalIDsFromArticleSources(sources), "source_node_ids": sourceIDs,
"productive_source_count": productionCount, "ai_source_count": aiCount, "production_ratio": productionRatio,
"generation_depth": generationDepth, "confidence": draft.Confidence,
"missing_information": plan.MissingInformation, "contradictions": plan.Contradictions,
"open_questions": draft.OpenQuestions, "research_query": plan.ResearchQuery, "research_evidence": evidence,
"review_notice": "Vor produktiver Nutzung im Editor prüfen, korrigieren und freigeben.",
},
}
bytes, err := json.MarshalIndent(doc, "", " ")
if err != nil {
return "", "", false, err
}
queued, err := e.Persistence.QueueFile(path, append(bytes, '\n'), 0o640)
if err != nil {
return "", "", false, err
}
return queued, articleID, true, nil
}
func (e *Engine) addRuntimeArticleNode(articleID string, sources []articleSource, plan model.ArticlePlanDecision, draft model.KnowledgeArticleDraft, productionCount, aiCount int, productionRatio float64, generationDepth int) {
nodeID := graph.ID("knowledge", articleID)
now := time.Now().UTC()
node := model.Node{
ID: nodeID, Kind: "ai-think", Label: draft.Title, Summary: clamp(strings.TrimSpace(draft.Text)+"\n\n"+formatArticleAnswer(draft), 1400),
Status: "staging", Origin: "knowledge-staging", ExternalID: articleID, URI: "brain://article/" + articleID,
Categories: unique(append([]string{"AI-THINK", "AI-Staging", "AI-Synthesis"}, draft.Categories...)), Keywords: unique(draft.Keywords), Weight: 1.45,
Metadata: map[string]any{"subtype": "knowledge_synthesis", "action": plan.Action, "target_node_id": plan.TargetArticleID, "generation_depth": generationDepth, "confidence": draft.Confidence, "source_node_ids": nodeIDsFromArticleSources(sources), "productive_source_count": productionCount, "ai_source_count": aiCount, "production_ratio": productionRatio}, UpdatedAt: now,
}
e.Graph.UpsertNode(node)
for _, source := range sources {
e.Graph.UpsertEdge(model.Edge{Source: nodeID, Target: source.Node.ID, Type: "synthesized_from", Origin: "knowledge-staging", Status: "staging", Confidence: draft.Confidence, Weight: .65, Explanation: plan.Reason})
}
if plan.TargetArticleID != "" {
e.Graph.UpsertEdge(model.Edge{Source: nodeID, Target: plan.TargetArticleID, Type: "proposes_" + plan.Action, Origin: "knowledge-staging", Status: "staging", Confidence: draft.Confidence, Weight: .8, Explanation: plan.Reason})
}
}
func (e *Engine) addResearchToSources(sources []articleSource, results []model.ResearchResult) {
for _, result := range results {
id := graph.ID("external", result.URL)
e.Graph.UpsertNode(model.Node{ID: id, Kind: "external", Label: result.Title, Summary: clamp(result.Content, 900), Status: "research", Origin: "research", ExternalID: result.URL, URI: result.URL, Weight: .8, UpdatedAt: time.Now().UTC()})
for _, source := range sources {
e.Graph.UpsertEdge(model.Edge{Source: id, Target: source.Node.ID, Type: "research_evidence", Origin: "research", Status: "staging", Confidence: .55, Weight: .4})
}
}
}
func formatArticleAnswer(draft model.KnowledgeArticleDraft) string {
var b strings.Builder
b.WriteString(strings.TrimSpace(draft.Answer))
appendListSection(&b, "Voraussetzungen", draft.Prerequisites)
appendListSection(&b, "Ergebnis prüfen", draft.Validation)
appendListSection(&b, "Fehlerbehandlung", draft.Troubleshooting)
return strings.TrimSpace(b.String())
}
func limitStrings(values []string, limit int) []string {
if limit > 0 && len(values) > limit {
return append([]string(nil), values[:limit]...)
}
return values
}
func appendListSection(b *strings.Builder, title string, items []string) {
clean := unique(items)
if len(clean) == 0 {
return
}
b.WriteString("\n\n## ")
b.WriteString(title)
for _, item := range clean {
b.WriteString("\n- ")
b.WriteString(strings.TrimSpace(item))
}
}
func articleSourceStats(sources []articleSource) (production, ai int, ratio float64, maxDepth int) {
for _, source := range sources {
if source.Node.Kind == "knowledge" && source.Node.Status == "production" {
production++
} else if source.Node.Kind == "ai-think" {
ai++
}
if source.Depth > maxDepth {
maxDepth = source.Depth
}
}
if production+ai > 0 {
ratio = float64(production) / float64(production+ai)
}
return
}
func nodeGenerationDepth(node model.Node) int {
if node.Kind != "ai-think" {
return 0
}
value, ok := node.Metadata["generation_depth"]
if !ok {
return 1
}
switch typed := value.(type) {
case int:
return typed
case int64:
return int(typed)
case float64:
return int(typed)
case json.Number:
n, _ := typed.Int64()
return int(n)
default:
return 1
}
}
func (e *Engine) hasEquivalentArticleDraft(sources []articleSource, plan model.ArticlePlanDecision) bool {
wanted := map[string]bool{}
for _, source := range sources {
wanted[source.Node.ID] = true
}
for _, node := range e.Graph.Snapshot().Nodes {
if node.Kind != "ai-think" || metadataString(node.Metadata, "subtype") != "knowledge_synthesis" {
continue
}
if (plan.Action == "update" || plan.Action == "merge") && metadataString(node.Metadata, "target_node_id") == plan.TargetArticleID {
return true
}
existing := metadataStringSlice(node.Metadata, "source_node_ids")
if len(existing) == 0 {
continue
}
intersection := 0
union := make(map[string]bool, len(wanted)+len(existing))
for id := range wanted {
union[id] = true
}
for _, id := range existing {
if wanted[id] {
intersection++
}
union[id] = true
}
if len(union) > 0 && float64(intersection)/float64(len(union)) >= .70 {
return true
}
}
return false
}
func validProductionTarget(id string, sources []articleSource) bool {
if strings.TrimSpace(id) == "" {
return false
}
for _, source := range sources {
if source.Node.ID == id && source.Node.Kind == "knowledge" && source.Node.Status == "production" {
return true
}
}
return false
}
func safeArticleAction(value string) string {
switch strings.ToLower(strings.TrimSpace(value)) {
case "create", "update", "merge", "skip":
return strings.ToLower(strings.TrimSpace(value))
default:
return "skip"
}
}
func filterArticleSources(sources []articleSource, ids []string) []articleSource {
wanted := map[string]bool{}
for _, id := range ids {
wanted[id] = true
}
var out []articleSource
for _, source := range sources {
if wanted[source.Node.ID] {
out = append(out, source)
}
}
return out
}
func nodeIDsFromArticleSources(sources []articleSource) []string {
out := make([]string, 0, len(sources))
for _, source := range sources {
out = append(out, source.Node.ID)
}
return unique(out)
}
func externalIDsFromArticleSources(sources []articleSource) []string {
out := make([]string, 0, len(sources))
for _, source := range sources {
out = append(out, nonempty(source.Node.ExternalID, source.Node.ID))
}
return unique(out)
}
func sortArticleSources(sources []articleSource) {
sort.Slice(sources, func(i, j int) bool {
if sources[i].Score == sources[j].Score {
return sources[i].Node.ID < sources[j].Node.ID
}
return sources[i].Score > sources[j].Score
})
}
func vectorCentroid(vectors [][]float64) []float64 {
if len(vectors) == 0 {
return nil
}
dim := len(vectors[0])
if dim == 0 {
return nil
}
out := make([]float64, dim)
count := 0
for _, vector := range vectors {
if len(vector) != dim {
continue
}
count++
for i, value := range vector {
out[i] += value
}
}
if count == 0 {
return nil
}
for i := range out {
out[i] /= float64(count)
}
return out
}
func cosineVector(a, b []float64) float64 {
if len(a) == 0 || len(a) != len(b) {
return 0
}
var dot, aa, bb float64
for i := range a {
dot += a[i] * b[i]
aa += a[i] * a[i]
bb += b[i] * b[i]
}
if aa == 0 || bb == 0 {
return 0
}
return dot / (math.Sqrt(aa) * math.Sqrt(bb))
}
func categoryAffinity(node model.Node, seeds []model.Node) float64 {
wanted := map[string]bool{}
for _, seed := range seeds {
for _, category := range seed.Categories {
wanted[strings.ToLower(strings.TrimSpace(category))] = true
}
}
var matches int
for _, category := range node.Categories {
if wanted[strings.ToLower(strings.TrimSpace(category))] {
matches++
}
}
return float64(matches)
}
func matchesCategories(node model.Node, filters []string) bool {
if len(filters) == 0 {
return true
}
wanted := map[string]bool{}
for _, filter := range filters {
wanted[strings.ToLower(strings.TrimSpace(filter))] = true
}
if wanted["*"] {
return true
}
if len(node.Categories) == 0 {
return wanted["__uncategorized__"]
}
for _, category := range node.Categories {
if wanted[strings.ToLower(strings.TrimSpace(category))] {
return true
}
}
return false
}
func metadataString(metadata map[string]any, key string) string {
if metadata == nil {
return ""
}
return strings.TrimSpace(fmt.Sprint(metadata[key]))
}
func metadataStringSlice(metadata map[string]any, key string) []string {
if metadata == nil {
return nil
}
switch value := metadata[key].(type) {
case []string:
return append([]string(nil), value...)
case []any:
out := make([]string, 0, len(value))
for _, item := range value {
if text := strings.TrimSpace(fmt.Sprint(item)); text != "" {
out = append(out, text)
}
}
return out
default:
return nil
}
}
func firstMapString(m map[string]any, keys ...string) string {
for _, key := range keys {
if value, ok := m[key]; ok {
text := strings.TrimSpace(fmt.Sprint(value))
if text != "" && text != "<nil>" {
return text
}
}
}
return ""
}
func isTaxonomyEdge(edgeType string) bool {
switch edgeType {
case "categorized_as", "mentions", "derived_from":
return true
default:
return false
}
}

View File

@@ -3,13 +3,10 @@ package engine
import (
"context"
"crypto/sha256"
"encoding/hex"
"encoding/json"
"errors"
"fmt"
"log/slog"
"math"
"os"
"path/filepath"
"sort"
"strings"
@@ -35,11 +32,14 @@ var (
)
type EnrichOutcome struct {
Result string
Candidate bool
Created bool
Rejected bool
Comparisons int
Result string
Candidate bool
Created bool
RelationCreated bool
ArticleCreated bool
ArticleSkipped bool
Rejected bool
Comparisons int
}
type Engine struct {
@@ -52,24 +52,27 @@ type Engine struct {
GLPIKB *ingest.GLPIKBSyncer
Persistence *persist.Coordinator
mu sync.Mutex
stateMu sync.RWMutex
lastScan time.Time
lastEnrich time.Time
lastAttempt time.Time
nextEnrich time.Time
ollamaOK bool
enrichRunning bool
enrichTrigger string
enrichResult string
enrichError string
enrichCycles uint64
enrichCreated uint64
enrichRejected uint64
enrichRequests chan string
runtimeMu sync.RWMutex
runtime RuntimeSettings
runtimePath string
mu sync.Mutex
stateMu sync.RWMutex
lastScan time.Time
lastEnrich time.Time
lastAttempt time.Time
nextEnrich time.Time
ollamaOK bool
enrichRunning bool
enrichTrigger string
enrichResult string
enrichError string
enrichCycles uint64
enrichCreated uint64
enrichRejected uint64
relationsCreated uint64
articlesCreated uint64
articlesSkipped uint64
enrichRequests chan string
runtimeMu sync.RWMutex
runtime RuntimeSettings
runtimePath string
}
func New(cfg config.Config, g *graph.Store, b *activity.Broker) *Engine {
@@ -84,6 +87,27 @@ func New(cfg config.Config, g *graph.Store, b *activity.Broker) *Engine {
if cfg.EnrichAnchors < 1 {
cfg.EnrichAnchors = 48
}
if cfg.ArticleMinSources < 2 {
cfg.ArticleMinSources = 3
}
if cfg.ArticleMaxSources < cfg.ArticleMinSources {
cfg.ArticleMaxSources = 8
}
if cfg.ArticleMinProductionRatio == 0 {
cfg.ArticleMinProductionRatio = .70
}
if cfg.ArticleMaxGenerationDepth < 1 {
cfg.ArticleMaxGenerationDepth = 2
}
if cfg.ArticleMinConfidence == 0 {
cfg.ArticleMinConfidence = .74
}
if cfg.ArticleMinTextChars < 1 {
cfg.ArticleMinTextChars = 180
}
if cfg.ArticleMinAnswerChars < 1 {
cfg.ArticleMinAnswerChars = 420
}
ollamaURLs := append([]string(nil), cfg.OllamaURLs...)
if len(ollamaURLs) == 0 && strings.TrimSpace(cfg.OllamaURL) != "" {
ollamaURLs = []string{cfg.OllamaURL}
@@ -235,6 +259,7 @@ func (e *Engine) runEnrichmentCycle(ctx context.Context, trigger string) {
e.Broker.Publish(model.Activity{Type: "think.cycle.started", Source: "brain", Phase: "autonomous", Message: fmt.Sprintf("Autonomer AI-THINK-Zyklus startet · bis zu %d sequenzielle Prüfungen", e.Cfg.EnrichBatchSize), Strength: .72, Metadata: map[string]any{"trigger": trigger, "batch_size": e.Cfg.EnrichBatchSize, "anchors": e.Cfg.EnrichAnchors}})
created, rejected, checked := 0, 0, 0
relationsCreated, articlesCreated, articlesSkipped := 0, 0, 0
result := "completed"
var cycleErr error
for step := 0; step < e.Cfg.EnrichBatchSize; step++ {
@@ -258,6 +283,15 @@ func (e *Engine) runEnrichmentCycle(ctx context.Context, trigger string) {
if outcome.Created {
created++
}
if outcome.RelationCreated {
relationsCreated++
}
if outcome.ArticleCreated {
articlesCreated++
}
if outcome.ArticleSkipped {
articlesSkipped++
}
if outcome.Rejected {
rejected++
}
@@ -282,15 +316,18 @@ func (e *Engine) runEnrichmentCycle(ctx context.Context, trigger string) {
}
e.enrichCreated += uint64(created)
e.enrichRejected += uint64(rejected)
e.relationsCreated += uint64(relationsCreated)
e.articlesCreated += uint64(articlesCreated)
e.articlesSkipped += uint64(articlesSkipped)
e.stateMu.Unlock()
metadata := map[string]any{"trigger": trigger, "checked": checked, "created": created, "rejected": rejected, "duration_ms": time.Since(started).Milliseconds(), "result": result}
metadata := map[string]any{"trigger": trigger, "checked": checked, "created": created, "relations_created": relationsCreated, "articles_created": articlesCreated, "articles_skipped": articlesSkipped, "rejected": rejected, "duration_ms": time.Since(started).Milliseconds(), "result": result}
if cycleErr != nil {
e.Broker.Publish(model.Activity{Type: "think.cycle.failed", Source: "brain", Phase: "autonomous", Message: "AI-THINK-Zyklus wurde mit Fehler beendet", Strength: .45, Metadata: metadata})
slog.Warn("enrichment cycle failed", "trigger", trigger, "error", cycleErr)
return
}
message := fmt.Sprintf("AI-THINK-Zyklus abgeschlossen · %d geprüft · %d erstellt · %d verworfen", checked, created, rejected)
message := fmt.Sprintf("AI-THINK-Zyklus abgeschlossen · %d Relationen · %d Artikel · %d verworfen", relationsCreated, articlesCreated, rejected)
if result == "no_candidate" {
message = "AI-THINK hat im aktuell geprüften Graphbereich keinen Kandidaten oberhalb des Schwellwerts gefunden"
}
@@ -525,8 +562,26 @@ func (e *Engine) EnrichOne(ctx context.Context) error {
if !e.ThinkingEnabled() {
return ErrThinkingDisabled
}
_, err := e.enrichOne(ctx, "direct")
return err
outcome, err := e.enrichOne(ctx, "direct")
if err != nil {
return err
}
e.stateMu.Lock()
if outcome.RelationCreated {
e.relationsCreated++
e.enrichCreated++
}
if outcome.ArticleCreated {
e.articlesCreated++
}
if outcome.ArticleSkipped {
e.articlesSkipped++
}
if outcome.Rejected {
e.enrichRejected++
}
e.stateMu.Unlock()
return nil
}
func (e *Engine) enrichOne(ctx context.Context, trigger string) (EnrichOutcome, error) {
@@ -547,7 +602,7 @@ func (e *Engine) enrichOne(ctx context.Context, trigger string) (EnrichOutcome,
e.setOllamaOK(true)
}
a, b, sim, ok, comparisons := e.Graph.NextPairFiltered(e.Cfg.SimilarityThreshold, e.Cfg.EnrichAnchors, e.thinkingCategories())
a, b, sim, ok, comparisons := e.Graph.NextPairFilteredDepth(e.Cfg.SimilarityThreshold, e.Cfg.EnrichAnchors, e.thinkingCategories(), e.Cfg.ArticleMaxGenerationDepth)
if !ok {
e.stateMu.Lock()
e.lastAttempt = time.Now().UTC()
@@ -563,7 +618,7 @@ func (e *Engine) enrichOne(ctx context.Context, trigger string) (EnrichOutcome,
e.stateMu.Unlock()
e.Broker.Publish(model.Activity{Type: "think.started", Source: "brain", Phase: "association", NodeIDs: []string{a.ID, b.ID}, Message: fmt.Sprintf("Verwandtschaft wird geprüft · %.0f%% semantische Nähe", sim*100), Strength: .88, Metadata: map[string]any{"trigger": trigger, "semantic_similarity": sim, "source_label": a.Label, "target_label": b.Label, "model": e.Cfg.ChatModel, "candidate_comparisons": comparisons}})
system := "Analysiere zwei interne Wissenseinträge. Erfinde keine Fakten. Entscheide, ob eine belastbare Beziehung besteht. Wenn externe Fakten fehlen, setze needs_research=true. Gib ausschließlich JSON nach Schema zurück."
system := "Du führst ausschließlich eine Relationserkennung für einen Wissensgraphen durch. Analysiere zwei interne Wissenseinträge, erfinde keine Fakten und entscheide, ob eine belastbare Beziehung besteht. Schreibe keinen Artikel und keine technische Synthese. Wenn externe Fakten zur Relationsentscheidung fehlen, setze needs_research=true. Gib ausschließlich JSON nach Schema zurück."
var decision model.RelationDecision
if err := e.Ollama.ChatJSON(ctx, system, relationContext(a, b, sim), relationSchema(), &decision); err != nil {
e.Broker.Publish(model.Activity{Type: "think.failed", Source: "brain", Phase: "inference", NodeIDs: []string{a.ID, b.ID}, Message: "Qwen-Beziehungsanalyse ist fehlgeschlagen; es wurde nichts gespeichert", Strength: .35, Metadata: map[string]any{"trigger": trigger, "error": err.Error(), "model": e.Cfg.ChatModel}})
@@ -603,12 +658,17 @@ func (e *Engine) enrichOne(ctx context.Context, trigger string) (EnrichOutcome,
edge.ID = graph.EdgeID(edge.Source, edge.Target, edge.Type, edge.Origin)
outcome := EnrichOutcome{Result: status, Candidate: true, Comparisons: comparisons}
if status == "staging" {
path, err := e.writeAIThink(a, b, decision, researchResults)
if err != nil {
return outcome, err
}
outcome.Created = true
e.Broker.Publish(model.Activity{Type: "think.created", Source: "brain", Phase: "staging", NodeIDs: []string{a.ID, b.ID}, EdgeIDs: []string{edge.ID}, Message: "Neuer AI-THINK-Beitrag wurde erzeugt und für den gebündelten Staging-Schreibvorgang vorgemerkt", Strength: 1, Metadata: map[string]any{"trigger": trigger, "path": path, "relation_type": safeRelation(decision.RelationType), "confidence": decision.Confidence, "semantic_similarity": sim, "research_result_count": len(researchResults), "title": decision.Title, "write_pending": true}})
outcome.RelationCreated = true
e.Broker.Publish(model.Activity{Type: "think.relation.created", Source: "brain", Phase: "relation", NodeIDs: []string{a.ID, b.ID}, EdgeIDs: []string{edge.ID}, Message: "Belastbare Wissensrelation wurde als überprüfbare Graph-Edge übernommen", Strength: .86, Metadata: map[string]any{"trigger": trigger, "relation_type": safeRelation(decision.RelationType), "confidence": decision.Confidence, "semantic_similarity": sim, "research_result_count": len(researchResults), "topic_label": decision.TopicLabel}})
articleOutcome, err := e.synthesizeKnowledgeArticle(ctx, trigger, []model.Node{a, b}, decision, researchResults)
if err != nil {
e.Broker.Publish(model.Activity{Type: "article.failed", Source: "brain", Phase: "knowledge-synthesis", NodeIDs: []string{a.ID, b.ID}, EdgeIDs: []string{edge.ID}, Message: "Die Relation bleibt erhalten, aber die Artikelsynthese ist fehlgeschlagen", Strength: .4, Metadata: map[string]any{"trigger": trigger, "error": err.Error()}})
outcome.ArticleSkipped = true
} else {
outcome.ArticleCreated = articleOutcome.Created
outcome.ArticleSkipped = articleOutcome.Skipped
}
} else {
outcome.Rejected = true
e.Broker.Publish(model.Activity{Type: "think.rejected", Source: "brain", Phase: "validation", NodeIDs: []string{a.ID, b.ID}, Message: "Ähnlichkeit geprüft, aber nicht als belastbare Edge übernommen", Strength: .42, Metadata: map[string]any{"trigger": trigger, "relation_type": safeRelation(decision.RelationType), "confidence": decision.Confidence, "semantic_similarity": sim, "explanation": decision.Explanation}})
@@ -625,37 +685,6 @@ func (e *Engine) addResearch(a, b model.Node, results []model.ResearchResult) {
e.Graph.UpsertEdge(model.Edge{Source: id, Target: b.ID, Type: "research_evidence", Origin: "research", Status: "staging", Confidence: .55, Weight: .4})
}
}
func (e *Engine) writeAIThink(a, b model.Node, d model.RelationDecision, results []model.ResearchResult) (string, error) {
if len(e.Cfg.StagingDirs) == 0 {
return "", fmt.Errorf("no BRAIN_STAGING_DIRS configured")
}
dir := e.Cfg.StagingDirs[0]
pair := a.ID + "\x00" + b.ID
sum := sha256.Sum256([]byte(pair))
short := strings.ToUpper(hex.EncodeToString(sum[:5]))
now := time.Now().UTC()
id := fmt.Sprintf("KB-AI-THINK-%s-%s", now.Format("20060102"), short)
cats := unique(append([]string{"AI-THINK", "AI-Staging"}, common(a.Categories, b.Categories)...))
keywords := unique(append(append([]string{}, d.Keywords...), first(a.Keywords, 4)...))
keywords = unique(append(keywords, first(b.Keywords, 4)...))
var evidence []map[string]any
for _, r := range results {
evidence = append(evidence, map[string]any{"title": r.Title, "url": r.URL, "excerpt": clamp(r.Content, 300)})
}
doc := map[string]any{"id": id, "title": nonempty(d.Title, "Zusammenhang: "+a.Label+" ↔ "+b.Label), "text": nonempty(d.Synthesis, d.Explanation), "answer": "Interne AI-THINK-Arbeitsnotiz. Vor produktiver Nutzung im Editor prüfen, korrigieren und freigeben.", "auto_reply": false, "min_score": 0.78, "categories": cats, "keywords": keywords, "source": "Neural Brain / " + e.Cfg.ChatModel + " (AI-THINK)", "source_uri": "brain://edge/" + short, "language": "de-DE", "communication_style": "formal", "ai_think": map[string]any{"status": "staging", "generated_at": now, "source_nodes": []string{a.ExternalID, b.ExternalID}, "source_node_ids": []string{a.ID, b.ID}, "relation_type": safeRelation(d.RelationType), "confidence": d.Confidence, "explanation": d.Explanation, "needs_research": d.NeedsResearch, "research_query": d.ResearchQuery, "research_evidence": evidence}}
bts, err := json.MarshalIndent(doc, "", " ")
if err != nil {
return "", err
}
path := filepath.Join(dir, strings.ToLower(id)+".json")
if e.Persistence.Pending(path) {
return path, nil
}
if _, err := os.Stat(path); err == nil {
return path, nil
}
return e.Persistence.QueueFile(path, append(bts, '\n'), 0o640)
}
func (e *Engine) Status() map[string]any {
s := e.Graph.Snapshot()
e.stateMu.RLock()
@@ -665,7 +694,11 @@ func (e *Engine) Status() map[string]any {
"next_enrich": e.nextEnrich, "ollama_ok": e.ollamaOK, "auto_enrich": e.Cfg.AutoEnrich,
"enrich_running": e.enrichRunning, "enrich_trigger": e.enrichTrigger, "enrich_result": e.enrichResult,
"enrich_error": e.enrichError, "enrich_cycles": e.enrichCycles, "enrich_created": e.enrichCreated,
"enrich_rejected": e.enrichRejected, "enrich_interval": e.Cfg.EnrichInterval.String(),
"enrich_rejected": e.enrichRejected, "relations_created": e.relationsCreated, "articles_created": e.articlesCreated,
"articles_skipped": e.articlesSkipped, "article_synthesis_enabled": e.Cfg.ArticleSynthesisEnabled,
"article_min_sources": e.Cfg.ArticleMinSources, "article_max_sources": e.Cfg.ArticleMaxSources,
"article_min_production_ratio": e.Cfg.ArticleMinProductionRatio, "article_max_generation_depth": e.Cfg.ArticleMaxGenerationDepth,
"enrich_interval": e.Cfg.EnrichInterval.String(),
"enrich_batch_size": e.Cfg.EnrichBatchSize, "enrich_anchors": e.Cfg.EnrichAnchors,
"research_enabled": e.Cfg.ResearchEnabled, "chat_model": e.Cfg.ChatModel, "embedding_model": e.Cfg.EmbeddingModel,
"ollama_pool": e.Ollama.PoolStatus(), "persistence": e.Persistence.Status(),
@@ -708,7 +741,7 @@ func relationContext(a, b model.Node, sim float64) string {
return fmt.Sprintf("SEMANTISCHE_NÄHE: %.4f\n\nA\nID: %s\nTitel: %s\nKategorien: %s\nInhalt: %s\n\nB\nID: %s\nTitel: %s\nKategorien: %s\nInhalt: %s", sim, a.ID, a.Label, strings.Join(a.Categories, ", "), a.Summary, b.ID, b.Label, strings.Join(b.Categories, ", "), b.Summary)
}
func relationSchema() map[string]any {
return map[string]any{"type": "object", "properties": map[string]any{"related": map[string]any{"type": "boolean"}, "relation_type": map[string]any{"type": "string", "enum": []string{"related_to", "depends_on", "supports", "contradicts", "extends", "same_topic", "caused_by"}}, "confidence": map[string]any{"type": "number", "minimum": 0, "maximum": 1}, "explanation": map[string]any{"type": "string"}, "needs_research": map[string]any{"type": "boolean"}, "research_query": map[string]any{"type": "string"}, "title": map[string]any{"type": "string"}, "synthesis": map[string]any{"type": "string"}, "keywords": map[string]any{"type": "array", "items": map[string]any{"type": "string"}}}, "required": []string{"related", "relation_type", "confidence", "explanation", "needs_research", "research_query", "title", "synthesis", "keywords"}}
return map[string]any{"type": "object", "properties": map[string]any{"related": map[string]any{"type": "boolean"}, "relation_type": map[string]any{"type": "string", "enum": []string{"related_to", "depends_on", "supports", "contradicts", "extends", "same_topic", "caused_by"}}, "confidence": map[string]any{"type": "number", "minimum": 0, "maximum": 1}, "explanation": map[string]any{"type": "string"}, "needs_research": map[string]any{"type": "boolean"}, "research_query": map[string]any{"type": "string"}, "topic_label": map[string]any{"type": "string"}, "keywords": map[string]any{"type": "array", "items": map[string]any{"type": "string"}}}, "required": []string{"related", "relation_type", "confidence", "explanation", "needs_research", "research_query", "topic_label", "keywords"}}
}
func answerSchema() map[string]any {
return map[string]any{"type": "object", "properties": map[string]any{"answer": map[string]any{"type": "string"}, "used_node_ids": map[string]any{"type": "array", "items": map[string]any{"type": "string"}}, "uncertainties": map[string]any{"type": "array", "items": map[string]any{"type": "string"}}}, "required": []string{"answer", "used_node_ids", "uncertainties"}}

View File

@@ -7,15 +7,18 @@ import (
"net/http/httptest"
"os"
"path/filepath"
"strings"
"testing"
"time"
"github.com/local/glpi-neural-brain/internal/activity"
"github.com/local/glpi-neural-brain/internal/config"
"github.com/local/glpi-neural-brain/internal/graph"
"github.com/local/glpi-neural-brain/internal/model"
)
func TestEnrichWritesAIThinkOnlyAfterStructuredQwenDecision(t *testing.T) {
func TestEnrichCreatesStructuredKnowledgeArticleFromThreeProductionSources(t *testing.T) {
var chatCalls int
mock := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
w.Header().Set("Content-Type", "application/json")
switch r.URL.Path {
@@ -28,11 +31,20 @@ func TestEnrichWritesAIThinkOnlyAfterStructuredQwenDecision(t *testing.T) {
_ = json.NewDecoder(r.Body).Decode(&req)
vectors := make([][]float64, len(req.Input))
for i := range vectors {
vectors[i] = []float64{1, float64(i) * .05, 0}
vectors[i] = []float64{1, float64(i) * .03, 0}
}
_ = json.NewEncoder(w).Encode(map[string]any{"embeddings": vectors})
case "/api/chat":
content := `{"related":true,"relation_type":"supports","confidence":0.91,"explanation":"Beide Einträge behandeln denselben VPN-Störungsablauf.","needs_research":false,"research_query":"","title":"VPN-Gateway und Remotezugriff","synthesis":"Der Gateway-Fehler ist ein konkreter Teilbereich des Remotezugriffs.","keywords":["VPN","Gateway"]}`
chatCalls++
var content string
switch chatCalls {
case 1:
content = `{"related":true,"relation_type":"supports","confidence":0.91,"explanation":"Die drei produktiven Quellen beschreiben zusammen Diagnose, Ursache und Behebung derselben VPN-Störung.","needs_research":false,"research_query":"","topic_label":"VPN-Gateway-Störung","keywords":["VPN","Gateway"]}`
case 2:
content = `{"action":"create","target_article_id":"","reason":"Die Quellen ergänzen sich zu einer vollständigen Anleitung, die noch nicht als einzelner Artikel existiert.","expected_value":"Ein durchgängiger Diagnose- und Lösungsablauf für den Support.","article_type":"troubleshooting","source_node_ids":[],"missing_information":[],"contradictions":[],"needs_research":false,"research_query":""}`
default:
content = `{"title":"VPN-Gateway-Störung systematisch beheben","text":"Dieser Artikel gilt für Remotezugriffe, bei denen der VPN-Client keine Verbindung zum Gateway herstellen kann. Typische Symptome sind Zeitüberschreitungen, ein nicht erreichbares Gateway oder ein Verbindungsabbruch unmittelbar nach dem Start. Die Anleitung grenzt lokale Clientfehler von einer Störung des zentralen Gateways ab.","answer":"1. Prüfen Sie zuerst, ob der betroffene Rechner eine funktionierende Internetverbindung besitzt. 2. Kontrollieren Sie, ob der konfigurierte Gateway-Name erreichbar ist. 3. Vergleichen Sie die Fehlermeldung mit dem dokumentierten Gateway-Ausfallbild. 4. Starten Sie den VPN-Client neu und führen Sie den Verbindungsversuch erneut aus. 5. Bleibt das Gateway nicht erreichbar, eskalieren Sie den Fall mit Zeitstempel und Fehlermeldung an das Netzwerkteam.","prerequisites":["Zugriff auf die VPN-Client-Konfiguration","Aktuelle Fehlermeldung des Benutzers"],"validation":["Der VPN-Tunnel wird aufgebaut","Die interne Zielressource ist erreichbar"],"troubleshooting":["Bei weiterhin nicht erreichbarem Gateway den dokumentierten Eskalationsweg verwenden"],"categories":["Netzwerk","VPN"],"keywords":["VPN","Gateway","Remotezugriff"],"source_node_ids":[],"confidence":0.92,"open_questions":[]}`
}
_ = json.NewEncoder(w).Encode(map[string]any{"message": map[string]any{"content": content}})
default:
http.NotFound(w, r)
@@ -54,15 +66,16 @@ func TestEnrichWritesAIThinkOnlyAfterStructuredQwenDecision(t *testing.T) {
t.Fatal(err)
}
}
write("vpn.json", `{"id":"KB-VPN","title":"VPN Gateway","text":"Gateway nicht erreichbar","categories":["Netzwerk"],"keywords":["VPN","Gateway"],"source":"internal-kb"}`)
write("remote.json", `{"id":"KB-REMOTE","title":"Remotezugriff und VPN","text":"Remotezugriff über VPN und Gateway","categories":["Netzwerk"],"keywords":["VPN","Remotezugriff"],"source":"internal-kb"}`)
write("vpn.json", `{"id":"KB-VPN","title":"VPN Gateway nicht erreichbar","text":"Der Client meldet, dass das Gateway nicht erreichbar ist.","answer":"Internetverbindung und Gateway-Adresse prüfen.","categories":["Netzwerk","VPN"],"keywords":["VPN","Gateway"],"source":"internal-kb"}`)
write("remote.json", `{"id":"KB-REMOTE","title":"Remotezugriff über VPN diagnostizieren","text":"Remotezugriff schlägt mit einem Timeout fehl.","answer":"Fehlermeldung erfassen, Client neu starten und Tunnel prüfen.","categories":["Netzwerk","VPN"],"keywords":["VPN","Remotezugriff"],"source":"internal-kb"}`)
write("escalation.json", `{"id":"KB-VPN-ESC","title":"VPN-Störungen eskalieren","text":"Anhaltende Gateway-Ausfälle benötigen eine strukturierte Eskalation.","answer":"Zeitstempel, Fehlermeldung und betroffene Benutzer an das Netzwerkteam übergeben.","categories":["Netzwerk","VPN"],"keywords":["VPN","Eskalation"],"source":"internal-kb"}`)
g, err := graph.Open(data)
if err != nil {
t.Fatal(err)
}
cfg := config.Config{DataDir: data, KnowledgeDirs: []string{knowledge}, StagingDirs: []string{staging}, OllamaURL: mock.URL, ChatModel: "qwen3:8b", EmbeddingModel: "embeddinggemma", ScanInterval: time.Minute, EnrichInterval: time.Minute, SimilarityThreshold: .5, RelationThreshold: .7, TopK: 5, MaxContextChars: 8000}
e := New(cfg, g, activity.New(20))
cfg := config.Config{DataDir: data, KnowledgeDirs: []string{knowledge}, StagingDirs: []string{staging}, OllamaURL: mock.URL, ChatModel: "qwen3:8b", EmbeddingModel: "embeddinggemma", ScanInterval: time.Minute, EnrichInterval: time.Minute, SimilarityThreshold: .5, RelationThreshold: .7, ArticleSynthesisEnabled: true, ArticleMinSources: 3, ArticleMaxSources: 6, ArticleMinProductionRatio: .7, ArticleMaxGenerationDepth: 2, ArticleMinConfidence: .7, ArticleMinTextChars: 80, ArticleMinAnswerChars: 180, TopK: 5, MaxContextChars: 16000}
e := New(cfg, g, activity.New(50))
if err := e.Scan(context.Background()); err != nil {
t.Fatal(err)
}
@@ -75,9 +88,12 @@ func TestEnrichWritesAIThinkOnlyAfterStructuredQwenDecision(t *testing.T) {
if err := e.Flush(context.Background()); err != nil {
t.Fatal(err)
}
if chatCalls != 3 {
t.Fatalf("expected relation, plan and draft calls, got %d", chatCalls)
}
files, err := filepath.Glob(filepath.Join(staging, "*.json"))
if err != nil || len(files) != 1 {
t.Fatalf("expected one AI-THINK file, files=%v err=%v", files, err)
t.Fatalf("expected one structured AI-THINK article, files=%v err=%v", files, err)
}
var doc map[string]any
b, _ := os.ReadFile(files[0])
@@ -87,15 +103,68 @@ func TestEnrichWritesAIThinkOnlyAfterStructuredQwenDecision(t *testing.T) {
if doc["auto_reply"] != false {
t.Fatalf("AI-THINK must be auto_reply=false: %#v", doc["auto_reply"])
}
cats, _ := doc["categories"].([]any)
found := false
for _, c := range cats {
if c == "AI-THINK" {
found = true
if got := doc["answer"].(string); len(got) < 180 || got == "Interne AI-THINK-Arbeitsnotiz. Vor produktiver Nutzung im Editor prüfen, korrigieren und freigeben." {
t.Fatalf("expected a real solution article, got %q", got)
}
ai, ok := doc["ai_think"].(map[string]any)
if !ok || ai["subtype"] != "knowledge_synthesis" || ai["action"] != "create" {
t.Fatalf("unexpected AI-THINK metadata: %#v", doc["ai_think"])
}
if ai["productive_source_count"].(float64) < 3 {
t.Fatalf("expected at least three productive sources: %#v", ai)
}
}
func TestEnrichRelationWithOnlyTwoSourcesDoesNotCreateArticle(t *testing.T) {
var chatCalls int
mock := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
w.Header().Set("Content-Type", "application/json")
switch r.URL.Path {
case "/api/tags":
_, _ = w.Write([]byte(`{"models":[{"name":"qwen3:8b","digest":"chat-digest"},{"name":"embeddinggemma:latest","digest":"embed-digest"}]}`))
case "/api/embed":
var req struct {
Input []string `json:"input"`
}
_ = json.NewDecoder(r.Body).Decode(&req)
vectors := make([][]float64, len(req.Input))
for i := range vectors {
vectors[i] = []float64{1, float64(i) * .02}
}
_ = json.NewEncoder(w).Encode(map[string]any{"embeddings": vectors})
case "/api/chat":
chatCalls++
content := `{"related":true,"relation_type":"same_topic","confidence":0.9,"explanation":"Beide Quellen behandeln dasselbe Gateway-Thema.","needs_research":false,"research_query":"","topic_label":"VPN","keywords":["VPN"]}`
_ = json.NewEncoder(w).Encode(map[string]any{"message": map[string]any{"content": content}})
}
}))
defer mock.Close()
root := t.TempDir()
knowledge, staging, data := filepath.Join(root, "knowledge"), filepath.Join(root, "staging"), filepath.Join(root, "data")
for _, dir := range []string{knowledge, staging, data} {
if err := os.MkdirAll(dir, 0o755); err != nil {
t.Fatal(err)
}
}
if !found {
t.Fatalf("AI-THINK category missing: %#v", cats)
_ = os.WriteFile(filepath.Join(knowledge, "a.json"), []byte(`{"id":"A","title":"VPN A","text":"Gateway","categories":["VPN"]}`), 0o644)
_ = os.WriteFile(filepath.Join(knowledge, "b.json"), []byte(`{"id":"B","title":"VPN B","text":"Gateway","categories":["VPN"]}`), 0o644)
g, _ := graph.Open(data)
e := New(config.Config{DataDir: data, KnowledgeDirs: []string{knowledge}, StagingDirs: []string{staging}, OllamaURL: mock.URL, ChatModel: "qwen3:8b", EmbeddingModel: "embeddinggemma", SimilarityThreshold: .5, RelationThreshold: .7, ArticleSynthesisEnabled: true, ArticleMinSources: 3, ArticleMaxSources: 8, ArticleMinProductionRatio: .7, ArticleMaxGenerationDepth: 2, ArticleMinConfidence: .7, ArticleMinTextChars: 80, ArticleMinAnswerChars: 180, MaxContextChars: 8000}, g, activity.New(20))
if err := e.Scan(context.Background()); err != nil {
t.Fatal(err)
}
if err := e.EnrichOne(context.Background()); err != nil {
t.Fatal(err)
}
if chatCalls != 1 {
t.Fatalf("article planning must not run with only two productive sources; calls=%d", chatCalls)
}
files, _ := filepath.Glob(filepath.Join(staging, "*.json"))
if len(files) != 0 {
t.Fatalf("unexpected article draft: %v", files)
}
if len(g.Snapshot().Edges) == 0 {
t.Fatal("relation edge should still be created")
}
}
@@ -132,11 +201,12 @@ func TestRuntimeSettingsDisableThinkingAndPersist(t *testing.T) {
DisplayCategories: []string{"GLPI KB"},
ThinkingCategories: []string{"Netzwerk"},
ViewMode: "honeycomb",
MaxDisplayNodes: 4321,
})
if err != nil {
t.Fatal(err)
}
if updated.LearningEnabled || updated.ThinkingEnabled || updated.ViewMode != "honeycomb" {
if updated.LearningEnabled || updated.ThinkingEnabled || updated.ViewMode != "honeycomb" || updated.MaxDisplayNodes != 4321 {
t.Fatalf("unexpected runtime settings: %+v", updated)
}
if e.RequestEnrich("manual") {
@@ -155,7 +225,91 @@ func TestRuntimeSettingsDisableThinkingAndPersist(t *testing.T) {
}
e2 := New(cfg, g2, activity.New(20))
loaded := e2.RuntimeSettings()
if loaded.LearningEnabled || loaded.ThinkingEnabled || loaded.ViewMode != "honeycomb" || len(loaded.DisplayCategories) != 1 {
if loaded.LearningEnabled || loaded.ThinkingEnabled || loaded.ViewMode != "honeycomb" || loaded.MaxDisplayNodes != 4321 || len(loaded.DisplayCategories) != 1 {
t.Fatalf("runtime settings were not restored: %+v", loaded)
}
}
func TestWriteKnowledgeArticleDraftPreservesUpdateTarget(t *testing.T) {
root := t.TempDir()
staging := filepath.Join(root, "staging")
data := filepath.Join(root, "data")
if err := os.MkdirAll(staging, 0o755); err != nil {
t.Fatal(err)
}
g, err := graph.Open(data)
if err != nil {
t.Fatal(err)
}
e := New(config.Config{DataDir: data, StagingDirs: []string{staging}, ChatModel: "qwen3:8b", ArticleMinSources: 3, ArticleMaxSources: 8, ArticleMinProductionRatio: .7, ArticleMaxGenerationDepth: 2, ArticleMinConfidence: .7, ArticleMinTextChars: 50, ArticleMinAnswerChars: 100}, g, activity.New(20))
target := model.Node{ID: graph.ID("knowledge", "KB-TARGET"), Kind: "knowledge", Status: "production", ExternalID: "KB-TARGET", Label: "Bestehender VPN-Artikel", Categories: []string{"VPN"}}
sources := []articleSource{
{Node: target, Content: "Bestehender Artikel", Depth: 0},
{Node: model.Node{ID: graph.ID("knowledge", "KB-SOURCE-2"), Kind: "knowledge", Status: "production", ExternalID: "KB-SOURCE-2", Label: "Diagnose", Categories: []string{"VPN"}}, Content: "Diagnosequelle"},
{Node: model.Node{ID: graph.ID("knowledge", "KB-SOURCE-3"), Kind: "knowledge", Status: "production", ExternalID: "KB-SOURCE-3", Label: "Prüfung", Categories: []string{"VPN"}}, Content: "Prüfquelle"},
}
plan := model.ArticlePlanDecision{Action: "update", TargetArticleID: target.ID, Reason: "Der bestehende Artikel benötigt Diagnose und Validierung.", ExpectedValue: "Vollständiger Ablauf", ArticleType: "troubleshooting", SourceNodeIDs: nodeIDsFromArticleSources(sources)}
draft := model.KnowledgeArticleDraft{Title: "VPN-Artikel vollständig diagnostizieren", Text: strings.Repeat("Problem und Geltungsbereich. ", 5), Answer: strings.Repeat("1. Konkreten Prüfschritt ausführen. ", 8), Validation: []string{"VPN-Tunnel ist aktiv"}, Categories: []string{"VPN"}, Keywords: []string{"VPN"}, SourceNodeIDs: nodeIDsFromArticleSources(sources), Confidence: .9}
path, _, created, err := e.writeKnowledgeArticleDraft(sources, plan, draft, nil, 3, 0, 1, 1)
if err != nil {
t.Fatal(err)
}
if !created {
t.Fatal("expected update draft to be queued")
}
if err := e.Flush(context.Background()); err != nil {
t.Fatal(err)
}
dataBytes, err := os.ReadFile(path)
if err != nil {
t.Fatal(err)
}
var doc map[string]any
if err := json.Unmarshal(dataBytes, &doc); err != nil {
t.Fatal(err)
}
ai := doc["ai_think"].(map[string]any)
if ai["action"] != "update" || ai["target_article_id"] != "KB-TARGET" || ai["target_node_id"] != target.ID {
t.Fatalf("update target metadata was not preserved: %#v", ai)
}
if !strings.Contains(doc["answer"].(string), "## Ergebnis prüfen") {
t.Fatalf("validation section missing from answer: %q", doc["answer"])
}
}
func TestSourceContentReloadsFullLocalKnowledgeDocument(t *testing.T) {
root := t.TempDir()
knowledge := filepath.Join(root, "knowledge")
data := filepath.Join(root, "data")
if err := os.MkdirAll(knowledge, 0o755); err != nil {
t.Fatal(err)
}
longProblem := strings.Repeat("Ausführliche Problembeschreibung. ", 80)
longAnswer := strings.Repeat("Konkreter Lösungsschritt. ", 80)
body, _ := json.Marshal(map[string]any{"id": "KB-LONG", "title": "Langer Artikel", "text": longProblem, "answer": longAnswer, "categories": []string{"Test"}})
if err := os.WriteFile(filepath.Join(knowledge, "long.json"), body, 0o644); err != nil {
t.Fatal(err)
}
g, err := graph.Open(data)
if err != nil {
t.Fatal(err)
}
e := New(config.Config{DataDir: data, KnowledgeDirs: []string{knowledge}, ArticleMinSources: 3, ArticleMaxSources: 8, ArticleMinProductionRatio: .7, ArticleMaxGenerationDepth: 2}, g, activity.New(20))
if _, err := e.Scanner.Scan(); err != nil {
t.Fatal(err)
}
var node model.Node
for _, candidate := range g.Snapshot().Nodes {
if candidate.ExternalID == "KB-LONG" {
node = candidate
break
}
}
if node.ID == "" {
t.Fatal("knowledge node not found")
}
full := e.sourceContent(node)
if !strings.Contains(full, "PROBLEM / BESCHREIBUNG:") || !strings.Contains(full, "LÖSUNG / ANTWORT:") || len(full) <= len(node.Summary) {
t.Fatalf("expected full source content, summary=%d full=%d", len(node.Summary), len(full))
}
}

View File

@@ -2,6 +2,7 @@ package engine
import (
"encoding/json"
"fmt"
"os"
"sort"
"strings"
@@ -18,6 +19,7 @@ type RuntimeSettings struct {
DisplayCategories []string `json:"display_categories"`
ThinkingCategories []string `json:"thinking_categories"`
ViewMode string `json:"view_mode"`
MaxDisplayNodes int `json:"max_display_nodes"`
}
type CategoryInfo struct {
@@ -33,6 +35,7 @@ func (e *Engine) defaultRuntimeSettings() RuntimeSettings {
DisplayCategories: append([]string(nil), e.Cfg.DisplayCategories...),
ThinkingCategories: append([]string(nil), e.Cfg.ThinkingCategories...),
ViewMode: e.Cfg.DefaultView,
MaxDisplayNodes: e.Cfg.MaxDisplayNodes,
})
}
@@ -47,6 +50,12 @@ func normalizeRuntimeSettings(in RuntimeSettings) RuntimeSettings {
if in.ViewMode != "neural" && in.ViewMode != "honeycomb" {
in.ViewMode = "neural"
}
if in.MaxDisplayNodes < 0 {
in.MaxDisplayNodes = 0
}
if in.MaxDisplayNodes > 500000 {
in.MaxDisplayNodes = 500000
}
return in
}
@@ -96,6 +105,9 @@ func (e *Engine) RuntimeSettings() RuntimeSettings {
}
func (e *Engine) SetRuntimeSettings(settings RuntimeSettings) (RuntimeSettings, error) {
if settings.MaxDisplayNodes < 0 || settings.MaxDisplayNodes > 500000 {
return e.RuntimeSettings(), fmt.Errorf("max_display_nodes must be between 0 and 500000")
}
settings = normalizeRuntimeSettings(settings)
e.runtimeMu.Lock()
previous := e.runtime
@@ -136,6 +148,7 @@ func (e *Engine) SetRuntimeSettings(settings RuntimeSettings) (RuntimeSettings,
"display_categories": len(settings.DisplayCategories),
"thinking_categories": len(settings.ThinkingCategories),
"view_mode": settings.ViewMode,
"max_display_nodes": settings.MaxDisplayNodes,
},
})
return settings, nil

View File

@@ -343,6 +343,10 @@ func (s *Store) NextPair(min float64, anchorLimit int) (model.Node, model.Node,
}
func (s *Store) NextPairFiltered(min float64, anchorLimit int, categories []string) (model.Node, model.Node, float64, bool, int) {
return s.NextPairFilteredDepth(min, anchorLimit, categories, 0)
}
func (s *Store) NextPairFilteredDepth(min float64, anchorLimit int, categories []string, maxAIDepth int) (model.Node, model.Node, float64, bool, int) {
s.mu.Lock()
defer s.mu.Unlock()
@@ -354,6 +358,9 @@ func (s *Store) NextPairFiltered(min float64, anchorLimit int, categories []stri
if !nodeMatchesCategories(n, categories) {
continue
}
if n.Kind == "ai-think" && maxAIDepth > 0 && graphNodeGenerationDepth(n) >= maxAIDepth {
continue
}
if v, ok := s.vectors[n.ID]; ok && len(v) > 0 {
nodes = append(nodes, n)
}
@@ -384,6 +391,9 @@ func (s *Store) NextPairFiltered(min float64, anchorLimit int, categories []stri
continue
}
right := nodes[j]
if left.Kind == "ai-think" && right.Kind == "ai-think" {
continue
}
if _, exists := blocked[pairKey(left.ID, right.ID)]; exists {
continue
}
@@ -482,6 +492,29 @@ func nodeMatchesCategories(n model.Node, filters []string) bool {
return false
}
func graphNodeGenerationDepth(n model.Node) int {
if n.Kind != "ai-think" {
return 0
}
value, ok := n.Metadata["generation_depth"]
if !ok {
return 1
}
switch typed := value.(type) {
case int:
return typed
case int64:
return int(typed)
case float64:
return int(typed)
case json.Number:
value, _ := typed.Int64()
return int(value)
default:
return 1
}
}
func edgeBetweenLocked(edges map[string]model.Edge, a, b string) bool {
for _, e := range edges {
if (e.Source == a && e.Target == b) || (e.Source == b && e.Target == a) {

View File

@@ -2,6 +2,8 @@ package ingest
import (
"context"
"crypto/sha256"
"encoding/hex"
"encoding/json"
"fmt"
htmlstd "html"
@@ -63,17 +65,19 @@ type GLPIKBSyncer struct {
documents int
nextSync time.Time
lastFingerprint string
contents map[string]string
}
type glpiKBCache struct {
SyncedAt time.Time `json:"synced_at"`
Path string `json:"path"`
Nodes []model.Node `json:"nodes"`
Edges []model.Edge `json:"edges"`
SyncedAt time.Time `json:"synced_at"`
Path string `json:"path"`
Nodes []model.Node `json:"nodes"`
Edges []model.Edge `json:"edges"`
Contents map[string]string `json:"contents,omitempty"`
}
func NewGLPIKBSyncer(cfg GLPIKBConfig, client GLPIKBSource, g *graph.Store, b *activity.Broker, p *persist.Coordinator) *GLPIKBSyncer {
return &GLPIKBSyncer{cfg: cfg, client: client, graph: g, broker: b, persistence: p}
return &GLPIKBSyncer{cfg: cfg, client: client, graph: g, broker: b, persistence: p, contents: map[string]string{}}
}
func (s *GLPIKBSyncer) Start(ctx context.Context) {
@@ -141,13 +145,14 @@ func (s *GLPIKBSyncer) LoadCache() error {
return err
}
s.graph.ReplaceOrigins([]string{"glpi-kb", "glpi-kb-taxonomy"}, cache.Nodes, cache.Edges)
fingerprint, _ := knowledgeFingerprint(cache.Nodes, cache.Edges)
fingerprint, _ := glpiKnowledgeFingerprint(cache.Nodes, cache.Edges, cache.Contents)
s.mu.Lock()
s.path = cache.Path
s.lastSync = cache.SyncedAt
s.documents = countKnowledge(cache.Nodes)
s.lastError = ""
s.lastFingerprint = fingerprint
s.contents = cloneStringMap(cache.Contents)
s.mu.Unlock()
return nil
}
@@ -171,8 +176,8 @@ func (s *GLPIKBSyncer) Sync(ctx context.Context, trigger string) error {
if err != nil {
return fmt.Errorf("load GLPI categories: %w", err)
}
nodes, edges := s.buildGraph(items, categories)
fingerprint, err := knowledgeFingerprint(nodes, edges)
nodes, edges, contents := s.buildGraph(items, categories)
fingerprint, err := glpiKnowledgeFingerprint(nodes, edges, contents)
if err != nil {
return err
}
@@ -193,7 +198,7 @@ func (s *GLPIKBSyncer) Sync(ctx context.Context, trigger string) error {
return nil
}
s.graph.ReplaceOrigins([]string{"glpi-kb", "glpi-kb-taxonomy"}, nodes, edges)
cache := glpiKBCache{SyncedAt: now, Path: path, Nodes: nodes, Edges: edges}
cache := glpiKBCache{SyncedAt: now, Path: path, Nodes: nodes, Edges: edges, Contents: contents}
if s.persistence != nil && s.cfg.CachePath != "" {
data, err := json.Marshal(cache)
if err != nil {
@@ -209,6 +214,7 @@ func (s *GLPIKBSyncer) Sync(ctx context.Context, trigger string) error {
s.lastError = ""
s.documents = len(items)
s.lastFingerprint = fingerprint
s.contents = cloneStringMap(contents)
s.mu.Unlock()
if s.broker != nil {
s.broker.Publish(model.Activity{Type: "glpi.kb.synced", Source: "glpi", Phase: "knowledge-sync", Message: fmt.Sprintf("%d GLPI-KB-Beiträge wurden in den Wissensgraphen übernommen", len(items)), Strength: .64, Metadata: map[string]any{"documents": len(items), "nodes": len(nodes), "edges": len(edges), "path": path, "trigger": trigger}})
@@ -225,7 +231,7 @@ func (s *GLPIKBSyncer) Status() GLPIKBStatus {
func (s *GLPIKBSyncer) setNextSync(t time.Time) { s.mu.Lock(); s.nextSync = t; s.mu.Unlock() }
func (s *GLPIKBSyncer) currentPath() string { s.mu.RLock(); defer s.mu.RUnlock(); return s.path }
func (s *GLPIKBSyncer) buildGraph(items []glpi.KnowledgeItem, categories []glpi.ITILCategory) ([]model.Node, []model.Edge) {
func (s *GLPIKBSyncer) buildGraph(items []glpi.KnowledgeItem, categories []glpi.ITILCategory) ([]model.Node, []model.Edge, map[string]string) {
kbCategoryNames := map[int64][]string{}
for _, category := range categories {
if category.KnowbaseCategoryID <= 0 {
@@ -241,6 +247,7 @@ func (s *GLPIKBSyncer) buildGraph(items []glpi.KnowledgeItem, categories []glpi.
}
now := time.Now().UTC()
nodesByID := map[string]model.Node{}
contents := map[string]string{}
var edges []model.Edge
sourceLabel := s.cfg.Source
if sourceLabel == "" {
@@ -262,6 +269,7 @@ func (s *GLPIKBSyncer) buildGraph(items []glpi.KnowledgeItem, categories []glpi.
categoriesForNode = uniqueStrings(categoriesForNode)
keywords := extractKeywords(item.Title, categoriesForNode)
nodesByID[nodeID] = model.Node{ID: nodeID, Kind: "knowledge", Label: strings.TrimSpace(item.Title), Summary: clamp(text, 1100), Status: "production", Origin: "glpi-kb", ExternalID: externalID, URI: "glpi://KnowbaseItem/" + strconv.FormatInt(item.ID, 10), Categories: categoriesForNode, Keywords: keywords, Weight: 1.35, Metadata: map[string]any{"source": sourceLabel, "glpi_id": item.ID, "glpi_category_ids": item.CategoryIDs, "language": item.Language, "modified_at": item.ModifiedAt}, UpdatedAt: now}
contents[nodeID] = text
edges = append(edges, model.Edge{Source: nodeID, Target: sourceID, Type: "derived_from", Origin: "glpi-kb", Status: "verified", Confidence: 1, Weight: .45})
for _, category := range categoriesForNode {
categoryID := graph.ID("category", strings.ToLower(category))
@@ -276,7 +284,44 @@ func (s *GLPIKBSyncer) buildGraph(items []glpi.KnowledgeItem, categories []glpi.
nodes = append(nodes, node)
}
sort.Slice(nodes, func(i, j int) bool { return nodes[i].ID < nodes[j].ID })
return nodes, edges
return nodes, edges, contents
}
func (s *GLPIKBSyncer) Content(nodeID string) (string, bool) {
s.mu.RLock()
defer s.mu.RUnlock()
value, ok := s.contents[nodeID]
return value, ok
}
func glpiKnowledgeFingerprint(nodes []model.Node, edges []model.Edge, contents map[string]string) (string, error) {
base, err := knowledgeFingerprint(nodes, edges)
if err != nil {
return "", err
}
keys := make([]string, 0, len(contents))
for key := range contents {
keys = append(keys, key)
}
sort.Strings(keys)
var b strings.Builder
b.WriteString(base)
for _, key := range keys {
b.WriteString("\x00")
b.WriteString(key)
b.WriteString("\x00")
b.WriteString(contents[key])
}
sum := sha256.Sum256([]byte(b.String()))
return hex.EncodeToString(sum[:]), nil
}
func cloneStringMap(in map[string]string) map[string]string {
out := make(map[string]string, len(in))
for key, value := range in {
out[key] = value
}
return out
}
var glpiTagRE = regexp.MustCompile(`(?s)<[^>]*>`)

View File

@@ -53,6 +53,14 @@ func TestGLPIKBSyncAddsGraphAndQueuesCache(t *testing.T) {
if !found {
t.Fatal("GLPI KB node missing")
}
for _, n := range snap.Nodes {
if n.ExternalID == "GLPI-KB-42" {
content, ok := s.Content(n.ID)
if !ok || content != "Gateway prüfen" {
t.Fatalf("full GLPI content missing: %q %v", content, ok)
}
}
}
if _, err := os.Stat(cachePath); !os.IsNotExist(err) {
t.Fatalf("cache must be buffered before flush: %v", err)
}
@@ -69,4 +77,20 @@ func TestGLPIKBSyncAddsGraphAndQueuesCache(t *testing.T) {
if _, err := os.Stat(cachePath); err != nil {
t.Fatal(err)
}
g2, err := graph.Open(filepath.Join(dir, "reload"))
if err != nil {
t.Fatal(err)
}
s2 := NewGLPIKBSyncer(GLPIKBConfig{Enabled: true, CachePath: cachePath}, fakeGLPIKB{}, g2, nil, p)
if err := s2.LoadCache(); err != nil {
t.Fatal(err)
}
for _, n := range g2.Snapshot().Nodes {
if n.ExternalID == "GLPI-KB-42" {
content, ok := s2.Content(n.ID)
if !ok || content != "Gateway prüfen" {
t.Fatalf("cached full GLPI content missing: %q %v", content, ok)
}
}
}
}

View File

@@ -129,11 +129,18 @@ func scanDir(root, origin, status string) ([]model.Node, []model.Edge, error) {
}
nodeID := graph.ID("knowledge", externalID)
rel, _ := filepath.Rel(root, path)
metadata := map[string]any{"path": filepath.ToSlash(rel), "source": source, "auto_reply": doc["auto_reply"], "min_score": doc["min_score"]}
if aiMeta, ok := doc["ai_think"].(map[string]any); ok {
for _, key := range []string{"subtype", "action", "target_article_id", "target_node_id", "generation_depth", "confidence", "source_node_ids", "productive_source_count", "ai_source_count", "production_ratio"} {
if value, exists := aiMeta[key]; exists {
metadata[key] = value
}
}
}
n := model.Node{
ID: nodeID, Kind: kind, Label: label, Summary: clamp(text, 900), Status: status, Origin: origin,
ID: nodeID, Kind: kind, Label: label, Summary: clamp(text, 1400), Status: status, Origin: origin,
ExternalID: externalID, URI: uri, Categories: categories, Keywords: keywords, Weight: 1.3,
Metadata: map[string]any{"path": filepath.ToSlash(rel), "source": source, "auto_reply": doc["auto_reply"], "min_score": doc["min_score"]},
UpdatedAt: time.Now().UTC(),
Metadata: metadata, UpdatedAt: time.Now().UTC(),
}
docs = append(docs, n)
for _, cat := range categories {

View File

@@ -21,7 +21,7 @@ func TestKnowledgeScannerIncludesAIThinkStaging(t *testing.T) {
if err := os.WriteFile(filepath.Join(prod, "vpn.json"), []byte(`{"id":"KB-VPN","title":"VPN","text":"Gateway","categories":["Netzwerk"],"keywords":["VPN"],"source":"internal-kb"}`), 0644); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(filepath.Join(stage, "think.json"), []byte(`{"id":"KB-AI-1","title":"VPN Zusammenhang","text":"Synthese","categories":["AI-THINK"],"keywords":["VPN"],"source":"Neural Brain"}`), 0644); err != nil {
if err := os.WriteFile(filepath.Join(stage, "think.json"), []byte(`{"id":"KB-AI-1","title":"VPN Zusammenhang","text":"Synthese","categories":["AI-THINK"],"keywords":["VPN"],"source":"Neural Brain","ai_think":{"subtype":"knowledge_synthesis","generation_depth":2,"source_node_ids":["a","b","c"]}}`), 0644); err != nil {
t.Fatal(err)
}
g, _ := graph.Open(filepath.Join(root, "data"))
@@ -37,6 +37,9 @@ func TestKnowledgeScannerIncludesAIThinkStaging(t *testing.T) {
}
if n.ExternalID == "KB-AI-1" && n.Kind == "ai-think" && n.Status == "staging" {
thinkOK = true
if n.Metadata["subtype"] != "knowledge_synthesis" || n.Metadata["generation_depth"].(float64) != 2 {
t.Fatalf("AI synthesis metadata missing: %#v", n.Metadata)
}
}
}
if !prodOK || !thinkOK {

View File

@@ -126,11 +126,37 @@ type RelationDecision struct {
Explanation string `json:"explanation"`
NeedsResearch bool `json:"needs_research"`
ResearchQuery string `json:"research_query"`
Title string `json:"title"`
Synthesis string `json:"synthesis"`
TopicLabel string `json:"topic_label"`
Keywords []string `json:"keywords"`
}
type ArticlePlanDecision struct {
Action string `json:"action"`
TargetArticleID string `json:"target_article_id"`
Reason string `json:"reason"`
ExpectedValue string `json:"expected_value"`
ArticleType string `json:"article_type"`
SourceNodeIDs []string `json:"source_node_ids"`
MissingInformation []string `json:"missing_information"`
Contradictions []string `json:"contradictions"`
NeedsResearch bool `json:"needs_research"`
ResearchQuery string `json:"research_query"`
}
type KnowledgeArticleDraft struct {
Title string `json:"title"`
Text string `json:"text"`
Answer string `json:"answer"`
Prerequisites []string `json:"prerequisites"`
Validation []string `json:"validation"`
Troubleshooting []string `json:"troubleshooting"`
Categories []string `json:"categories"`
Keywords []string `json:"keywords"`
SourceNodeIDs []string `json:"source_node_ids"`
Confidence float64 `json:"confidence"`
OpenQuestions []string `json:"open_questions"`
}
type AnswerDecision struct {
Answer string `json:"answer"`
UsedNodeIDs []string `json:"used_node_ids"`

View File

@@ -36,7 +36,7 @@ func TestRuntimeSettingsAndCategoriesAPI(t *testing.T) {
}, g, broker)
h := (&Server{Engine: eng, Graph: g, Broker: broker}).Handler()
body := `{"learning_enabled":false,"thinking_enabled":false,"learning_categories":["GLPI"],"display_categories":["Ollama"],"thinking_categories":["GLPI"],"view_mode":"honeycomb"}`
body := `{"learning_enabled":false,"thinking_enabled":false,"learning_categories":["GLPI"],"display_categories":["Ollama"],"thinking_categories":["GLPI"],"view_mode":"honeycomb","max_display_nodes":1500}`
req := httptest.NewRequest(http.MethodPut, "/api/runtime-settings", strings.NewReader(body))
req.Header.Set("Content-Type", "application/json")
res := httptest.NewRecorder()
@@ -48,7 +48,7 @@ func TestRuntimeSettingsAndCategoriesAPI(t *testing.T) {
if err := json.NewDecoder(res.Body).Decode(&settings); err != nil {
t.Fatal(err)
}
if settings.LearningEnabled || settings.ThinkingEnabled || settings.ViewMode != "honeycomb" {
if settings.LearningEnabled || settings.ThinkingEnabled || settings.ViewMode != "honeycomb" || settings.MaxDisplayNodes != 1500 {
t.Fatalf("unexpected settings: %+v", settings)
}

View File

@@ -45,3 +45,5 @@ body.honeycomb-view .legend{opacity:.58}body.honeycomb-view .mode-status small:a
.settings-footer{display:flex;align-items:center;justify-content:space-between;gap:12px;padding-top:13px}.settings-footer span{font-size:9px;color:#7f9aac;line-height:1.35}.settings-footer button{border:1px solid rgba(82,231,255,.3);background:rgba(82,231,255,.1);color:var(--cyan);border-radius:11px;padding:10px 13px;font-size:9px;font-weight:800;letter-spacing:.13em;cursor:pointer}.settings-footer button:disabled{opacity:.5;cursor:wait}
@media(max-width:1100px){.dock{max-width:calc(100vw - 40px)}.settings-panel{right:10px;top:10px;bottom:10px}}
@media(max-width:780px){.settings-panel{left:10px;width:auto}.dock{max-width:calc(100vw - 20px)}.dock-separator{display:none}}
.performance-settings{flex:0 0 auto}.number-setting{display:flex;align-items:center;justify-content:space-between;gap:16px;padding:10px;margin:7px 0;border:1px solid rgba(133,200,255,.1);border-radius:12px;background:rgba(255,255,255,.022)}.number-setting b{display:block;font-size:11px}.number-setting small{display:block;margin-top:3px;max-width:250px;color:#718b9e;font-size:9px;line-height:1.4}.number-setting input{width:112px;flex:0 0 auto;border:1px solid rgba(82,231,255,.2);background:rgba(2,8,17,.72);color:var(--text);border-radius:10px;padding:9px 8px;text-align:right;font-variant-numeric:tabular-nums;outline:0}.number-setting input:focus{border-color:rgba(82,231,255,.48);box-shadow:0 0 0 3px rgba(82,231,255,.07)}.node-limit-presets{display:flex;flex-wrap:wrap;gap:5px;margin-top:8px}.node-limit-presets button{border:1px solid rgba(133,200,255,.12);background:rgba(255,255,255,.025);color:#7893a6;border-radius:999px;padding:5px 8px;font-size:8px;cursor:pointer}.node-limit-presets button:hover{color:var(--cyan);border-color:rgba(82,231,255,.3);background:rgba(82,231,255,.08)}.setting-hint{margin:8px 2px 0;color:#6f899d;font-size:9px;line-height:1.4}

View File

@@ -41,9 +41,9 @@
lodOpenUntil: new Map(), lodHotUntil: new Map(), lodDirty: true, lodLastBuild: 0, lodNextExpiry: 0, lodZoomBand: 2,
renderNodes: [], renderEdges: [], renderIdleEdges: [], renderNodeById: new Map(), renderEdgeById: new Map(), visibleForNode: new Map(),
edgeRenderMap: new Map(), renderActive: new Map(), renderEdgeActive: new Map(), renderStats: {nodes: 0, edges: 0, hiddenNodes: 0, hiddenEdges: 0},
fullSnapshot: null, runtimeSettings: {learning_enabled: true, thinking_enabled: true, learning_categories: [], display_categories: [], thinking_categories: [], view_mode: 'neural'},
fullSnapshot: null, fullNodeById: new Map(), runtimeSettings: {learning_enabled: true, thinking_enabled: true, learning_categories: [], display_categories: [], thinking_categories: [], view_mode: 'neural', max_display_nodes: 0},
availableCategories: [], viewMode: 'neural', honeycombNodes: [], honeycombSpacing: 0, settingsOpen: false, settingsDraft: null,
graphVersion: null, displaySignature: ''
forcedDisplayUntil: new Map(), nextDisplayLimitExpiry: 0, graphVersion: null, displaySignature: '', displayLimitStats: {limit: 0, eligible: 0, shown: 0}
};
function resize() {
@@ -75,24 +75,113 @@
return nodeCategories.some(category => wanted.has(category));
}
function activeForcedDisplayIDs(now = Date.now()) {
const active = new Set();
let next = 0;
for (const [id, until] of state.forcedDisplayUntil) {
if (until <= now) {
state.forcedDisplayUntil.delete(id);
continue;
}
active.add(id);
if (!next || until < next) next = until;
}
state.nextDisplayLimitExpiry = next;
return active;
}
function nodeLimitScore(node, degree, forced) {
if (forced.has(node.id)) return 1e12 + degree;
let score = Math.min(5000, degree * 8) + Math.min(100, Number(node.weight || 1) * 4);
if (node.kind === 'knowledge') score += 120;
if (node.kind === 'ai-think') score += 230;
if (node.status === 'staging') score += 180;
if (node.kind === 'external') score += 105;
if (node.kind === 'category') score += 45;
if (node.kind === 'source') score += 30;
if (state.active.has(node.id)) score += 500000;
if (state.selected?.id === node.id || state.hover?.id === node.id) score += 750000;
return score + pseudo(node.id, 71) * 5;
}
function limitSnapshot(snapshot, rawLimit) {
const limit = Math.max(0, Math.min(500000, Math.trunc(Number(rawLimit) || 0)));
const eligible = snapshot.nodes.length;
if (!limit || eligible <= limit) {
state.displayLimitStats = {limit, eligible, shown: eligible};
return snapshot;
}
const forced = activeForcedDisplayIDs();
const degree = new Map(snapshot.nodes.map(node => [node.id, 0]));
for (const edge of snapshot.edges) {
degree.set(edge.source, (degree.get(edge.source) || 0) + 1);
degree.set(edge.target, (degree.get(edge.target) || 0) + 1);
}
const selected = new Set();
const buckets = new Map();
const candidates = snapshot.nodes.map(node => ({node, score: nodeLimitScore(node, degree.get(node.id) || 0, forced)}));
candidates.sort((a, b) => b.score - a.score || a.node.id.localeCompare(b.node.id));
for (const candidate of candidates) {
if (!forced.has(candidate.node.id) || selected.size >= limit) continue;
selected.add(candidate.node.id);
}
for (const candidate of candidates) {
if (selected.has(candidate.node.id)) continue;
const key = categoryKey(candidate.node) || 'Sonstige';
if (!buckets.has(key)) buckets.set(key, []);
buckets.get(key).push(candidate);
}
const bucketList = [...buckets.values()].sort((a, b) => (b[0]?.score || 0) - (a[0]?.score || 0));
for (const bucket of bucketList) {
if (selected.size >= limit || !bucket.length) break;
selected.add(bucket.shift().node.id);
}
const remainder = [];
for (const bucket of bucketList) remainder.push(...bucket);
remainder.sort((a, b) => b.score - a.score || a.node.id.localeCompare(b.node.id));
for (const candidate of remainder) {
if (selected.size >= limit) break;
selected.add(candidate.node.id);
}
const nodes = snapshot.nodes.filter(node => selected.has(node.id));
const edges = snapshot.edges.filter(edge => selected.has(edge.source) && selected.has(edge.target));
state.displayLimitStats = {limit, eligible, shown: nodes.length};
return {...snapshot, nodes, edges};
}
function filteredSnapshot(snapshot) {
const filters = state.runtimeSettings.display_categories || [];
if (!filters.length) return snapshot;
const visible = new Set();
const noteKinds = new Set(['knowledge', 'ai-think', 'external']);
for (const node of snapshot.nodes) {
if (noteKinds.has(node.kind) && categoryFilterMatches(node, filters)) visible.add(node.id);
if (node.kind === 'category' && filters.some(value => String(value).toLowerCase() === String(node.label || '').toLowerCase())) visible.add(node.id);
let filtered = snapshot;
if (filters.length) {
const visible = new Set();
const noteKinds = new Set(['knowledge', 'ai-think', 'external']);
for (const node of snapshot.nodes) {
if (noteKinds.has(node.kind) && categoryFilterMatches(node, filters)) visible.add(node.id);
if (node.kind === 'category' && filters.some(value => String(value).toLowerCase() === String(node.label || '').toLowerCase())) visible.add(node.id);
}
for (const edge of snapshot.edges) {
if (visible.has(edge.source)) visible.add(edge.target);
if (visible.has(edge.target)) visible.add(edge.source);
}
filtered = {
...snapshot,
nodes: snapshot.nodes.filter(node => visible.has(node.id)),
edges: snapshot.edges.filter(edge => visible.has(edge.source) && visible.has(edge.target))
};
}
for (const edge of snapshot.edges) {
if (visible.has(edge.source)) visible.add(edge.target);
if (visible.has(edge.target)) visible.add(edge.source);
}
return {
...snapshot,
nodes: snapshot.nodes.filter(node => visible.has(node.id)),
edges: snapshot.edges.filter(edge => visible.has(edge.source) && visible.has(edge.target))
};
return limitSnapshot(filtered, state.runtimeSettings.max_display_nodes);
}
function currentDisplaySignature() {
const forced = [...activeForcedDisplayIDs()].sort();
return JSON.stringify({categories: state.runtimeSettings.display_categories || [], limit: Number(state.runtimeSettings.max_display_nodes || 0), forced});
}
async function loadRuntimeConfiguration() {
@@ -120,12 +209,27 @@
if ($('settingsThinking')) $('settingsThinking').checked = Boolean(panelSettings.thinking_enabled);
if ($('settingsViewNeural')) $('settingsViewNeural').classList.toggle('active', panelSettings.view_mode !== 'honeycomb');
if ($('settingsViewHoneycomb')) $('settingsViewHoneycomb').classList.toggle('active', panelSettings.view_mode === 'honeycomb');
if ($('settingsMaxDisplayNodes')) $('settingsMaxDisplayNodes').value = String(Math.max(0, Number(panelSettings.max_display_nodes || 0)));
updateDisplayLimitHint(panelSettings.max_display_nodes);
const enrichButton = $('enrichNow');
if (enrichButton && !state.brainStatus?.enrich_running) enrichButton.disabled = !thinking;
if (!learning && !thinking) setVisualMode('living');
updateViewButtons();
}
function updateDisplayLimitHint(value = state.runtimeSettings.max_display_nodes) {
const hint = $('displayLimitHint');
if (!hint) return;
const limit = Math.max(0, Math.trunc(Number(value) || 0));
if (!limit) {
hint.textContent = 'Unbegrenzt · Kategorie-Filter und LOD bestimmen die Renderlast.';
return;
}
const stats = state.displayLimitStats || {};
const eligible = Number(stats.eligible || state.fullSnapshot?.nodes?.length || 0);
hint.textContent = `Maximal ${limit.toLocaleString('de-DE')} Nodes · aktuell ${Math.min(limit, eligible).toLocaleString('de-DE')} von ${eligible.toLocaleString('de-DE')} auswählbar.`;
}
function categoryLabel(name) {
return name === '__uncategorized__' ? 'Ohne Kategorie' : name;
}
@@ -165,15 +269,19 @@
learning_categories: [...(settings.learning_categories || [])],
display_categories: [...(settings.display_categories || [])],
thinking_categories: [...(settings.thinking_categories || [])],
view_mode: settings.view_mode === 'honeycomb' ? 'honeycomb' : 'neural'
view_mode: settings.view_mode === 'honeycomb' ? 'honeycomb' : 'neural',
max_display_nodes: Math.max(0, Math.min(500000, Math.trunc(Number(settings.max_display_nodes) || 0)))
};
const previousDisplay = JSON.stringify(state.runtimeSettings.display_categories || []);
const previousDisplay = JSON.stringify({categories: state.runtimeSettings.display_categories || [], limit: Number(state.runtimeSettings.max_display_nodes || 0)});
const updated = await api('/api/runtime-settings', {method: 'PUT', body: JSON.stringify(normalized)});
state.runtimeSettings = {...normalized, ...updated};
state.viewMode = state.runtimeSettings.view_mode;
syncRuntimeControls();
applyViewMode(state.viewMode, false);
if (previousDisplay !== JSON.stringify(state.runtimeSettings.display_categories || [])) await loadGraph();
if (previousDisplay !== JSON.stringify({categories: state.runtimeSettings.display_categories || [], limit: Number(state.runtimeSettings.max_display_nodes || 0)})) {
state.displaySignature = '';
await loadGraph();
}
await loadStatus();
return state.runtimeSettings;
}
@@ -194,45 +302,78 @@
$('settingsBackdrop')?.classList.add('hidden');
}
function applyGraphSnapshot(snap, force = false) {
const displaySignature = currentDisplaySignature();
if (!force && state.fullSnapshot && state.graphVersion === snap.version && state.displaySignature === displaySignature) return false;
state.graphVersion = snap.version;
state.displaySignature = displaySignature;
state.fullSnapshot = snap;
state.fullNodeById = new Map(snap.nodes.map(node => [node.id, node]));
const filtered = filteredSnapshot(snap);
const old = state.nodeById;
const oldClusters = state.clusterByKey;
state.nodes = filtered.nodes.map(n => ({...n, glow: old.get(n.id)?.glow || 0, screen: null, clusterKey: '', clusterColor: old.get(n.id)?.clusterColor || ''}));
state.edges = filtered.edges;
state.nodeById = new Map(state.nodes.map(n => [n.id, n]));
state.edgeById = new Map(state.edges.map(e => [e.id, e]));
state.adjacency = new Map();
for (const e of state.edges) {
if (!state.adjacency.has(e.source)) state.adjacency.set(e.source, []);
if (!state.adjacency.has(e.target)) state.adjacency.set(e.target, []);
state.adjacency.get(e.source).push(e);
state.adjacency.get(e.target).push(e);
}
buildLayout(old, oldClusters);
for (const node of state.nodes) {
node.neuralX = node.x;
node.neuralY = node.y;
node.neuralZ = node.z;
}
buildHoneycombLayout();
buildLODHierarchy();
applyViewMode(state.runtimeSettings.view_mode || state.viewMode, false);
$('nodeCount').textContent = state.nodes.length.toLocaleString('de-DE');
const limit = Number(state.runtimeSettings.max_display_nodes || 0);
const capText = limit ? ` · GPU-Limit ${limit.toLocaleString('de-DE')}` : '';
$('nodeCount').parentElement.title = `${state.nodes.length.toLocaleString('de-DE')} angezeigt · ${snap.nodes.length.toLocaleString('de-DE')} insgesamt${capText}`;
$('edgeCount').textContent = state.edges.length.toLocaleString('de-DE');
updateDisplayLimitHint(limit);
return true;
}
async function loadGraph() {
try {
const snap = await api('/api/graph');
const displaySignature = JSON.stringify(state.runtimeSettings.display_categories || []);
if (state.fullSnapshot && state.graphVersion === snap.version && state.displaySignature === displaySignature) return;
state.graphVersion = snap.version;
state.displaySignature = displaySignature;
state.fullSnapshot = snap;
const filtered = filteredSnapshot(snap);
const old = state.nodeById;
const oldClusters = state.clusterByKey;
state.nodes = filtered.nodes.map(n => ({...n, glow: old.get(n.id)?.glow || 0, screen: null, clusterKey: '', clusterColor: old.get(n.id)?.clusterColor || ''}));
state.edges = filtered.edges;
state.nodeById = new Map(state.nodes.map(n => [n.id, n]));
state.edgeById = new Map(state.edges.map(e => [e.id, e]));
state.adjacency = new Map();
for (const e of state.edges) {
if (!state.adjacency.has(e.source)) state.adjacency.set(e.source, []);
if (!state.adjacency.has(e.target)) state.adjacency.set(e.target, []);
state.adjacency.get(e.source).push(e);
state.adjacency.get(e.target).push(e);
}
buildLayout(old, oldClusters);
for (const node of state.nodes) {
node.neuralX = node.x;
node.neuralY = node.y;
node.neuralZ = node.z;
}
buildHoneycombLayout();
buildLODHierarchy();
applyViewMode(state.runtimeSettings.view_mode || state.viewMode, false);
$('nodeCount').textContent = state.nodes.length.toLocaleString('de-DE');
$('nodeCount').parentElement.title = `${state.nodes.length.toLocaleString('de-DE')} sichtbar · ${snap.nodes.length.toLocaleString('de-DE')} insgesamt`;
$('edgeCount').textContent = state.edges.length.toLocaleString('de-DE');
applyGraphSnapshot(snap, false);
} catch {
setSystem('offline', false);
}
}
function forceDisplayNodes(ids, duration = 36000) {
const limit = Number(state.runtimeSettings.max_display_nodes || 0);
if (!limit || !state.fullSnapshot || !ids?.length) return false;
const known = state.fullNodeById;
const now = Date.now();
let changed = false;
for (const id of ids) {
if (!known.has(id)) continue;
const until = now + duration;
if ((state.forcedDisplayUntil.get(id) || 0) < until) state.forcedDisplayUntil.set(id, until);
if (!state.nodeById.has(id)) changed = true;
}
activeForcedDisplayIDs(now);
if (changed) applyGraphSnapshot(state.fullSnapshot, true);
return changed;
}
function expireForcedDisplayNodes() {
if (!state.nextDisplayLimitExpiry || Date.now() < state.nextDisplayLimitExpiry) return;
const before = state.forcedDisplayUntil.size;
activeForcedDisplayIDs();
if (before !== state.forcedDisplayUntil.size && state.fullSnapshot) applyGraphSnapshot(state.fullSnapshot, true);
}
function setSystem(text, ok = true) {
$('systemState').lastChild.textContent = ' ' + text;
$('systemState').style.color = ok ? 'var(--green)' : 'var(--red)';
@@ -267,7 +408,7 @@
text = `Living-only · Thinking pausiert${learningEnabled ? '' : ' · Learning pausiert'}.`;
cls = 'waiting';
} else if (status.enrich_running) {
text = `AI-THINK läuft · ${status.enrich_trigger === 'manual' ? 'manuell' : 'automatisch'} · Batch ${status.enrich_batch_size || 1}`;
text = `AI-THINK läuft · Relationen und KB-Synthese · ${status.enrich_trigger === 'manual' ? 'manuell' : 'automatisch'} · Batch ${status.enrich_batch_size || 1}`;
cls = 'running';
} else if (queued) {
text = 'AI-THINK ist eingeplant und wartet auf den sequenziellen Worker.';
@@ -282,7 +423,7 @@
text = `Im zuletzt geprüften Graphbereich kein Kandidat über dem Schwellwert · nächster Versuch ${nextRunText(status.next_enrich)}.`;
cls = 'waiting';
} else {
text = `Autonom aktiv · ${status.enrich_batch_size || 1} Prüfungen pro Zyklus · nächster Lauf ${nextRunText(status.next_enrich)}.`;
text = `Autonom aktiv · ${status.enrich_batch_size || 1} Relationsprüfungen pro Zyklus · Artikel ab ${status.article_min_sources || 3} produktiven Quellen · nächster Lauf ${nextRunText(status.next_enrich)}.`;
cls = 'waiting';
}
panel.className = `autonomy-status ${cls}`;
@@ -1085,7 +1226,7 @@
if (!state.runtimeSettings.learning_enabled && !state.runtimeSettings.thinking_enabled) return 'living';
if (!evt || evt.type === 'brain.idle') return 'living';
if (evt.type?.includes('research')) return state.runtimeSettings.thinking_enabled ? 'researching' : 'living';
if (evt.type?.includes('think')) return state.runtimeSettings.thinking_enabled ? 'thinking' : 'living';
if (evt.type?.includes('think') || evt.type?.startsWith('article.')) return state.runtimeSettings.thinking_enabled ? 'thinking' : 'living';
if (evt.source === 'agent' || evt.source === 'knowledgebase' || evt.type?.includes('query')) return 'processing';
if (evt.type === 'graph.updated' || evt.type === 'scan.started' || evt.type === 'embedding.batch' || evt.type?.startsWith('glpi.kb')) return state.runtimeSettings.learning_enabled ? 'learning' : 'living';
return 'processing';
@@ -1479,6 +1620,7 @@
const dt = Math.min(0.05, (now - state.last) / 1000);
state.last = now;
updateVisualState(now, dt);
expireForcedDisplayNodes();
if (state.viewMode === 'neural') updateLOD(now);
drawBackground(now);
if (state.viewMode === 'neural') {
@@ -1508,6 +1650,8 @@
requestAnimationFrame(frame);
function activate(evt) {
const requestedDuration = evt?.type?.includes('think') || evt?.type?.includes('research') ? 45000 : 32000;
forceDisplayNodes(evt?.node_ids || [], requestedDuration);
const strength = Math.max(0.15, Math.min(1.4, evt.strength || 0.6));
const mode = eventMode(evt);
const substep = evt.type === 'node.activated' || evt.type === 'edges.traversed';
@@ -1558,12 +1702,12 @@
}
addLog(evt);
if (evt.type === 'graph.updated') loadGraph();
if (evt.type?.startsWith('think.')) loadStatus();
if (evt.type?.startsWith('think.') || evt.type?.startsWith('article.')) loadStatus();
}
function shouldLog(evt) {
if (!evt || evt.type === 'brain.idle' || evt.type === 'node.activated' || evt.type === 'edges.traversed') return false;
const important = new Set(['scan.started', 'graph.updated', 'embedding.batch', 'query.started', 'query.completed', 'think.queued', 'think.cycle.started', 'think.cycle.completed', 'think.cycle.failed', 'think.no_candidate', 'think.started', 'think.created', 'think.rejected', 'think.failed', 'think.paused', 'research.started', 'agent.run', 'glpi.kb.synced', 'glpi.kb.failed', 'persistence.flushed', 'persistence.failed']);
const important = new Set(['scan.started', 'graph.updated', 'embedding.batch', 'query.started', 'query.completed', 'think.queued', 'think.cycle.started', 'think.cycle.completed', 'think.cycle.failed', 'think.no_candidate', 'think.started', 'think.relation.created', 'think.rejected', 'think.failed', 'think.paused', 'research.started', 'article.plan.started', 'article.plan.skipped', 'article.research.started', 'article.research.failed', 'article.draft.started', 'article.draft.rejected', 'article.created', 'article.duplicate', 'article.skipped', 'article.failed', 'agent.run', 'glpi.kb.synced', 'glpi.kb.failed', 'persistence.flushed', 'persistence.failed']);
if (!important.has(evt.type) && !(evt.source === 'agent' || evt.source === 'knowledgebase' || evt.source === 'external' || evt.query)) return false;
const fingerprint = `${evt.type}|${evt.message || ''}|${evt.query || ''}|${evt.source || ''}`;
const last = state.lastLogFingerprint.get(fingerprint) || 0;
@@ -1591,11 +1735,21 @@
'think.cycle.failed': 'AI-THINK-Zyklus fehlgeschlagen',
'think.no_candidate': 'Keine neue Beziehung',
'think.started': 'AI-THINK prüft Zusammenhang',
'think.created': 'AI-THINK erstellt Beitrag',
'think.relation.created': 'Relation übernommen',
'think.rejected': 'AI-THINK verwirft Edge',
'think.failed': 'AI-THINK Fehler',
'think.paused': 'AI-THINK pausiert',
'research.started': 'Recherche gestartet',
'research.started': 'Relationsrecherche gestartet',
'article.plan.started': 'Artikelmehrwert wird geprüft',
'article.plan.skipped': 'Artikelsynthese übersprungen',
'article.research.started': 'Artikelrecherche gestartet',
'article.research.failed': 'Artikelrecherche fehlgeschlagen',
'article.draft.started': 'KB-Entwurf wird geschrieben',
'article.draft.rejected': 'KB-Entwurf abgelehnt',
'article.created': 'KB-Entwurf erstellt',
'article.duplicate': 'KB-Entwurf bereits vorhanden',
'article.skipped': 'Noch kein belastbarer Artikel',
'article.failed': 'Artikelsynthese fehlgeschlagen',
'agent.run': 'Agent-Lauf',
'glpi.kb.synced': 'GLPI-KB synchronisiert',
'glpi.kb.failed': 'GLPI-KB Fehler',
@@ -1621,6 +1775,14 @@
if (evt.metadata?.batch_size) meta.push(`${Number(evt.metadata.batch_size)} Schritte`);
if (evt.metadata?.checked !== undefined) meta.push(`${Number(evt.metadata.checked)} geprüft`);
if (evt.metadata?.created !== undefined) meta.push(`${Number(evt.metadata.created)} erstellt`);
if (evt.metadata?.relations_created !== undefined) meta.push(`${Number(evt.metadata.relations_created)} Relationen`);
if (evt.metadata?.articles_created !== undefined) meta.push(`${Number(evt.metadata.articles_created)} Artikel`);
if (evt.metadata?.articles_skipped !== undefined) meta.push(`${Number(evt.metadata.articles_skipped)} Artikel übersprungen`);
if (evt.metadata?.productive_sources !== undefined) meta.push(`${Number(evt.metadata.productive_sources)} produktive Quellen`);
if (evt.metadata?.ai_sources !== undefined) meta.push(`${Number(evt.metadata.ai_sources)} AI-Quellen`);
if (evt.metadata?.production_ratio !== undefined) meta.push(`${Math.round(Number(evt.metadata.production_ratio) * 100)}% Produktionswissen`);
if (evt.metadata?.generation_depth !== undefined) meta.push(`Tiefe ${Number(evt.metadata.generation_depth)}`);
if (evt.metadata?.action) meta.push(String(evt.metadata.action).toUpperCase());
if (evt.metadata?.rejected !== undefined) meta.push(`${Number(evt.metadata.rejected)} verworfen`);
if (evt.metadata?.comparisons) meta.push(`${Number(evt.metadata.comparisons).toLocaleString('de-DE')} Vergleiche`);
if (evt.metadata?.candidate_comparisons) meta.push(`${Number(evt.metadata.candidate_comparisons).toLocaleString('de-DE')} Vergleiche`);
@@ -1633,12 +1795,12 @@
const eventQuery = evt.query || evt.metadata?.research_query || '';
if (eventQuery) meta.push('Query');
let message = evt.message || eventQuery || evt.type;
if ((evt.type === 'think.started' || evt.type === 'think.created' || evt.type === 'think.rejected' || evt.type === 'research.started') && nodes.length >= 2) {
if ((evt.type === 'think.started' || evt.type === 'think.relation.created' || evt.type === 'think.rejected' || evt.type === 'research.started' || evt.type?.startsWith('article.')) && nodes.length >= 2) {
message = `${message} · ${nodes.slice(0, 2).join(' ↔ ')}`;
} else if (evt.type === 'query.started' && evt.query) {
message = `${evt.source === 'agent' ? 'Agent' : evt.source === 'knowledgebase' ? 'Knowledgebase' : 'Brain'} verarbeitet eine Anfrage.`;
}
return {time, title, message, meta, query: eventQuery, regions, cls: evt.type?.includes('think') ? 'think' : evt.type?.includes('research') ? 'research' : evt.type === 'graph.updated' || evt.type === 'scan.started' || evt.type?.startsWith('glpi.kb') || evt.type?.startsWith('persistence.') ? 'graph' : evt.source === 'agent' ? 'agent' : ''};
return {time, title, message, meta, query: eventQuery, regions, cls: evt.type?.includes('think') || evt.type?.startsWith('article.') ? (evt.type?.includes('research') ? 'research' : 'think') : evt.type?.includes('research') ? 'research' : evt.type === 'graph.updated' || evt.type === 'scan.started' || evt.type?.startsWith('glpi.kb') || evt.type?.startsWith('persistence.') ? 'graph' : evt.source === 'agent' ? 'agent' : ''};
}
function addLog(evt) {
@@ -1703,6 +1865,11 @@
$('categorySearch').addEventListener('input', e => renderCategoryFilters(e.currentTarget.value));
$('settingsLearning').addEventListener('change', e => { if (state.settingsDraft) state.settingsDraft.learning_enabled = e.currentTarget.checked; });
$('settingsThinking').addEventListener('change', e => { if (state.settingsDraft) state.settingsDraft.thinking_enabled = e.currentTarget.checked; });
$('settingsMaxDisplayNodes').addEventListener('input', e => {
if (!state.settingsDraft) return;
state.settingsDraft.max_display_nodes = Math.max(0, Math.min(500000, Math.trunc(Number(e.currentTarget.value) || 0)));
updateDisplayLimitHint(state.settingsDraft.max_display_nodes);
});
$('settingsViewNeural').addEventListener('click', () => {
if (state.settingsDraft) state.settingsDraft.view_mode = 'neural';
$('settingsViewNeural').classList.add('active');
@@ -1728,6 +1895,14 @@
state.settingsDraft[`${button.dataset.clearFilter}_categories`] = [];
renderCategoryFilters($('categorySearch').value);
}));
document.querySelectorAll('[data-node-limit]').forEach(button => button.addEventListener('click', () => {
if (!state.settingsDraft) return;
const value = Math.max(0, Math.min(500000, Math.trunc(Number(button.dataset.nodeLimit) || 0)));
state.settingsDraft.max_display_nodes = value;
$('settingsMaxDisplayNodes').value = String(value);
updateDisplayLimitHint(value);
}));
$('saveSettings').addEventListener('click', async e => {
if (!state.settingsDraft) return;
const button = e.currentTarget;
@@ -1737,6 +1912,7 @@
try {
state.settingsDraft.learning_enabled = $('settingsLearning').checked;
state.settingsDraft.thinking_enabled = $('settingsThinking').checked;
state.settingsDraft.max_display_nodes = Math.max(0, Math.min(500000, Math.trunc(Number($('settingsMaxDisplayNodes').value) || 0)));
await persistRuntimeSettings(state.settingsDraft);
feedback.textContent = 'Aktiv · Speicherung erfolgt gebündelt mit dem nächsten Flush.';
setTimeout(closeSettingsPanel, 550);

View File

@@ -92,6 +92,22 @@
</div>
</section>
<section class="settings-section performance-settings">
<div class="settings-section-title"><h2>GPU-Limit</h2><span>0 = unbegrenzt</span></div>
<label class="number-setting" for="settingsMaxDisplayNodes">
<span><b>Maximal angezeigte Nodes</b><small>Begrenzt Neural- und Honeycomb-Ansicht. Aktive Notes werden bei Bedarf temporär eingeblendet und ersetzen inaktive Nodes.</small></span>
<input id="settingsMaxDisplayNodes" type="number" min="0" max="500000" step="100" inputmode="numeric" value="0">
</label>
<div class="node-limit-presets" aria-label="Voreinstellungen für das Node-Limit">
<button type="button" data-node-limit="1000">1.000</button>
<button type="button" data-node-limit="5000">5.000</button>
<button type="button" data-node-limit="10000">10.000</button>
<button type="button" data-node-limit="25000">25.000</button>
<button type="button" data-node-limit="0">Unbegrenzt</button>
</div>
<p id="displayLimitHint" class="setting-hint">Unbegrenzt · Kategorie-Filter und LOD bestimmen die Renderlast.</p>
</section>
<section class="settings-section category-settings">
<div class="settings-section-title"><h2>Kategorie-Filter</h2><span>Leer = alle Kategorien</span></div>
<label class="filter-search"><span>Kategorien durchsuchen</span><input id="categorySearch" type="search" placeholder="z. B. GLPI, Netzwerk, Ollama"></label>

Binary file not shown.