Kategorie-Problem bei Typ String bezüglich Daten aus dem KB-System
This commit is contained in:
47
README.md
47
README.md
@@ -143,6 +143,53 @@ KNOWLEDGE_AUTO_REPLY_SOURCES=internal-kb,glpi-kb
|
||||
|
||||
`KNOWLEDGE_AUTO_REPLY_SOURCES` muss eine Teilmenge von `KNOWLEDGE_ALLOWED_SOURCES` sein. Mit `KNOWLEDGE_AUTO_REPLY_SOURCES=none` kann die Quellenfreigabe für Auto-Replies vollständig deaktiviert werden. Dokumente aus nicht erlaubten Quellen werden nicht in die Suchmenge aufgenommen und damit auch nicht an Ollama übergeben. Ein Knowledge-Dokument ohne `source` führt absichtlich zu einem Startfehler, damit die Herkunft nicht implizit geraten wird.
|
||||
|
||||
### Gemeinsame KB-Dateien mit fremden Kategorien
|
||||
|
||||
Lokale KB-Dateien dürfen in `categories` neben numerischen GLPI-IDs jetzt auch String-Kategorien aus einer anderen Anwendung enthalten. Die Quelldatei muss dafür nicht verändert werden. Beispiel:
|
||||
|
||||
```json
|
||||
{
|
||||
"id": "KB-SEC-ATTCK-AN-0001",
|
||||
"categories": ["Security", "MITRE ATT&CK", "Account Access"]
|
||||
}
|
||||
```
|
||||
|
||||
Empfohlener Standard:
|
||||
|
||||
```env
|
||||
KNOWLEDGE_CATEGORY_MODE=unscoped
|
||||
KNOWLEDGE_CATEGORY_MAP_FILE=/app/data/knowledge-category-map.json
|
||||
KNOWLEDGE_IGNORE_GLOBS=
|
||||
```
|
||||
|
||||
`unscoped` lädt auch Artikel mit unbekannten externen Kategorien. Diese Labels werden als `external_categories` im Agenten behalten und für das lexikalische Retrieval mitbenutzt. Solange mindestens eine externe Kategorie nicht auf GLPI abgebildet ist, wird `auto_reply` für diesen Artikel **fail-closed deaktiviert**. Der Artikel bleibt aber für RAG und Klassifizierung verfügbar.
|
||||
|
||||
Eine Mapping-Datei kann externe Kategorien ohne Änderung der KB-Dateien auf eine oder mehrere GLPI-ITIL-Kategorie-IDs abbilden:
|
||||
|
||||
```json
|
||||
{
|
||||
"Security": 17,
|
||||
"Account Access": [2, 17],
|
||||
"Microsoft Office": 23,
|
||||
"Docker": 31
|
||||
}
|
||||
```
|
||||
|
||||
Alternativ ist auch `{ "mappings": { ... } }` erlaubt. Mapping-Schlüssel werden ohne Beachtung der Groß-/Kleinschreibung verglichen. Numerische Strings in `categories`, z. B. `"17"`, werden direkt als GLPI-ID verstanden.
|
||||
|
||||
Weitere Modi:
|
||||
|
||||
- `KNOWLEDGE_CATEGORY_MODE=skip`: Eine Datei mit mindestens einer unbekannten externen Kategorie wird komplett ignoriert.
|
||||
- `KNOWLEDGE_CATEGORY_MODE=strict`: Eine unbekannte externe Kategorie verhindert den Start. Das entspricht dem alten strengen Verhalten.
|
||||
|
||||
Bestimmte gemeinsame Dateien können unabhängig davon per Dateimuster ausgeschlossen werden:
|
||||
|
||||
```env
|
||||
KNOWLEDGE_IGNORE_GLOBS=KB-SEC-ATTCK-*.json,external-only-*.json
|
||||
```
|
||||
|
||||
Im Dashboard werden externe und nicht gemappte Kategorien sowie die Zahl ignorierter Dateien angezeigt.
|
||||
|
||||
### GLPI Knowledge Base als echter Connector
|
||||
|
||||
Die GLPI-Wissensdatenbank kann jetzt direkt read-only synchronisiert werden. Der Agent ermittelt bei `GLPI_KB_PATH=auto` den lesbaren `KnowbaseItem`-Collection-Endpunkt aus `/api.php/doc.json`. GLPI selbst entscheidet anhand der Rechte des OAuth-Service-Accounts, welche Artikel sichtbar sind.
|
||||
|
||||
13
UPGRADE.md
13
UPGRADE.md
@@ -129,3 +129,16 @@ KNOWLEDGE_RETRIEVAL_FLOOR=0.30
|
||||
```
|
||||
|
||||
`KNOWLEDGE_TOP_K` ist die maximale Anzahl von Artikeln im Ollama-Prompt. Artikel werden nur übergeben, wenn sie mindestens den Retrieval-Floor erreichen und nicht mehr als `KNOWLEDGE_CANDIDATE_MAX_GAP` unter dem besten Treffer liegen. `KNOWLEDGE_AUDIT_TOP_K` steuert separat, wie viele Treffer für Dashboard/Audit aufbewahrt werden. Bestehende `.env`-Dateien sollten die drei neuen/angepassten Werte explizit ergänzen.
|
||||
|
||||
|
||||
## Shared KB category compatibility
|
||||
|
||||
Local knowledge JSON files may now use external string labels in `categories`. Recommended migration settings:
|
||||
|
||||
```env
|
||||
KNOWLEDGE_CATEGORY_MODE=unscoped
|
||||
KNOWLEDGE_CATEGORY_MAP_FILE=/app/data/knowledge-category-map.json
|
||||
KNOWLEDGE_IGNORE_GLOBS=
|
||||
```
|
||||
|
||||
Unmapped labels no longer crash startup in `unscoped` mode. Such documents remain searchable but their `auto_reply` is disabled until all external labels are mapped. Use `skip` to ignore those documents or `strict` to retain fail-fast behavior.
|
||||
|
||||
@@ -72,6 +72,7 @@ func main() {
|
||||
k, err := knowledge.Load(ctx, cfg.KnowledgeDir, cfg.DataDir, o, cfg.RAGEnabled, cfg.KnowledgeAllowedSources, knowledge.ScoringConfig{
|
||||
SemanticWeight: cfg.KnowledgeSemanticWeight, TitleWeight: cfg.KnowledgeTitleWeight, LexicalWeight: cfg.KnowledgeLexicalWeight, KeywordWeight: cfg.KnowledgeKeywordWeight, CategoryWeight: cfg.KnowledgeCategoryWeight,
|
||||
EmbeddingProfile: embeddingProfile, EmbeddingIdentity: cfg.OllamaEmbeddingModel, ChunkWords: cfg.KnowledgeChunkWords, ChunkOverlap: cfg.KnowledgeChunkOverlapWords, MaxChunksPerDoc: cfg.KnowledgeMaxChunksPerDoc, MaxQueryChunks: cfg.KnowledgeMaxQueryChunks,
|
||||
CategoryMode: cfg.KnowledgeCategoryMode, CategoryMapFile: cfg.KnowledgeCategoryMapFile, IgnoreGlobs: cfg.KnowledgeIgnoreGlobs,
|
||||
})
|
||||
if err != nil {
|
||||
slog.Error("knowledge store initialization failed",
|
||||
@@ -87,6 +88,10 @@ func main() {
|
||||
slog.Error("learning store initialization failed", "error", err)
|
||||
os.Exit(1)
|
||||
}
|
||||
stats := k.LoadStats()
|
||||
if stats.IgnoredFiles > 0 || stats.UnmappedCategoryFiles > 0 {
|
||||
slog.Warn("knowledge loaded with compatibility rules", "ignored_files", stats.IgnoredFiles, "unmapped_category_files", stats.UnmappedCategoryFiles, "unmapped_categories", stats.UnmappedCategories, "category_mode", cfg.KnowledgeCategoryMode)
|
||||
}
|
||||
m := metrics.New()
|
||||
m.SetKnowledgeDocs(k.Count())
|
||||
if cfg.GLPIKBEnabled {
|
||||
|
||||
@@ -5,6 +5,7 @@ import (
|
||||
"fmt"
|
||||
"net/url"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strconv"
|
||||
"strings"
|
||||
"time"
|
||||
@@ -54,6 +55,9 @@ type Config struct {
|
||||
KnowledgeAllowedSources []string
|
||||
KnowledgeAutoReplySources []string
|
||||
KnowledgeWebEditEnabled bool
|
||||
KnowledgeCategoryMode string
|
||||
KnowledgeCategoryMapFile string
|
||||
KnowledgeIgnoreGlobs []string
|
||||
KnowledgeSemanticWeight float64
|
||||
KnowledgeTitleWeight float64
|
||||
KnowledgeLexicalWeight float64
|
||||
@@ -165,6 +169,9 @@ func Load() (Config, error) {
|
||||
KnowledgeAllowedSources: envStringList("KNOWLEDGE_ALLOWED_SOURCES", "internal-kb"),
|
||||
KnowledgeAutoReplySources: envStringList("KNOWLEDGE_AUTO_REPLY_SOURCES", "internal-kb"),
|
||||
KnowledgeWebEditEnabled: envBool("KNOWLEDGE_WEB_EDIT_ENABLED", false),
|
||||
KnowledgeCategoryMode: strings.ToLower(env("KNOWLEDGE_CATEGORY_MODE", "unscoped")),
|
||||
KnowledgeCategoryMapFile: strings.TrimSpace(os.Getenv("KNOWLEDGE_CATEGORY_MAP_FILE")),
|
||||
KnowledgeIgnoreGlobs: envStringListPreserveCase("KNOWLEDGE_IGNORE_GLOBS", ""),
|
||||
KnowledgeSemanticWeight: envFloat("KNOWLEDGE_WEIGHT_SEMANTIC", 0.45),
|
||||
KnowledgeTitleWeight: envFloat("KNOWLEDGE_WEIGHT_TITLE", 0.20),
|
||||
KnowledgeLexicalWeight: envFloat("KNOWLEDGE_WEIGHT_LEXICAL", 0.20),
|
||||
@@ -298,6 +305,16 @@ func (c Config) Validate() error {
|
||||
if c.KnowledgeWebEditEnabled && c.WebAllowAnonymous {
|
||||
return errors.New("KNOWLEDGE_WEB_EDIT_ENABLED requires authenticated dashboard access; WEB_ALLOW_ANONYMOUS must be false")
|
||||
}
|
||||
switch c.KnowledgeCategoryMode {
|
||||
case "", "unscoped", "skip", "strict":
|
||||
default:
|
||||
return errors.New("KNOWLEDGE_CATEGORY_MODE must be one of: unscoped, skip, strict")
|
||||
}
|
||||
for _, pattern := range c.KnowledgeIgnoreGlobs {
|
||||
if _, err := filepath.Match(pattern, "probe.json"); err != nil {
|
||||
return fmt.Errorf("invalid KNOWLEDGE_IGNORE_GLOBS pattern %q: %w", pattern, err)
|
||||
}
|
||||
}
|
||||
if c.KnowledgeTopK != 0 && (c.KnowledgeTopK < 1 || c.KnowledgeTopK > 20) {
|
||||
return errors.New("KNOWLEDGE_TOP_K must be between 1 and 20")
|
||||
}
|
||||
|
||||
@@ -10,6 +10,7 @@ import (
|
||||
"os"
|
||||
"path/filepath"
|
||||
"sort"
|
||||
"strconv"
|
||||
"strings"
|
||||
"sync"
|
||||
"unicode"
|
||||
@@ -32,6 +33,21 @@ type ScoringConfig struct {
|
||||
ChunkOverlap int
|
||||
MaxChunksPerDoc int
|
||||
MaxQueryChunks int
|
||||
CategoryMode string
|
||||
CategoryMapFile string
|
||||
IgnoreGlobs []string
|
||||
}
|
||||
|
||||
type LoadOptions struct {
|
||||
CategoryMode string
|
||||
CategoryMapFile string
|
||||
IgnoreGlobs []string
|
||||
}
|
||||
|
||||
type LoadStats struct {
|
||||
IgnoredFiles int `json:"ignored_files"`
|
||||
UnmappedCategoryFiles int `json:"unmapped_category_files"`
|
||||
UnmappedCategories []string `json:"unmapped_categories,omitempty"`
|
||||
}
|
||||
|
||||
type Store struct {
|
||||
@@ -51,6 +67,9 @@ type Store struct {
|
||||
cachePath string
|
||||
allowedSources map[string]struct{}
|
||||
scoring ScoringConfig
|
||||
loadOptions LoadOptions
|
||||
loadStats LoadStats
|
||||
categoryMap map[string][]int64
|
||||
}
|
||||
type cacheFile struct {
|
||||
Version int `json:"version,omitempty"`
|
||||
@@ -108,22 +127,34 @@ func Load(ctx context.Context, dir, dataDir string, embedder Embedder, rag bool,
|
||||
if len(scoring) > 0 {
|
||||
scoreCfg = normalizeScoring(scoring[0])
|
||||
}
|
||||
s := &Store{dir: dir, managedDir: managedDir, titleVectors: map[string][]float64{}, chunkVectors: map[string][][]float64{}, chunks: map[string][]string{}, files: map[string]string{}, managed: map[string]bool{}, external: map[string]string{}, staticDocs: map[string]model.KnowledgeDoc{}, embedder: embedder, rag: rag, cachePath: filepath.Join(dataDir, "embeddings.json"), allowedSources: map[string]struct{}{}, scoring: scoreCfg}
|
||||
for _, source := range allowedSources {
|
||||
s.allowedSources[strings.ToLower(strings.TrimSpace(source))] = struct{}{}
|
||||
loadOpts := LoadOptions{CategoryMode: scoreCfg.CategoryMode, CategoryMapFile: scoreCfg.CategoryMapFile, IgnoreGlobs: scoreCfg.IgnoreGlobs}
|
||||
if strings.TrimSpace(loadOpts.CategoryMode) == "" {
|
||||
loadOpts.CategoryMode = "unscoped"
|
||||
}
|
||||
static, staticFiles, err := readDocs(dir, s.allowedSources)
|
||||
categoryMap, err := loadCategoryMap(loadOpts.CategoryMapFile)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
s := &Store{dir: dir, managedDir: managedDir, titleVectors: map[string][]float64{}, chunkVectors: map[string][][]float64{}, chunks: map[string][]string{}, files: map[string]string{}, managed: map[string]bool{}, external: map[string]string{}, staticDocs: map[string]model.KnowledgeDoc{}, embedder: embedder, rag: rag, cachePath: filepath.Join(dataDir, "embeddings.json"), allowedSources: map[string]struct{}{}, scoring: scoreCfg, loadOptions: loadOpts, categoryMap: categoryMap}
|
||||
for _, source := range allowedSources {
|
||||
s.allowedSources[strings.ToLower(strings.TrimSpace(source))] = struct{}{}
|
||||
}
|
||||
static, staticFiles, stats, err := readDocs(dir, s.allowedSources, loadOpts, categoryMap)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
s.loadStats = stats
|
||||
for i, d := range static {
|
||||
s.staticDocs[d.ID] = d
|
||||
s.files[d.ID] = staticFiles[i]
|
||||
}
|
||||
managed, managedFiles, err := readDocs(managedDir, s.allowedSources)
|
||||
managed, managedFiles, managedStats, err := readDocs(managedDir, s.allowedSources, LoadOptions{CategoryMode: "strict"}, categoryMap)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
s.loadStats.IgnoredFiles += managedStats.IgnoredFiles
|
||||
s.loadStats.UnmappedCategoryFiles += managedStats.UnmappedCategoryFiles
|
||||
s.loadStats.UnmappedCategories = mergeStrings(s.loadStats.UnmappedCategories, managedStats.UnmappedCategories)
|
||||
merged := map[string]model.KnowledgeDoc{}
|
||||
order := []string{}
|
||||
for _, d := range static {
|
||||
@@ -154,35 +185,48 @@ func Load(ctx context.Context, dir, dataDir string, embedder Embedder, rag bool,
|
||||
return s, nil
|
||||
}
|
||||
|
||||
func readDocs(dir string, allowed map[string]struct{}) ([]model.KnowledgeDoc, []string, error) {
|
||||
func readDocs(dir string, allowed map[string]struct{}, opts LoadOptions, categoryMap map[string][]int64) ([]model.KnowledgeDoc, []string, LoadStats, error) {
|
||||
entries, err := os.ReadDir(dir)
|
||||
if err != nil {
|
||||
return nil, nil, fmt.Errorf("read knowledge directory %q: %w", dir, err)
|
||||
return nil, nil, LoadStats{}, fmt.Errorf("read knowledge directory %q: %w", dir, err)
|
||||
}
|
||||
var docs []model.KnowledgeDoc
|
||||
var files []string
|
||||
stats := LoadStats{}
|
||||
for _, e := range entries {
|
||||
if e.IsDir() || !strings.HasSuffix(strings.ToLower(e.Name()), ".json") {
|
||||
continue
|
||||
}
|
||||
if matchesAnyGlob(e.Name(), opts.IgnoreGlobs) {
|
||||
stats.IgnoredFiles++
|
||||
continue
|
||||
}
|
||||
path := filepath.Join(dir, e.Name())
|
||||
b, err := os.ReadFile(path)
|
||||
if err != nil {
|
||||
return nil, nil, err
|
||||
return nil, nil, stats, err
|
||||
}
|
||||
var d model.KnowledgeDoc
|
||||
if err := json.Unmarshal(b, &d); err != nil {
|
||||
return nil, nil, fmt.Errorf("%s: %w", e.Name(), err)
|
||||
d, unmapped, skip, err := decodeKnowledgeDoc(b, opts.CategoryMode, categoryMap)
|
||||
if err != nil {
|
||||
return nil, nil, stats, fmt.Errorf("%s: %w", e.Name(), err)
|
||||
}
|
||||
if len(unmapped) > 0 {
|
||||
stats.UnmappedCategoryFiles++
|
||||
stats.UnmappedCategories = mergeStrings(stats.UnmappedCategories, unmapped)
|
||||
}
|
||||
if skip {
|
||||
stats.IgnoredFiles++
|
||||
continue
|
||||
}
|
||||
if d.ID == "" || d.Title == "" {
|
||||
return nil, nil, fmt.Errorf("%s: id/title required", e.Name())
|
||||
return nil, nil, stats, fmt.Errorf("%s: id/title required", e.Name())
|
||||
}
|
||||
if !safeID(d.ID) {
|
||||
return nil, nil, fmt.Errorf("%s: invalid id %q", e.Name(), d.ID)
|
||||
return nil, nil, stats, fmt.Errorf("%s: invalid id %q", e.Name(), d.ID)
|
||||
}
|
||||
d.Source = strings.ToLower(strings.TrimSpace(d.Source))
|
||||
if d.Source == "" {
|
||||
return nil, nil, fmt.Errorf("%s: source required", e.Name())
|
||||
return nil, nil, stats, fmt.Errorf("%s: source required", e.Name())
|
||||
}
|
||||
if _, ok := allowed[d.Source]; !ok {
|
||||
continue
|
||||
@@ -192,7 +236,229 @@ func readDocs(dir string, allowed map[string]struct{}) ([]model.KnowledgeDoc, []
|
||||
docs = append(docs, d)
|
||||
files = append(files, path)
|
||||
}
|
||||
return docs, files, nil
|
||||
return docs, files, stats, nil
|
||||
}
|
||||
|
||||
func decodeKnowledgeDoc(b []byte, mode string, categoryMap map[string][]int64) (model.KnowledgeDoc, []string, bool, error) {
|
||||
var raw map[string]json.RawMessage
|
||||
if err := json.Unmarshal(b, &raw); err != nil {
|
||||
return model.KnowledgeDoc{}, nil, false, err
|
||||
}
|
||||
catRaw := raw["categories"]
|
||||
raw["categories"] = json.RawMessage(`[]`)
|
||||
normalized, err := json.Marshal(raw)
|
||||
if err != nil {
|
||||
return model.KnowledgeDoc{}, nil, false, err
|
||||
}
|
||||
var d model.KnowledgeDoc
|
||||
if err := json.Unmarshal(normalized, &d); err != nil {
|
||||
return model.KnowledgeDoc{}, nil, false, err
|
||||
}
|
||||
ids, labels, unmapped, err := parseKnowledgeCategories(catRaw, categoryMap)
|
||||
if err != nil {
|
||||
return model.KnowledgeDoc{}, nil, false, err
|
||||
}
|
||||
d.Categories = ids
|
||||
d.ExternalCategories = labels
|
||||
d.UnmappedExternalCategories = append([]string(nil), unmapped...)
|
||||
if len(unmapped) == 0 {
|
||||
return d, nil, false, nil
|
||||
}
|
||||
switch strings.ToLower(strings.TrimSpace(mode)) {
|
||||
case "strict":
|
||||
return model.KnowledgeDoc{}, unmapped, false, fmt.Errorf("unmapped external categories: %s", strings.Join(unmapped, ", "))
|
||||
case "skip":
|
||||
return d, unmapped, true, nil
|
||||
default:
|
||||
// Unmapped external taxonomies remain searchable, but may not trigger an automatic reply.
|
||||
d.AutoReply = false
|
||||
return d, unmapped, false, nil
|
||||
}
|
||||
}
|
||||
|
||||
func parseKnowledgeCategories(raw json.RawMessage, categoryMap map[string][]int64) ([]int64, []string, []string, error) {
|
||||
if len(raw) == 0 || string(raw) == "null" {
|
||||
return nil, nil, nil, nil
|
||||
}
|
||||
var v any
|
||||
if err := json.Unmarshal(raw, &v); err != nil {
|
||||
return nil, nil, nil, err
|
||||
}
|
||||
items, ok := v.([]any)
|
||||
if !ok {
|
||||
items = []any{v}
|
||||
}
|
||||
ids := []int64{}
|
||||
labels := []string{}
|
||||
unmapped := []string{}
|
||||
for _, item := range items {
|
||||
itemIDs, label, err := parseCategoryItem(item, categoryMap)
|
||||
if err != nil {
|
||||
return nil, nil, nil, err
|
||||
}
|
||||
ids = append(ids, itemIDs...)
|
||||
if label != "" {
|
||||
labels = appendUniqueString(labels, label)
|
||||
if len(itemIDs) == 0 {
|
||||
unmapped = appendUniqueString(unmapped, label)
|
||||
}
|
||||
}
|
||||
}
|
||||
return uniqueInt64(ids), labels, unmapped, nil
|
||||
}
|
||||
|
||||
func parseCategoryItem(v any, categoryMap map[string][]int64) ([]int64, string, error) {
|
||||
switch x := v.(type) {
|
||||
case nil:
|
||||
return nil, "", nil
|
||||
case float64:
|
||||
if x <= 0 || math.Trunc(x) != x {
|
||||
return nil, "", fmt.Errorf("category id must be a positive integer")
|
||||
}
|
||||
return []int64{int64(x)}, "", nil
|
||||
case string:
|
||||
label := strings.TrimSpace(x)
|
||||
if label == "" {
|
||||
return nil, "", nil
|
||||
}
|
||||
if n, err := strconv.ParseInt(label, 10, 64); err == nil && n > 0 {
|
||||
return []int64{n}, "", nil
|
||||
}
|
||||
return append([]int64(nil), categoryMap[normalizeCategoryLabel(label)]...), label, nil
|
||||
case map[string]any:
|
||||
if id, ok := x["id"]; ok {
|
||||
ids, _, err := parseCategoryItem(id, categoryMap)
|
||||
if err == nil && len(ids) > 0 {
|
||||
return ids, "", nil
|
||||
}
|
||||
}
|
||||
for _, key := range []string{"name", "label", "title"} {
|
||||
if val, ok := x[key].(string); ok {
|
||||
return parseCategoryItem(val, categoryMap)
|
||||
}
|
||||
}
|
||||
return nil, "", fmt.Errorf("unsupported category object")
|
||||
default:
|
||||
return nil, "", fmt.Errorf("unsupported category value type %T", v)
|
||||
}
|
||||
}
|
||||
|
||||
func loadCategoryMap(path string) (map[string][]int64, error) {
|
||||
out := map[string][]int64{}
|
||||
path = strings.TrimSpace(path)
|
||||
if path == "" {
|
||||
return out, nil
|
||||
}
|
||||
b, err := os.ReadFile(path)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("read knowledge category map %q: %w", path, err)
|
||||
}
|
||||
var root map[string]json.RawMessage
|
||||
if err := json.Unmarshal(b, &root); err != nil {
|
||||
return nil, fmt.Errorf("parse knowledge category map %q: %w", path, err)
|
||||
}
|
||||
if nested, ok := root["mappings"]; ok {
|
||||
var m map[string]json.RawMessage
|
||||
if err := json.Unmarshal(nested, &m); err != nil {
|
||||
return nil, fmt.Errorf("parse mappings in %q: %w", path, err)
|
||||
}
|
||||
root = m
|
||||
}
|
||||
for label, rv := range root {
|
||||
ids, err := parseMappingIDs(rv)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("category mapping %q: %w", label, err)
|
||||
}
|
||||
if len(ids) == 0 {
|
||||
return nil, fmt.Errorf("category mapping %q contains no positive GLPI ids", label)
|
||||
}
|
||||
out[normalizeCategoryLabel(label)] = uniqueInt64(ids)
|
||||
}
|
||||
return out, nil
|
||||
}
|
||||
|
||||
func parseMappingIDs(raw json.RawMessage) ([]int64, error) {
|
||||
var v any
|
||||
if err := json.Unmarshal(raw, &v); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
items, ok := v.([]any)
|
||||
if !ok {
|
||||
items = []any{v}
|
||||
}
|
||||
ids := []int64{}
|
||||
for _, item := range items {
|
||||
switch x := item.(type) {
|
||||
case float64:
|
||||
if x <= 0 || math.Trunc(x) != x {
|
||||
return nil, fmt.Errorf("id must be a positive integer")
|
||||
}
|
||||
ids = append(ids, int64(x))
|
||||
case string:
|
||||
n, err := strconv.ParseInt(strings.TrimSpace(x), 10, 64)
|
||||
if err != nil || n <= 0 {
|
||||
return nil, fmt.Errorf("%q is not a positive GLPI category id", x)
|
||||
}
|
||||
ids = append(ids, n)
|
||||
default:
|
||||
return nil, fmt.Errorf("unsupported mapping value type %T", item)
|
||||
}
|
||||
}
|
||||
return ids, nil
|
||||
}
|
||||
|
||||
func matchesAnyGlob(name string, patterns []string) bool {
|
||||
for _, pattern := range patterns {
|
||||
if ok, _ := filepath.Match(pattern, name); ok {
|
||||
return true
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
func normalizeCategoryLabel(s string) string {
|
||||
return strings.ToLower(strings.Join(strings.Fields(s), " "))
|
||||
}
|
||||
func uniqueInt64(in []int64) []int64 {
|
||||
seen := map[int64]struct{}{}
|
||||
out := []int64{}
|
||||
for _, v := range in {
|
||||
if v <= 0 {
|
||||
continue
|
||||
}
|
||||
if _, ok := seen[v]; ok {
|
||||
continue
|
||||
}
|
||||
seen[v] = struct{}{}
|
||||
out = append(out, v)
|
||||
}
|
||||
return out
|
||||
}
|
||||
func appendUniqueString(in []string, s string) []string {
|
||||
for _, v := range in {
|
||||
if strings.EqualFold(v, s) {
|
||||
return in
|
||||
}
|
||||
}
|
||||
return append(in, s)
|
||||
}
|
||||
func mergeStrings(a, b []string) []string {
|
||||
out := append([]string(nil), a...)
|
||||
for _, s := range b {
|
||||
out = appendUniqueString(out, s)
|
||||
}
|
||||
sort.Strings(out)
|
||||
return out
|
||||
}
|
||||
|
||||
func (s *Store) LoadStats() LoadStats {
|
||||
if s == nil {
|
||||
return LoadStats{}
|
||||
}
|
||||
s.mu.RLock()
|
||||
defer s.mu.RUnlock()
|
||||
out := s.loadStats
|
||||
out.UnmappedCategories = append([]string(nil), s.loadStats.UnmappedCategories...)
|
||||
return out
|
||||
}
|
||||
|
||||
func (s *Store) Count() int {
|
||||
@@ -1206,7 +1472,7 @@ func cosine(a, b []float64) float64 {
|
||||
}
|
||||
func lexical(text string, d model.KnowledgeDoc) float64 {
|
||||
q := tokens(text)
|
||||
hay := tokens(d.Title + " " + d.Text + " " + strings.Join(d.Keywords, " "))
|
||||
hay := tokens(d.Title + " " + d.Text + " " + strings.Join(d.Keywords, " ") + " " + strings.Join(d.ExternalCategories, " "))
|
||||
if len(q) == 0 {
|
||||
return 0
|
||||
}
|
||||
|
||||
@@ -310,3 +310,85 @@ func TestRegressionShortGermanLoginTicketGetsStrongLexicalEvidence(t *testing.T)
|
||||
t.Fatalf("anmelden/Benutzeranmeldung should match through a German support stem, got %f", sim)
|
||||
}
|
||||
}
|
||||
|
||||
func TestExternalStringCategoryLoadsUnscoped(t *testing.T) {
|
||||
dir := t.TempDir()
|
||||
data := t.TempDir()
|
||||
body := `{"id":"KB-EXT-1","title":"Docker Test","text":"Docker Fehler","answer":"Pruefen","auto_reply":false,"min_score":0.7,"categories":["Docker","Security"],"keywords":["docker"],"source":"internal-kb","language":"de-DE","communication_style":"formal"}`
|
||||
if err := os.WriteFile(filepath.Join(dir, "ext.json"), []byte(body), 0o644); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
s, err := Load(context.Background(), dir, data, nil, false, []string{"internal-kb"}, ScoringConfig{CategoryMode: "unscoped"})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
doc, ok := s.ByID("KB-EXT-1")
|
||||
if !ok {
|
||||
t.Fatal("document not loaded")
|
||||
}
|
||||
if len(doc.Categories) != 0 {
|
||||
t.Fatalf("expected no GLPI ids, got %v", doc.Categories)
|
||||
}
|
||||
if len(doc.ExternalCategories) != 2 {
|
||||
t.Fatalf("external categories=%v", doc.ExternalCategories)
|
||||
}
|
||||
stats := s.LoadStats()
|
||||
if stats.UnmappedCategoryFiles != 1 {
|
||||
t.Fatalf("stats=%+v", stats)
|
||||
}
|
||||
}
|
||||
|
||||
func TestExternalStringCategoryMapsToGLPI(t *testing.T) {
|
||||
dir := t.TempDir()
|
||||
data := t.TempDir()
|
||||
mapPath := filepath.Join(data, "category-map.json")
|
||||
if err := os.WriteFile(mapPath, []byte(`{"Docker":[12,13],"Security":7}`), 0o644); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
body := `{"id":"KB-EXT-2","title":"Docker Test","text":"Docker Fehler","answer":"Pruefen","auto_reply":false,"min_score":0.7,"categories":["Docker","Security"],"keywords":["docker"],"source":"internal-kb","language":"de-DE","communication_style":"formal"}`
|
||||
if err := os.WriteFile(filepath.Join(dir, "ext.json"), []byte(body), 0o644); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
s, err := Load(context.Background(), dir, data, nil, false, []string{"internal-kb"}, ScoringConfig{CategoryMode: "unscoped", CategoryMapFile: mapPath})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
doc, _ := s.ByID("KB-EXT-2")
|
||||
want := []int64{12, 13, 7}
|
||||
if len(doc.Categories) != len(want) {
|
||||
t.Fatalf("categories=%v", doc.Categories)
|
||||
}
|
||||
for _, id := range want {
|
||||
found := false
|
||||
for _, got := range doc.Categories {
|
||||
if got == id {
|
||||
found = true
|
||||
}
|
||||
}
|
||||
if !found {
|
||||
t.Fatalf("missing id %d in %v", id, doc.Categories)
|
||||
}
|
||||
}
|
||||
if s.LoadStats().UnmappedCategoryFiles != 0 {
|
||||
t.Fatalf("unexpected unmapped stats: %+v", s.LoadStats())
|
||||
}
|
||||
}
|
||||
|
||||
func TestKnowledgeIgnoreGlobs(t *testing.T) {
|
||||
dir := t.TempDir()
|
||||
data := t.TempDir()
|
||||
bad := `{"id":"KB-BAD","title":"Foreign","text":"x","answer":"x","auto_reply":false,"min_score":0.7,"categories":[{"unsupported":true}],"source":"internal-kb","language":"de-DE","communication_style":"formal"}`
|
||||
if err := os.WriteFile(filepath.Join(dir, "KB-SEC-ATTCK-AN-0001.json"), []byte(bad), 0o644); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
s, err := Load(context.Background(), dir, data, nil, false, []string{"internal-kb"}, ScoringConfig{CategoryMode: "strict", IgnoreGlobs: []string{"KB-SEC-ATTCK-*.json"}})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if s.Count() != 0 {
|
||||
t.Fatalf("count=%d", s.Count())
|
||||
}
|
||||
if s.LoadStats().IgnoredFiles != 1 {
|
||||
t.Fatalf("stats=%+v", s.LoadStats())
|
||||
}
|
||||
}
|
||||
|
||||
@@ -60,17 +60,19 @@ type KnowledgeDoc struct {
|
||||
Answer string `json:"answer"`
|
||||
// AnswerHTML contains trusted rich text from a synchronized GLPI KB item.
|
||||
// It is never sent to the LLM or used for embeddings.
|
||||
AnswerHTML string `json:"answer_html,omitempty"`
|
||||
AutoReply bool `json:"auto_reply"`
|
||||
MinScore float64 `json:"min_score"`
|
||||
Categories []int64 `json:"categories"`
|
||||
Keywords []string `json:"keywords"`
|
||||
Source string `json:"source"`
|
||||
SourceURI string `json:"source_uri,omitempty"`
|
||||
SourceCategoryIDs []int64 `json:"source_category_ids,omitempty"`
|
||||
SourceModifiedAt string `json:"source_modified_at,omitempty"`
|
||||
Language string `json:"language"`
|
||||
CommunicationStyle string `json:"communication_style"`
|
||||
AnswerHTML string `json:"answer_html,omitempty"`
|
||||
AutoReply bool `json:"auto_reply"`
|
||||
MinScore float64 `json:"min_score"`
|
||||
Categories []int64 `json:"categories"`
|
||||
ExternalCategories []string `json:"external_categories,omitempty"`
|
||||
UnmappedExternalCategories []string `json:"unmapped_external_categories,omitempty"`
|
||||
Keywords []string `json:"keywords"`
|
||||
Source string `json:"source"`
|
||||
SourceURI string `json:"source_uri,omitempty"`
|
||||
SourceCategoryIDs []int64 `json:"source_category_ids,omitempty"`
|
||||
SourceModifiedAt string `json:"source_modified_at,omitempty"`
|
||||
Language string `json:"language"`
|
||||
CommunicationStyle string `json:"communication_style"`
|
||||
}
|
||||
|
||||
// GLPIKnowledgeItem is the normalized read-only representation returned by
|
||||
|
||||
@@ -18,6 +18,7 @@ import (
|
||||
"time"
|
||||
|
||||
"github.com/example/glpi-ai-agent/internal/config"
|
||||
knowledgepkg "github.com/example/glpi-ai-agent/internal/knowledge"
|
||||
"github.com/example/glpi-ai-agent/internal/metrics"
|
||||
"github.com/example/glpi-ai-agent/internal/model"
|
||||
"github.com/example/glpi-ai-agent/internal/queue"
|
||||
@@ -34,6 +35,7 @@ type KnowledgeManager interface {
|
||||
Delete(string) error
|
||||
IsManaged(string) bool
|
||||
Origin(string) string
|
||||
LoadStats() knowledgepkg.LoadStats
|
||||
}
|
||||
type FeedbackManager interface {
|
||||
Categories(context.Context) ([]model.Category, error)
|
||||
@@ -107,11 +109,12 @@ func (s *Server) dashboard(w http.ResponseWriter, r *http.Request) {
|
||||
func (s *Server) status(w http.ResponseWriter, r *http.Request) {
|
||||
g, o := s.metrics.Health()
|
||||
kbOK, kbDocs, kbLastSync, kbLastErr := s.metrics.GLPIKBStatus()
|
||||
loadStats := s.knowledge.LoadStats()
|
||||
respondJSON(w, map[string]any{
|
||||
"uptime_seconds": int(time.Since(s.metrics.Started).Seconds()), "dry_run": s.cfg.DryRun, "auto_reply": s.cfg.AutoReply, "auto_category": s.cfg.AutoCategory,
|
||||
"processed": s.metrics.Processed.Load(), "skipped": s.metrics.Skipped.Load(), "errors": s.metrics.Errors.Load(), "category_changes": s.metrics.CategoryChanged.Load(), "replies": s.metrics.Replies.Load(), "queue_depth": s.q.Len(),
|
||||
"glpi_ok": g, "ollama_ok": o, "knowledge_docs": s.metrics.KnowledgeDocs(), "last_poll": s.metrics.LastPoll(),
|
||||
"communication_language": s.cfg.CommunicationLanguage, "communication_style": s.cfg.CommunicationStyle, "knowledge_allowed_sources": s.cfg.KnowledgeAllowedSources, "knowledge_auto_reply_sources": s.cfg.KnowledgeAutoReplySources,
|
||||
"communication_language": s.cfg.CommunicationLanguage, "communication_style": s.cfg.CommunicationStyle, "knowledge_allowed_sources": s.cfg.KnowledgeAllowedSources, "knowledge_auto_reply_sources": s.cfg.KnowledgeAutoReplySources, "knowledge_category_mode": s.cfg.KnowledgeCategoryMode, "knowledge_category_map_configured": strings.TrimSpace(s.cfg.KnowledgeCategoryMapFile) != "", "knowledge_ignore_globs": s.cfg.KnowledgeIgnoreGlobs, "knowledge_ignored_files": loadStats.IgnoredFiles, "knowledge_unmapped_category_files": loadStats.UnmappedCategoryFiles, "knowledge_unmapped_categories": loadStats.UnmappedCategories,
|
||||
"category_confidence": s.cfg.CategoryConfidence, "reply_confidence": s.cfg.ReplyConfidence, "knowledge_min_score": s.cfg.KnowledgeMinScore, "knowledge_retrieval_floor": s.cfg.KnowledgeRetrievalFloor, "knowledge_evidence_weight_retrieval": s.cfg.KnowledgeEvidenceRetrievalWeight, "knowledge_evidence_weight_ai": s.cfg.KnowledgeEvidenceAIWeight, "knowledge_evidence_weight_category": s.cfg.KnowledgeEvidenceCategoryWeight,
|
||||
"knowledge_weight_semantic": s.cfg.KnowledgeSemanticWeight, "knowledge_weight_title": s.cfg.KnowledgeTitleWeight, "knowledge_weight_lexical": s.cfg.KnowledgeLexicalWeight, "knowledge_weight_keywords": s.cfg.KnowledgeKeywordWeight, "knowledge_weight_category": s.cfg.KnowledgeCategoryWeight, "knowledge_embedding_profile": s.cfg.KnowledgeEmbeddingProfile,
|
||||
"knowledge_chunk_words": s.cfg.KnowledgeChunkWords, "knowledge_chunk_overlap_words": s.cfg.KnowledgeChunkOverlapWords, "knowledge_max_chunks_per_doc": s.cfg.KnowledgeMaxChunksPerDoc,
|
||||
|
||||
@@ -118,7 +118,7 @@ function configNotice(text,kind=''){return `<div class="notice ${kind}">${text}<
|
||||
function renderStatusChrome(){const g=!!statusData.glpi_ok,o=!!statusData.ollama_ok;$('#glpiChip').innerHTML=`<span class="status-dot ${g?'ok':'bad'}"></span>GLPI ${g?'OK':'Fehler'}`;$('#ollamaChip').innerHTML=`<span class="status-dot ${o?'ok':'bad'}"></span>Ollama ${o?'OK':'Fehler'}`;$('#sideMode').innerHTML=`${statusData.dry_run?badge('DRY RUN','warn'):badge('LIVE','good')} ${statusData.auto_reply?badge('Auto-Reply','good'):badge('Auto-Reply aus','warn')}<div style="margin-top:8px">${esc(statusData.ollama_model||'–')} · ${esc(statusData.communication_language||'–')} / ${esc(statusData.communication_style||'–')}</div>`;$('#lastRefresh').textContent=new Date().toLocaleTimeString('de-DE')}
|
||||
function renderOverview(){const stats=[['Verarbeitet',fmtNum(statusData.processed),'seit Start'],['Fehler',fmtNum(statusData.errors),statusData.errors?'prüfen':'keine'],['Queue',fmtNum(statusData.queue_depth),`von ${fmtNum(statusData.queue_size)}`],['Knowledge',fmtNum(statusData.knowledge_docs),`${fmtNum(statusData.glpi_kb_documents)} aus GLPI`],['Lernbeispiele',fmtNum(statusData.learning_examples),`${fmtNum(statusData.learning_examples_per_category)} je Kategorie im Prompt`],['Auto-Aktionen',`${fmtNum(statusData.category_changes)} / ${fmtNum(statusData.replies)}`,'Kategorie / Antwort']];$('#overviewStats').innerHTML=stats.map(x=>`<div class="stat"><div class="stat-label">${esc(x[0])}</div><div class="stat-value">${esc(x[1])}</div><div class="stat-foot">${esc(x[2])}</div></div>`).join('');
|
||||
const recent=runsData.slice(0,6);$('#recentRuns').innerHTML=recent.length?recent.map(x=>{const score=x.knowledge_score?` · KB ${pct(x.knowledge_score)}`:'';return `<div class="health-row click-row" data-run-id="${esc(x.run_id)}"><div><div class="ticket-title">#${esc(x.ticket_id)} ${esc(x.ticket_name||'')}</div><div class="health-meta">${esc(fmtDate(x.finished_at))}${esc(score)}</div></div>${outcomeBadge(x.outcome)}</div>`}).join(''):'<div class="empty">Noch keine Verarbeitungen.</div>';
|
||||
const notices=[];if(statusData.dry_run)notices.push(configNotice('<strong>Dry Run aktiv.</strong> Änderungen und Antworten werden nur simuliert.','warn'));if(!statusData.auto_reply)notices.push(configNotice('<strong>Auto-Reply global deaktiviert.</strong> KB-Treffer werden bewertet, aber nicht gesendet.','warn'));if(statusData.knowledge_min_score>=.8)notices.push(configNotice(`<strong>Hoher Evidenz-Schwellwert:</strong> ${pct(statusData.knowledge_min_score)}. Dieser gilt erst nach der KI-Auswahl.`,'warn'));if(statusData.knowledge_retrieval_floor>=.5)notices.push(configNotice(`<strong>Hoher Retrieval-Floor:</strong> ${pct(statusData.knowledge_retrieval_floor)}. Kurze Tickets könnten bereits vor dem Evidenz-Reranking blockiert werden.`,'warn'));if(statusData.glpi_kb_enabled&&!statusData.glpi_kb_ok)notices.push(configNotice(`<strong>GLPI-KB-Sync gestört.</strong> ${esc(statusData.glpi_kb_last_error||'Kein Fehlertext verfügbar.')}`,'bad'));if(statusData.rag_enabled&&!statusData.knowledge_docs)notices.push(configNotice('<strong>RAG aktiv, aber keine Knowledge-Dokumente geladen.</strong>','bad'));if(statusData.context_fail_closed)notices.push(configNotice('Kontextquellen arbeiten <strong>fail-closed</strong>: Fehler können Auto-Replies blockieren.'));if(!notices.length)notices.push(configNotice('Keine auffälligen Konfigurationshinweise erkannt.','good'));$('#diagnosticNotices').innerHTML=notices.join('');
|
||||
const notices=[];if(statusData.dry_run)notices.push(configNotice('<strong>Dry Run aktiv.</strong> Änderungen und Antworten werden nur simuliert.','warn'));if(!statusData.auto_reply)notices.push(configNotice('<strong>Auto-Reply global deaktiviert.</strong> KB-Treffer werden bewertet, aber nicht gesendet.','warn'));if(statusData.knowledge_min_score>=.8)notices.push(configNotice(`<strong>Hoher Evidenz-Schwellwert:</strong> ${pct(statusData.knowledge_min_score)}. Dieser gilt erst nach der KI-Auswahl.`,'warn'));if(statusData.knowledge_retrieval_floor>=.5)notices.push(configNotice(`<strong>Hoher Retrieval-Floor:</strong> ${pct(statusData.knowledge_retrieval_floor)}. Kurze Tickets könnten bereits vor dem Evidenz-Reranking blockiert werden.`,'warn'));if(statusData.glpi_kb_enabled&&!statusData.glpi_kb_ok)notices.push(configNotice(`<strong>GLPI-KB-Sync gestört.</strong> ${esc(statusData.glpi_kb_last_error||'Kein Fehlertext verfügbar.')}`,'bad'));if(statusData.knowledge_unmapped_category_files>0)notices.push(configNotice(`<strong>${esc(statusData.knowledge_unmapped_category_files)} KB-Datei(en) mit nicht zugeordneten Fremdkategorien.</strong> Diese Artikel bleiben suchbar, Auto-Reply ist dafür fail-closed deaktiviert. Nicht zugeordnet: ${esc((statusData.knowledge_unmapped_categories||[]).join(', ')||'–')}`,'warn'));if(statusData.knowledge_ignored_files>0)notices.push(configNotice(`<strong>${esc(statusData.knowledge_ignored_files)} KB-Datei(en) durch Kompatibilitäts-/Ignore-Regeln übersprungen.</strong>`,'warn'));if(statusData.rag_enabled&&!statusData.knowledge_docs)notices.push(configNotice('<strong>RAG aktiv, aber keine Knowledge-Dokumente geladen.</strong>','bad'));if(statusData.context_fail_closed)notices.push(configNotice('Kontextquellen arbeiten <strong>fail-closed</strong>: Fehler können Auto-Replies blockieren.'));if(!notices.length)notices.push(configNotice('Keine auffälligen Konfigurationshinweise erkannt.','good'));$('#diagnosticNotices').innerHTML=notices.join('');
|
||||
const health=[['GLPI API',statusData.glpi_ok,statusData.glpi_api_version||''],['Ollama',statusData.ollama_ok,statusData.ollama_model||''],['GLPI Knowledge Base',!statusData.glpi_kb_enabled||statusData.glpi_kb_ok,statusData.glpi_kb_enabled?`${statusData.glpi_kb_documents||0} Artikel · Sync ${fmtDate(statusData.glpi_kb_last_sync)}`:'deaktiviert'],['Uptime Kuma',true,statusData.uptime_kuma_enabled?`aktiv · ${statusData.uptime_kuma_mode}`:'deaktiviert'],['Change Calendar',true,statusData.change_calendar_enabled?'aktiv':'deaktiviert'],['Major Incidents',true,statusData.major_incidents_enabled?'aktiv':'deaktiviert'],['Benutzer ↔ Geräte',true,statusData.user_device_context_enabled?'aktiv':'deaktiviert']];$('#integrationHealth').innerHTML=health.map(x=>`<div class="health-row"><div class="health-name"><span class="status-dot ${x[1]?'ok':'bad'}"></span><div>${esc(x[0])}<div class="health-meta">${esc(x[2])}</div></div></div>${x[1]?badge('OK','good'):badge('Fehler','bad')}</div>`).join('');
|
||||
$('#scoringOverview').innerHTML=`${progress('Retrieval: Semantik',statusData.knowledge_weight_semantic)}${progress('Retrieval: Titel',statusData.knowledge_weight_title)}${progress('Retrieval: Lexikalisch',statusData.knowledge_weight_lexical)}${progress('Retrieval: Keywords',statusData.knowledge_weight_keywords)}${progress('Retrieval: Kategorie/Lernen',statusData.knowledge_weight_category)}<div style="margin-top:14px" class="health-row"><span>Retrieval-Floor</span><strong>${esc(pct(statusData.knowledge_retrieval_floor))}</strong></div><div class="health-row"><span>Finaler Evidenz-Schwellwert</span><strong>${esc(pct(statusData.knowledge_min_score))}</strong></div><div class="health-row"><span>Evidenz: Retrieval / KI / Kategorie</span><strong>${esc(pct(statusData.knowledge_evidence_weight_retrieval))} / ${esc(pct(statusData.knowledge_evidence_weight_ai))} / ${esc(pct(statusData.knowledge_evidence_weight_category))}</strong></div><div class="health-row"><span>Kategorie-Confidence</span><strong>${esc(pct(statusData.category_confidence))}</strong></div><div class="health-row"><span>Reply-Confidence</span><strong>${esc(pct(statusData.reply_confidence))}</strong></div>`}
|
||||
function categoryMini(x){if(!x.ai_recommended_category_id)return `<div class="decision-mini">${badge('Keine Empfehlung','warn')}<div class="muted small">KI-Sicherheit ${pct(x.ai_category_confidence)}</div></div>`;const p=policyLabel(x.category_decision);return `<div class="decision-mini"><div class="decision-line"><strong>${esc(x.ai_recommended_category_name||`#${x.ai_recommended_category_id}`)}</strong> <span class="score ${scoreClass(x.ai_category_confidence)}">${esc(pct(x.ai_category_confidence))}</span></div>${badge(p[0],p[1])}<div class="muted small">Schwellwert ${esc(pct(x.category_threshold))}</div></div>`}
|
||||
@@ -139,11 +139,11 @@ function renderRunDrawer(x){currentRun=x;$('#runDrawerTitle').textContent=`#${x.
|
||||
function openRunDrawer(){ $('#runBackdrop').classList.add('show');$('#runDrawer').classList.add('show') } function closeRunDrawer(){ $('#runBackdrop').classList.remove('show');$('#runDrawer').classList.remove('show');currentRun=null }
|
||||
function kbStatsData(){const managed=kbDocs.filter(x=>x.managed).length,glpi=kbDocs.filter(x=>x.source==='glpi-kb').length,auto=kbDocs.filter(x=>x.auto_reply).length;return [['Gesamt',kbDocs.length,'geladene Artikel'],['Web-verwaltet',managed,'editierbar'],['GLPI-KB',glpi,'read-only synchronisiert'],['Auto-Reply',auto,'grundsätzlich freigegeben']]}
|
||||
function filteredKB(){const q=$('#kbSearch').value.trim().toLowerCase(),src=$('#kbSourceFilter').value,mode=$('#kbManageFilter').value;return kbDocs.filter(d=>(!src||d.source===src)&&(!mode||(mode==='managed'&&d.managed)||(mode==='readonly'&&!d.managed)||(mode==='autoreply'&&d.auto_reply))&&(!q||[d.id,d.title,d.text,d.answer,(d.keywords||[]).join(' '),(d.categories||[]).join(' ')].join(' ').toLowerCase().includes(q)))}
|
||||
function renderKB(){const st=kbStatsData();$('#kbStats').innerHTML=st.map(x=>`<div class="stat"><div class="stat-label">${esc(x[0])}</div><div class="stat-value">${fmtNum(x[1])}</div><div class="stat-foot">${esc(x[2])}</div></div>`).join('');const docs=filteredKB();$('#kbGrid').innerHTML=docs.length?docs.map(d=>`<article class="kb-card"><div style="display:flex;justify-content:space-between;gap:10px"><div><div class="kb-card-title">${esc(d.title)}</div><div class="muted small mono">${esc(d.id)}</div></div>${d.managed?badge('Web','info'):badge(d.origin==='glpi-kb'?'GLPI':'Read-only')}</div><div class="kb-meta">${badge(d.source||'–')}${d.answer_html?badge('Rich Text','good'):badge('Plaintext')}${d.auto_reply?badge('Auto-Reply','good'):badge('kein Auto-Reply','warn')}${badge(`Min ${pct(d.min_score)}`)}</div><div class="kb-snippet">${esc((d.text||'').slice(0,260))}${(d.text||'').length>260?'…':''}</div><div class="muted small" style="margin-top:8px">Kategorien: ${esc((d.categories||[]).join(', ')||'alle')}${(d.source_category_ids||[]).length?` · GLPI-KB: ${esc(d.source_category_ids.join(', '))}`:''}</div><div class="kb-actions">${d.managed?`<button class="btn small" data-kb-edit="${esc(d.id)}">Bearbeiten</button><button class="btn small danger" data-kb-delete="${esc(d.id)}">Löschen</button>`:`<span class="muted small">${d.origin==='glpi-kb'?'wird aus GLPI synchronisiert':'über Datei/Git verwalten'}</span>`}</div></article>`).join(''):'<div class="empty">Keine passenden Knowledge-Einträge.</div>'}
|
||||
function renderKB(){const st=kbStatsData();$('#kbStats').innerHTML=st.map(x=>`<div class="stat"><div class="stat-label">${esc(x[0])}</div><div class="stat-value">${fmtNum(x[1])}</div><div class="stat-foot">${esc(x[2])}</div></div>`).join('');const docs=filteredKB();$('#kbGrid').innerHTML=docs.length?docs.map(d=>`<article class="kb-card"><div style="display:flex;justify-content:space-between;gap:10px"><div><div class="kb-card-title">${esc(d.title)}</div><div class="muted small mono">${esc(d.id)}</div></div>${d.managed?badge('Web','info'):badge(d.origin==='glpi-kb'?'GLPI':'Read-only')}</div><div class="kb-meta">${badge(d.source||'–')}${d.answer_html?badge('Rich Text','good'):badge('Plaintext')}${d.auto_reply?badge('Auto-Reply','good'):badge('kein Auto-Reply','warn')}${badge(`Min ${pct(d.min_score)}`)}</div><div class="kb-snippet">${esc((d.text||'').slice(0,260))}${(d.text||'').length>260?'…':''}</div><div class="muted small" style="margin-top:8px">Kategorien: ${esc((d.categories||[]).join(', ')||'alle')}${(d.external_categories||[]).length?` · Extern: ${esc(d.external_categories.join(', '))}`:''}${(d.unmapped_external_categories||[]).length?` · ⚠ nicht zugeordnet: ${esc(d.unmapped_external_categories.join(', '))}`:''}${(d.source_category_ids||[]).length?` · GLPI-KB: ${esc(d.source_category_ids.join(', '))}`:''}</div><div class="kb-actions">${d.managed?`<button class="btn small" data-kb-edit="${esc(d.id)}">Bearbeiten</button><button class="btn small danger" data-kb-delete="${esc(d.id)}">Löschen</button>`:`<span class="muted small">${d.origin==='glpi-kb'?'wird aus GLPI synchronisiert':'über Datei/Git verwalten'}</span>`}</div></article>`).join(''):'<div class="empty">Keine passenden Knowledge-Einträge.</div>'}
|
||||
function renderLearning(){const q=$('#learningSearch').value.trim().toLowerCase(),rows=learningRows.filter(x=>!q||[x.ticket_id,x.subject,x.text,x.category_name,x.category_id].join(' ').toLowerCase().includes(q));const corrections=learningRows.filter(x=>x.correction).length;$('#learningStats').innerHTML=[['Gesamt',learningRows.length,'bestätigte Beispiele'],['Korrekturen',corrections,'KI lag anders'],['Bestätigungen',learningRows.length-corrections,'KI wurde bestätigt']].map(x=>`<div class="stat"><div class="stat-label">${esc(x[0])}</div><div class="stat-value">${fmtNum(x[1])}</div><div class="stat-foot">${esc(x[2])}</div></div>`).join('');$('#learningTable').innerHTML=rows.length?rows.map(x=>`<tr><td><div class="ticket-title">#${esc(x.ticket_id)} ${esc(x.subject)}</div><div class="muted small">${esc((x.text||'').slice(0,220))}</div></td><td><strong>${esc(x.category_name)}</strong> (#${esc(x.category_id)})${x.ai_recommended_category_id?`<div class="muted small">KI: #${esc(x.ai_recommended_category_id)} · ${esc(pct(x.ai_confidence))}</div>`:''}</td><td>${x.correction?badge('Korrektur','warn'):badge('Bestätigung','good')}</td><td class="nowrap">${esc(fmtDate(x.created_at))}</td><td><button class="btn small danger" data-learning-delete="${esc(x.id)}">Löschen</button></td></tr>`).join(''):'<tr><td colspan="5" class="empty">Keine Lernbeispiele.</td></tr>'}
|
||||
function configCard(title,subtitle,rows){return `<div class="panel config-card"><div class="panel-head"><div><div class="panel-title">${esc(title)}</div><div class="panel-sub">${esc(subtitle)}</div></div></div><div class="config-list">${rows.map(([k,v])=>`<div class="config-row"><div class="config-key">${esc(k)}</div><div class="config-val">${v}</div></div>`).join('')}</div></div>`}
|
||||
function val(v){if(typeof v==='boolean')return v?badge('aktiv','good'):badge('aus','warn');if(Array.isArray(v))return esc(v.length?v.join(', '):'–');return esc(v??'–')}
|
||||
function renderConfig(){const s=statusData;const groups=[configCard('Agent & GLPI','Polling, Worker und Schreibmodus',[['Dry Run',val(s.dry_run)],['Auto-Kategorie',val(s.auto_category)],['Auto-Reply',val(s.auto_reply)],['Worker',val(s.workers)],['Queue-Größe',val(s.queue_size)],['API-Version',val(s.glpi_api_version)],['Poll-Intervall',val(s.glpi_poll_interval)],['Poll-Limit',val(s.glpi_poll_limit)],['Ticket-Filter gesetzt',val(s.glpi_ticket_filter_configured)],['Erlaubte Status',val(s.glpi_allowed_status_ids)],['GLPI-Timeout',val(s.glpi_timeout)]]),configCard('Ollama','Modelle und Inferenzbudget',[['Chat-Modell',val(s.ollama_model)],['Embedding-Modell',val(s.ollama_embedding_model)],['Embedding-Profil',val(s.knowledge_embedding_profile)],['Timeout',val(s.ollama_timeout)],['Num Predict',val(s.ollama_num_predict)],['Keep Alive',val(s.ollama_keep_alive)],['Thinking',val(s.ollama_think)],['Max. parallel',val(s.ollama_max_concurrent)],['JSON-Retries',val(s.ollama_json_retries)]]),configCard('Knowledge / RAG','Retrieval, Chunking und Ranking',[['RAG',val(s.rag_enabled)],['Max. Kandidaten an KI',val(s.knowledge_top_k)],['Audit Top K',val(s.knowledge_audit_top_k)],['Max. Abstand zum Top-Treffer',pct(s.knowledge_candidate_max_gap)],['Finaler Evidenz-Schwellwert',`<strong>${pct(s.knowledge_min_score)}</strong>`],['Retrieval-Floor',`<strong>${pct(s.knowledge_retrieval_floor)}</strong>`],['Evidenzgewicht Retrieval',pct(s.knowledge_evidence_weight_retrieval)],['Evidenzgewicht KI',pct(s.knowledge_evidence_weight_ai)],['Evidenzgewicht Kategorie',pct(s.knowledge_evidence_weight_category)],['Retrieval: Semantik',pct(s.knowledge_weight_semantic)],['Retrieval: Titel',pct(s.knowledge_weight_title)],['Retrieval: Lexikalisch',pct(s.knowledge_weight_lexical)],['Retrieval: Keywords',pct(s.knowledge_weight_keywords)],['Retrieval: Kategorie/Lernen',pct(s.knowledge_weight_category)],['Chunk-Wörter',val(s.knowledge_chunk_words)],['Overlap-Wörter',val(s.knowledge_chunk_overlap_words)],['Max. KB-Chunks',val(s.knowledge_max_chunks_per_doc)],['Max. Ticket-Chunks',val(s.knowledge_max_query_chunks)],['Erlaubte Quellen',val(s.knowledge_allowed_sources)],['Auto-Reply-Quellen',val(s.knowledge_auto_reply_sources)]]),configCard('GLPI Knowledge Base','Synchronisation der GLPI-Wissensdatenbank',[['Aktiv',val(s.glpi_kb_enabled)],['Sync OK',val(s.glpi_kb_ok)],['Dokumente',val(s.glpi_kb_documents)],['Letzter Sync',val(fmtDate(s.glpi_kb_last_sync))],['Intervall',val(s.glpi_kb_sync_interval)],['Pfad',val(s.glpi_kb_path)],['Filter gesetzt',val(s.glpi_kb_filter_configured)],['Limit',val(s.glpi_kb_limit)],['Auto-Reply',val(s.glpi_kb_auto_reply)],['Auto-Reply-Kategorien',val(s.glpi_kb_auto_reply_category_ids)],['Letzter Fehler',val(s.glpi_kb_last_error||'–')]]),configCard('Policy & Kommunikation','Entscheidungsschwellen und Sprache',[['Kategorie-Confidence',`<strong>${pct(s.category_confidence)}</strong>`],['Reply-Confidence',`<strong>${pct(s.reply_confidence)}</strong>`],['Sprache',val(s.communication_language)],['Stil',val(s.communication_style)],['KB-Webeditor',val(s.knowledge_edit_enabled)],['Lernen',val(s.learning_enabled)],['Max. Lernbeispiele',val(s.learning_max_examples)],['Beispiele/Kategorie',val(s.learning_examples_per_category)]]),configCard('Kontextquellen','Störungen, Changes, Incidents und Geräte',[['Kontext aktiv',val(s.context_enabled)],['Timeout',val(s.context_timeout)],['Relevanz-Minimum',pct(s.context_relevance_min_score)],['Fail-closed',val(s.context_fail_closed)],['Incident blockiert Reply',val(s.context_incident_block)],['Change Calendar',val(s.change_calendar_enabled)],['Lookback',val(s.change_lookback)],['Lookahead',val(s.change_lookahead)],['Major Incidents',val(s.major_incidents_enabled)],['Benutzer-Geräte',val(s.user_device_context_enabled)],['Uptime Kuma',val(s.uptime_kuma_enabled)],['Uptime-Modus',val(s.uptime_kuma_mode)]] )];$('#configGroups').innerHTML=groups.join('')}
|
||||
function renderConfig(){const s=statusData;const groups=[configCard('Agent & GLPI','Polling, Worker und Schreibmodus',[['Dry Run',val(s.dry_run)],['Auto-Kategorie',val(s.auto_category)],['Auto-Reply',val(s.auto_reply)],['Worker',val(s.workers)],['Queue-Größe',val(s.queue_size)],['API-Version',val(s.glpi_api_version)],['Poll-Intervall',val(s.glpi_poll_interval)],['Poll-Limit',val(s.glpi_poll_limit)],['Ticket-Filter gesetzt',val(s.glpi_ticket_filter_configured)],['Erlaubte Status',val(s.glpi_allowed_status_ids)],['GLPI-Timeout',val(s.glpi_timeout)]]),configCard('Ollama','Modelle und Inferenzbudget',[['Chat-Modell',val(s.ollama_model)],['Embedding-Modell',val(s.ollama_embedding_model)],['Embedding-Profil',val(s.knowledge_embedding_profile)],['Timeout',val(s.ollama_timeout)],['Num Predict',val(s.ollama_num_predict)],['Keep Alive',val(s.ollama_keep_alive)],['Thinking',val(s.ollama_think)],['Max. parallel',val(s.ollama_max_concurrent)],['JSON-Retries',val(s.ollama_json_retries)]]),configCard('Knowledge / RAG','Retrieval, Chunking und Ranking',[['RAG',val(s.rag_enabled)],['Max. Kandidaten an KI',val(s.knowledge_top_k)],['Audit Top K',val(s.knowledge_audit_top_k)],['Max. Abstand zum Top-Treffer',pct(s.knowledge_candidate_max_gap)],['Finaler Evidenz-Schwellwert',`<strong>${pct(s.knowledge_min_score)}</strong>`],['Retrieval-Floor',`<strong>${pct(s.knowledge_retrieval_floor)}</strong>`],['Evidenzgewicht Retrieval',pct(s.knowledge_evidence_weight_retrieval)],['Evidenzgewicht KI',pct(s.knowledge_evidence_weight_ai)],['Evidenzgewicht Kategorie',pct(s.knowledge_evidence_weight_category)],['Retrieval: Semantik',pct(s.knowledge_weight_semantic)],['Retrieval: Titel',pct(s.knowledge_weight_title)],['Retrieval: Lexikalisch',pct(s.knowledge_weight_lexical)],['Retrieval: Keywords',pct(s.knowledge_weight_keywords)],['Retrieval: Kategorie/Lernen',pct(s.knowledge_weight_category)],['Chunk-Wörter',val(s.knowledge_chunk_words)],['Overlap-Wörter',val(s.knowledge_chunk_overlap_words)],['Max. KB-Chunks',val(s.knowledge_max_chunks_per_doc)],['Max. Ticket-Chunks',val(s.knowledge_max_query_chunks)],['Erlaubte Quellen',val(s.knowledge_allowed_sources)],['Auto-Reply-Quellen',val(s.knowledge_auto_reply_sources)],['Fremdkategorie-Modus',val(s.knowledge_category_mode)],['Kategorie-Mapping',val(s.knowledge_category_map_configured?'konfiguriert':'–')],['Ignore-Globs',val(s.knowledge_ignore_globs)],['Ignorierte Dateien',val(s.knowledge_ignored_files)],['KBs mit ungemappten Kategorien',val(s.knowledge_unmapped_category_files)],['Ungemappte Kategorien',val(s.knowledge_unmapped_categories)]]),configCard('GLPI Knowledge Base','Synchronisation der GLPI-Wissensdatenbank',[['Aktiv',val(s.glpi_kb_enabled)],['Sync OK',val(s.glpi_kb_ok)],['Dokumente',val(s.glpi_kb_documents)],['Letzter Sync',val(fmtDate(s.glpi_kb_last_sync))],['Intervall',val(s.glpi_kb_sync_interval)],['Pfad',val(s.glpi_kb_path)],['Filter gesetzt',val(s.glpi_kb_filter_configured)],['Limit',val(s.glpi_kb_limit)],['Auto-Reply',val(s.glpi_kb_auto_reply)],['Auto-Reply-Kategorien',val(s.glpi_kb_auto_reply_category_ids)],['Letzter Fehler',val(s.glpi_kb_last_error||'–')]]),configCard('Policy & Kommunikation','Entscheidungsschwellen und Sprache',[['Kategorie-Confidence',`<strong>${pct(s.category_confidence)}</strong>`],['Reply-Confidence',`<strong>${pct(s.reply_confidence)}</strong>`],['Sprache',val(s.communication_language)],['Stil',val(s.communication_style)],['KB-Webeditor',val(s.knowledge_edit_enabled)],['Lernen',val(s.learning_enabled)],['Max. Lernbeispiele',val(s.learning_max_examples)],['Beispiele/Kategorie',val(s.learning_examples_per_category)]]),configCard('Kontextquellen','Störungen, Changes, Incidents und Geräte',[['Kontext aktiv',val(s.context_enabled)],['Timeout',val(s.context_timeout)],['Relevanz-Minimum',pct(s.context_relevance_min_score)],['Fail-closed',val(s.context_fail_closed)],['Incident blockiert Reply',val(s.context_incident_block)],['Change Calendar',val(s.change_calendar_enabled)],['Lookback',val(s.change_lookback)],['Lookahead',val(s.change_lookahead)],['Major Incidents',val(s.major_incidents_enabled)],['Benutzer-Geräte',val(s.user_device_context_enabled)],['Uptime Kuma',val(s.uptime_kuma_enabled)],['Uptime-Modus',val(s.uptime_kuma_mode)]] )];$('#configGroups').innerHTML=groups.join('')}
|
||||
function renderSourceOptions(){const filterOld=$('#kbSourceFilter').value,sourceOld=$('#kbSource').value;const sources=[...new Set(kbDocs.map(x=>x.source).filter(Boolean))].sort();$('#kbSourceFilter').innerHTML='<option value="">Alle Quellen</option>'+sources.map(x=>`<option value="${esc(x)}">${esc(x)}</option>`).join('');if([...$('#kbSourceFilter').options].some(o=>o.value===filterOld))$('#kbSourceFilter').value=filterOld;const allowed=statusData.knowledge_allowed_sources||[];$('#kbSource').innerHTML=allowed.map(x=>`<option value="${esc(x)}">${esc(x)}</option>`).join('');if([...$('#kbSource').options].some(o=>o.value===sourceOld))$('#kbSource').value=sourceOld;else if([...$('#kbSource').options].some(o=>o.value==='internal-kb'))$('#kbSource').value='internal-kb'}
|
||||
function renderCategoryPicker(filter=''){const q=filter.toLowerCase();$('#kbCategoryList').innerHTML=categories.filter(c=>!q||(c.completename||c.name||'').toLowerCase().includes(q)).map(c=>`<label class="category-item"><input type="checkbox" value="${Number(c.id)}" ${kbCategorySelection.has(Number(c.id))?'checked':''}><span>${esc(c.completename||c.name)} <span class="muted">#${Number(c.id)}</span></span></label>`).join('')||'<div class="muted small" style="padding:8px">Keine Kategorie gefunden.</div>'}
|
||||
function clearKbForm(){currentKbId='';kbCategorySelection=new Set();$('#kbForm').reset();$('#kbId').disabled=false;$('#kbId').value='';$('#kbLanguage').value=statusData.communication_language||'de-DE';$('#kbStyle').value=statusData.communication_style||'formal';$('#kbScore').value=Number(statusData.knowledge_min_score||.70).toFixed(2);if([...$('#kbSource').options].some(x=>x.value==='internal-kb'))$('#kbSource').value='internal-kb';$('#kbModalTitle').textContent='Neuen Artikel anlegen';$('#kbModalEyebrow').textContent='Interne Knowledge Base';$('#kbEditState').textContent='Neuer Artikel';$('#kbFormMessage').className='form-message';$('#kbCategorySearch').value='';renderCategoryPicker();updateCounts()}
|
||||
|
||||
6
knowledge-category-map.example.json
Normal file
6
knowledge-category-map.example.json
Normal file
@@ -0,0 +1,6 @@
|
||||
{
|
||||
"Security": 17,
|
||||
"Account Access": [2, 17],
|
||||
"Microsoft Office": 23,
|
||||
"Docker": 31
|
||||
}
|
||||
Reference in New Issue
Block a user