Operational On-Call and Internal Ticket Q&A for Incident Management and Root Cause Analysis: Knowledge Base Retrieval and Recall

This category of data comes from incident timelines, operational logs, monitoring metrics, and code commit records. Data is updated after each major

What This Category of Data Looks Like

This category of data comes from incident timelines, operational logs, monitoring metrics, and code commit records. Data is updated after each major incident, with monthly cumulative archiving. Core documents include RCA reports, incident tickets, post-incident review minutes, and rectification tracking sheets. Fields include unique incident identifier, root cause positioning description, associated monitoring metrics, and rectification closure status. No additional statistical quantitative fields are included.

What Constraints These Characteristics Impose on Knowledge Base Retrieval and Recall

This category of data primarily consists of long-text incident review documents, and relies on time-series and metric-related data. Retrieval and recall must support cross-document associated matching. Since update frequency is low, frequent incremental knowledge base synchronization is not required, but recall accuracy for historical incident data must be maintained. Document structures contain fixed core fields including root cause description and rectification items. Retrieval logic must prioritize matching root cause-related keywords to avoid interference from irrelevant content. Additionally, data sources include logs and monitoring metrics, so multi-dimensional retrieval matching rules must be supported.

Configuration Settings

The following configurations are compatible with FastGPT v4.17.0.

Configuration ItemRecommended Value RangeRationale
Similarity Threshold0.75–0.85Matches core semantics of root cause descriptions to avoid recalling irrelevant incident documents
Single Search LimitTop 8–12 resultsAdapts to the length of individual RCA reports, controls total context usage
Search ModeKeyword + Semantic HybridMatches both root cause keywords and semantic associations, covering retrieval needs for logs and review documents
RerankingEnabledPerforms secondary sorting on recall results to raise the display priority of root cause-related documents
Query OptimizationEnabledAutomatically expands synonyms related to root causes to cover wording differences across different operations staff
Target Retrieval CollectionBound to incident review and ticket collection IDsLimits retrieval scope to exclude irrelevant knowledge base content outside this category
Maximum Context12000–15000 tokenAdapts to context requirements for long-text RCA reports, avoids truncation of core root cause content

Three Common Misconfigurations

  • When testing a workflow, the reply will always include a fixed preset prompt at the end. This occurs because the default system prompt output is not disabled in the conversation configuration.
  • The number of knowledge base retrieval and recall results does not match the configured single search limit. This occurs because the target retrieval collection is not correctly bound, leading to a retrieval scope that exceeds the preset range.
  • After deploying with docker-compose, the knowledge base and workflow configurations become blank after a period of time, but the API remains available. This occurs because the mounted configuration volume does not use persistent storage, and configuration files are lost after container restart.

How to Confirm Proper Configuration

  • Manually enter a known incident keyword associated with a root cause, and verify that the top 3 recall results include matching RCA reports of the same type.
  • View the retrieval configuration panel, confirm that the switch states for similarity threshold, reranking, and query optimization match the preset configuration.
  • Export the current knowledge base collection ID list, and verify that the target retrieval collection configuration only includes incident review and ticket collections for this category.
  • Trigger a workflow test, and check that no default system prompt output appears at the end of the reply.

The values provided are common starting points and should be measured against the reader's own samples.

Question material comes from public community discussions. Configuration values are common starting points and should be measured against your own samples. Verified on 2026-10-19.