Operations Duty and Internal Ticket Q&A for Performance Testing and Capacity Evaluation: Knowledge Base Retrieval and Recall

Data sources for this category include operating metrics from pressure testing platforms, system performance data monitored by APM tools, query

What data for this category consists of

Data sources for this category include operating metrics from pressure testing platforms, system performance data monitored by APM tools, query records from database slow logs, and traffic baseline data from production environments. Daily documents include pressure testing plans, capacity evaluation reports, pressure testing data records, and optimization suggestion sheets. Pressure testing plans contain test scenarios and target thresholds. Capacity evaluation reports include baseline analysis and bottleneck conclusions. Pressure testing data records include operating metrics for each node. Optimization suggestion sheets contain adjusted parameters and expected effects. Data update frequency is typically once per month, or after major system version iterations.

What constraints these characteristics impose on knowledge base retrieval and recall

This category’s data covers structured metrics, semi-structured reports, and unstructured plans. Retrieval and recall must support matching across multiple content formats. Update cycles follow monthly or major version milestones. Incremental updates must avoid full synchronization to reduce redundant calculations. Documents contain clear performance metric fields and scenario binding relationships. During retrieval, metric units and business scenarios must be associated to avoid returning irrelevant results. Queries in operations duty and ticket scenarios focus primarily on historical pressure testing data or capacity thresholds. Recall must prioritize matching scenario-related historical records to improve response efficiency.

How to configure settings

Configuration ItemRecommended SettingRationale
similarity0.75–0.85Matches the precision requirements for performance metrics, and avoids recalling irrelevant non-performance testing documents
limitTop 8 entriesAdapts to the need for quick access to core data during operations duty and ticket scenarios, and avoids excessive results interfering with judgment
searchModehybridCombines keyword and semantic retrieval, and supports mixed queries for structured metrics, semi-structured reports, and unstructured plans
usingReRankEnabledPerforms secondary sorting on recall results, prioritizes content strongly related to performance metrics, and improves result relevance
datasetSearchUsingExtensionQueryEnabledExpands the semantic scope of queries, covers vague questions from operations staff, such as expanding "pressure test timeout" to corresponding scenario optimization solutions
collectionIdsBind exclusive pressure testing knowledge base collectionsIsolates business data, avoids retrieving documents from other categories, and narrows the retrieval scope

Three common configuration mistakes

  • Issue: A large number of non-performance testing related documents appear in retrieval results. Cause: The collectionIds binding for the exclusive knowledge base collection is not configured, leading to retrieval scope covering other business data.
  • Issue: An undefined model error is returned when calling the retrieval function. Cause: The embedding model is not configured correctly, or an unsupported model type is used, resulting in vector generation failure.
  • Issue: Recall results for multi-column Excel format pressure testing data are chaotic. Cause: Adapted segmentation rules are not configured for structured table data, leading to incorrect field splitting and inability to accurately match metric queries.

How to confirm correct configuration

  • Check that the retrieval scope only covers pressure testing related knowledge base collections, and confirm that the collectionIds configuration is correct.
  • Submit a simulated query related to performance metrics, verify that the returned result similarity falls within the 0.75–0.85 range, and confirm that the similarity configuration is active.
  • After enabling the re-ranking function, check that the returned results are sorted to prioritize content related to performance metrics, and confirm that the usingReRank configuration is active.
  • Submit a vague query, verify that semantic expansion is triggered, and confirm that the datasetSearchUsingExtensionQuery configuration is active.
  • Complete the above tests in the FastGPT v4.17.0 environment to ensure version compatibility.

Note: The configuration values provided here are common starting points. Adjustments should be made based on specific samples and testing data for each deployment.

Question material comes from public community discussions. Configuration values are common starting points and should be measured against your own samples. Verified on 2026-10-19.