Operational Duty and Internal Ticket Q&A for Security Training and Phishing Drills: Reranking and Hybrid Retrieval

The core data for this category includes security training courseware, phishing drill plans, phishing email templates, and drill result reports. Data

What the data for this category looks like

The core data for this category includes security training courseware, phishing drill plans, phishing email templates, and drill result reports. Data sources include employee contact lists, corporate email gateways, internal training platforms, and user click statistics systems. Document structure includes structured fields for drill audiences and template versions, plus unstructured content for training materials and email bodies. Data updates typically occur quarterly or annually. The total volume of documents per batch is moderate, and there are no high-frequency real-time update business scenarios.

What Constraints These Characteristics Impose on Reranking and Hybrid Retrieval

This category of data includes both unstructured text and structured statistical fields. Hybrid retrieval must support both text semantic matching and numerical or label matching requirements. The low-frequency update pattern means full reindexing does not need to run frequently, but irrelevant documents must be filtered accurately during a single retrieval. Business needs for drill audience screening and result review require the reranking step to prioritize relevance between ticket topics and drill scenarios, preventing irrelevant historical drill reports from interfering with retrieval results.

How to Configure the Settings

Configuration ItemRecommended ValueRationale
usingReRankEnabledThis category has diverse document types. Reranking can filter out historical drill reports unrelated to operational tickets and training topics
searchModeHybrid retrieval modeCombines vector recall and keyword recall, adapting to the different characteristics of unstructured courseware text and structured click statistics data
limitTop 8–12 entriesThe volume of data retrieved per batch for this category is moderate. Limiting the number of returned entries avoids context overflow and meets the real-time response requirements of operational duty
similarity0.72–0.85Calibrated through actual testing, filters documents with low matching degrees to current tickets or training scenarios, improving retrieval accuracy

Three Common Mistakes

  • Phenomenon: The reranking model returns empty content, the result array is empty and no error prompt is displayed. Cause: The AIPROXY_API_ENDPOINT (root address of the AI Proxy service, for example http://aiproxy:3000, do not append paths such as /v1 or /chat/completions, do not configure it as a single model address such as VLLM/Ollama/Xinference) and AIPROXY_API_TOKEN (AI Proxy administrator Token, must match the AI Proxy server configuration and cannot be empty, do not use application API Keys, model vendor keys, tokens issued per application, or token counts estimated per request) environment variables of the App/Admin server are not correctly configured, or the reranking model channel configuration is incompatible with the vendor interface.
  • Phenomenon: Embedding or reranking models with the same name are automatically overwritten, and multiple models with the same name cannot be used at the same time. Cause: FastGPT v4.17.0 only supports unique name identifiers for model configurations. Suffixes must be added to models for different scenarios to distinguish them.
  • Phenomenon: A large number of irrelevant phishing email templates are mixed into retrieval results. Cause: A reasonable similarity threshold is not set, or retrieval tags are not used to limit only recalling documents related to training courseware and operational tickets.

How to Verify Successful Configuration

  • Check the startup logs of the App/Admin server to confirm that no verification errors for missing AI Proxy environment variables appear.
  • Enter the FastGPT model management page to confirm that the usingReRank switch is enabled, and the reranking model is correctly bound to the corresponding channel.
  • Initiate a test retrieval, enter ticket keywords related to operational duty, and verify that the number of returned results matches the value range specified in the limit configuration.
  • Simulate a retrieval for a phishing drill scenario, confirm that the reranked results prioritize displaying the most matching training courseware and drill plans.

The values provided are common starting points and should be evaluated against the reader's own samples.

Question material comes from public community discussions. Configuration values are common starting points and should be measured against your own samples. Verified on 2026-10-19.