Operational On-Call and Internal Ticket Q&A for Log Platforms and Observability: Re-Ranking and Hybrid Retrieval

The data for this category primarily comes from logging systems, distributed tracing (Trace) platforms, Metrics indicator systems, and application SDK

What the data for this category looks like

The data for this category primarily comes from logging systems, distributed tracing (Trace) platforms, Metrics indicator systems, and application SDK instrumentation points. Regularly maintained documentation includes logging collection specifications, tracing guidelines, indicator dictionaries, and dashboard configuration records. Data updates occur daily or align with application version iterations. Data fields include log timestamps, levels, service identifiers, Trace link IDs, node latency, indicator names and their corresponding units, among others. The structure varies slightly across different collection sources.

Multi-source heterogeneous data structures require the hybrid retrieval link to adapt to different content types such as log text, link IDs, and indicator values with units. Field weights for each data source must be clarified. Frequently updated data means the retrieval and re-ranking links need low latency to avoid disrupting operational on-call troubleshooting efficiency due to data lag. Differences in documents from various collection sources require hybrid retrieval to cover both vector semantic matching and keyword matching, ensuring comprehensive recalled content. Ticket Q&A in operational scenarios requires associating contextual evidence, and the re-ranking link must prioritize returning links and log fragments strongly relevant to the current question, filtering out irrelevant indicator data.

How to configure the settings

Configuration ItemRecommended SettingRationale
usingReRankEnabledPerforms secondary sorting on mixed results from multi-source recall to improve the accuracy of operational problem matching, and adapts to the re-ranking logic of FastGPT v4.17.0
similarity0.70–0.85Filters recall results with low semantic matching, adapts to the semantic characteristics of log and tracing text, and balances recall coverage and relevance
limitTop 8–12 entriesControls the total number of results returned by a single retrieval to avoid exceeding the model's context processing limit, and meets the scenario requirements of rapid operational troubleshooting
searchModehybridEnables both vector retrieval and keyword retrieval, integrating retrieval results from multiple data sources such as logs, tracing, and indicators
Re-ranking model call timeout300 secondsAdapts to the retrieval scenario of batch data from log platforms, preventing operational on-call troubleshooting workflows from being interrupted by re-ranking delays

Three common configuration mistakes

  • Symptom: The order of retrieval results does not change after configuring the re-ranking model, and the model parameters cannot be selected in the interface. Cause: The re-ranking model has not been configured for access in the system settings, or the usingReRank switch has not been enabled.
  • Symptom: The number of retrieved recall results exceeds expectations, resulting in excessively long context that prevents normal answer generation. Cause: The limit parameter has not been set correctly, and the number of results returned by a single retrieval has not been controlled.
  • Symptom: A large number of low-correlation non-log or non-link data is mixed into the hybrid retrieval results. Cause: The searchMode has not been set to hybrid mode, and vector and keyword retrieval have not been enabled simultaneously to integrate multi-source data.

How to confirm the configuration is complete

  • Access the FastGPT admin backend, navigate to the configuration page of the corresponding application, and confirm that the usingReRank switch is enabled.
  • Initiate a test retrieval for an operational ticket, compare the result ranking before and after enabling re-ranking, and verify whether the re-ranking function operates correctly.
  • Review the number of results returned by the retrieval to confirm that it matches the quantity range set by the limit parameter.
  • Review the application's retrieval logs to confirm that the re-ranking model call request has been sent normally and returned valid results.

The values provided are common starting points and should be measured against the user’s own samples.

Question material comes from public community discussions. Configuration values are common starting points and should be measured against your own samples. Verified on 2026-10-19.