What this category of data looks like
Data sources for petroleum refining production ledger data include DCS/APC control systems, LIMS laboratory systems, tank farm metering systems, crude oil evaluation reports, equipment management systems, and energy management systems. Data update rhythms fall into three categories: real-time, shift-based, and daily. Covered documents include operating procedures, production plans, material balance sheets, distillate export quality reports, interlock ledgers, shift handover records, and change management documents. Document fields include unit number, crude oil density, distillation range temperature, tank farm liquid level, interlock trigger threshold, and others. Units include MPa, ℃, m³, t/h, and other industrial general and specialized units.
Constraints on knowledge base retrieval and recall
Heterogeneous data from multiple sources requires the retrieval system to adapt to different document structures, to avoid losing structured parameter information during chunking. Differentiated update rhythms (real-time, shift-based, daily) require retrieval configurations to support incremental indexing triggered by update frequency, ensuring the timeliness of recalled data. The business characteristic of multiple units coupled via material and hydrogen networks requires retrieval results to be filtered by unit and material node, preventing invalid cross-unit data interference. Fields with specialized units require unit information matching during retrieval, to avoid incorrect recall caused by unit confusion.
How to set the configuration
The following configurations are based on FastGPT v4.17.0, adapted to retrieval and recall needs for petroleum refining production ledgers:
| Configuration Item | Recommended Setting | Rationale |
|---|---|---|
similarity | 0.75–0.85 | Petroleum refining process parameters have high precision requirements. This range filters low-match irrelevant documents and avoids incorrect calls of interlock and material balance data |
limit | Top 8–12 results | The total volume of ledger data for coupled multiple units is large. Limiting the number of recalled results avoids context overflow while covering core information of major associated units |
searchMode | mixedRecall | Process documents contain both structured parameters and unstructured operating steps. Hybrid retrieval matches both field keywords and semantic content |
usingReRank | Enabled | Semantic similarity of multi-source data is easily disturbed by units and unit numbers. Reranking improves the sorting priority of high-match results |
datasetSearchUsingExtensionQuery | Enabled | Production ledgers contain a large number of industry-specific terms. Extended queries supplement synonyms and unit aliases to improve recall coverage |
collectionIds | Configured by unit and document type grouping | Petroleum refining multiple units operate in a coupled manner. Group-based filtering avoids invalid cross-unit data recall and improves retrieval accuracy |
Three common errors
- Symptom: An error occurs during question-answer splitting, with a prompt that the document structure cannot be recognized. Cause: Petroleum refining ledgers contain mixed structured tables and unstructured text. The default splitting rule cannot adapt to multi-format documents, leading to chunking failure.
- Symptom: A 503 error is displayed when uploading files via the dialog box, while uploading to the knowledge base has no abnormalities. Cause: Files uploaded via the dialog box are not verified against industrial document-specific formats. Additionally, temporary cache resources are insufficient when processing multi-source heterogeneous data in real time, triggering service current limiting.
- Symptom: Retrieval accuracy for complex process questions is low, with only scattered data from a single unit returned. Cause:
usingReRankanddatasetSearchUsingExtensionQueryare not enabled, resulting in recalled documents that do not cover cross-unit coupling logic and failing to filter low-match content.
How to verify the configuration is complete
- Upload a typical interlock ledger document, initiate a query containing interlock thresholds, and check if the recall results include the interlock trigger condition fields for the corresponding unit, and if the match rate falls within the range set by
similarity. - Initiate a query involving a multi-unit coupling scenario, and check if the returned results cover ledger data for at least two associated units, and if the sorting matches the expected priority of
usingReRank. - Upload a real-time updated DCS data document, check if the index update status matches the configured update frequency, and confirm that the incremental index has not missed newly uploaded data.
- Test uploading a single process procedure document via the dialog box, confirm no 503 error occurs, and that the core content of the document can be normally recalled via knowledge base retrieval.
Note: The configuration values provided are common starting points. They should be evaluated against the reader's own samples.