What this category’s data includes
Petroleum refining scenarios cover production operations, process, scheduling, equipment, and laboratory departments. Daily documents include operating procedures, production plans, material balances, distillate outlet quality reports, interlock ledgers, shift handover records, and change management documents. Data sources include DCS/APC systems, LIMS laboratory systems, tank area metering systems, crude oil evaluation databases, equipment management systems, and energy management systems. Data updates follow three frequencies: real-time, shift-based, and daily. Each document includes fields such as process parameters, material flow rates, quality indicators, and equipment status. Corresponding units include tons, cubic meters, degrees Celsius, ppm, and others.
Constraints on vector models and indexing
Mixed access to multi-source heterogeneous data requires indexes to support parallel storage of structured and unstructured data. Real-time updated DCS data requires vector indexes to support high-frequency incremental synchronization, avoiding excessive time consumption from full reconstruction. Business logic coupled across multiple units requires vector recall to link material and energy network connections. Business field association rules must be embedded during indexing. Data with different update frequencies must be divided into index partitions, managed by real-time, shift-based, and daily categories. This ensures retrieval efficiency matches data freshness. Document fields contain a large number of process parameters and quality indicators. Dedicated vector encoding rules must be designed for numeric fields to avoid loss of business semantics from general-purpose encoding.
Configuration settings
This configuration applies to FastGPT v4.17.0, optimized for production ledger data characteristics in petroleum refining scenarios.
| Configuration Item | Recommended Setting | Rationale |
|---|---|---|
| Chunk Length | 800–1200 characters | Matches the typical paragraph length of a single ledger document, avoiding cutting across business logic |
| Overlap Length | 100–200 characters | Covers process parameter association information across chunks, preventing key logic from breaking |
| Similarity Threshold | 0.75–0.85 | Filters low-correlation historical ledgers and process plans, adapting to the strict requirements of refining operations |
| Single Search Limit | Top 10 results | Controls the scale of retrieval results, avoiding context overflow, and aligning with decision reference needs in production scenarios |
defaultToken | 8000 | Adapts to the total length requirement after concatenating multiple documents, covering the content volume of a single batch of ledger summaries |
maxToken | 16000 | Supports complete context input for complex report generation, meeting the integration needs of multi-unit data |
Three common configuration mistakes
- Connection failure prompts when importing a custom vector database. The access whitelist for the corresponding database is not configured in the system, or port mapping parameters do not match the deployment environment.
- No retrieval results generated after enabling the vector model channel. The API key and access address for the model channel are not configured correctly, causing requests to fail to route normally to the model service.
- Far fewer retrieval results than expected after index construction. The chunk length is set too large, cutting complete shift handover records and interlock ledgers into too few chunks, or the similarity threshold is set too high, filtering out a large amount of valid historical data.
How to confirm proper configuration
- Navigate to the vector model and index configuration page, verify that the connection status of the added custom vector database or model channel shows normal.
- Upload a typical petroleum refining production ledger document, trigger index construction, and check that the index progress log has no error messages.
- Enter a process-related query statement, perform retrieval, and confirm that the returned result document types and business fields match expectations.
- Adjust the similarity threshold or chunk length parameters, compare changes in retrieval results, and confirm that the configuration takes effect.
The values provided are common starting points and should be measured against the reader's own samples.