What data for this category looks like
Data sources for petroleum refining scenarios include DCS/APC control systems, LIMS laboratory information management systems, tank farm metering systems, crude oil evaluation databases, equipment management systems, and energy management systems. Data update frequencies fall into three categories: real-time, shift-based, and daily. These correspond to process parameters, shift handover records, and daily material balance reports. Daily documents include operating procedures, production plans, material balance documents, distillate outlet quality reports, interlock ledgers, shift handover records, and change management documents. Fields include crude oil property parameters, unit load indicators, distillate temperature, tank farm liquid level, and product quality test values. Units include kilograms per cubic meter, tons, degrees Celsius, percentage, and others.
What constraints these characteristics impose on deployment and upgrade
This category’s data is multi-source and heterogeneous, with layered update rhythms and diverse document structures. This creates three core constraints for deployment and upgrade. First, data access across multiple systems must adapt to different interface formats. Deploy with unified data access adaptation rules during setup. Second, layered update requirements for real-time, shift-based, and daily data require configured layered scheduled tasks to avoid resource conflicts. Third, diverse document types and exclusive fields require pre-configured custom parsing templates to ensure accurate extraction of ledger fields. During upgrades, verify the adaptability of new data sources and the stability of scheduling logic to avoid disrupting existing production ledger aggregation workflows.
Configuration Settings
| Configuration Item | Recommended Value | Rationale |
|---|---|---|
MONGODB_URI | mongodb://username:password@address:port/database_name?authSource=admin | The ledger data volume for petroleum refining is large. Specify the authentication source to ensure connection security, and adapt to storage needs for multiple documents and historical data. |
REDIS_URL | redis://:password@address:port/database_number | Cache real-time process parameters and batch-parsed document fragments to avoid repeated calculations and improve the efficiency of aggregated report generation. |
STORAGE_S3_ENDPOINT | http://object storage address:port | Store a large number of process documents and historical ledger data, adapting to the private deployment scenario of private object storage. |
DB_MAX_LINK | 20–30 | Control the number of concurrent connections for multiple data sources to avoid exhausting database resources and ensure stable system operation. |
| Document parsing template | Configure separately by document type. Extract distillation range and density fields for quality reports, extract unit load and inspection status fields for shift handover records | Petroleum refining documents have diverse types, and exclusive fields need precise matching to avoid missing key business information in parsing results. |
| Data synchronization scheduling rules | Synchronize real-time data every 30 seconds, synchronize shift-based data within 10 minutes after the end of each shift, synchronize daily data at 2:00 AM daily | Match the update frequencies of different data types, balancing real-time performance and system resource usage. |
Three Common Configuration Mistakes
- After uploading a petroleum refining process PDF document, the knowledge base parsing result is empty or only extracts a small amount of meaningless text. The cause is that no custom parsing template is configured for petroleum refining exclusive documents. Default parsing rules cannot recognize exclusive fields such as distillation range and unit load.
- The system reports a database connection timeout error (status code 504). The cause is that the
DB_MAX_LINKconfiguration value is too low. Concurrent connections from multiple data sources exceed the upper limit, causing the database to reject requests. - Real-time process data synchronization delay exceeds the preset threshold. The cause is that the interval set in the data synchronization scheduling rules is too long, not matching the update frequency of real-time data.
How to Confirm Proper Configuration
- Log in to the database management tool, verify that the connections configured with
MONGODB_URIandPG_URLare normal. Execute basic query statements to confirm data read and write functions. - Upload a typical petroleum refining shift handover record document, check if the parsing result includes exclusive fields such as unit load and inspection status, matching the configured parsing template rules.
- View the data synchronization log to confirm that tasks with different update frequencies execute according to preset rules, with no abnormal errors.
- Access the front-end domain name
FE_DOMAIN, verify that FastGPT v4.17.0 loads the interface normally and can normally call knowledge base query functions.
The values provided here are common starting points and should be measured against the reader's own samples.