What data for this category looks like
This category covers the full oil refining production process. Data sources include DCS/APC control systems, LIMS laboratory information management systems, tank farm metering systems, crude oil evaluation databases, equipment management systems, and energy management systems. Daily documents include operating procedures, production plans, material balance sheets, distillate export quality reports, interlock ledgers, shift handover records, and change management documents. Data update frequencies cover three ranges: real-time for process parameters, shift-based for shift handover records, and daily for material balance and quality reports. Each data entry includes fields such as process variables, material components, quality indicators, and equipment status. Units are adapted to the source systems.
What constraints these characteristics impose on databases and operations & maintenance
Data for this category is multi-source, heterogeneous, and has widely varying update frequencies. This requires databases to support mixed read/write operations and multi-format data ingestion. Real-time process data has high throughput. The system must accommodate high-concurrency write requests and low-latency query needs. Both multiple types of documents and structured data coexist. The system must support linkage between structured indexing and unstructured vector storage. Data anomalies can directly lead to off-spec products or unplanned shutdowns. The operations & maintenance link must include data verification, real-time alerting, and rapid traceability capabilities. It must also ensure full-link data traceability.
Configuration Settings
This setup is adapted for FastGPT v4.17.0. Specific settings are listed below:
| Configuration Item | Recommended Setting | Rationale |
|---|---|---|
MONGODB_URI | mongodb://user:pass@host:27017/炼油质量库?replicaSet=rs0 | Adapts to storage needs for multi-source unstructured data; replica set configuration ensures high availability |
PG_URL | postgresql://user:pass@host:5432/工艺元数据库 | Stores structured shift handover records and change management document metadata; supports complex relational queries |
DB_MAX_LINK | 200–300 | Matches concurrent connection needs for real-time data and bulk document processing; prevents service interruptions caused by exhausted connections |
REDIS_URL | redis://:authpass@host:6379/0 | Caches real-time process alerts and frequently queried quality indicator data; reduces load on primary databases |
Three Common Configuration Errors
- Symptom: A
MongoDB connection timeouterror is thrown on startup, or no results are returned when querying historical process data. Cause: The MongoDB replica set is not configured, so it cannot support high-concurrency write requests for real-time data. - Symptom: After bulk uploading shift handover records, some structured fields are empty or cannot be retrieved with conditional queries. Cause: Structured metadata is not stored using
PG_URL. Relying solely on vector database storage leads to loss of structured information. - Symptom: The system experiences query timeouts or request rejections during production peak hours. Cause:
DB_MAX_LINKis not set appropriately. Concurrent connections exceed the configured limit, causing request queuing and timeouts.
How to Verify Correct Configuration
- Run the official database connection test script to confirm that
MONGODB_URIandPG_URLcan establish normal connections, with no authentication or network errors. - Import one real-time process data entry and one shift handover record. Confirm that the data writes successfully to the corresponding databases and can be queried using preset conditions.
- Check the system monitoring dashboard to confirm that concurrent connection counts do not exceed the limit set by
DB_MAX_LINK. - Trigger a complete quality traceability query. Confirm that structured metadata and unstructured documents are linked to return complete results.
The values provided here are common starting points, and should be evaluated against the reader's own samples.