What the data for this category looks like
Refinery and chemical production ledger data comes from three sources: device DCS real-time collection systems, tank farm batch metering systems, and offline quality inspection reports. Data update frequencies vary: device operating parameters update every 1 minute, material in/out ledgers update every 8 hours per shift, and quality inspection reports update daily. Fixed fields for a single ledger document include device number, operating parameters (temperature unit ℃, pressure unit MPa, flow unit m³/h), material name, in/out volume, quality inspection indicators (such as sulfur content unit ppm), and record time. Some batch ledgers also include corresponding quality inspection PDF attachments.
What constraints these characteristics impose on deployment and upgrade
Decentralized data sources and varying update frequencies require adapting to multi-source data interface connection configurations during deployment, to avoid data synchronization interruptions. The high-frequency read/write requirements for real-time parameters require ensuring database concurrent connection capacity, to prevent ledger refresh failures caused by insufficient connections. The large size of attached PDF attachments requires storage services to support large file uploads, while also configuring secure access permission rules. Strict requirements for fields and units require presetting verification rules in initial configuration, to avoid format errors during data import.
How to set the configurations
Reference only; verify against the target FastGPT release and deployment documentation.
| Configuration Item | Recommended Value | Rationale |
|---|---|---|
DB_MAX_LINK | 20–30 | Matches the high-frequency read/write concurrency requirements of refinery and chemical ledgers, ensuring stable database connections |
STORAGE_S3_ENDPOINT | Intranet address adapted to the enterprise’s internal object storage | Ensures intranet storage security for sensitive refinery data, supports upload of large-capacity quality inspection PDF attachments |
REDIS_URL | Intranet-accessible Redis instance with 8 GB or more memory | Caches frequently read real-time device parameters, reducing read/write pressure on the core database |
MONGODB_URI | Replica set connection address configured | Ensures high availability of read/write operations for refinery ledger data, avoiding synchronization interruptions caused by single-node failures |
FILE_TOKEN_KEY | Random string generated in accordance with enterprise confidentiality specifications | Ensures security of download permissions for ledger-related files, preventing unauthorized access |
Three common mistakes
- Symptom: A database connection limit exceeded error is returned when batch importing monthly refinery and chemical ledgers. Cause: The
DB_MAX_LINKconfiguration value is lower than the connection requirements for actual concurrent imports, and cannot support read/write requests for batch data. - Symptom: FastGPT returns a timeout error when parsing quality inspection PDFs attached to ledgers, and the parsing service log shows the task has completed. Cause: No timeout parameter for Redis cache task status is configured, causing the frontend to trigger a timeout return.
- Symptom: An insufficient permissions prompt is displayed when downloading ledger files. Cause: The
FILE_TOKEN_KEYconfiguration does not match the key of the file storage service, resulting in permission verification failure.
How to confirm configurations are correct
- Run a batch import test task for refinery and chemical ledgers, check database connection number monitoring, confirm that connection limit exceeded is not triggered, and adjust
DB_MAX_LINKto a value matching concurrency requirements. - Upload a single large-capacity ledger attachment, check storage service logs, confirm there are no upload failure records, and verify that the
STORAGE_S3_ENDPOINTconfiguration is valid. - Generate a ledger summary report and attempt to download the attached files, verify that link permissions are normal, and confirm that the
FILE_TOKEN_KEYconfiguration is compliant. - Check the refresh records of real-time device parameters, confirm that they match the preset update frequency, and verify that the data synchronization configuration is valid.
The values provided are common starting points and should be measured against the reader's own samples.
Question material comes from public community discussions. Configuration values are common starting points and should be measured against your own samples. Verified on 2026-09-28.