What the data for this category looks like
Typical data sources for continuous basic chemical production include DCS/SIS systems, online analyzers, LIMS systems, equipment condition monitoring, energy metering devices, and raw material tank sensors. Daily associated documents include operating procedures, process index cards, DCS trend charts, laboratory reports, interlock bypass records, special work permits, and shift logs.
Data update frequencies are divided into three categories: real-time (such as DCS pressure and temperature data), shift-based (such as shift logs and special work records), and daily (such as full-batch laboratory reports). Data fields include process parameters (units: MPa, ℃, m³/h), test results (units: %, ppm), and equipment operating status codes. Both structured and unstructured data are stored together.
What constraints these characteristics impose on the database and operations maintenance link
Real-time high-frequency process data streams require the database to support high-concurrency writes to avoid data packet loss or delays. Mixed storage of multi-source heterogeneous data requires separating storage media for time-series data and structured document data to adapt to different query requirements.
Data with different update frequencies need tiered storage strategies. Real-time data should retain 7 days of high-frequency query replicas, while historical data should be archived to low-cost storage. Key event data such as interlock bypass records require strong consistency guarantees to avoid traceability deviations caused by data inconsistency.
Associated queries across multiple types of documents require the database to support cross-table association, ensuring that quality problems can be traced end-to-end from process parameters, test results, to operation records.
How to set the configurations
| Configuration Item | Recommended Setting | Rationale for This Setting |
|---|---|---|
MONGODB_URI | mongodb://<db-user>:<db-pass>@<mongo-host>:27017/chemical_quality | Real-time DCS and online analyzer time-series data for continuous basic chemical production is primarily unstructured streaming data. MongoDB is suitable for high-concurrency write scenarios, and specifying the business database name separates production data |
PG_URL | postgresql://<db-user>:<db-pass>@<pg-host>:5432/chemical_trace | Structured historical data such as process index cards, laboratory reports, and shift logs requires strong consistency queries. PostgreSQL is suitable for complex associated query requirements |
DB_MAX_LINK | 100–150 | Real-time data streams generate thousands of records per second. Sufficient connection pool support supports concurrent read/write operations. Adjust peak load based on on-site measurement calibration |
REDIS_URL | redis://:<redis-pass>@<redis-host>:6379/0 | Cache interlock alarms and real-time process threshold check results to reduce repeated query pressure on the database |
Three common mistakes
- Symptom: After deploying FastGPT via Docker, startup fails with
connection refusederror. Logs show failure to connect to the specified database instance. Cause: TheMONGODB_URIconfiguration was not modified correctly. The default local test address was used, and production environment database connection information was not replaced. - Symptom: Connecting to the built-in MongoDB of FastGPT via MongoDB Compass fails, and time-series data related to quality traceability cannot be viewed. Cause: The 27017 port access permission of the MongoDB instance was not opened, or the configured connection username and password were incorrect.
- Symptom: After generating an SQL query, the number of returned results does not match expectations, and corresponding batch laboratory data cannot be matched. Cause: Cross-table associated query rules were not configured for multi-source data in basic chemical production. The database connection did not enable cross-database associated query permissions, resulting in failure to associate DCS data with LIMS laboratory data.
How to confirm the configuration is complete
- Run the database connectivity test script built into FastGPT v4.17.0 to verify that instances corresponding to
MONGODB_URIandPG_URLcan establish normal connections, with no timeout or permission errors. - Import a simulated real-time DCS pressure data set and the corresponding batch laboratory report data set to confirm that the data can be written to the corresponding database, and that complete associated information can be matched via FastGPT's retrieval function.
- Adjust the value of
DB_MAX_LINKand observe the connection pool usage when the system performs high-concurrency writes, to confirm that no connection overflow or read/write blocking occurs. - Check the real-time alarm data cached by Redis to confirm that it can be updated and queried normally, with no cache expiration or loss issues.
The values provided here are common starting points, and should be evaluated against the reader's own actual samples.