What this category of data looks like
Data for this category comes from multiple types of business documents of specialty chemical enterprises including coatings, resins, additives, and agrochemical intermediates. These documents include batch production instructions, feeding review sheets, reaction process records, cleaning records, test reports, and deviation change orders. Data sources cover the formula and process library, raw material batch ledger, reactor DCS system, weighing system, LIMS laboratory management system, and warehouse management system. Updates are triggered per production batch and each key process step. Data must be associated with fields such as batch number, raw material batch number, process parameters, and test results. Due to production switching between multiple product varieties, production records and cleaning confirmation data for different batches must be distinguished to ensure the confidentiality of formula and process information.
What constraints do these characteristics impose on the database and operations link
The high-frequency update feature of multiple batches requires the database to support high-concurrency writes and fast queries by batch dimension. The need for multi-source heterogeneous data integration requires configuring unified data validation rules and permission isolation mechanisms to prevent leakage of sensitive information such as formulas. For batch data associated across systems, the operations link must ensure consistency of field names and formats to avoid cross-system data conflicts. The data switching requirements brought by frequent product variety switching require the database to support fast index switching and batch-level data isolation. The business goal of full-link quality traceability requires the operations link to retain complete historical data versions and support combined retrieval by multiple conditions such as process steps and raw material batch numbers.
How to set the configurations
| Configuration Item | Recommended Value | Rationale |
|---|---|---|
MONGODB_URI | mongodb://[username:password@]host1:port1,host2:port2/[authSource=admin] | Adapts to multi-source data access from reactor DCS, LIMS and other multi-node deployed systems, supports replica sets to ensure data availability, and matches the multi-source data integration logic of FastGPT v4.17.0 |
DB_MAX_LINK | 200–300 | Matches concurrent write and query requests for a single production batch, and avoids service interruptions caused by exhausted connections |
REDIS_URL | redis://:[password]@[host]:[port]/0 | Caches process parameters and test results associated with batches, speeds up retrieval for quality traceability, and reduces database load |
PG_URL | postgresql://[user]:[password]@[host]:[port]/fastgpt | Stores structured batch production records and deviation change orders, and supports associated queries with complex conditions |
| Database connection timeout | 600 seconds | Matches the collection interval of reactor DCS data and the batch write cycle of batch data, and avoids failed normal data writes caused by short timeouts |
Three common mistakes
- Symptom: A timeout error occurs when FastGPT connects to MongoDB, but normal connection is possible using the mongosh command. Cause: The correct authSource parameter is not configured in
MONGODB_URI, or the database connection pool limit is not adapted to the concurrent requirements of multi-source data. - Symptom: When querying application historical versions, earlier historical version documents show the
tmbIdfield as empty. Cause: No field completion script was executed for existing batch data, and thetmbIdfield was not mandatory in the data writing process to associate batch information. - Symptom: Database connection exhaustion errors occur when multiple people initiate multi-batch quality traceability requests at the same time. Cause: The
DB_MAX_LINKparameter was not configured based on the concurrent request volume of a single production batch, causing the connection pool to be fully occupied.
How to confirm the configurations are correct
- Use the built-in database connectivity test function of FastGPT to verify the connectivity of configurations such as
MONGODB_URIandPG_URL. - Import feeding review sheets, reaction process records and other data for a single batch, check that all required fields are fully written, and confirm that permission isolation for sensitive data is effective.
- Initiate multiple concurrent batch data query requests, observe the usage of the database connection pool, and confirm that the number of connections does not exceed the preset range.
- Query existing and newly generated application historical version data, and confirm that the
tmbIdfield correctly associates batch information.
The values provided here are common starting points and should be measured against the reader's own samples.