What data for this category looks like
Data covers end-to-end processes at carbonated drink, juice, tea, functional beverage, and packaged drinking water factories. Daily documents include formula sheets, water treatment records, blending records, sterilization or aseptic filling records, packaging material inspection records, and finished product inspection records. Data sources include water source and raw material test results, formula system data, online sugar and acidity monitoring data, filling equipment logs, visual inspection results, and LIMS system data. Data updates occur per batch, per shift, or via continuous collection. Fields include feeding parameters, water quality indicators, filling liquid level, sealing strength, batch number, shelf life, and more. Units include mg/L, %, mm, days, and more. Packaging and formula information must be linked synchronously.
Constraints on database and operations
The continuous collection of equipment data, multi-source heterogeneous records, and batched update rhythm for this category impose multiple constraints on database operations. High-speed real-time online monitoring data generates high-frequency write requests. A read-write separation architecture must be configured to distribute load. Batched data linked by batch numbers requires a secondary index based on batch number, to quickly locate full-link information for a single batch during quality traceability. Multi-source physical and chemical, equipment, and LIMS data need unified field mapping rules to avoid field mismatches during cross-source queries. Frequent product switches require the database to support dynamic partitioning or rapid table structure adjustments, to adapt to differences in formula and inspection fields across product categories. Historical data related to microbial contamination must be retained for a sufficient duration, and support fast time-range retrieval.
Configuration settings
| Configuration Item | Recommended Setting | Rationale |
|---|---|---|
MONGODB_URI | mongodb://fastgpt:securepass@db-host:27017/fastgpt_db?maxPoolSize=60 | Stores unstructured or semi-structured data such as equipment logs and visual inspection results, adapting to high-concurrency write requirements |
PG_URL | postgresql://lims_user:limspass@pg-host:5432/lims_db?sslmode=disable | Stores structured batch data such as formula sheets and inspection records, supporting complex associated queries |
DB_MAX_LINK | 60–100 | Adapts to batched data writes per batch or per shift and concurrent requests from real-time monitoring, avoiding connection exhaustion |
REDIS_URL | redis://:redispass@redis-host:6379/0?maxmemory=1000 MB | Caches batched information for high-frequency queries and real-time online monitoring data, reducing database read and write pressure |
| Batch write threshold | No more than 4000 entries per batch | Adapts to the total volume of inspection records and equipment logs per batch at beverage factories, avoiding single-write timeouts |
This document was written based on FastGPT v4.17.0.
Three common configuration mistakes
- Phenomenon: The database experiences sustained high write load during Docker deployment, with rapid disk space growth, even triggering disk space exhaustion errors. Cause: No
DB_MAX_LINKconfigured to limit connection pool size, leading to a large number of concurrent requests to create database connections, triggering high-frequency disk read and write operations. - Phenomenon: Milvus deployment container fails to start, prompting a PostgreSQL database connection authentication error. Cause: Incorrectly wrote non-vector database PostgreSQL configurations into the Milvus Compose file, and did not only configure necessary vector database and associated storage components according to category requirements.
- Phenomenon: Corresponding formula data cannot be found during batch traceability, with empty fields in retrieval results. Cause: Unified field mapping rules for multi-source data were not implemented, leading to a mismatch between formula fields from the LIMS system and storage fields in FastGPT.
How to confirm the configuration is correct
- Run a database connection test script to verify that
MONGODB_URIandPG_URLhave normal connection status, with no authentication or address errors. - Simulate single-batch data writes, check that write time and storage occupancy meet expected values, and confirm that the connection count for
DB_MAX_LINKdoes not trigger exhaustion. - Retrieve historical batch data, verify that multi-source data field mapping is correct, and that formula, inspection, and equipment log information can be properly associated.
- Check Redis cache monitoring, confirm that high-frequency query data has been cached, and that database read and write pressure matches configuration expectations.
The values provided are common starting points and should be measured against the reader's own samples.