What the Data for This Category Looks Like
The data for this category covers the entire production chain of liquid milk, yogurt, milk powder, and dairy ingredients. Daily documentation includes raw milk acceptance records, standardized ingredient lists, pasteurization/UHT processing records, CIP cleaning records, filling logs, reserved sample records, and microbial test reports. Data comes from raw milk testing equipment, MES systems, homogenization and heat treatment equipment, online temperature and flow sensors, CIP systems, and LIMS laboratory management systems. Updates are synced in real time per individual milk tank and production batch. Fields include raw material physicochemical indicators, equipment operating parameters, batch numbers, test results, and more. Units follow professional food testing metrology standards for temperature, flow, total bacterial count, and other relevant metrics.
Constraints for Database and Operations
Real-time, high-frequency batch-level updates require the database to support low-latency writes and high-concurrency queries, to avoid production data backups. Multi-source heterogeneous data requires unified structured storage to accommodate varying field formats such as raw milk indicators, equipment parameters, and test reports. The perishable nature of liquid raw materials requires full-chain batch data to be fully traceable and quickly accessible, to support timely anomaly troubleshooting. Continuous, non-stop production requires database operations to avoid service interruptions, while adapting to scenarios with large fluctuations in single-batch data volume to prevent resource exhaustion. Cross-batch associated query requirements demand that the database supports efficient multi-table join operations, to ensure fast response times for full-chain traceability.
Configuration Settings
This configuration applies to FastGPT v4.17.0.
| Configuration Item | Recommended Setting | Rationale |
|---|---|---|
| Database connection pool size | 20–50 connections | Adapts to resource requirements for real-time writes per milk tank and concurrent queries across multiple batches, and prevents connection exhaustion |
| Primary database connection address | Configure MONGODB_URI as mongodb://<user>:<pass>@<host>:<port>/dairy_production?replicaSet=rs0 | Adapts to multi-source heterogeneous batch data storage, and supports replica sets to ensure high availability in production environments |
| Structured data storage address | Configure PG_URL as postgresql://<user>:<pass>@<host>:<port>/dairy_batch?sslmode=require | Stores fixed-format documents such as raw milk acceptance records and microbial test reports, and supports cross-table associated queries |
| Cache service address | Configure REDIS_URL as redis://<host>:<port>/0?maxmemory=1000 MB | Caches frequently accessed basic batch information and equipment operating parameters, to reduce database query pressure |
| Database write timeout | 600 seconds | Adapts to the waiting requirements for real-time data writes, and prevents normal write failures caused by short timeouts |
Three Common Configuration Errors
- Scenario: When calling workflows 2-3 times per second, database query nodes return empty results or time out. Cause: The database connection pool is configured too small. High concurrency exhausts available connections, preventing subsequent requests from being processed.
- Scenario: After using a DB node to query the database, additional code is required to parse the returned content. Cause: The database return format is not set to structured mapping. The default returns a raw JSON string, which requires additional processing to extract fields.
- Scenario: Excessive database write latency causes production data to back up. Cause: The database replica set is not enabled, or the connection timeout setting is too short, failing to meet the write requirements for real-time batch data.
How to Confirm Proper Configuration
- Perform single-batch data write and query operations, verify that data synchronization delay meets production requirements, and adjust timeout settings to match the scenario.
- Simulate multiple concurrent requests, check that database connection pool usage does not reach the threshold, and confirm that connection resources are sufficient.
- Call the database node to query historical batch data, verify that associated query results are complete, and confirm that the structured storage configuration is active.
- Check the cache service's memory usage, confirm that frequent data cache hit rates meet expectations, and adjust cache capacity to match the scenario.
The values provided are common starting points and should be measured against the reader's own samples.