What the data for this category looks like
Business data for dairy processing covers the entire production process of liquid milk, yogurt, milk powder, and dairy ingredients. Daily documents include raw milk acceptance records, standardized batching sheets, pasteurization or UHT heat treatment records, CIP cleaning records, filling records, sample retention and micro-test reports. Data sources include raw milk testing systems, MES production systems, homogenization and heat treatment equipment, online temperature and flow monitoring tools, CIP systems, and LIMS laboratory management systems. Data is synced in real time per milk tank and per production batch. Fields include raw milk antibiotic and physicochemical indicators, heat treatment time and temperature, CIP cleaning parameters, and more. As a liquid biological raw material, strict batch isolation is required for traceability.
What constraints do these characteristics impose on knowledge base retrieval and recall
The real-time, batch-based data characteristics of dairy processing require knowledge base retrieval to support precise filtering by production batch, to avoid cross-batch data interference. Mixed storage of multiple document types requires retrieval to cover both structured fields such as raw milk indicator values and unstructured process descriptions such as CIP operation steps. High data update frequency requires retrieval indexes to quickly sync the latest documents including raw milk acceptance records and micro-test reports. The perishable liquid raw material and strict isolation requirements mean retrieval results must be accurately bound to the current batch, with no abnormal data from other batches included. Fast response is also needed to support the timeliness demands of on-site defect analysis.
How to set configurations
These configurations are based on FastGPT v4.17.0.
| Configuration Item | Recommended Value | Rationale |
|---|---|---|
similarity | 0.75–0.85 | Matches structured fields such as antibiotic and physicochemical indicators in raw milk acceptance records, balancing accuracy and recall coverage |
limit | Top 8 entries | Associated documents for each dairy production batch typically do not exceed 8, avoiding redundant information interfering with defect analysis |
searchMode | mixedRecall | Matches both text keywords such as "pasteurization temperature" and vector semantics such as compliant cleaning processes, adapting to multiple types of business documents |
usingReRank | Enabled | Re-ranks retrieved documents by business relevance, prioritizing key analysis fields such as heat treatment time and CIP processes |
datasetSearchUsingExtensionQuery | Enabled | Expands retrieval scope to cover batch-associated data in sample retention and micro-test reports, improving traceability accuracy |
collectionIds | Knowledge base collections bound by production batch | Precise limitation of retrieval scope to avoid cross-batch data interfering with current batch defect analysis |
Three Common Mistakes
- Phenomenon: Retrieval results do not display embedded images in docx format documents. Cause: Image recognition configuration for the knowledge base is not enabled, and the image parser is not bound to the corresponding knowledge base collection.
- Phenomenon: Number of associated documents returned by retrieval is far lower than expected. Cause: The
similaritythreshold is set too high, or thelimitvalue is too small, failing to cover all necessary business documents for the current batch. - Phenomenon: Unable to call the maker tool to complete image parsing. Cause: The maker tool association is not enabled in the knowledge base configuration, and the call permissions for the image parser are not correctly configured.
How to Verify the Configuration is Correct
- Upload a docx document of raw milk acceptance records with embedded images to the target knowledge base, initiate a retrieval with the keyword "raw milk antibiotic test results", confirm that the results include the image content in the document.
- View the knowledge base configuration page, confirm that
usingReRankanddatasetSearchUsingExtensionQueryare enabled, and thesimilaritythreshold is set within the 0.75–0.85 range. - Initiate a retrieval for a specific production batch, confirm that the returned results only include documents corresponding to this batch such as CIP records and filling records, with no cross-batch data mixed in.
- Test the retrieval response time, confirm that it meets the real-time requirements of on-site defect analysis, adjust relevant configurations to optimize response speed.
The values provided are common starting points and should be measured against the reader's own samples.