What data looks like for this category
Typical entities include liquid milk, yogurt, milk powder, and dairy ingredient enterprises. Daily documents cover raw milk acceptance records, standardized ingredient lists, pasteurization or UHT processing records, CIP cleaning records, filling records, retained sample and microbial testing reports. Data sources include milk source testing systems, MES systems, homogenization and heat treatment equipment, online temperature and flow systems, CIP systems, and LIMS laboratory management systems. Data updates are performed per milk tank or production batch. Some online monitoring parameters are updated in real time. Document fields include raw milk physical and chemical indicators, heat treatment time and temperature, CIP process parameters, and microbial test results. Some fields have clear units, such as temperature ℃, flow L/h, and time seconds.
What constraints do these characteristics impose on knowledge base retrieval and recall
Dairy processing data has high update frequency and strong batch correlation. Retrieval must accurately match production batch parameters for the current tender project to avoid mixing cross-batch data. Many document fields include units, so retrieval must match both fields and unit features to prevent irrelevant data from being recalled. Parameters from raw milk acceptance, heat treatment, and other links directly affect quotation compliance and cost accounting for tenders. The accuracy of retrieval results directly ties to the compliance of tender documents, so the scope and quality of recalled data must be strictly controlled. Document structures vary widely across different production links, so retrieval rules must be configured specifically to ensure documents from corresponding links are prioritized for recall. This supports full-process compliance verification for tender documents.
Configuration Settings
These configurations are based on FastGPT v4.17.0.
| Configuration Item | Recommended Value | Rationale |
|---|---|---|
similarity | 0.75–0.85 | Dairy processing data has high field precision requirements. Low-match irrelevant documents must be filtered to ensure recalled results are strongly correlated with core fields such as raw milk indicators and heat treatment parameters |
limit | Top 10–15 entries | Tender documents need to cover compliance data from multiple links. Too many recalled entries will exceed the context window, while too few will fail to cover full-process parameters. Adjust based on actual testing |
searchMode | mixedRecall | Balances semantic matching and field keyword matching, enabling accurate recall of structured data such as physical and chemical indicators with units, time and temperature parameters |
usingReRank | Enabled | Dairy processing data has strong batch correlation. Reranking prioritizes documents from the same batch and production link, improving the integration efficiency of tender data |
datasetSearchUsingExtensionQuery | Disabled | Tender scenarios require strictly limiting recalled content to uploaded compliant production documents, avoiding generation of irrelevant extended content that affects the accuracy of quotations and documents |
collectionIds | Bind exclusive dataset IDs corresponding to production links | Classify documents from raw milk acceptance, heat treatment, filling and other links into different datasets, enabling precise targeted recall of specific link data required for tenders |
Three Common Misconfigurations
- Phenomenon: When
limitis set to 3000, retrieval results cannot be passed to the large model context. Reason: FastGPT has a default upper limit for the context window. Recalled entries exceeding the limit will be automatically truncated, making it impossible to fully transmit parameter content. - Phenomenon: Retrieval results have many duplicate fragments, and the number of recalled files cannot be accurately controlled. Reason: No chunking duplicate rate filtering rules are set for dairy processing batch documents, and the number of recalled entries is not limited via the
limitparameter, resulting in repeated recall of multiple document fragments from the same batch and exceeding required quantities. - Phenomenon: Retrieval responses include general content outside the knowledge base, which does not meet the compliance requirements of tender documents. Reason: No strict knowledge base restriction rules are configured, causing the large model to call external knowledge to generate responses.
How to Verify Proper Configuration
- Upload a single batch of raw milk acceptance records and ingredient lists, initiate a retrieval targeting raw milk antibiotic indicators, and confirm that recalled results only include relevant documents from this batch.
- Adjust the
similarityparameter to 0.8, initiate a retrieval, and check whether the matching degree of returned results meets the preset threshold and filters out low-correlation content. - Enable
usingReRank, retrieve multiple documents from the same production batch, and confirm that recalled results are sorted by batch and link priority. - Disable
datasetSearchUsingExtensionQuery, initiate a retrieval for a question outside the knowledge base scope, and confirm that responses only return compliant data within the knowledge base.
The values listed here are common starting points. They should be evaluated against the reader's own samples.