Data Characteristics for This Category
Data sources for this category include approved formulation processes, material batches, MES/electronic batch records, equipment and environmental monitoring data, LIMS systems, and quality management systems. Data is updated per production batch, each key process step, and according to fixed monitoring plans. Daily documents include master production documents, batch production records, batch packaging records, cleaning records, inspection records, deviation/CAPA/change documents, and batch release review forms. These documents contain fields such as material batch numbers, process parameters, test results, and deviation descriptions, with no additional custom statistical values.
Constraints for Vector Models and Indexes
The batch-based, stepwise update pattern of this category requires indexes to support incremental updates by production batch, to avoid time costs from full index reconstruction. The coexistence of multiple document types requires vector models to adapt to both structured parameter fields and unstructured deviation description text, ensuring consistent vector representations across different data formats. The requirement for data synchronization with production operations means the index’s timestamp association logic must be strictly bound to process steps. This ensures that all data for the exact link can be accurately matched during traceability. The high confidence requirement for quality traceability also requires index recall results to cover all associated deviations, inspections, and release records, preventing critical information from being missed.
Configuration Recommendations
| Configuration Item | Recommended Setting | Rationale |
|---|---|---|
| Chunk Length | 800–1200 characters | Adapts to the text length of individual process steps in batch production records, avoiding information fragmentation within chunks |
| Overlap Length | 100–200 characters | Retains associated process parameter information between adjacent chunks, preventing cross-chunk step logic from being split |
maxToken | 4096 token | Matches the input limit of most commercial embedding models, adapting to the long text content of batch release review forms |
| Similarity Threshold | 0.75–0.85 | Filters low-relevance deviation records while ensuring recall rates for critical quality issues |
| Single Search Limit | Top 10 entries | Covers the main associated link data for a single production batch, avoiding excessive results that interfere with analysis |
Common Misconfiguration Issues
- Phenomenon: After deploying FastGPT v4.17.0 locally, no automatic image indexing option appears when creating a knowledge base and uploading documents. Cause: This feature requires enabling commercial edition integration modules, and relevant plugins are not configured by default in local open-source deployments.
- Phenomenon: The embedding model cannot connect to OneAPI, returning connection timeout or authentication errors. Cause: The API address and key for the embedding model are not configured correctly, or the interface permissions of OneAPI have not been granted to FastGPT.
- Phenomenon: Search results only return a small number of deviation records, failing to cover inspection and release data associated with the full batch. Cause: The single search limit is set too low, not matching the business requirement of covering multi-link quality data for this category.
How to Confirm Proper Configuration
- Upload a typical single batch production record, check if the number of text chunks after splitting matches the number of process steps in the document, and adjust the chunk length to a reasonable range.
- Enter quality issue keywords, view the similarity distribution of recall results, and adjust the similarity threshold according to the business’s requirements for recall accuracy.
- Test incremental upload of new records for a single batch, confirm that the index only updates data for the corresponding batch, and no full reconstruction task is triggered.
- Verify the embedding model connection status, check whether document vectors can be generated normally, with no connection errors or abnormal logs.
The values provided are common starting points and should be measured against the reader's own samples.