What the data for this category looks like
Data sources for this category include design history files, device and material batch records, MES systems, clean environment monitoring equipment, sterilization equipment, and inspection and calibration systems. Data is generated once per product batch or equipment, with regular updates per environmental monitoring plans. Daily documents cover production process specifications, batch production records, clean zone operation records, sterilization batch records, inspection reports, software version records, adverse event and traceability materials. Fields include unique device identification, material batch numbers, clean environment parameters, sterilization parameters, test results, and more.
What constraints these characteristics impose on the vector model and indexing link
Batch-based high-frequency updated data requires vector indexes to support incremental synchronization mechanisms, to avoid server load fluctuations caused by full reconstruction. Mixed multi-type documents, including process specification text, parameter records, monitoring values, etc., require indexes to support vector mapping of unstructured text and structured fields, while matching maxToken and defaultToken settings to adapt to documents of different lengths. Strongly associated fields such as unique device identification and material batch numbers need to be bound with corresponding metadata during the indexing phase to support subsequent precise traceability. High-precision clean environment and sterilization parameter values require the vector model to adapt to the encoding accuracy of numerical features, to avoid feature loss.
Configuration Settings
The following configurations are based on FastGPT v4.17.0.
| Configuration Item | Recommended Range | Rationale |
|---|---|---|
| Chunk Length | 800–1200 characters | Adapts to the text length of process specifications and records in this category, avoiding breakage of key information within chunks |
| Overlap Length | 100–200 characters | Retains contextual association between adjacent chunks, covering cross-chunk key information such as batch numbers and sterilization parameters |
| Similarity Threshold | 0.75–0.85 | Balances retrieval accuracy and recall rate, adapting to the demand for high-precision parameter and identification matching |
maxToken | 8192 token | Meets the vector encoding requirements for long documents such as complete batch production records |
defaultToken | 2048 token | Adapts to the conventional encoding length of single unstructured records |
| Single Retrieval Limit | Top 10 entries | Controls retrieval return volume to avoid server overload, adapting to batch traceability query scenarios |
Three Common Misconfigurations
- Phenomenon: After batch uploading multiple batch production records and clean zone records, the server restarts. The knowledge base status shows not ready, and no automatic indexing action occurs. Cause: Incremental indexing trigger rules are not configured. The full indexing task is interrupted after the server abnormally restarts, and does not automatically recover.
- Phenomenon: After updating the FastGPT version, the indexes of original sterilization batch records cannot return retrieval results. Cause: The vector encoding dimension of indexes from the old version does not match the default model of the new version. Corresponding vectors are not regenerated.
- Phenomenon: When retrieving adverse event traceability materials, the returned results include irrelevant process specification text. Cause: Metadata filtering rules for unique device identification are not bound. Vector retrieval does not associate batch fields for precise matching.
How to Verify Proper Configuration
- Upload a single batch production record, check the index generation progress, confirm no abnormal errors occur.
- Enter a query containing a specific material batch number, confirm whether retrieval results are associated with documents of the corresponding batch.
- Adjust the similarity threshold, confirm whether the number of retrieval results changes as expected.
- Restart the server, confirm whether the knowledge base status automatically recovers to ready, without requiring manual triggering of indexing.
The values provided are common starting points, and should be evaluated against the reader's own samples.