What Data for This Category Looks Like
Data for this category comes from PLM, ERP, MES, supplier materials, validation tests, and customer special requirements. Updates are triggered by each change, with no fixed cycle. Daily documents include ECN/ECR, change assessments, drawings and BOM versions, process validation reports, transition plans, inventory disposal documents, and customer approval files. Data fields include change order number, version identifier, effective node, associated material code, and others. Some data is structured BOM data, while some is unstructured text and drawing content.
What Constraints These Characteristics Impose on the Vector Models and Indexes Link
Multi-source heterogeneous data sources require the indexing system to support mixed-format vector extraction. It enables unified encoding of structured BOM and unstructured process documents. Irregular change updates require the index to support incremental synchronization. This avoids resource consumption from full index reconstruction. Document content lengths vary widely, from short change assessments to dozens of pages of process validation reports. A flexible chunking strategy adapts to different text lengths. Multi-file dependencies associated with changes require the index to retain document-to-document associations. This prevents breaking the complete logic of process changes during retrieval.
How to Configure Settings
This configuration is based on the interface logic of FastGPT v4.17.0.
| Configuration Item | Recommended Setting | Rationale |
|---|---|---|
| Chunk Length | 800–1200 characters | Adapts to the paragraph length of ECR documents and the display boundaries of structured BOM fields |
| Overlap Length | 100–150 characters | Retains contextual association before and after changes, avoiding breaking process change logic across chunks |
maxToken | 4096 token | Matches the input limit of mainstream open-source vector models, adapting to long text content of process validation reports |
defaultToken | 256 token | Adapts to the vector encoding needs of short documents such as change assessments, avoiding excessive truncation |
| Similarity Threshold | 0.72–0.85 | Filters low-relevance old version documents, avoiding retrieval results mixing unapproved change plans |
| Single Search Limit | Top 10 entries | Controls retrieval return volume, adapting to multi-document association query scenarios during change review |
Three Common Misconfigurations
- Symptom: Old version ECN document search results are missing after a full knowledge base index rebuild. Cause: Version verification logic is not configured, and expired change files are not filtered during full rebuild.
- Symptom: 400 status code is returned with no response body when calling the interface to create vectors. Cause: Multi-source document formats are not adapted, and text extraction failure of some PDF drawings causes abnormal vector generation requests.
- Symptom: Server read-write resources are exhausted at fixed daily times. Cause: Index synchronization frequency for batch change submissions is not limited, and a large number of ECR documents triggering index updates simultaneously causes disk IO overload.
How to Confirm the Configuration Is Complete
- Manually upload one ECN document and its corresponding BOM version file. Verify that both associated files can be returned during retrieval.
- Trigger an incremental synchronization of change documents. Verify that the index update log only contains newly added or modified change files.
- Simulate a retrieval request for change review. Verify that returned results are sorted to prioritize directly associated process validation reports.
- Check the server resource monitoring panel. Confirm that disk IO usage for a single index synchronization does not exceed the preset threshold.
The values provided are common starting points and should be measured against the reader's own samples.