What the data for this category looks like
Business data for SMT mounting and electronic assembly comes from customer BOMs, Gerber coordinate files, MES systems, solder paste and component batch ledgers, mounter operation logs, SPI/AOI inspection reports, reflow oven temperature records, first article confirmation sheets, and repair traceability tags. Data syncs in real time per work order and each line change. Some oven temperature and inspection data updates every second. Single documents include structured and semi-structured fields such as part numbers, reference designators, component batch numbers, oven temperature curve parameters, and defect types. The total number of documents associated with a single work order changes dynamically with production scale.
Constraints imposed by these characteristics on vector models and indexes
Real-time high-frequency updates require indexes to support incremental synchronization, avoiding performance loss from full index reconstruction. Multi-source heterogeneous data formats require unified chunking rules to ensure structured fields from BOMs and text descriptions from SPI reports are correctly vectorized. Work order-level granularity requires chunking to associate production batch information, preventing vector confusion across work orders. Long-text oven temperature curves and AOI defect descriptions need adapted larger context windows to avoid truncation of critical information. Large volumes of repeated component part numbers and defect type text also require indexes to support deduplication and similarity filtering.
How to set configurations
| Configuration Item | Recommended Value | Rationale |
|---|---|---|
defaultToken | Set from representative sample documents | defaultToken is the default token count for direct text chunking; tune it on representative source documents. See the v4.17.2 model-field reference. It is not a vector dimension, embedding-model input cap, or retrieval/LLM context. |
maxToken | 16384 | Covers the maximum text length of AOI inspection reports with complete oven temperature curves, prevents critical defect descriptions from being truncated |
| Chunk length | 800–1200 characters | Adapts to the field density of SMT documents, ensures each chunk contains complete production parameters or defect records |
| Overlap length | 100–150 characters | Prevents critical information from breaking across chunks, such as consecutive component reference designators and defect descriptions |
| Similarity threshold | 0.72–0.85, calibrated via actual testing | Distinguishes similar component batches and defect types, reduces irrelevant entries in traceability results |
| Single search limit | Top 10 entries | Matches the query requirements of work order-level data, balances recall accuracy and response latency |
Three common misconfigurations
- Symptom: The ollama qwen2.5 language model and bge-m3 index model have been added, and the available model list displays them, but the text understanding model dropdown menu has no corresponding options when creating a knowledge base. Cause: The models were not bound to the text understanding model permission group for the knowledge base, only global model channel addition was completed.
- Symptom: When uploading a single document, it shows 8 chunks. After it shows as ready, viewing again shows 13 chunks with duplicate fragments. Cause: Deduplication rules for incremental indexing were not configured, and real-time updated work order data repeatedly triggered chunking tasks, leading to the same document being indexed multiple times.
- Symptom: Only single-file adjustment of vector model training parameters and re-upload is supported; batch processing of all work order documents is not available. Cause: Batch indexing training batch task configuration was not enabled, and the default mode is only single-file operations.
How to verify the configuration is properly set
- Upload a test BOM and coordinate file, check that the number of chunks matches expectations and there are no duplicate fragments.
- Initiate a quality defect traceability query, confirm that the returned results include matching SPI/AOI report fragments, and that the similarity meets the set threshold.
- View the model binding list, confirm that the added ollama qwen2.5 and bge-m3 models appear in the text understanding model dropdown menu.
- On the batch task management page of FastGPT v4.17.0, submit an indexing task for all work order documents, confirm that the task executes in batches and there are no duplicate chunk logs.
The values provided are common starting points and should be measured against the reader's own samples.