This document is written based on FastGPT v4.17.0.
What this category of data looks like
This category of data primarily originates from wafer test maps, manufacturing execution systems (MES), bonders and molding equipment, ATE test machines, X-ray/SAM inspection equipment, and customer specification documents. Daily generated documents include packaging process cards, bonding and molding parameter tables, wafer incoming material maps, test program version files, yield reports, and failure analysis reports. Data updates follow production batches, single-chip test results, and daily operational rhythms, covering full-process records from wafer incoming material to finished product testing.
What constraints these characteristics impose on the vector models and indexes workflow
Aligning wafer incoming material maps with physical batches requires precise matching of coordinate and batch information. This demands indexes to effectively support mixed features of structured fields and unstructured documents. High-frequency daily updated ATE test data and yield reports require indexes to support incremental updates, avoiding time-consuming full reconstruction. Packaging parameters vary significantly across different batches, so vector models must capture device states corresponding to subtle parameter differences. Failure analysis reports contain a large number of professional terms and failure modes, requiring indexes to deliver higher accuracy in vector representation of domain-specific terms. Mixed storage of multi-source heterogeneous data also increases index retrieval complexity, requiring targeted optimization of chunking and matching logic.
How to set configurations
| Configuration Item | Recommended Setting | Rationale |
|---|---|---|
| Chunk Length | 800–1200 characters | The content length of packaging process cards and parameter forms mostly falls within this range, preventing damage to parameter associations across chunks |
| Overlap Length | 100–150 characters | Preserves parameter continuity across chunks, avoiding context breaks for bonding parameters |
maxToken | 4096 token | Adapts to the maximum input length of bce-embedding-v1, preventing truncation of critical failure analysis content |
defaultToken | Calibrated based on actual measurements | Adjusts the default token threshold for short-text data from single-chip tests to balance retrieval accuracy and speed |
| Similarity Threshold | 0.72–0.80 | Professional terms in the semiconductor field require a high threshold for vector matching, filtering irrelevant equipment inspection records |
| Single Search Limit | Top 8 results | ATE test data has strong batch relevance, and a small number of highly matched results can cover typical maintenance scenarios |
Three common configuration errors
- Symptom: After using chunk mode to call the pushdata API to upload data, the interface remains in the "Indexing" state with no updates for a long time. Cause: No incremental update trigger rule is configured, or chunk parsing fails because the chunk length exceeds the model's supported upper limit.
- Symptom: The last set of indexes generated after Q&A splitting fails to complete vectorization. Cause: The token count of the last chunk set exceeds the
maxTokenlimit, and no automatic truncation or completion logic is triggered. - Symptom: Retrieval results have low matching accuracy after connecting to bce-embedding-v1. Cause: The similarity threshold is not adjusted to match the vector distribution characteristics of this model, or the input token requirements of the model are not aligned.
How to verify correct configuration
- Upload a single packaging process card document, and check whether the chunking results retain complete parameter entries and process steps.
- Call the pushdata API to upload incremental test data, and verify whether the index update progress matches the expected rhythm.
- Enter professional questions related to equipment inspection, and verify whether the retrieval results include yield reports and failure analysis reports corresponding to the target batch.
- View the vector model call logs, and confirm that the actual usage values of
defaultTokenandmaxTokenmeet the configured requirements.
The values provided here are common starting points and should be measured against the reader's own samples.