What the data for this category looks like
This category includes data from process integration, module, equipment, yield, and manufacturing departments at wafer fabs. Daily documents include process flow cards, batch run records, equipment recipes, SPC charts, exception handling sheets, defect maps, and yield reports. Data sources include MES systems, equipment FDC, defect detection and metrology equipment, wafer maps, process recipe libraries, and experimental data. Update frequencies cover per wafer batch, real-time, and daily. Fields include wafer batch numbers, nanoscale process parameters, defect spatial coordinates, yield values, and more. Units and precision vary significantly across different documents.
What constraints do these characteristics impose on the vector model and indexing stage?
The multi-source heterogeneous data structure, high-frequency update rhythm, and nanoscale precision requirements of this category impose multiple constraints on the vector model and indexing stage. Mixed multi-source text, numerical, and chart data require embedding models to support semantic alignment across cross-type features. Real-time updates per wafer batch and daily batch synchronization require indexes to support incremental construction and rapid refresh. Long process associations across hundreds of steps require a reasonable chunking strategy to avoid context breaks. The demand for cross-module yield attribution requires indexes to store complete metadata association information to ensure retrieval results are traceable.
Configuration Settings
| Configuration Item | Recommended Value | Rationale |
|---|---|---|
| Chunk Length | 800–1200 characters | Adapts to the single-process description length of wafer process documents, avoiding context breaks across processes |
| Overlap Length | 100–150 characters | Retains association information between adjacent processes, improving cross-step retrieval accuracy |
| maxToken | 4096 | Matches the input limit of mainstream embedding models, adapting to long documents containing defect map text |
| Similarity Threshold | 0.75–0.85 | Distinguishes process parameter drift from normal fluctuations, adapting to the matching precision of nanoscale parameters |
| Single Search Limit | Top 8 results | Covers the association retrieval requirements of hundreds of processes, avoiding omission of key batch records |
| defaultToken | Set from representative sample documents | defaultToken is the default token count for direct text chunking; tune it on representative source documents. See the v4.17.2 model-field reference. It is not a vector dimension, embedding-model input cap, or retrieval/LLM context. |
Three Common Misconfigurations
- Index status remains unready for extended periods, and retrieval cannot be triggered. The cause is failure to configure incremental indexing to adapt to real-time updated wafer batch data, resulting in excessively long full index construction time.
- Embedding models cannot properly handle requests in the Doubao-embedding format. The cause is failure to adjust the input format of the embedding model to adapt to the multimodal data structure of mixed wafer parameter values and text.
- Retrieval results cannot display the corresponding document source information. The cause is failure to enable document traceability index configuration, resulting in the vector index not storing metadata fields of original documents.
How to Confirm Proper Configuration
- Upload a standard wafer process flow card, wait for FastGPT v4.17.0 to complete indexing, and check that the interface status shows ready.
- Enter a query statement about process parameter drift, and check if retrieval results return corresponding equipment FDC records and batch run logs.
- Click the traceability button on retrieval results, confirm jumping to the corresponding original document and displaying metadata information such as batch number and equipment number.
- Test embedding model calls, confirm returned vector result dimensions match preset parameters.
The values provided are common starting points and should be measured against the reader's own samples.