What the data for this category looks like
Daily documents for semiconductor wafer fabrication include process flow cards, lot run records, equipment recipes, SPC charts, exception handling sheets, defect maps, and yield reports. Data comes primarily from MES systems, equipment FDC modules, defect detection and metrology devices, process recipe libraries, and experimental data. Three update frequency categories apply: per wafer lot, real-time, and daily. Professional fields include wafer lot numbers, process parameters, defect coordinates, and more. Data structure varies by process type and document type.
What constraints these characteristics impose on vector models and indexes
Multi-source, multi-type data requires unified preprocessing logic to avoid embedding deviations caused by terminology and format differences. Mixed update frequencies need index strategies adapted to both real-time and batch update scenarios to maintain data timeliness. Ultra-long process chains and nanoscale parameter couplings require retaining contextual associations between processes during chunking, while improving recall accuracy to match subtle process drift differences. Exception reports and yield reports are long-form text, so vector models and indexes must handle large token capacities.
How to set configurations
| Configuration Item | Recommended Setting | Rationale |
|---|---|---|
| Chunk Length | 800–1200 characters | Matches the typical content length of single-process documents, adapts to the information density of wafer fabrication process documents |
| Overlap Length | 100–150 characters | Retains contextual associations between adjacent processes, avoids logical breaks in the process chain |
maxToken | 4096 token | Adapts to the long-text content of exception reports and yield reports, covers complete lot analysis data |
| Similarity Threshold | 0.75–0.85 | Distinguishes process parameter drift from normal fluctuations, adapts to subtle differences in nanoscale parameters |
| Single Search Limit | Top 10 results | Meets recall requirements for cross-process attribution, avoids interference from redundant data |
defaultToken | 512 | Serves as the default length for basic chunking, adapts to short-text exception handling sheets |
This document was written based on FastGPT v4.17.0.
Three common mistakes
- Phenomenon: After batch re-embedding the knowledge base, the recall rate of exception reports containing multilingual professional terminology fails to meet expectations. Cause: The embedding model was not fine-tuned for wafer fabrication-specific terminology such as process parameter units and defect types, or the index was not fully rebuilt.
- Phenomenon: The index corresponding to manually inserted exception report documents disappears several hours later. Cause: Persistent storage for incremental indexing was not configured, or the index refresh cycle did not match the update frequency of wafer lots, leading to overwriting.
- Phenomenon: After connecting qwen3-embedding-8b in FastGPT v4.9.11, the process document status remains "Indexing". Cause: The input format of this embedding model was not adapted, or the total token count of single-batch documents exceeded the model's supported limit, causing the indexing process to block.
How to confirm the configuration is properly set
- Upload a typical wafer lot run record, verify that chunked text fragments match key process descriptions in the original document.
- After replacing the embedding model, retrieve exception reports containing professional parameters, check that recall results are sorted in line with business attribution logic.
- Submit a batch re-embedding task, confirm that the total number of indexes matches the total number of documents to be processed after the task completes.
- Simulate real-time updated equipment FDC data uploads, verify that the index can respond to data updates in a timely manner.
The values provided are common starting points and should be measured against the reader's own samples.