What the Data for This Category Looks Like
Typical data sources include wafer fab MES systems, equipment FDC modules, defect detection and metrology equipment, process recipe libraries, experimental data, and more. Daily generated documents cover process flow cards, batch run records, equipment recipes, SPC charts, exception handling sheets, defect maps, yield reports, and others. Some documents contain fields such as nanoscale process parameters, batch numbers, defect spatial coordinates, and more. Data update frequencies cover three rhythms: per batch wafer output, real-time equipment parameter synchronization, and daily yield summary. Some multi-process linked documents circulate across multiple departments, with complex structures including nested headers, timing curves, image annotations, and other formats.
Constraints Imposed on Document Parsing and Chunking by These Characteristics
The document characteristics of semiconductor wafer fabrication impose multiple constraints on the parsing and chunking process. First, multi-source and heterogeneous document formats (MES tables, FDC timing charts, defect map images) require parsing tools to support non-standard structures such as nested headers and image annotations, to avoid losing the association between nanoscale parameters and their corresponding units. Second, data updated by batch or in real time needs chunking to retain contextual associations, to avoid splitting continuous data required for cross-process process logic and parameter drift analysis. Third, frequently updated documents need to adapt to streaming parsing logic, to ensure that newly uploaded batch records can be quickly chunked and stored. Finally, long documents circulating across departments require chunking to not damage the rework path association between processes, otherwise it will affect the attribution accuracy of subsequent cross-review.
How to Configure the Settings
These configurations are based on FastGPT v4.17.0.
| Configuration Item | Recommended Setting | Rationale |
|---|---|---|
chunkSize | 800–1200 characters | Semiconductor documents contain nanoscale parameters and continuous process steps. This value range balances context retention and chunk granularity, avoiding splitting process logic |
chunkSplitter | By paragraph + semantic boundary | Documents such as process flow cards and exception handling sheets have clear paragraph logic. Semantic boundary chunking avoids forced splitting of process steps by character count |
customPdfParse | Enabled | Wafer fabrication documents contain non-standard PDF elements such as SPC charts and wafer maps. Custom parsing preserves the association between images and their accompanying text |
PARSE_FILE_TIMEOUT_SECONDS | 600 seconds | Parsing large batch run records and full-process yield reports takes a long time. This setting avoids timeout interruptions for standard parsing tasks |
UPLOAD_FILE_MAX_SIZE | 1000 MB | Full-process record files for a single batch may be large. This setting covers the document upload needs of most wafer fabs |
chunkSettingMode | Manual configuration | Different document types (equipment recipe cards / defect maps) require differentiated chunking parameters. Manual configuration offers stronger adaptability |
Three Common Misconfigurations
- Symptom: After uploading an Excel-format equipment recipe file, the parsing result only retains headers, and the fields corresponding to process parameters and units are empty. Cause: Custom parsing rules for nested headers and unit row formats in the semiconductor industry are not enabled. The default Excel parsing logic breaks the association between parameters and units.
- Symptom: When parsing a PDF file containing multiple SPC timing charts, the task returns a timeout error. Cause: The value set for
PARSE_FILE_TIMEOUT_SECONDSis shorter than the actual parsing time of large timing chart documents, and does not match the parsing needs of wafer fabrication documents. - Symptom: Process parameters of adjacent processes after chunking are split into different chunks, which cannot support cross-process drift analysis. Cause:
chunkSizeis set too small, and semantic boundary chunking is not enabled. Forced splitting by character count breaks the coherence of process logic.
How to Verify Proper Configuration
- Upload a single typical document (such as a single batch run record) and check the parsed chunk results, confirm that core process parameters and process steps are not split.
- Upload a PDF file containing SPC charts and wafer maps, confirm that the parsing result preserves the association between images and their accompanying text.
- Upload a test file with a size close to
UPLOAD_FILE_MAX_SIZE, confirm that the upload and parsing process is not interrupted. - Adjust the
chunkSizeparameter, compare chunk results under different values, confirm that they meet the logical association requirements of the documents.
The values listed above are common starting points and should be evaluated against the reader's own samples.