What This Category of Data Looks Like
Daily documents for semiconductor wafer fabrication include process flow cards, batch run records, equipment recipes, SPC charts, exception handling sheets, defect maps, and yield reports. Data sources include MES systems, equipment FDC modules, defect detection and metrology equipment, wafer map files, process recipe libraries, and experimental datasets. Data update cycles cover each wafer batch production, real-time equipment parameter synchronization, and daily yield report archiving. Document fields include process step numbers, nanoscale process parameters, batch identifiers, defect coordinates, yield values, and some documents embed industrial charts and structured data.
What Constraints These Characteristics Impose on Document Parsing and Chunking
Ultra-long process chain documents contain hundreds of steps, so chunking must avoid splitting parameter association logic across processes. Real-time updated exception reports and equipment parameters require incremental parsing to prevent missing the latest exception data. The coupling of nanoscale parameters requires that chunking does not break the corresponding relationship between parameters, otherwise subsequent root cause analysis will be affected. Large differences in document formats across different departments require adaptation to non-standard PDFs and embedded chart parsing. Limited rework windows require the parsing process to complete in a short time to avoid delaying exception handling.
How to Set Configurations
| Configuration Item | Recommended Setting | Rationale |
|---|---|---|
chunkSize | 800–1200 characters | Wafer fabrication documents contain nanoscale parameters and process steps. This range avoids splitting that breaks parameter associations and contextual logic |
chunkSplitter | By paragraph + semantic boundary | Documents such as SPC charts and defect maps have fixed formats. Semantic chunking preserves the association between embedded tables and defect data |
PARSE_FILE_TIMEOUT_SECONDS | 300 seconds | Large batch run records and yield report files have significant size, requiring sufficient time for complete parsing |
customPdfParse | Enabled | Non-standard PDF formats such as equipment recipes and SPC charts. Custom parsing can extract embedded industrial charts and structured data |
UPLOAD_FILE_MAX_SIZE | 1000 MB | Batch run record files archived in wafer fabs may reach large sizes. Adapting to this range avoids upload blocking |
chunkSettingMode | Manual configuration | Different document types (process flow cards, exception handling sheets) require differentiated chunking parameters. Manual configuration allows flexible adjustments |
Three Common Misconfigurations
- An error indicating the file exceeds upload limits occurs when uploading large batch run records. The cause is failure to adjust the
UPLOAD_FILE_MAX_SIZEparameter to match the size of archived files from wafer fabs. - When parsing PDF documents containing SPC charts, extracted table data and defect coordinates are missing. The cause is failure to enable
customPdfParse; the default parsing cannot recognize embedded industrial chart formats. - Unexpected results are returned when calling the API to retrieve chunked index content. The cause is failure to explicitly specify
chunkSettingModeas manual configuration; the default chunking parameters cannot adapt to the special format of wafer fabrication documents.
How to Verify Proper Configuration
- Upload a single wafer yield report under 1000 MB, check that the parsing status completes within 300 seconds with no timeout error prompts.
- Import a PDF document containing SPC charts and defect coordinates, check that the parsing result retains table data and defect location fields.
- Call the FastGPT v4.17.0 API, verify that the chunked index can be retrieved by specifying parameters, and the returned results match the configured
chunkSizerange. - For two document types: process flow cards and exception handling sheets, switch
chunkSettingModeto manually adjust chunking parameters, confirm that the semantic integrity of the parsing results has no obvious breaks.
The values provided are common starting points and should be measured against the reader's own samples.