What This Category of Data Looks Like
This category of data primarily originates from wafer fab MES systems, equipment FDC modules, defect detection and metrology equipment, process recipe libraries, and experimental data. Daily documents include process flow cards, batch run records, equipment recipes, SPC charts, exception handling sheets, defect maps, and yield reports. Data update rhythms follow wafer batch progress. Key parameters such as etch depth and point coordinates synchronize in real time during production. Bulk documents are summarized and archived daily. Most document structures include standardized process step fields, equipment identifiers, and batch numbers. Some chart documents include spatial distribution coordinates and value ranges.
What Constraints Do These Characteristics Impose on Document Parsing and Chunking
Ultra-long process chains and nanoscale parameters require retaining logical relationships between processes to prevent splitting that breaks process path integrity. Real-time updated batch data requires chunking to adapt to high-frequency incremental synchronization, avoiding data lag caused by overly long chunk intervals. Mixed document types require parsing rule adjustments for documents with spatial coordinates such as SPC charts and defect maps, to ensure chart elements bind to corresponding numerical fields. Cleanroom environments use mostly standard industrial document archive formats, requiring compatibility with non-standard PDF and Excel templates, while handling chunk boundaries for strongly associated fields like batch numbers and equipment IDs. The limited rework window business characteristic requires parsing and chunking to quickly locate abnormally associated data, preventing redundant content from interfering with retrieval efficiency.
How to Configure Settings
This document is written based on FastGPT v4.17.0 configuration guidelines.
| Configuration Item | Recommended Value | Rationale |
|---|---|---|
chunkSize | 800–1200 characters | Adapts to the process segment length of wafer manufacturing documents, avoiding splitting that disrupts process logic |
chunkSplitter | By paragraph + semantic boundary | Matches natural paragraph breaks of process flow cards and exception handling sheets, retaining integrity of associated fields |
customPdfParse | Enabled | Compatibility with non-standard layouts of industrial PDFs, accurately extracting values and coordinates from equipment recipes and SPC charts |
PARSE_FILE_TIMEOUT_SECONDS | 120 seconds | Addresses parsing time for large batch run records, avoiding timeout interruptions |
UPLOAD_FILE_MAX_SIZE | 1000 MB | Supports uploading archived documents for bulk wafer batches |
chunkSettingMode | Manual configuration | Adjusts chunking rules for different document types such as defect maps and yield reports |
Three Common Configuration Mistakes
- Importing Feishu multi-dimensional documents and Excel tables results in missing fields in parsed results. Cause: Custom parsing rules are not enabled for table documents, and default chunking does not bind table columns to corresponding business fields.
- Uploading industrial PDF documents results in missing SPC chart coordinate fields in parsed results. Cause: The
customPdfParseconfiguration is not enabled, and the default parsing rule cannot recognize non-standard chart layouts in industrial PDFs. - Request errors are returned when calling the Marker service to parse PDFs. Cause:
PARSE_FILE_TIMEOUT_SECONDSis not set to a reasonable duration, and large document parsing timeouts do not trigger a retry mechanism.
How to Confirm the Configuration Is Correct
- Upload a standard process flow card PDF, check if the parsed result contains complete process step numbers and corresponding parameter descriptions, confirm that chunks do not split critical process logic.
- Upload a single batch of Excel run records, check the upload progress and parsing status, confirm that no file size limit exceeded error is triggered.
- Upload a defect map PDF, check if the parsed result retains defect point coordinates and corresponding annotation text, confirm that chart elements and fields are fully bound.
- View the system configuration page, confirm that key parameters such as
customPdfParseandPARSE_FILE_TIMEOUT_SECONDShave been set according to business requirements.
The values listed are common starting points and should be evaluated against the reader's own samples.