What this category of data looks like
Daily documents in semiconductor wafer fabrication scenarios include process flow cards, batch operation records, equipment recipes, SPC charts, exception handling sheets, defect maps, yield reports, and more. Data sources cover MES systems, equipment FDC modules, defect detection and metrology devices, wafer map files, process recipe libraries, and experimental datasets. Updates align with wafer batch generation, real-time updates during equipment operation, or daily archiving. Document fields include nanoscale process parameters, batch identifiers, defect coordinates, yield ratios, and other relevant fields. Units include nanometers, ppm, and similar units.
What constraints do these characteristics impose on the document parsing and chunking stage?
The data characteristics of this category create multiple constraints for the parsing and chunking workflow: Multi-source and heterogeneous document formats require the parsing module to support mixed text, table, and chart content. High-frequency update requirements mean the chunking process must support incremental parsing and fast indexing. Nanoscale parameters and cross-process associated content require chunking to retain full context and avoid truncating critical logical chains. Cross-departmental document reuse needs require chunking rules to be flexibly adjusted for different document types.
Configuration Settings
These configurations are based on FastGPT v4.17.0.
| Configuration Item | Recommended Setting | Rationale |
|---|---|---|
chunkSize | 800–1200 characters | Adapts to documents with nanoscale process parameters and cross-process associations, retains complete single-process logic, and avoids truncating critical parameter descriptions |
chunkSplitter | Split by semantic paragraphs | Matches the paragraph structure of process flow cards and exception handling sheets, prevents splitting cross-process logical chains |
PARSE_FILE_TIMEOUT_SECONDS | 600 seconds | Reserves sufficient parsing time for large equipment recipe documents containing multi-page SPC charts and defect maps |
customPdfParse | Enabled | For structured equipment recipe documents, custom parsing rules preserve tables and parameter fields, avoiding loss of text within charts |
chunkSettingMode | Manual configuration | Adjust chunking parameters for document types such as batch operation records and yield reports to match differentiated content structures |
Three Common Configuration Mistakes
- Phenomenon: Custom-chunked batch operation records stored in the knowledge base have duplicate blocks automatically removed. The index order does not match the preset configuration. Cause: The global document deduplication rule is enabled by default, and the setting is not disabled for wafer fabrication documents with legally duplicate fields.
- Phenomenon: When parsing exception handling sheets containing SPC charts, parameter text inside the charts cannot be extracted normally. Cause: The structured chart parsing module of
customPdfParseis not enabled, and only the default plain text parsing rules are used. - Phenomenon: Submitting a large yield report document causes the parsing task to time out and fail. Cause: The
PARSE_FILE_TIMEOUT_SECONDSparameter is not adjusted, and the default timeout duration is insufficient for processing large documents with multi-page charts and tables.
How to Verify Proper Configuration
- Upload a typical equipment recipe document. Review the parsed chunking results to confirm nanoscale process parameter descriptions are not truncated.
- Submit custom-chunked batch operation records. Check that all preset chunks are retained in the knowledge base, with no unexpectedly removed duplicate blocks.
- Call the API to submit a document parsing task. Use the status query interface to obtain real-time parsing progress and result status.
- Upload an exception handling sheet containing SPC charts. Confirm the parsing results include parameter text and annotation information from the charts.
The values provided are common starting points and should be measured against the reader's own samples.