What data for this category looks like
Data for this category mainly comes from wafer test maps, MES systems, bonding and molding equipment, ATE test machines, X-ray/SAM inspection equipment, and customer specification documents. Daily documents include packaging process cards, bonding and molding parameter tables, wafer incoming material maps, test program version files, yield reports, and failure analysis reports. Data updates occur per production batch, after each chip test, and daily updates. Field differences across different documents are significant. For example, parameter tables include process parameters such as temperature, pressure, and time. Wafer maps include point coordinates and yield markers. Test program files include version numbers and test item lists.
What constraints these characteristics impose on the "document parsing and chunking" link
Multi-source heterogeneous data formats require the parsing module to adapt to multiple file types. These include CSV files exported by equipment, process card PDFs, wafer bitmaps, and more. Custom parsing rules must be supported to handle structured parameter tables and graph-associated content. High-frequency updated per-batch or per-chip test data requires retaining batch association information during chunking. This prevents cross-batch data mixing that causes Q&A deviations. Mixed graphic and text content in wafer maps and failure analysis reports requires retaining visually associated text and graphic descriptions during chunking. Forced splitting that breaks semantic integrity must be avoided. Large differences in fields across documents require retaining field context during chunking. This ensures subsequent Q&A can accurately match equipment parameters and test results.
How to set the configurations
| Configuration Item | Recommended Setting | Rationale |
|---|---|---|
chunkSize | 800–1200 characters | Adapts to the text and parameter block length of packaging process cards and yield reports, avoids splitting process steps or test items |
chunkSplitter | Split by paragraph + title | Matches process nodes and test item titles in documents, retains semantic integrity |
customPdfParse | Enable structured table parsing | Adapts to structured data in bonding and molding parameter tables, extracts fields and corresponding values |
PARSE_FILE_TIMEOUT_SECONDS | 120 seconds | Adapts to the parsing duration of PDFs associated with large-size wafer incoming material maps, avoids timeout failures |
chunkSettingMode | Manual configuration | Adjust chunking rules for different document types (process cards, test reports, wafer maps) to adapt to heterogeneous data |
Three common errors
- Phenomenon: After uploading a Word format process document containing images, the knowledge base can display the image link, but the Q&A module cannot call the image content. Cause: Image-text association parsing for
customPdfParseis not enabled. Only the image link is extracted without binding context. - Phenomenon: When parsing CSV format test program files exported by ATE test machines, an error with empty fields occurs. Cause:
chunkSplitteris not configured to adapt to CSV delimiters, leading to loss of column field association during chunking. - Phenomenon: When uploading a single wafer incoming material map PDF larger than 1000 MB, the parsing task times out and fails. Cause: The value of
PARSE_FILE_TIMEOUT_SECONDSis lower than the actual required parsing duration, and the timeout parameter is not adjusted for large-size files.
How to confirm the configuration is correct
- Use the FastGPT v4.17.0 platform, upload a single packaging process card PDF, and check if the parsed chunks retain complete process step paragraphs without forced splitting.
- Upload a document containing a structured parameter table, and verify that complete values of fields such as temperature and pressure are extracted after parsing.
- Upload a large-size wafer incoming material map PDF, confirm that the parsing task does not have a timeout error, and adjust
PARSE_FILE_TIMEOUT_SECONDSbased on the actual parsing duration. - Upload a Word document containing images, and check if the Q&A module can associate image content with corresponding text descriptions.
The values provided are common starting points and should be measured against the reader's own samples.