What This Category of Data Looks Like
Documents generated daily by process, equipment, quality control, and production departments at crystalline silicon cell manufacturing enterprises include process cards, diffusion and coating parameter tables, printing and sintering records, EL detection images, IV grading data, yield reports, and more. Data sources include silicon wafer incoming material records, MES systems, operating data from diffusion furnaces, coating equipment, and printing presses, EL and IV test results, and laboratory test data. Update frequencies cover per-batch production, per-wafer testing, and daily summaries. Documents contain detailed process parameters, wafer-level test results, and batch-related information.
Constraints for Document Parsing and Chunking
The multi-source heterogeneous document structure and high-frequency update rhythm of this category create multiple constraints for document parsing and chunking. Mixed layouts of multi-column parameter tables, wafer-level test data, and associated images require parsing logic to fully extract structured content, and prevent field misalignment or context loss. The fine granularity of wafer-level data means chunk size cannot be overly large, as this will introduce information from unrelated batches. At the same time, the need for complete single process window parameters means chunks cannot be split excessively for key content. Additionally, associated parsing of EL detection images and IV data requires retaining batch mapping relationships within documents, to ensure subsequent Q&A can accurately link images and their corresponding parameters.
Configuration Settings
| Configuration Item | Recommended Approach | Rationale |
|---|---|---|
chunkSize | 800–1200 characters | Adapts to the length of single blocks of parameter tables and test records in photovoltaic process documents, avoiding splitting context of process window parameters |
chunkSplitter | Split by paragraph + table boundaries | Matches the structured layout of process cards and test reports, retains complete structure of multi-column tables |
customPdfParse | Enable multi-column text merging mode | Handles multi-column layout of diffusion furnace parameter tables and printing and sintering records, avoiding content misalignment |
PARSE_FILE_TIMEOUT_SECONDS | 300–600 seconds | Compatible with parsing time for large-volume documents such as EL detection images and IV grading data |
chunkSettingMode | Manual fine-tuned configuration | Adjusts chunk granularity to meet the needs of fine chunking for wafer-level test data |
Three Common Misconfigurations
- Symptom: Parsed multi-column process parameter tables have column misalignment or field merging. Cause: Multi-column text merging mode in
customPdfParsewas not enabled, and the default parsing logic splits content as single-column. - Symptom: After uploading a PDF document containing EL detection images, the parsed result lacks batch information linked to the images. Cause: The image association parsing switch in
customPdfParsewas not configured, so only plain text content was extracted. - Symptom: A single diffusion process window parameter is split into two separate chunks during chunking. Cause:
chunkSizewas set too small, failing to accommodate the full length of a single process parameter and breaking key context.
How to Confirm Correct Configuration
- Upload a test document containing a multi-column diffusion furnace parameter table, and check that the parsed table structure is complete with no column misalignment or field merging.
- Upload a PDF document containing EL detection images and corresponding batch numbers, and check that the parsed result includes association information between images and batches.
- Review the chunked text blocks, and confirm that a single process window parameter is not split across multiple blocks.
- In the FastGPT v4.17.0 environment, test uploading an IV grading data document under 100MB, and confirm that no timeout error occurs during parsing.
The values provided are common starting points and should be measured against the reader’s own samples.