What this category of data looks like
Data related to photovoltaic cell manufacturing typically comes from wafer incoming material records, MES production systems, real-time logs from diffusion furnaces, coating equipment, and printing presses, EL and IV test results, and laboratory test data. Daily documents include process cards, diffusion and coating parameter tables, printing and sintering records, EL defect detection images, IV performance grading data, and yield reports. Data update frequencies cover per-batch production, per-wafer inspection, and daily summaries. Document structures combine structured tables, plain text descriptions, and image-based test results. Fields include process parameters, test indicators, and their corresponding physical units.
Constraints Imposed on Document Parsing and Chunking
Mixed-document structures require parsing and chunking to retain both the field associations of structured data and the semantic binding of image content. This avoids splitting critical process window information for the same production batch. The mix of wafer-level and batch-level data means chunking must not break the association between production batches and their corresponding test results. Coexisting high-frequency real-time device logs and low-frequency summary reports require adaptation to documents of varying lengths. Additionally, manufacturing processes are sensitive to surface microstructures. Parsed chunks must retain complete combined process parameter information, to avoid losing context from splitting a single parameter alone.
Configuration Settings
This configuration is based on FastGPT v4.17.0.
| Configuration Item | Recommended Value | Rationale |
|---|---|---|
chunkSize | 800–1200 characters | Photovoltaic cell documents contain closely linked process parameters and test data, to avoid splitting critical process window information for the same production batch |
chunkSplitter | Split by semantic paragraphs + table blocks | Documents mix structured tables (IV grading data) and unstructured process descriptions, to retain data associations |
customPdfParse | Enable OCR + structured parsing | Contains EL detection images and printing and sintering record images, requires extracting defect annotation text within images |
UPLOAD_FILE_MAX_SIZE | 1000 MB | Single-batch process documents may include logs and test data from multiple devices, to support large-volume batch uploads |
PARSE_FILE_TIMEOUT_SECONDS | 600 seconds | When parsing documents with multiple EL detection images, reserve sufficient time for OCR and structured extraction |
chunkSettingMode | Manual configuration | For the mixed structure of photovoltaic process documents, the default automatic mode is not needed; custom chunk rules deliver more precise results |
Three Common Mistakes
- Symptom: When uploading a Java interface document with a modified file extension (such as changing it to TXT format), the parsing result has no valid fields. Cause: The document format does not match the actual content, and the FastGPT document parsing engine cannot recognize structured code content in non-native formats.
- Symptom: When using chunk mode to call the pushdata API to upload documents, the task stays in the indexing state for a long time. Cause: The
chunkSizesetting is too large, or thePARSE_FILE_TIMEOUT_SECONDSvalue is insufficient, leading to parsing timeout without completion. - Symptom: When uploading a PDF document containing EL detection images, defect annotations within the images are not extracted. Cause: The OCR parsing switch for
customPdfParseis not enabled, and only plain text content is extracted.
How to Verify Correct Configuration
- Upload a single test document that includes process parameter tables and EL detection images, check if the parsing result includes the parameters within the table and the annotation text within the images.
- View the parsing log of the upload task, confirm that no timeout error is triggered, and the task status shows parsing completed.
- Randomly sample parsed chunk content, check if critical process parameters for the same batch are retained within the same chunk.
- Upload a test document containing structured tables, confirm that the parsing result fully retains the fields and their corresponding values within the table.
The values provided are common starting points, and should be evaluated against samples relevant to the specific deployment.