What the Data for This Category Looks Like
This data covers daily documentation from process, equipment, quality, and production departments of crystalline silicon cell manufacturers. Included documents are process cards, diffusion and coating parameter tables, printing and sintering records, EL inspection images, IV grading data, and yield reports. Data sources include silicon wafer incoming ledgers, production MES systems, real-time operating data from diffusion furnaces, coating equipment, and printers, EL and IV test results, and lab test data. Update frequency follows production batches, single-piece test results, and daily summaries. Documents contain numeric parameters, image-based inspection results, and grading statistics fields. Units include degrees Celsius, meters per minute, conversion efficiency percentage, and others.
Constraints for Document Parsing and Chunking
Multi-source, heterogeneous document types require the parsing workflow to support structured tables, plain-text parameter descriptions, and image-based inspection data simultaneously. A single parsing rule cannot cover all content types. High-frequency update cycles require the parsing process to have low latency, to avoid delaying cross-review efficiency in production workflows. Fine-grained per-piece data attributes require chunks to remain compact. Larger chunks will lose critical parameter association information for process windows. It is also necessary to retain OCR text associations for inspection images, otherwise traceability analysis for defects such as cracks and color differences cannot be completed. Differences in field units across documents require retaining contextual associations during chunking, to avoid information invalidation caused by separating parameters from their units.
Configuration Settings
The following configurations are based on FastGPT v4.17.0.
| Configuration Item | Recommended Setting | Rationale |
|---|---|---|
chunkSize | 800–1200 characters | Photovoltaic process documents include continuous process descriptions and independent parameter blocks. This range balances contextual association and the integrity of information per chunk |
chunkSplitter | Semantic priority + fixed length hybrid | Process cards have continuous process logic blocks and scattered equipment parameter entries. Hybrid splitting balances logical coherence and chunking accuracy |
customPdfParse | Enable custom parsing | Documents include EL inspection images and Excel-format table datasets. Targeted configuration of table recognition and OCR rules is required |
PARSE_FILE_TIMEOUT_SECONDS | 300–600 seconds | Single PDF files containing multiple batches of process records and EL inspection images have large file sizes. This duration avoids parsing timeouts |
chunkSettingMode | Manual configuration mode | Adjustment of chunk overlap rate for per-piece data is required, to ensure that process window parameters are not split into two independent chunks |
Three Common Configuration Mistakes
- Symptom: After uploading an Excel table dataset, the knowledge base displays normally, but table content is missing during answer generation. Cause: Table structured parsing configuration in
customPdfParseis not enabled, and only the default plain-text extraction rule is used. - Symptom: Garbled Chinese characters appear when OCR recognizes EL inspection images. Cause: Chinese language support for OCR is not configured, or parsing timeout prevents complete character recognition for some image areas.
- Symptom: The number of returned results does not match expectations when querying a single technical document. Cause:
chunkSizeis set outside the reasonable range for document parameter blocks, leading to critical parameters being split or merged, and failing to accurately match query content.
How to Verify Correct Configuration
- Upload a test document containing printing and sintering parameter tables, EL inspection images, and yield reports. Confirm that parsed text retains table fields and values, as well as defect description content extracted via OCR.
- Adjust
chunkSizeto 800 characters and 1200 characters separately. Submit queries for specific process window parameters, to verify that returned results include complete parameter association information. - Upload a single multi-batch process compilation document with an approximate size of 150 MB. Confirm that the parsing process does not trigger timeout errors, and that all document content is correctly chunked.
- Submit queries for specific EL defect levels in a single document, to verify that returned results include corresponding inspection image descriptions and production data for the relevant batch.
The values provided are common starting points and should be validated against local sample data.