What this category of data looks like
It typically consists of documents daily used by process, equipment, quality, and production departments of crystalline silicon cell manufacturers, including process cards, diffusion and coating parameter tables, printing and sintering records, EL detection images, IV grading data, yield reports, and similar materials. Data sources include silicon wafer incoming materials, MES systems, diffusion furnace/coating equipment/printer operating data, EL and IV test results, and laboratory test data. The update frequency is usually per production batch, per piece detection, with daily updates. Documents contain structured parameter fields, semi-structured process descriptions, unstructured chart data, and fields with dedicated process parameters and units.
What constraints do these characteristics impose on the "document parsing and chunking" link
The multi-type structure of process documents requires chunking rules to adapt to different semantic units, avoiding splitting parameter combinations across processes. Fine-grained wafer-level data and high-frequency update rhythm require chunking to retain the association between processes and batches, ensuring that subsequent Q&A can accurately trace single-batch process windows. Unstructured EL detection images and IV data require high-precision OCR parsing to extract valid parameters; otherwise, key basis for Q&A will be missing. The specificity of document fields requires retaining the binding relationship between parameters and units during chunking, avoiding confusion of process parameters across different processes.
How to set the configurations
| Configuration Item | Recommended Setting | Rationale |
|---|---|---|
chunkSize | 800–1200 characters | Adapts to the length of parameter sections and process descriptions in photovoltaic process documents, avoiding splitting parameter groups across processes |
chunkSplitter | Split by title + paragraph | Process documents have clear process titles such as texturing and diffusion, retaining semantic integrity |
customPdfParse | Enable high-precision OCR mode | Documents containing EL detection images and IV data charts require accurate extraction of parameter labels next to charts |
PARSE_FILE_TIMEOUT_SECONDS | 600 seconds | Parsing large EL detection image sets or multi-batch yield reports takes a long time, preventing premature termination of requests |
UPLOAD_FILE_MAX_SIZE | 1000 MB | Multi-process archived documents for a single batch have large volume, meeting batch upload requirements |
chunkSettingMode | Manual configuration | Different document types (parameter tables, test reports) require differentiated chunking rules to adapt to diverse document structures |
Three common errors
- An OCR Error occurs after enabling
customPdfParse. This is because some old low-resolution EL detection images or over-compressed PDFs cannot be recognized by the high-precision OCR engine. - Not configuring
chunkSettingModecauses the large model to fail to trigger parsing in simple mode. This is because the default chunking rules do not adapt to the parameter-intensive structure of process documents, and the large model cannot recognize valid semantic chunks. - For private deployments, setting
PARSE_FILE_TIMEOUT_SECONDStoo short results in a timeout error in FastGPT even if the parsing service succeeds. This is because the parsing service takes longer than the configured timeout threshold, and FastGPT terminates the request early.
How to confirm the configuration is correct
- Upload a single process card document, check if the parsed chunks are split by process titles, with no parameter sections split across processes.
- Upload a PDF containing EL detection images, confirm that the chart parameters extracted by OCR match the original text, with no missing fields.
- Upload an archived document within 1000 MB, confirm that the parsing task completes within 600 seconds with no timeout errors.
- Upload a process parameter document in FastGPT v4.17.0 simple mode, confirm that the large model triggers the file parsing action and returns matching process content.
The values provided are common starting points and should be measured against the reader's own samples.