This document is compatible with FastGPT v4.17.0.
What the data for this category looks like
Data for this category comes from silicon wafer incoming materials, MES systems, diffusion furnaces, coating equipment, printing presses, EL and IV testing equipment, and laboratories. Daily documents include process cards, diffusion and coating parameter tables, printing and sintering records, EL detection images, IV grading data, and yield reports. Data updates occur per production batch, per individual detection step, and are synchronized daily. Document fields include batch identifiers, process parameter values, test results, and timestamps, with units such as temperature, current, power, and size.
What constraints do these characteristics impose on document parsing and chunking
This category’s data characteristics impose multiple constraints on document parsing and chunking. A high share of documents use mixed formats, including text, tables, and embedded detection charts. Precise chunking for multi-format content is required to avoid splitting table rows across chunks or breaking chart information. Data updates frequently by production batch, single device run, and single test item. Chunks must bind to corresponding batch identifiers and device numbers to prevent cross-batch content confusion. Fields use dedicated units. Chunks must retain the binding between fields and their units, to avoid separating process parameters from their corresponding units after splitting.
Configuration settings
| Configuration Item | Recommended Setting | Rationale |
|---|---|---|
chunkSize | 800–1200 characters | Photovoltaic cell process parameters and test records are mostly short text blocks. This range can retain complete parameter groups for a single batch process or single device. |
chunkSplitter | Split by line breaks and table rows | Adapts to this category’s tabular documents, enables separate chunking for single records, and resolves multi-row merging issues during Excel import. |
customPdfParse | Enabled | This category’s documents contain embedded EL detection images and IV data charts. Enabling this retains text annotations associated with charts, avoiding parsing loss. |
PARSE_FILE_TIMEOUT_SECONDS | 300–600 seconds | Parsing multiple documents for a single batch (such as a process card plus test report) requires a long duration. This range covers parsing times for most conventional documents. |
UPLOAD_FILE_MAX_SIZE | 2000 MB | This category’s bulk production data documents have large volume. This value supports uploading multiple batch documents in a single upload. |
chunkSettingMode | Manual configuration | Chunking parameters must be adjusted for different document types, such as process cards and test reports. Manual mode enables fine-grained control. |
Three common mistakes
- Phenomenon: When importing a process parameter table from Excel, the automatic splitting result merges multiple consecutive rows into one chunk. Cause: The
chunkSplittersetting is not specified to split by table rows. The default chunking rules do not adapt to table structures. - Phenomenon: When uploading the same IV grading data document, the recall rate is abnormal after the first import, and returns to normal after re-importing. Cause:
customPdfParseis not enabled, resulting in incomplete parsing of text annotations associated with embedded charts. Chunked content deviates from actual data. - Phenomenon: After batch uploading multiple EL test reports, the workflow cannot extract hidden crack data from each report individually. Cause: Chunks are not bound to batch identifiers. Chunks do not retain context association for individual documents, making it impossible to match corresponding reports during extraction.
How to confirm correct configuration
- Upload a single Excel process parameter table, check if each chunk corresponds to one single record, to verify that the
chunkSplitterconfiguration takes effect. - Upload a PDF test report containing embedded charts, check if chunks retain text annotations next to the charts, to verify that
customPdfParseis enabled. - Batch upload multiple documents of the same type, check if the upload queue loads normally, to verify that the
UPLOAD_FILE_MAX_SIZEconfiguration matches the total document volume. - Wait for parsing to complete, check the chunking logs to confirm that the parsing duration does not exceed the
PARSE_FILE_TIMEOUT_SECONDSconfiguration range, to verify that the timeout setting is reasonable.
The values provided are common starting points and should be evaluated against the reader's own samples.