What this category of data looks like
Data sources for the photovoltaic module encapsulation process include battery cell grading records, glass adhesive film and backsheet batch ledgers, production MES systems, string welding and lamination equipment logs, and output from EL defect detectors and IV power testers. Data is updated on a per-unit basis: exclusive test reports and process records are generated for each module, and batch ledgers are generated for each batch of raw materials. Daily documents include layout drawings, material batch records, lamination process records, EL defect images, IV test reports, insulation withstand voltage records, and packing lists. Fields covered include batch numbers, current grades, process parameters, test values, and equipment numbers. Units used include professional production parameters such as amperes, watts, and megapascals.
What constraints do these characteristics impose on the document parsing and chunking link
The coexistence of multiple document types requires parsing and chunking to support structured tables, unstructured PDFs, and image formats. Using a single split logic will lead to loss of field associations. High-frequency updated data per module or batch requires chunk size to be controlled within a reasonable range. This prevents mixing content across batches or processes, which reduces retrieval accuracy. Multi-column structured tables and image documents with metadata require chunking to retain binding relationships between fields. For example, material information for the same batch must be bound to corresponding lamination process records. EL defect images must be linked to IV test reports. Additionally, field overlap across different documents is low. A generic chunking logic will cause context breaks, which harms subsequent inspection and audit matching.
How to set the configurations
| Configuration Item | Recommended Setting | Rationale |
|---|---|---|
chunkSize | 800–1200 characters | Adapts to the field density of photovoltaic module-related documents. Ensures a single chunk contains complete associated data for one batch or one module, while balancing retrieval efficiency |
chunkSplitter | Split by paragraph and table row | Adapts to multi-column Excel material batch records, lamination process tables and other structured documents. Retains binding relationships between fields, and prevents multi-column data from being truncated |
customPdfParse | Enabled | Supports parsing PDF test reports linked to EL defect images, extracts test values and metadata, and adapts to parsing requirements for unstructured documents |
PARSE_FILE_TIMEOUT_SECONDS | 300–600 seconds | Adapts to the parsing duration of large EL defect images and batch test reports, prevents parsing task failures due to timeout |
UPLOAD_FILE_MAX_SIZE | 1000 MB | Adapts to upload requirements for large EL defect images and batch test reports, prevents upload failures due to oversized files |
chunkSettingMode | Manual configuration | Adjusts chunking rules for specific document types in photovoltaic module encapsulation, adapts to parsing and chunking requirements for multi-format documents |
Three common errors
- Phenomenon: After uploading a multi-column Excel material batch record, only the first two columns of content can be matched during retrieval. Cause: The
chunkSplitterconfiguration was not adjusted to adapt to multi-column tables. The default chunking logic only recognizes the first two columns of data. - Phenomenon: When calling a conditional judgment component to process parsed boolean qualified parameters, the parameters become empty values. Cause: The original data type association of the parameters was not retained during document parsing, and the context binding between the parameters and the corresponding test report was lost during chunking.
- Phenomenon: Enabling PDF enhanced parsing fails to call the mineru API normally, resulting in parsing task failure. Cause: The associated parameters for the mineru API were not configured. The corresponding parsing interface was not bound after
customPdfParsewas enabled.
How to confirm the configuration is correct
This verification process is compatible with FastGPT v4.17.0.
- Upload a photovoltaic module material batch Excel table, check whether the parsed content includes all column field information.
- Upload a PDF test report linked to EL defect images, confirm whether the parsing task completes within the duration set by
PARSE_FILE_TIMEOUT_SECONDS. - View the chunk preview interface, confirm that a single chunk does not span multiple batches or test data, and matches the
chunkSizeconfiguration range. - After enabling
customPdfParse, check whether the parsed result includes associated metadata and test values for EL defect images.
The values listed here are common starting points, and should be evaluated against the reader's own samples.