What the data for this category looks like
Data sources for photovoltaic module encapsulation scenarios include battery cell grading records, glass adhesive film and backplane batch ledgers, MES systems, string welding and lamination equipment logs, EL/IV tester test reports, insulation withstand voltage records, and packing lists. One record is generated per module or per batch of materials. Daily documents include structured tables, process parameter text, EL defect image annotations, IV power test data, and more. Fields cover current gear, lamination temperature, test power, insulation withstand voltage value, and other items. Corresponding units include ampere, degree Celsius, watt, megaohm, and others.
What constraints these characteristics impose on the "document parsing and chunking" link
Multi-source heterogeneous data formats require parsing configurations to adapt to structured tables, image annotations, and plain text parameters simultaneously. High-frequency single-record updates require chunk granularity that preserves parameter relevance while avoiding excessive chunking that increases retrieval costs. Tender documents require precise matching of battery cell batches and power grades, so chunking must not excessively split critical parameter groups. Large format differences across documents require targeted adjustments to parsing rules; otherwise, table field misalignment or image text loss may occur, directly affecting the accuracy of subsequent quotation matching.
How to set the configurations
| Configuration Item | Recommended Setting | Rationale |
|---|---|---|
chunkSize | 800–1200 characters | Adapts to the text length of mixed process parameters and test data in photovoltaic module encapsulation documents, avoiding splitting critical parameter groups |
chunkSplitter | By paragraph + semantic boundary | Adapts to the mixed text format of EL image annotations and IV test reports, preserving the relevance between parameters and their corresponding descriptions |
customPdfParse | Enable table + image text extraction | Adapts to the table format of packing lists and lamination process records, as well as defect location annotation text attached to EL images |
PARSE_FILE_TIMEOUT_SECONDS | 300–600 seconds | Addresses the time required for image parsing of EL images and IV test reports, avoiding parsing timeout for large test files |
UPLOAD_FILE_MAX_SIZE | 100 MB | Adapts to the need for uploading multiple documents packaged for a single batch of materials, aligning with the common packaging scale of tender documents |
chunkSettingMode | Manual configuration | Flexibly adjust chunking parameters for different document types such as structured records and image reports |
Three common errors
- After uploading a knowledge base packaged as a single Excel file, importing it to a new server fails to retain the original document hierarchy. Cause: The document hierarchy retention option in the chunking configuration is not enabled, and all content within the Excel is merged into a single chunk by default.
- After uploading a PDF file containing EL images, no parsing results are generated. Cause: The image text extraction function of
customPdfParseis not enabled, and only plain text content within the PDF is parsed. - After configuring
chunkSizeto an excessively small value, continuous parameters for battery cell current gears are split across two chunks. Cause: Insufficient chunk length causes critical process parameter groups to be forcibly truncated, making it impossible to fully meet the matching requirements for tender quotations.
How to confirm the configuration is correct
- Upload a single-batch packing list table file supported by FastGPT v4.17.0, and check whether the parsed chunks retain the row and column structure of the table and the corresponding field relationships.
- Upload a PDF file containing EL image annotations, and check whether the parsing results include descriptions of crack locations in EL images and corresponding test data.
- Adjust
chunkSizeto different ranges, upload a text file containing continuous lamination process parameters, and confirm that parameter groups are not split. - Upload a packaged document under 100 MB in size, and confirm that the parsing task completes within the time set by
PARSE_FILE_TIMEOUT_SECONDS.
The values provided are common starting points and should be measured against the reader's own samples.