What Data for This Category Looks Like
Daily documents for this category include winding and core assembly records, vacuum drying records, insulating oil reports, partial discharge and withstand voltage test reports, and nameplate parameter tables. Data sources cover design BOM, silicon steel sheet and insulating material data, vacuum drying equipment, oil test results, and test station test data. Update frequency is once per product or per material batch. Most documents contain specialized fields such as insulation moisture content, partial discharge amplitude, tap position parameters, with units including ppm, pC, kV, etc. A single product corresponds to multiple associated documents, which must be associated by equipment number or batch.
What Constraints Do These Characteristics Impose on the Document Parsing and Chunking Link
The document characteristics of this category create multiple constraints for the parsing and chunking process. A single product corresponds to multiple associated documents. Chunk association must be done by equipment number or batch to avoid mixing data across batches. Specialized fields such as insulation moisture content and partial discharge amplitude have specific units. Retain the complete binding relationship between fields and units during chunking to prevent misalignment between parameters and their units. High-frequency updates for single-product documents require the parsing process to adapt to small-batch, high-frequency upload requirements. Ensure core parameters of each test report are not truncated. Multi-column formatted test data tables require accurate identification of column correspondences to avoid data misalignment.
How to Set the Configuration
The following configurations are based on FastGPT v4.17.0, adapted to the document characteristics of transformer and high-voltage electrical equipment manufacturing:
| Configuration Item | Recommended Value | Rationale |
|---|---|---|
chunkSize | 800–1200 characters | Adapts to the single-paragraph parameter length of documents such as insulating oil reports and withstand voltage test reports, avoiding splitting specialized parameter combinations |
chunkSplitter | Recognize by paragraph + title | Matches the structure of documents in this category, which are divided by process steps and test items, and retains logically associated content blocks |
customPdfParse | Enable multi-column table recognition mode | Adapts to the multi-column layout of insulating oil reports and test data tables, accurately extracting parameters from each column |
PARSE_FILE_TIMEOUT_SECONDS | 120 seconds | Covers the parsing time of single large-capacity test reports, preventing parsing failure due to timeout |
UPLOAD_FILE_MAX_SIZE | 1000 MB | Adapts to the batch upload requirement of multiple large-capacity test reports for a single device |
chunkSettingMode | Manual configuration mode | For the special document structure of this category, flexibly adjust chunk boundaries to meet the requirements of associating multiple documents for a single device |
Three Common Misconfigurations
- Issue: Multi-column test data tables have misaligned columns after parsing, with some fields empty. Cause: The multi-column recognition mode of
customPdfParseis not enabled, and the default parsing logic cannot adapt to the multi-column parameter layout of insulating oil reports. - Issue: Parsing tasks time out, returning status code 504. Cause: The value set for
PARSE_FILE_TIMEOUT_SECONDSis too small, failing to cover the full parsing time of large-capacity withstand voltage test reports. - Issue: Chunked content includes unrelated document fragments from other devices. Cause: Chunk association rules are not set based on equipment number, mixing test reports from different batches into the same chunks.
How to Confirm the Configuration Is Correct
- Upload an insulating oil report PDF, and verify that the parsed table columns align with the original document, with no field misalignment or missing content.
- Upload a large-capacity withstand voltage test report, and confirm that the parsing task completes within 120 seconds with no timeout errors.
- Import 3 associated documents for a single device, and verify that the chunked results are categorized by equipment number, with no mixed content from other devices.
- Adjust
chunkSizeto different ranges, and verify that the chunked content retains complete specialized parameter combinations with no parameter splitting or truncation.
The values provided are common starting points and should be measured against the reader's own samples.