What the Data for This Category Looks Like
The daily documents for this category include winding and core assembly records, vacuum drying records, insulating oil reports, partial discharge and withstand voltage test reports, and nameplate parameter tables. Data sources include design BOMs, silicon steel sheet and insulating material data, vacuum and drying equipment, oil test results, and test station test records. The update frequency is once per piece of equipment or per batch of materials. Most documents are structured records exclusive to a single unit, containing dedicated fields such as insulation resistance, partial discharge value, equipment number, and batch number. Full-link traceability is required per single unit.
What Constraints Do These Characteristics Impose on the Document Parsing and Chunking Stage?
The single-unit exclusive document structure requires that parsing and chunking must retain the association between fields and processes, and content must not be split across batches. Scenarios with mixed document types require chunking rules adapted to both long text paragraphs and nested table formats. Parameter fields with specific units require the parsing process to retain the binding relationship between units and corresponding parameters, to avoid semantic loss after splitting. The high-frequency single-unit update requirement requires the parsing process to have sufficient fault tolerance and efficiency, to prevent overall process blocking due to abnormalities in a single-unit document.
How to Configure Settings
| Configuration Item | Recommended Setting | Rationale |
|---|---|---|
chunkSize | 800–1200 characters | Adapt to the parameter table length of test reports and paragraph length of assembly records, avoid splitting associated insulation parameter items |
chunkSplitter | Split by paragraph + title | Match the chapter structure of documents, retain the contextual integrity of processes such as winding, drying, and testing |
customPdfParse | Enabled | Adapt to the complex nested table format of insulating oil reports and test reports, retain field associations from original typesetting |
PARSE_FILE_TIMEOUT_SECONDS | 120–180 seconds | Process multi-page test reports for a single piece of equipment, avoid parsing timeout for long documents |
UPLOAD_FILE_MAX_SIZE | 50 MB | Adapt to batch material documents and complete record packages for a single piece of equipment |
chunkSettingMode | Manual configuration | Adjust chunking rules for different document types such as assembly records and test reports, adapt to the precise requirements of single-unit traceability |
Three Common Misconfigurations
- Uploading an insulating oil report results in parsing failure, with a 504 status code returned. The cause is that
customPdfParseis not enabled, making it impossible to parse nested table fields in the report. - When batch uploading assembly records for multiple pieces of equipment, the parsing result only returns the first 3 chunks. The cause is that the
chunkSizeparameter is not adjusted, and overly long paragraphs are forcibly truncated, causing chunking exceptions. - Parsing times out during conversation invocation, with no error prompt. The cause is that
PARSE_FILE_TIMEOUT_SECONDSis set to 60 seconds, which cannot cover the parsing duration of the complete test report for a single piece of equipment.
How to Confirm Correct Configuration
- Upload a standard insulating oil report, and check whether the parsed chunks retain the context of associated fields including test date, insulating oil model, and breakdown voltage.
- Navigate to the system settings page of FastGPT v4.17.0, confirm that the
customPdfParseswitch is enabled, and the value ofPARSE_FILE_TIMEOUT_SECONDSmeets the configuration requirements. - Upload a withstand voltage test report with more than 10 pages, and check whether the number of parsed chunks matches the number of document pages, with no missing or duplicate content.
- Test uploading a batch document package within 50 MB, and confirm that the upload progress bar completes normally, with no abnormal interruption prompts.
The values provided are common starting points and should be measured against samples appropriate for the specific use case.