What the Data for This Category Looks Like
Core data for this category comes from PLM, ERP, MES, supplier materials, validation tests, and customer special requirements. Update frequency is irregular, triggered each time a process or engineering change occurs. Commonly processed documents include ECN/ECR files, change assessment reports, drawings and BOM version records, process validation reports, production switching plans, inventory disposal documents, and customer approval files. Document fields include change number, version number, effective date, material code, inspection items, qualified judgment standards, and more. Units include pieces, batches, dates, percentages, and more. Some documents embed vector drawings or multi-format attachments.
What Constraints Do These Characteristics Impose on the Document Parsing and Chunking Step
Documents in this category have a mixed structure of text paragraphs, hierarchical headings, and embedded attachments. Data associated with each change has tight relational links, so context between change items, corresponding validation results, and BOM versions must be retained. The irregular update rhythm requires parsing and chunking rules to be flexibly adjusted to adapt to export formats from different PLM/ERP sources. Large BOM comparison files and validation reports with drawings will increase parsing time. Reasonable timeout and chunking parameters must be configured to avoid splitting critical associated information.
How to Configure
The following configurations are written for FastGPT v4.17.0.
| Configuration Item | Recommended Setting | Rationale |
|---|---|---|
chunkSize | 800–1200 characters | Adapts to the typical paragraph length of ECN/ECR documents, retains context between change entries and associated validation information |
chunkSplitter | Split by paragraph + heading | Matches the hierarchical structure of change documents, avoids splitting change items from their corresponding BOM versions and inspection records |
customPdfParse | Enabled | Processes process validation reports with embedded vector drawings, retains the association between text and drawings |
PARSE_FILE_TIMEOUT_SECONDS | 600 seconds | Adapts to the parsing time of large PLM export files, prevents parsing interruptions for large BOM comparison documents |
UPLOAD_FILE_MAX_SIZE | 1000 MB | Accommodates change documents with multi-page drawings, prevents failures when uploading large attachments |
chunkSettingMode | Manual configuration | Adjust chunking rules to adapt to scenarios for documents exported from different PLM and ERP formats |
Three Common Mistakes
- Symptom: A
net::ERR_INCOMPnetwork error occurs when parsing ECN documents with large-size drawings, and upload progress is interrupted. Cause: TheUPLOAD_FILE_MAX_SIZEconfiguration was not adjusted, and the default limit cannot accommodate large attachment files. - Symptom: Calling the API to retrieve content for a specified chunk index returns an empty value. Cause: The
chunkSizeconfiguration value does not match the actual paragraph length of the document, causing some change entries to not be correctly chunked. - Symptom: Chunking results for process validation reports lose the association between inspection items and qualified judgment standards. Cause: The
chunkSplitterconfiguration is set to split only by line breaks, without retaining the document's hierarchical heading structure, which splits associated information.
How to Confirm Proper Configuration
- A typical ECR document may be uploaded, and the parsed chunk list reviewed to confirm that each change entry and its corresponding BOM version are in the same chunk.
- The largest-size PLM export file may be uploaded, and parsing progress checked to confirm completion within the time set by the
PARSE_FILE_TIMEOUT_SECONDSconfiguration. - Chunk-related APIs may be called to verify that returned chunk indexes correspond consistently to actual document paragraphs.
- After enabling the
customPdfParseconfiguration, a process validation report with embedded drawings may be uploaded, and the parsing result checked to confirm that the association between drawings and text is retained.
The values provided are common starting points and should be evaluated against samples utilized in the specific deployment context.