This document is written based on the configuration logic of FastGPT v4.17.0.
What the data for this category looks like
Data for this category comes from manufacturing enterprise PLM, ERP, MES systems, supplier materials, validation test results, and customer special requirement documents. Commonly processed documents include ECN/ECR forms, change assessment reports, engineering drawings, BOM version lists, process validation reports, production switching plans, inventory disposal documents, and customer approval files.
Data updates do not follow a fixed cycle. Updates are triggered each time a process or engineering change occurs. Document structures include specific business fields such as version number, material code, change impact scope, and approval nodes. Some documents include tables, drawings, and scanned file formats.
What constraints these characteristics impose on the document parsing and chunking workflow
Since the document types cover forms, tables, drawings, and scanned files, the parsing link must support content extraction across multiple formats. Data updates have no fixed cycle, and single batch file sizes vary widely. The system must support parsing large-sized batch files.
Documents include specific business fields. When chunking, associated context for change impacts must be retained. Splitting must not damage core business logic such as version numbers and approval nodes. Data formats from multiple sources are inconsistent. The system must support custom parsing rules to adapt to file formats exported by different systems.
Configuration Settings
| Configuration Item | Recommended Value | Rationale |
|---|---|---|
chunkSize | 800–1200 characters | Adapts to the paragraph length of ECN and change assessment reports, retains complete context of change impact scope |
chunkSplitter | Split by paragraph + title | Matches the hierarchical structure of change documents, avoids splitting that damages associated fields such as version numbers and approval nodes |
customPdfParse | Enabled | Supports parsing image-format PDFs such as scanned engineering drawings and process validation reports, extracts structured text content |
UPLOAD_FILE_MAX_SIZE | 1000 MB | Meets the file size requirements for change packages containing multiple drawings and large BOM lists |
PARSE_FILE_TIMEOUT_SECONDS | 600 seconds | Addresses parsing time for complex change documents, prevents task failure due to timeout |
chunkSettingMode | Manual configuration | Adjusts chunking rules to adapt to specific business fields for files exported from different sources such as PLM and ERP |
Three Common Mistakes
- Symptom: After uploading a scanned process validation report, the parsing result is empty or returns garbled text, and the console shows a parsing failure status. Cause: The
customPdfParseconfiguration is not enabled. The default parsing only supports text-based PDFs, and cannot recognize text content in scanned images. - Symptom: After uploading an Excel-format BOM list, the vectorized indexing is incorrect. Some material code and version number fields are missing or misaligned. Cause: Custom parsing rules are not configured for merged cells in the Excel file. This causes the field splitting logic to not match the business format.
- Symptom: After uploading a change package containing multiple drawings via the API, a timeout error is triggered. Cause: The
PARSE_FILE_TIMEOUT_SECONDSvalue is not adjusted to accommodate large files. The default timeout duration is insufficient to complete complex parsing tasks.
How to Verify Correct Configuration
A typical ECN document is uploaded. The parsed chunking results are checked to confirm that paragraphs covering the change impact scope are not forcibly split. A scanned engineering drawing PDF is uploaded. The parsed text content is verified to be complete, with no garbled text or missing content. A change package of approximately 200 MB is uploaded. The parsing task is confirmed to complete normally, with no timeout error triggered. An Excel BOM list containing merged cells is uploaded. The parsed fields are checked to correspond one-to-one with the original document.
The values listed here are common starting points, and should be evaluated against corresponding samples.