This document is formulated based on the document parsing and chunking logic of FastGPT v4.17.0.
What data is included in this category
Core data for this category comes from manufacturing enterprise equipment departments, workshop maintenance teams and public utility departments. It covers documents such as equipment ledgers, inspection checklists, maintenance plans, work orders, fault analysis, spare parts lists, and special equipment inspection records. Data sources include EAM/CMMS systems, equipment sensor data, operator inspection records, maintenance history archives, spare parts inventory ledgers, and statutory inspection information. Update cycles cover shift-based inspection submissions, daily/weekly/monthly maintenance plan releases, and emergency work order entry when faults occur. Documents include fixed fields and units, such as temperature and pressure values for inspection items, spare part models, and inventory quantities.
What constraints do these characteristics impose on document parsing and chunking
Heterogeneous document formats (tabular inspection checklists, text-based fault analysis, form-style inspection records) require the parsing module to adapt to multiple layout types. Flattening content must be avoided, as this loses field associations. Frequently updated data sources require the parsing process to have efficient triggering capabilities. This supports real-time processing needs for emergency fault work orders. Document structures with fixed fields and units require chunking to retain contextual associations. The correspondence between inspection items and inspection standards, or between spare part models and inventory quantities, must not be broken. Heterogeneous data from multiple sources also requires unified post-parsing data formats. This facilitates subsequent recall matching for process specification and technical standard Q&A.
How to configure these settings
| Configuration Item | Recommended Setting | Rationale |
|---|---|---|
chunkSize | 800–1200 characters | Equipment inspection documents contain multiple field combinations. Excessively large chunks will lose field associations, while excessively small chunks will disrupt inspection item logic |
chunkSplitter | Split by table + paragraph | Inspection checklists and spare parts lists mostly use tabular formats. Prioritize splitting by table units then splicing context, to retain field integrity |
PARSE_FILE_TIMEOUT_SECONDS | 120 seconds | Long documents such as special equipment inspection records require sufficient time for parsing, to avoid parsing failures caused by timeout interruptions |
customPdfParse | Enabled | Most equipment documents are PDF files with forms. Enabling custom parsing retains the original layout and the correspondence between fields and units |
UPLOAD_FILE_MAX_SIZE | 500 MB | Supports batch import of historical maintenance work order archive files exported from EAM, to cover full historical data synchronization |
Three common misconfigurations
- Attempting to push files via an external system and failing to directly trigger parsing and chunking, resulting in empty knowledge base entries. The cause is that automatic parsing trigger rules after active pushing are not configured. Only manual uploads or pre-set scheduled synchronization tasks are supported.
- Calling the workflow API to upload images, and receiving a parameter error in the response. The cause is failing to use the multipart/form-data format to pass image file fields, and mistakenly passing file content as a JSON parameter.
- Losing unit fields such as temperature and pressure after parsing inspection checklist documents during chunking. The cause is not enabling the
customPdfParseconfiguration. The default parsing flattens table content, losing the correspondence between fields and units.
How to confirm correct configuration
- Upload a typical equipment inspection checklist PDF, and check if the parsed and chunked content retains complete fields and corresponding units such as inspection items, inspection standards, and allowable deviations.
- Call the file parsing API, pass a test maintenance work order file, and check if the number of returned chunked results matches the number of document paragraphs and tables.
- After setting
PARSE_FILE_TIMEOUT_SECONDSto 120 seconds, upload a 100 MB special equipment inspection record document, and confirm that the parsing task is not interrupted by a timeout within 120 seconds. - Test pushing files via an external system, wait for automatic synchronization to complete, and search for the keyword "bearing inspection" in the knowledge base to confirm that relevant chunked content can be retrieved normally.
The values provided are common starting points, and should be evaluated against the user’s own samples.