This document is compatible with FastGPT v4.17.0.
What this category of data looks like
Data sources include raw material quality certificates, heating furnace systems, press collection data, MES systems, non-destructive testing equipment, and customer forging part drawings. Daily documents cover forging process specifications, cutting cards, heating furnace logs, forging process cards, mold life records, heat treatment work orders, and ultrasonic testing reports. Data is updated per part, per batch, or per furnace run. Single document fields include forging part number, heat batch number, heating temperature, holding time, testing results, and some fields have physical units attached.
What constraints these characteristics impose on the "document parsing and chunking" step
Since data is updated per furnace run or batch and individual document fields are clearly defined, cross-batch content must be avoided during chunking to ensure accurate batch association during ledger summary. Multi-source heterogeneous documents have significant format differences: some are structured Excel spreadsheets, others are semi-structured text records. Different parsing logic must be adapted to preserve field integrity. Frequently updated small-volume documents require parsing speed that matches production cycles. Large files such as complete workshop logs require controlled chunk granularity to avoid context breaks. Associated fields such as forging part number and heat batch number must be used as chunk anchors to ensure complete aggregation of data from the same batch. Otherwise, batch matching errors will occur in subsequent summary reports.
How to set the configurations
| Configuration Item | Recommended Setting | Rationale |
|---|---|---|
chunkSize | 800–1200 characters | Matches the field count and length of single ledger entries for forging, avoids including cross-batch data in a single chunk |
chunkSplitter | Mixed structured table and text mode | Compatible with mixed-format documents such as cutting cards and heating furnace logs, preserves integrity of table rows and paragraphs |
customPdfParse | Enable table recognition enhanced mode | Improves structured data extraction accuracy for parameter tables in ultrasonic testing reports and process specifications |
UPLOAD_FILE_MAX_SIZE | 1000 MB | Accommodates complete workshop production logs for a single batch and multiple associated documents |
PARSE_FILE_TIMEOUT_SECONDS | 600 seconds | Matches parsing time for large files under 1000 MB, avoids timeout interruptions |
chunkSettingMode | Manual configuration mode | Allows adjustment of chunking rules for different document types, adapting to heterogeneous documents in forging scenarios |
Three common mistakes
- Issue: Multiple forging part records are merged into a single chunk after Excel import, and cannot be split per individual record. Cause: Custom delimiters are not enabled, or line breaks are not configured as the split basis. The default chunking logic does not recognize table row structures.
- Issue: Word documents larger than 10 MB take excessive time to parse, or generate parsing timeout errors. Cause: The
PARSE_FILE_TIMEOUT_SECONDSparameter is not adjusted, and large file parsing logic is not adapted, leading to single-threaded parsing exceeding thresholds. - Issue: After importing raw material quality certificates, key fields such as heat batch number and testing results are empty or identified incorrectly. Cause: The dedicated document parsing mode of
customPdfParseis not enabled. The general parsing logic cannot match the fixed header format of quality certificates.
How to confirm the configuration is correct
- Upload a single cutting card Excel file, check if chunking results generate separate chunks for each forging part record. Adjust configurations until the expected outcome is achieved.
- Upload a 100 MB workshop heating furnace log document, verify that parsing progress completes within the preset duration. Adjust the corresponding timeout parameter if a timeout occurs.
- Import a PDF ultrasonic testing report, confirm that fields such as testing results and forging part number are fully extracted and retain their original format. Verify the effect of enabling the dedicated parsing mode.
- Check the knowledge base upload file limit settings, confirm that multiple associated documents matching the business scale can be uploaded. Verify the file size limit configuration.
The values provided are common starting points and should be evaluated against the reader's own samples.