What the data for this category looks like
Data sources for this scenario include material grade archives, customer technical requirement documents, temperature control instrument collection data, furnace temperature uniformity test reports, test results from hardness testers and metallographic equipment, and workshop MES systems. The update frequency is per furnace load and per batch. Daily generated documents include heat treatment process cards, furnace loading layout drawings, real-time furnace temperature curves, hardness and metallographic test reports, equipment inspection sheets, and exception handling sheets. Document fields include heating temperature (℃), holding time (h), hardness value (HV), metallographic structure type, furnace number, batch number, and others. Some documents embed static charts and numerical curves.
What constraints these characteristics impose on the "document parsing and chunking" link
The high-frequency update requirement per furnace load and per batch means the chunking logic must support batch fast processing, to avoid parsing timeouts caused by overly long single chunks. The mixed multi-type document scenario requires chunking rules to support differentiated adaptation: process cards must retain complete parameter combinations, furnace temperature curves must associate time axis data corresponding to the furnace number, and exception handling sheets must bind batch information from the original process. Documents with embedded charts and numerical curves require the parsing link to retain the association between text and graphics, to avoid lost image links or separation of text and charts. Differences in multi-source data formats require support for custom parsing rules, to adapt to non-standard document formats such as CSV exported from temperature control instruments and JSON exported from MES systems.
How to set the configuration
This document applies to FastGPT v4.17.0.
| Configuration Item | Recommended Setting | Rationale |
|---|---|---|
chunkSize | 800–1200 characters | Adapts to the core parameter length of a single furnace load process card, avoiding splitting associated fields such as heating temperature and holding time |
chunkSplitter | Split by paragraph + fixed character truncation | Adapts to the segment logic of furnace temperature curves and metallographic reports, retaining the context of chart descriptions and corresponding detection data |
customPdfParse | Enabled | Adapts to documents with embedded static charts such as furnace loading drawings and furnace temperature curves, retaining the binding relationship between image links and associated text |
PARSE_FILE_TIMEOUT_SECONDS | 300 seconds | Covers the time requirements for batch parsing of multiple documents per batch, avoiding timeouts triggered by large documents such as furnace temperature curves |
chunkSettingMode | Manual configuration | Adjust chunking rules separately for different document types, ensuring that associated batch information on exception handling sheets is not split |
Three common mistakes
- After importing a Markdown-format test report, the interface shows that images fail to load, and image links lose domain information. The cause is that when
customPdfParseis not enabled, FastGPT parses only plain text content by default, and does not retain the complete path association of Markdown images. - When uploading furnace temperature curve documents on a local area network, the error
GET request failed net::ERR_INCOMPLETE_CHUNKED_ENCODINGappears. The cause is that theUPLOAD_FILE_MAX_SIZEconfiguration value is too low, which cannot accommodate a single large-size furnace temperature curve document, leading to upload interruption. - In the chunking results, the furnace number and batch number on the exception handling sheet are split into two independent chunks, making it impossible to associate parameter information from the corresponding process card. The cause is that the
chunkSizevalue is too small, without considering key associated fields across paragraphs on the exception handling sheet.
How to confirm the configuration is correct
- Upload a single heat treatment process card PDF, check that the parsed text blocks contain complete heating temperature and holding time parameters, with no split breaks in fields.
- Upload a document with embedded furnace loading drawings, confirm that the image links in the parsing results retain complete domain information, and can load normally in the preview interface.
- Batch upload 10 furnace temperature curve documents from the same batch, check that the parsing time does not exceed the configured
PARSE_FILE_TIMEOUT_SECONDSvalue, and no timeout error logs are generated. - View the chunk index list, confirm that the furnace number and batch number on the exception handling sheet match the batch information of the corresponding process card block, and that complete content of the chunk index can be obtained via the API.
The values provided here are common starting points, and should be evaluated against the user's own samples.