What this category of data looks like
Daily documents for iron and steel smelting and continuous casting include heat plans, batching lists, converter smelting records, composition reports, continuous casting process records, and slab judgment sheets. Data sources cover hot metal and scrap steel data, L2 process control, laboratories, MES, energy medium systems, and equipment status. Updates trigger per heat, real-time, or per shift. Document structures include structured tables, long-text process descriptions, and image-text combined process records. Fields include smelting parameters, composition data, equipment status, and judgment results, corresponding to common industrial physical units.
What constraints do these characteristics impose on the document parsing and chunking link
This category has high real-time update requirements. Document parsing must be completed quickly to avoid delaying bid quotation preparation. Heat and casting sequences are tightly coupled. Chunking must retain complete data associations for individual heats. Splitting must not break business logic. Mixed document types have varying structures. Parsing logic must adapt to structured tables, long text, and image-text combined content. Documents come from multiple systems with different formats. Parsing tools must support Excel with multiple sheets, PDF with mixed images and text, and other formats. Core fields such as composition data and process parameters must not be omitted. Process errors have severe consequences. Parsing accuracy directly impacts the credibility of subsequent quotations and bid documents.
How to set the configuration
The following configurations are based on FastGPT v4.17.0:
| Configuration Item | Recommended Setting | Rationale |
|---|---|---|
chunkSize | 800–1200 characters | Adapts to the long text of converter smelting records and table fields of composition reports. Prevents splitting from breaking complete data associations for individual heats |
chunkSplitter | Split by paragraph + table block | Matches structured tables in batching lists and slab judgment sheets, as well as long text in process records. Retains logical integrity of content |
customPdfParse | Enable original image retention mode | Avoids misparsing of image-text combined continuous casting process records via OCR. Retains the correspondence between original images and corresponding text |
PARSE_FILE_TIMEOUT_SECONDS | 120–180 seconds | Adapts to large-volume continuous casting process records and Excel batching lists with multiple sheets. Prevents parsing interruptions due to timeout |
chunkSettingMode | Dynamic adjustment by document type | Distinguishes between structured short text in heat plans and long text in continuous casting process records. Optimizes chunking accuracy |
UPLOAD_FILE_MAX_SIZE | 1000 MB | Supports batch upload of heat ledgers and historical process records. Meets requirements for calling historical data during bid quotation preparation |
Three common misconfigurations
- Original image extraction fails after parsing PDF process records. This occurs when only basic OCR parsing is enabled, and the original image retention mode of
customPdfParseis not turned on. - Only the first sheet content is returned when uploading an Excel batching list with multiple sheets. This happens when the
chunkSplitteris not configured to support multi-sheet parsing logic, and splitting is performed only on the entire file. - A timeout error occurs when parsing large-volume heat plans. This is because the set
PARSE_FILE_TIMEOUT_SECONDSvalue is less than the actual time required for parsing, preventing complete data extraction.
How to verify correct configuration
- Upload a single PDF of image-text combined continuous casting process records. Check if the parsing result retains the correspondence between original images and corresponding text.
- Upload an Excel batching list with multiple sheets. Check if the parsing result includes all sheet names and their corresponding content.
- Upload a converter smelting record document of approximately 10 MB. Check that no timeout error is triggered during parsing.
- In the split chunked content, the heat number, composition data, and process parameters for a single heat are not split into different chunks.
The values provided are common starting points and should be measured against the reader's own samples.