What data for this category looks like
Data sources include hot metal and scrap steel ledgers, L2 process control data, laboratory composition reports, MES system records, energy medium monitoring data, and equipment status logs. Daily documents include heat plans, batching lists, converter smelting records, continuous casting process parameter records, slab quality judgment sheets, and others. Fields in single documents cover temperature, component proportion, equipment rotation speed, timestamps, and more. Units include degrees Celsius, percentage, revolutions per minute, minutes, and others. Update triggers follow per heat, real-time, or per shift schedules. Data volume per heat increases as the number of process steps rises. Some packaged documents integrate multi-step data and have larger file sizes.
What constraints these characteristics impose on document parsing and chunking
Documents for iron and steel smelting and continuous casting are mostly structured process data, including real-time timestamps and precise numerical fields. Chunking must retain field relevance to avoid splitting critical process parameter groups. High-temperature continuous processes have high data update frequencies. Chunk boundaries must match heat and casting sequence nodes to prevent mixing cross-process data. Single documents have many fields and limited unit consistency. Parsing must adapt to mixed multi-field formats to ensure accurate binding of values and units. Real-time data timeliness requires that the parsing process does not experience excessive timeouts, otherwise it will affect the efficiency of subsequent exception report drafting.
How to configure settings
This setup is compatible with FastGPT v4.17.0.
| Configuration Item | Recommended Setting | Rationale |
|---|---|---|
chunkSize | 800–1200 characters | Adapts to the field density of core process data per heat, avoids splitting critical parameter groups |
chunkSplitter | Chunk by paragraph + semantic meaning | Matches the paragraph structure divided by production links in documents, retains process logic integrity |
PARSE_FILE_TIMEOUT_SECONDS | 120–180 seconds | Addresses parsing time for single multi-step documents, prevents timeout failures due to large data volumes |
customPdfParse | Enable structured parsing mode | Adapts to documents with tables such as batching lists and composition reports, accurately extracts fields and corresponding values |
UPLOAD_FILE_MAX_SIZE | 500 MB | Compatible with packaged documents of full-process records per heat, meets multi-step data integration needs |
chunkSettingMode | Manual configuration | Adjusts chunking rules for different document types such as smelting records vs. judgment sheets, adapts to business scenarios |
Three common mistakes
- Phenomenon: A large number of empty fields appear when parsing batching lists with tables. Cause: The structured parsing mode of
customPdfParseis not enabled, so nested component data in tables cannot be recognized. - Phenomenon: An error "file too large to upload" is triggered when uploading continuous casting process records. Cause: The
UPLOAD_FILE_MAX_SIZEconfiguration is not adjusted, and uploads are interrupted due to exceeding the default limit. - Phenomenon: After uploading a slab judgment sheet with a process flow chart, the retrieval results do not return the annotation information in the image. Cause: The image OCR extraction switch for
customPdfParseis not configured, so process annotation data in the image is not parsed.
How to confirm that configuration is set correctly
- Upload a single converter smelting record document, check that all preset fields and corresponding values are included in the parsing result.
- Upload a packaged document of around 100 MB, confirm that upload and parsing processes have no timeout interruptions.
- Adjust the
chunkSizeparameter, verify that chunking results do not split the core process parameter group of a single heat. - After enabling
customPdfParse, check that parsing results of batching lists with tables retain the corresponding relationship of cells.
The values given are common starting points, and should be measured against the reader’s own samples.