What data for this category looks like
Data for this category comes from hot metal and scrap steel datasets, L2 process control systems, laboratories, MES systems, energy medium systems, and equipment condition monitoring modules. Data update cycles include per heat, real-time, and shift-based periods. Daily documents include heat plans, batching lists, converter smelting records, composition reports, continuous casting process records, and slab judgment sheets. Fields cover process parameters, equipment operating status, and test results, with associated units such as degrees Celsius, tons, meters per minute, and others.
Constraints for Document Parsing and Chunking
Documents with multiple update cycles require chunking logic to support both short real-time logs and long-period reports, to avoid excessive adaptation costs. The connection between professional process fields and equipment status must retain context during chunking. This prevents splitting from disrupting process logic and reducing subsequent retrieval accuracy. Continuous process records in single-heat documents may exceed ideal chunk lengths, so chunking rules need adjustment to accommodate long texts. Large continuous casting process documents also increase parsing time. Set reasonable timeout and size limits to avoid task interruptions that impact business processes.
Configuration Settings
The following configurations are based on FastGPT v4.17.0, adapted to document parsing and chunking requirements for iron and steel smelting and continuous casting scenarios.
| Configuration Item | Recommended Value | Rationale |
|---|---|---|
chunkSize | 800–1200 characters | Matches the field association length of single-heat process records, avoiding splitting that disrupts process logic |
chunkSplitter | Fixed delimiter + semantic segmentation | Aligns with structured segmentation rules for heat plans and smelting records, retaining process context |
PARSE_FILE_TIMEOUT_SECONDS | 600 seconds | Accommodates parsing time for large continuous casting process documents, preventing task interruptions from timeouts |
UPLOAD_FILE_MAX_SIZE | 500 MB | Supports batch uploads of multi-heat summary documents per batch |
customPdfParse | Enable structured parsing | Adapts to tabular documents such as converter smelting records and composition reports, extracting accurate fields |
chunkSettingMode | Manual configuration | Adjusts chunking parameters for different document types, adapting to data characteristics with multiple update cycles |
Three Common Misconfigurations
- A timeout error occurs when uploading PDF documents of approximately 10 MB. This is caused by failure to adjust the
PARSE_FILE_TIMEOUT_SECONDSparameter; the default value is insufficient to cover parsing time for large continuous casting process documents. - Individual DOCX documents fail to parse. This is caused by failure to enable the compatibility mode of
customPdfParse, which cannot recognize non-standard table and field formats in iron and steel industry documents. - Long process record segments are not truncated correctly. This is caused by failure to configure a reasonable range for
chunkSize, or failure to enable semantic segmentation rules, resulting in over-length segments not being split.
How to Confirm Configurations Are Correctly Set
- Upload a single continuous casting process record document larger than 100 MB, and check if the parsing task completes within 600 seconds.
- Upload a batching list containing multi-heat data, and check if the chunking results retain complete field associations for each single heat.
- After enabling
customPdfParse, upload a converter smelting record PDF, and check if the parsing results extract complete composition and temperature fields. - Adjust
chunkSizeto 800 characters, upload a process log with a single segment of 1500 characters, and check if the chunking results are truncated into two segments while retaining context.
The values provided are common starting points and should be measured against the reader's own samples.