What Data for This Category Looks Like
This category of data originates from ironmaking, steelmaking, continuous casting, quality control, and scheduling departments of integrated iron and steel enterprises. It covers daily documents including heat plans, batching lists, converter smelting records, composition reports, continuous casting process records, and slab inspection sheets. Data sources include hot metal and scrap metal ledgers, L2 process control data, laboratory test results, MES systems, energy medium systems, and equipment condition monitoring data. Update frequencies are per heat, real-time, or per shift. Documents include structured fields such as heat number, element content (ppm level), smelting temperature, and casting sequence duration. Some slab inspection sheets are in image format, requiring extraction of table and text information.
Constraints for Document Parsing and Chunking
Documents from multiple sources with high update frequencies require parsing and chunking workflows to support batch and fast processing. This avoids delaying quality audit progress due to parsing delays. Multiple document types, including text tables and scanned images, coexist. The workflow must support both structured text parsing and image content extraction. The tightly coupled business logic between heats and casting sequences requires chunking to retain cross-document association information. It must not arbitrarily split the binding relationship between heat numbers, composition data, and process records. Real-time updated data requires parsing workflows to have low latency. This prevents critical data from failing to enter the repository in a timely manner due to improper chunking configuration.
Configuration Settings
These configurations are based on FastGPT v4.17.0.
| Configuration Item | Recommended Setting | Rationale |
|---|---|---|
chunkSize | 800–1200 characters | Aligns with the length of core single-segment information for heat-related documents, retaining complete associations between compositions, process parameters, and their corresponding heats |
chunkSplitter | Split by paragraph + fixed title blocks | Matches structured titles such as heat numbers, composition reports, and process records in documents, retaining complete business context after splitting |
customPdfParse | Enabled | Adapts to image-format documents such as scanned slab inspection sheets, calling dedicated parsing capabilities to extract table and text information from images |
UPLOAD_FILE_MAX_SIZE | 500 MB | Accommodates batch upload requirements for complete multi-document packages of single-heat process records, preventing interception due to oversized files |
PARSE_FILE_TIMEOUT_SECONDS | 300–600 seconds | Matches the parsing time required for large smelting record documents, avoiding interruptions to the parsing process due to timeouts |
chunkSettingMode | Manual mode | Supports adjusting chunking rules for different document types such as batching lists and composition reports, adapting to multi-format data processing |
Three Common Misconfigurations
- Symptom: Parsing results are empty after uploading an image-format slab inspection sheet. Cause: The
customPdfParseconfiguration is not enabled, so dedicated image parsing capabilities cannot be called to extract table and text information from images. - Symptom: A 504 timeout error is returned when calling the document parsing API. Cause: The
PARSE_FILE_TIMEOUT_SECONDSsetting is too low, failing to match the actual parsing time required for large heat record documents. - Symptom: Parsing fails when an image URL is passed in. Cause: No accessible internal image storage domain whitelist is configured, so the parsing service cannot pull the target image resource.
How to Verify Proper Configuration
- Upload a single slab inspection sheet image document, confirm that the parsing result contains core fields such as element content and slab grade from the original document.
- Call the test interface with the timeout threshold corresponding to
PARSE_FILE_TIMEOUT_SECONDS, upload a 100 MB converter smelting record, and confirm that no timeout error is triggered. - After configuring
UPLOAD_FILE_MAX_SIZE, upload a multi-document compression package that meets the upper limit, and confirm that the upload and parsing processes complete normally. - Switch
chunkSettingModeto manual mode, adjustchunkSizeto 1000 characters, and confirm that the chunking result length matches the expected setting.
The values provided are common starting points and should be measured against the reader's own samples.