What the data for this category looks like
Data for this category comes from the full production flow of hot rolling, cold rolling, section steel, and steel pipe manufacturers. Core documents include rolling plans, heating schedules, rolling specifications, plate shape and thickness records, surface quality reports, and quality certificates. Data sources cover slab/billet batches, heating furnaces, mill automatic control systems, online thickness and width measuring equipment, surface inspection devices, and MES systems. Updates are generated in real time per coil or per steel piece. Document fields include millimeter-level thickness values, surface defect grades, production batch numbers, and process parameters. Supported formats include PDF reports, XLS production logs, and PPT process descriptions.
Constraints on Document Parsing and Chunking
High-frequency real-time per-coil or per-piece document generation requires parsing and chunking workflows to support fast batch processing, and avoid single-file parsing timeouts. Multi-format document coverage—including PDF process reports, XLS production logs, and PPT process descriptions—requires adapting parsing rules for different structures, particularly XLS multi-row tables and PPT mixed text and image content. The coil-level traceability business requirement mandates that chunking retain production batch association information, and prevent mixing content across batches. Documents contain fields with clear units such as millimeter-level thickness values and defect grades, so parsing must preserve the binding relationship between field units and corresponding values to ensure complete semantics after chunking.
Configuration Recommendations
| Configuration Item | Recommended Value Range | Rationale |
|---|---|---|
chunkSize | 800–1200 characters | Ensures semantic integrity for unit-bearing process parameters and batch information in this category, avoids splitting single content blocks |
chunkSplitter | By paragraph + semantic boundary | Adapts to documents with logical paragraphs such as rolling specifications and surface quality reports, retains the coherence of process descriptions |
customPdfParse | Enable table structured parsing | Adapts to multi-column tables in PDF reports of this category, such as plate shape and thickness records and quality certificates, extracts complete fields and values |
UPLOAD_FILE_MAX_SIZE | 1000 MB | Supports large-size quality certificates and rolling plan logs uploaded in single-batch batches |
PARSE_FILE_TIMEOUT_SECONDS | 600 seconds | Adapts to parsing of high-frequency real-time per-coil documents, avoids timeouts during small-batch multi-file processing |
chunkSettingMode | Manual configuration | Adjust chunking parameters for different document types (XLS logs, PPT process descriptions) to adapt to the multi-format document characteristics of this category |
This configuration applies to FastGPT v4.17.0.
Three Common Configuration Mistakes
- Symptom: When uploading PPT process descriptions, the parsing process throws the error "the argument ‘windows-1252’ is invalid encoding". Cause: No encoding parsing rules adapted for Chinese content are configured, the default encoding fails to recognize Chinese text and causes parsing failure.
- Symptom: After uploading XLS production logs, the chunking result splits and misplaces batch numbers and thickness values in multi-row tables. Cause: The XLS structured parsing function of
customPdfParseis not enabled, default line-by-line splitting destroys the association logic of the table. - Symptom: When batch uploading multiple documents for a single coil of steel, some parsing tasks time out and fail. Cause:
PARSE_FILE_TIMEOUT_SECONDSis not adjusted to a value suitable for high-frequency real-time document processing, the default threshold cannot cover the time consumption of parallel multi-file parsing.
How to Verify Correct Configuration
- Upload a typical XLS production log for this category, check whether the chunking result retains the complete association between batch numbers and thickness values, with no field misplacement.
- Upload a PPT document containing Chinese process descriptions, confirm that the parsing process has no encoding errors, and the chunked content covers complete process paragraphs.
- Batch upload 10 documents for a single coil of steel, check whether parsing tasks complete within the threshold set by
PARSE_FILE_TIMEOUT_SECONDS, with no timeout failures. - Adjust
chunkSizeto different values, verify that the chunked content retains the binding relationship between process parameters and units, with no semantic breaks.
The values given are common starting points and should be measured against the reader's own samples.