What this category of data looks like
The main entities in this category are hot rolling, cold rolling, section steel, and steel pipe enterprises. Daily documents include rolling plans, heating schedules, rolling specifications, plate shape and thickness records, surface quality reports, and quality certificates. Data sources cover slab/billet batches, heating furnaces, mill automatic control systems, online thickness and width measuring equipment, surface inspection devices, and MES systems. Update frequency is per coil or per steel piece batch level, real-time. Most documents are tables or parameter pages with fixed fields, including thickness (unit: millimeters), plate shape index, surface defect level, and others. A single document is linked to the full process data of the corresponding production batch.
What constraints do these characteristics impose on the "document parsing and chunking" link
The batch-level real-time updated data in this category requires document parsing and chunking to split units by production batch, to avoid cross-batch mixing of associated data. Fixed-field process and inspection documents require chunking to retain the contextual association of fields and parameters, preventing loss of corresponding relationships between key indicators such as thickness and plate shape after splitting. The feature that a single document is linked to full process production data requires chunk length to adapt to the contextual needs of retrieval scenarios, while avoiding reduced retrieval efficiency caused by overly large chunks. In addition, defect descriptions and corresponding detection points in surface quality reports need to retain spatial association information during chunking. If chunking errors cause loss of key data, subsequent audit errors may occur, leading to steel coil downgrading, equipment damage, or customer complaints.
How to set the configurations
These configurations are based on FastGPT v4.17.0.
| Configuration Item | Recommended Setting | Rationale |
|---|---|---|
chunkSize | 800–1200 characters | Adapts to the average length of single-batch process and inspection data in this category, retains the contextual association of key parameters, and avoids losing field correspondence after splitting |
chunkSplitter | Split by paragraph + fixed header | Matches the fixed format structure of documents in this category, ensuring that chunks contain complete process parameter or inspection report units |
customPdfParse | Enabled | Adapts to the fixed typesetting of quality certificates and surface quality reports in this category, retains the original layout information of tables and parameters |
PARSE_FILE_TIMEOUT_SECONDS | 600 seconds | Covers the parsing time of single large rolling specification or batch production batch documents, avoiding parsing failure due to timeout |
chunkSettingMode | Manual configuration | Allows adjustment of chunking rules for different document types such as rolling plans and surface quality reports, adapting to the differences of multiple types of documents in this category |
Three common mistakes
- Phenomenon: When uploading Word or Excel documents for parsing, an error is returned indicating that only .txt and .m formats are supported. Cause: The parsing configuration adapted to office documents is not enabled, or custom parsing rules are not configured correctly.
- Phenomenon: The answers output by the large model do not include image content from the documents. Cause: The image text extraction configuration is not enabled in the document parsing and chunking link, only plain text content is extracted, and the defect description information in the images is not associated.
- Phenomenon: The configured value of the custom chunk length is unreasonable, resulting in failure to match complete batch process data during retrieval. Cause: The
chunkSizeis not adjusted based on the average length of single-batch documents in this category. Too short chunks lead to parameter context breakage, while too long chunks lead to reduced retrieval efficiency.
How to confirm the configuration is correct
- Upload a typical hot rolled coil rolling specification document, check the parsed chunk list, and confirm that each chunk contains a complete process parameter segment and the corresponding steel coil batch information.
- Test uploading a Word-format surface quality report, confirm that there are no format errors in the parsing result, and that fields such as thickness and defect level in the table are retained.
- Adjust the
chunkSizeconfiguration value, upload documents of different lengths, and verify that the parsed chunk length matches the expected setting. - Check the parsing task logs, confirm that the
PARSE_FILE_TIMEOUT_SECONDSconfiguration does not trigger timeout errors, and that the document parsing is complete.
The values provided here are common starting points and should be measured against the reader's own samples.