What this category of data looks like
Daily documents for iron and steel smelting and continuous casting include heat plans, batching lists, converter smelting records, composition reports, continuous casting process records, and slab judgment sheets. Data sources cover hot metal and scrap steel data, L2 process control systems, laboratories, MES, energy medium systems, and equipment status. Update frequencies are per heat, real-time, and per shift. Each single document includes fields such as heat number, cast sequence number, composition proportion, and temperature value. Some documents present structured parameters in table form.
What constraints do these characteristics impose on the document parsing and chunking link?
Documents for this category are generated in batches per heat and updated frequently. Parsing and chunking must adapt to the high-frequency writing business rhythm. Structured table fields and continuous process descriptions coexist. Chunking must not destroy associated context such as heat number and cast sequence number. Chunking of key process parameters such as final composition and temperature will affect subsequent Q&A accuracy. Loss of these parameters will lead to process judgment deviation. At the same time, document parsing must not occupy production system resources. Parsing duration and file size must be limited.
How to set the configuration
| Configuration Item | Recommended Setting | Rationale |
|---|---|---|
chunkSize | 800–1200 characters | Adapt to the length of key process parameters and continuous descriptions in single-heat documents, avoid splitting complete process steps |
chunkSplitter | Mixed paragraph and fixed-length splitting | Balance the chunking logic for structured table fields and continuous process records, retain field associations |
customPdfParse | Enabled | Retain table formats and field associations in PDF documents, avoid loss of structured data |
UPLOAD_FILE_MAX_SIZE | 1000 MB | Adapt to the maximum volume of single-heat continuous casting process archive documents |
PARSE_FILE_TIMEOUT_SECONDS | 600 seconds | Meet the complete parsing time requirements for large process documents |
chunkSettingMode | Custom mode | Allow targeted configuration of chunking rules that retain associated fields such as heat number and cast sequence number |
Three common errors
- No search results or parsing error messages after uploading documents: The cause is that
customPdfParseis not enabled, resulting in structured table fields in composition reports and slab judgment sheets being incorrectly split or lost. - Unable to locate the specified parsing function entry: The cause is that the
customPdfParsemode is not activated. The default parsing mode cannot adapt to structured tables in iron and steel smelting documents. - Unable to retrieve key process parameters after chunking: The cause is that
chunkSizeis set too small, splitting continuous continuous casting process steps into multiple independent chunks and destroying context association.
How to confirm the configuration is correct
- Upload a single converter smelting record PDF, check whether the parsed text completely retains structured fields such as heat number, composition proportion, and temperature value.
- Upload a continuous casting process archive document within 1000 MB, confirm that the parsing status shows success with no timeout errors.
- Adjust
chunkSizeto 800–1200 characters, check whether the chunking result does not split continuous continuous casting process description paragraphs. - Run the FastGPT v4.17.0 knowledge base search test, input "Converter end temperature control", confirm that the returned chunks include relevant process parameters.
The values provided here are common starting points and should be measured against the reader's own samples.