What Data for This Category Looks Like
Core data for this category comes from ironmaking, steelmaking, continuous casting, quality control, and scheduling departments of integrated iron and steel enterprises. Daily documents include heat plans, batching lists, converter smelting records, composition reports, continuous casting process records, and slab judgment sheets. Data sources cover hot metal and scrap steel data, L2 process control systems, laboratories, MES systems, energy medium systems, and equipment condition monitoring systems. Update frequency follows three patterns: real-time records generated per heat, real-time synchronization of process data, and shift summary reports. Most documents are structured tables or text formats with fixed process parameters. Fields include heat identifiers, composition test values, process temperature and flow parameters, with units matching industry general standards based on field type.
What Constraints These Characteristics Impose on Document Parsing and Chunking
Real-time data updates per heat require document parsing and chunking to support asynchronous batch processing, to avoid blocking production scheduling workflows. Documents contain structured tables and process parameter text. Chunking must avoid splitting rows that span critical parameters, as this breaks the association logic for quality traceability. Heterogeneous document formats from multiple sources require the parsing module to adapt to different export formats while preserving associations between fields. Since errors can lead to heat downgrades, continuous casting interruptions, or batch losses, the parsing link must include failure retry and exception marking capabilities, and limit single-task timeout periods to prevent impact on subsequent production workflows.
How to Configure Settings
These configurations are adapted for FastGPT v4.17.0:
| Configuration Item | Recommended Setting | Rationale |
|---|---|---|
chunkSize | 800–1200 characters | This category of documents contains closely linked process parameters and test data. This range preserves the integrity of key information for a single heat, avoiding splitting associated content across fields |
chunkSplitter | Split by paragraph + table block | Adapts to the mixed format of structured tables and process text in this category of documents, preventing splitting of critical parameter rows within tables |
customPdfParse | Enabled | Most documents in this category are formatted PDFs exported from industrial L2 systems. Enabling this preserves table structures and field associations, avoiding OCR recognition errors |
PARSE_FILE_TIMEOUT_SECONDS | 600 seconds | The total size of documents associated with a single heat is usually within tens of MB. This duration covers the time required for batch parsing, avoiding task interruption due to timeout |
UPLOAD_FILE_MAX_SIZE | 500 MB | The total of multiple heat-related documents associated with a single cast usually does not exceed this threshold, meeting batch upload and parsing requirements |
chunkSettingMode | Manual configuration | The document structure for this category is fixed. Manual configuration allows precise matching of field association logic, avoiding incorrect splitting from automatic chunking |
Three Common Misconfigurations
- Symptom: Logs show
ocr error, and parsed results lack table content or have empty fields. Cause: PDFs in this category are mostly formatted with encrypted headers exported from industrial systems, and default OCR parsing cannot recognize embedded structured table text. - Symptom: Normal chat functions work normally, but an abnormal error occurs after adding file parsing. Cause: The
customPdfParseconfiguration is not enabled, making it impossible to parse the dedicated PDF format exported from industrial systems, triggering a context exception. - Symptom: Parsing returns a JSON format error. Cause: The multi-field association structure of this category of documents is not adapted, and chunking splits associated parameters across heats, causing the generated retrieval context to fail to match query logic.
How to Verify Proper Configuration
- Upload a converter smelting record PDF, and check if the parsed result retains complete table structure and field associations.
- Check background task logs to confirm that parsing time for documents under 500 MB does not exceed
600 seconds. - Test the chunked content to confirm that consecutive rows containing heat numbers and corresponding composition data are not split.
- Verify that after enabling
customPdfParse, encrypted PDFs exported from industrial systems can be parsed normally with no OCR errors.
The values provided are common starting points and should be measured against the reader's own samples.