What data for this category looks like
This category's data originates from ERP material master data, melting furnaces and thermometers, spectrometers, MES heat records, mold management systems, customer drawings and technical specifications. Update frequency is per heat, per batch, or daily. Daily documents include casting process cards, batching lists, heat records, mold histories, first-piece inspection records, material reports, and nonconforming product review forms. Fields include heat number, mold number, alloy composition values, melting temperature, pouring parameters, defect types, and more. Document formats mix structured tables and unstructured text, with some being non-editable PDFs scanned from paper.
Constraints on document parsing and chunking
The mixed-format documents and high-frequency update characteristics of this category impose multiple constraints on parsing and chunking. First, a large number of scanned non-editable documents exist, so OCR recognition must be supported to extract complete text. Second, the structure of different documents varies significantly. For example, batching lists are structured tables, while process cards are paragraph-based explanations. Chunking logic must adapt to mixed formats to preserve field associations. Third, single-heat data is an independent business unit. Chunking must avoid splicing content across heats, ensuring that associated fields such as heat number and corresponding composition appear in the same chunk. Finally, high-frequency updated ledger data requires the parsing process to have sufficient timeliness to prevent task interruptions due to timeouts.
Configuration settings
The following configurations are adapted for FastGPT v4.17.0.
| Configuration Item | Recommended Setting | Rationale |
|---|---|---|
chunkSize | 800–1200 characters | Adapts to the average content length of a single-heat ledger in foundry scenarios, avoiding splitting key associated fields such as heat number and alloy composition |
chunkSplitter | Chunk by document structure + semantics | Adapts to mixed-format documents, preserves field context in tables and paragraphs, and avoids splitting merged cells, paragraph titles, and content |
customPdfParse | Enable OCR mode | Processes non-editable PDF documents such as scanned material reports and first-piece inspection records to accurately extract text content from images |
PARSE_FILE_TIMEOUT_SECONDS | 300–600 seconds | Adapts to the parsing time of batch heat ledgers and large-volume spectrometer data files, preventing parsing task interruptions due to timeouts |
UPLOAD_FILE_MAX_SIZE | 1000 MB | Supports uploading multi-heat ledger files archived in batches from workshops, meeting daily data aggregation needs |
chunkSettingMode | Manual configuration | Adjusts chunking rules for different document types such as Excel batching lists and PDF process cards, adapting to the field association requirements of foundry scenarios |
Three common mistakes
- Symptom: Parsing results are empty or garbled after a scanned PDF is uploaded. Cause: OCR mode for
customPdfParseis not enabled, so text content in scanned images cannot be recognized. - Symptom: Merged cell batching ratio fields are empty or misaligned after an Excel batching list is uploaded. Cause: Chunking is not configured by document structure, and direct character-based chunking destroys the merged cell associations of the table.
- Symptom: Cross-heat content splicing appears in chunking results after multiple heat ledgers are uploaded in batch. Cause:
chunkSizeis set too small, splitting continuous document content from different heats into the same chunk, losing context association of the heat number.
How to verify correct configuration
- A scanned material report is uploaded, and the parsed text is checked to confirm it contains complete alloy composition values and heat number.
- An Excel batching list with merged cells is uploaded, and the parsed table fields are verified to match the original document, with no misalignment or missing content.
- Two consecutive heat record documents are uploaded, and the chunking results are checked to confirm they are split separately by heat, with no cross-heat content splicing.
- A single heat archive file of approximately 100 MB is uploaded, and the parsing process is confirmed to have no timeout errors.
The values given are common starting points and should be measured against the user's own samples.