What this category of data looks like
Document data for metal material processing mainly comes from production workshop process specifications, raw material material certificates, finished product inspection reports, and equipment operation logs. Data updates follow production batches; corresponding documents are updated immediately after a single production batch is completed. Most documents contain structured tables, such as alloy composition proportion tables and mechanical performance parameter tables, alongside long-form process operation instructions. Core fields include material grade, heat batch number, tensile strength, and elongation. Common units are MPa, %, and mass proportion values. Some documents also include process diagrams or equipment parameter images that are bound to the text content.
What constraints do these characteristics impose on the document parsing and chunking workflow
The high proportion of structured tables requires the parsing step to retain the row and column structure of tables, and avoid splitting complete composition or performance parameter tables during chunking. The long-form process instructions and batch-bound update rhythm require chunk length to match the information density of single-batch process content, and avoid mixing cross-batch content in the same chunk. Fixed unit and field associations require chunking to retain the binding relationship between fields, corresponding parameters, and units, and avoid separating parameters from their units. Attached process diagrams require parsing tools to associate them with corresponding text content, and avoid separating images from their explanatory text, which would affect subsequent retrieval accuracy.
How to configure the settings
All following configuration items apply to FastGPT v4.17.0.
| Configuration Item | Recommended Setting | Rationale |
|---|---|---|
chunkSize | 800–1200 characters | Matches the information density of process instructions and parameter tables in metal material processing documents, avoiding overloading or over-splitting single chunks |
chunkSplitter | Split by paragraph + table | Matches the mixed structure of long-form process instructions and structured tables in documents, retaining content integrity |
customPdfParse | Enable MinerU parsing mode | Supports parsing metal material processing documents with tables, embedded images, and complex layouts, and adapts to the formats of material certificates and inspection reports |
PARSE_FILE_TIMEOUT_SECONDS | 600 seconds | Covers parsing time for long documents such as monthly process summaries, and avoids parsing failure due to timeout |
chunkSettingMode | Manual configuration | Flexibly adjust chunking rules and parsing parameters for the mixed structure of metal material processing documents |
Three common configuration mistakes
- Phenomenon: When uploading documents with external image links or embedded process diagrams, the parsing task fails, returning status code 400 or "Failed to parse non-standard file format". Cause: The MinerU parsing mode in
customPdfParseis not enabled, and the default parsing tool cannot process document content associated with image addresses. - Phenomenon: After locally deploying MinerU and verifying port connectivity via curl, FastGPT still cannot call the parsing service after enabling enhanced PDF parsing. Cause: The local API access address and port of MinerU are not filled in the FastGPT system configuration, and parsing requests are not correctly forwarded to the MinerU container.
- Phenomenon: The parsed metal composition table only generates a single vector, and cannot split into multiple vectors according to table rows and columns. Cause:
chunkSplitteris not configured to a mode that supports table splitting, and the chunking logic does not recognize the row and column structure of structured tables.
How to confirm the configuration is set correctly
- Upload a standard metal material certificate PDF, and check whether the tables in the parsing result retain their complete row and column structure.
- In the knowledge base settings interface, check the enabled status of the
customPdfParseoption, and confirm that the MinerU API address is configured correctly. - Test parsing a document containing long-form process instructions, and check whether the chunked content does not split complete process step paragraphs.
- View the parsing task run logs, and confirm that no timeout-related errors occur, and that the task execution duration falls within the range of the preset timeout parameter.
The values provided here are common starting points, and should be evaluated against the applicable on-site samples.
Question material comes from public community discussions. Configuration values are common starting points and should be measured against your own samples. Verified on 2026-09-28.