What the Data for This Category Looks Like
Data for this category primarily comes from steel coil labels and quality certificates, punch press and feeder operation data, mold management systems, MES systems, and coordinate measuring machine and fixture inspection data. Updates are triggered per raw material coil, each production batch, and each mold change. Daily documents include stamping work instructions, material utilization rate tables, mold inspection logs, first/last piece records, dimensional inspection reports, and mold change records. Most document fields include material grade, nominal material thickness, mold number, measured dimensional values, and equipment operation parameters, with units including millimeters, tons, hours, and others.
What Constraints These Characteristics Impose on Document Parsing and Chunking
Documents in this category have high update frequency and large batch differences. Chunking must adapt to the independence of single-batch data to avoid mixing cross-batch information. Daily documents contain a large amount of tabular data, such as material utilization rate tables and dimensional inspection reports. Parsing must retain complete table structure, otherwise corresponding parameters cannot be associated after chunking. Data sources cover equipment operation and quality inspection records, with high field correlation. Chunking must retain contextual association between fields to avoid losing parameter correspondence after splitting. A single document may contain data from multiple molds and multiple batches of raw materials. It is necessary to accurately identify batch boundaries within the document to prevent incorrect merging of cross-batch content.
Configuration Settings
The following configurations are based on FastGPT v4.17.0.
| Configuration Item | Recommended Value | Rationale |
|---|---|---|
chunkSize | 800–1200 characters | Adapts to the paragraph length with multiple associated fields in documents of this category, avoiding splitting complete data of a single inspection batch |
chunkSplitter | By document paragraph + custom delimiter | Documents of this category often include work instruction steps and table blocks. Using paragraphs and custom delimiters such as "Batch X" and "mold change record" enables precise chunking |
customPdfParse | Enabled | PDF documents of this category often include equipment diagrams, tables and dimension annotations. Enabling this option retains typesetting and table structure, improving parsing completeness |
PARSE_FILE_TIMEOUT_SECONDS | 120 seconds | Some documents of this category include multi-batch data and long-term equipment operation logs, requiring sufficient time to complete full parsing |
chunkSettingMode | Custom mode | Adapts to the variable document structure characteristics of this category, allowing manual adjustment of chunking rules to match chunking requirements of different document types |
UPLOAD_FILE_MAX_SIZE | 1000 MB | Some documents of this category include complete batch equipment operation logs, requiring support for large file uploads |
Three Common Misconfigurations
- Phenomenon: No image content is displayed after uploading a PDF document, and the large language model cannot reference image information. Cause: The
customPdfParseconfiguration is not enabled. The default parsing only extracts text content and does not retain image information such as mold diagrams and dimension annotations in the document. - Phenomenon: No results are returned during knowledge base search testing, only an error prompt is displayed. Cause: The
chunkSizevalue is too large, causing single-block content to exceed context limits, or keyword field associations are not retained during chunking, making it impossible to match search keywords. - Phenomenon: Content across mold change batches is merged after chunking. Cause: The
chunkSplitteris not configured to use custom delimiters, and boundary identifiers such as "Batch X" and "mold change record" in the document are not identified, causing parameters of different batches to be merged into the same chunk.
How to Verify Correct Configuration
- Upload a test document that includes a mold inspection log and mold change records, check whether the parsed text retains column alignment and field correspondence of the table.
- Enter the knowledge base search test page, enter keywords including material grade and measured dimensional values, confirm that corresponding document fragments can be returned.
- Check the parsing task log, confirm that the parsing time does not exceed the duration set by
PARSE_FILE_TIMEOUT_SECONDS, and no timeout errors occur. - Adjust the
chunkSizevalue, compare chunking results under different lengths, confirm that single-block content does not split complete inspection data of a single batch.
The values provided here are common starting points and should be evaluated against the user's own samples.