What Data Looks Like for This Category
Typical entities are enterprises producing steel structures, pressure vessels, construction machinery, and rail vehicle structural components. Daily documents include welding procedure qualifications, welding procedure specifications, welding consumable issue records, welder qualification records, weld inspection reports, repair orders, and more. Data sources include design drawings, material quality certificates, welding equipment parameters, welding consumable warehouses, non-destructive testing systems, and personnel qualification ledgers. Update frequency is per weld, per batch, or per shift. Documents mix structured tables and unstructured process text, with fields including weld number, consumable model and heat number, welder certificate number, non-destructive testing results, and more. Common units are millimeters, hours, shifts, and heat numbers.
What Constraints These Characteristics Impose on the Document Parsing and Chunking Link
Frequently updated documents require the parsing process to have sufficient efficiency to avoid delays in the audit link caused by timeouts. Mixed-format document types require parsing logic compatible with multiple file formats including PDF, Word, and Excel, while also preserving the complete structure of tables to prevent breakage of business field associations caused by split tables. Clear business fields and numbers require chunking to retain contextual associations, avoiding splitting qualification records and test results for the same weld into different chunks, which affects subsequent audit verification. Multi-source data also requires that the parsing process does not lose key qualification parameters such as welder certificate validity period and consumable batch number, otherwise compliance issues will occur in the audit link.
How to Set the Configurations
The following configurations are based on the logic of FastGPT v4.17.0.
| Configuration Item | Recommended Setting | Rationale |
|---|---|---|
chunkSize | 800–1200 characters | Welded structural component documents include short fields such as weld numbers and consumable parameters. This value range balances contextual integrity and chunk density |
chunkSplitter | Split by paragraph + table | Daily documents mix process text and structured ledgers. Splitting by paragraph would destroy table structure. Retaining complete table chunks prevents breakage of business field associations |
customPdfParse | Enabled | Design drawings and inspection reports are mostly in PDF format. Enabling custom parsing preserves drawing annotations and table typesetting, avoiding content loss |
PARSE_FILE_TIMEOUT_SECONDS | 60–90 seconds | Single batch documents such as a single batch of consumable issues plus corresponding inspection records have relatively large content. This duration reserves sufficient parsing time to avoid timeout errors |
chunkSettingMode | Custom mode | Chunking rules need to be adjusted for different document types such as process specifications, ledgers, and inspection reports. General mode cannot adapt to the contextual requirements of structured fields |
UPLOAD_FILE_MAX_SIZE | 100 MB | Design drawings for welded structural components have relatively large file sizes. This setting allows complete upload of large-size drawing documents |
Three Common Mistakes
- Scenario: After uploading ledger files in Word or Excel format, the knowledge base search results do not include table or image content from the document. Cause: The
chunkSplitterconfiguration was not adjusted. The default chunking logic splits table rows and image-associated content, leading to loss of key business information. - Scenario: When calling the knowledge base API to return a file reading link, clicking the link prompts the error "Only support .txt, .m". Cause: The uploaded file format is not supported by the system, or the
UPLOAD_FILE_MAX_SIZEsetting is too small, causing the file to be truncated and its format to become abnormal. - Scenario: After setting a custom chunk length, the welder qualification record and inspection result for the same weld are split into different chunks. Cause: The
chunkSizewas not adjusted based on the field correlation within the document. A value that is too large causes cross-paragraph splitting, or a value that is too small causes chunking to destroy contextual associations.
How to Confirm the Configuration Is Properly Set
- Upload a typical weld inspection report PDF, and check whether the parsed chunks retain the complete weld number and non-destructive testing result tables, with no chaotic content splitting.
- Upload a 10 MB welding consumable issue Excel file, verify that the parsing process does not have timeout errors, and confirm that the time consumption falls within the range set by
PARSE_FILE_TIMEOUT_SECONDS. - Search for content related to "welder qualification records", and confirm that the returned results include complete certificate numbers and validity period information, without being incorrectly split into different chunks.
- Click any file reading link returned by the knowledge base, and confirm that the corresponding document can be opened normally, with no format unsupported errors.
The values provided are common starting points and should be measured against the reader's own samples.