What this category of data looks like
Typical stakeholders include incoming quality control (IQC), supplier quality engineering (SQE), procurement quality, and supplier engineering departments at manufacturing enterprises. Daily processed documents include inspection specifications, sampling plans, incoming material reports, supplier audit reports, exception contact sheets, 8D reports, concession approval forms, and similar materials. Data sources include purchase orders, drawings and specifications, supplier quality certificates, inspection equipment data, historical quality data, and supplier portals. Corresponding documents are generated for each batch of incoming materials. Supplier evaluations are conducted monthly or quarterly, and audits are carried out irregularly. Documents contain fields such as batch number, sampling quantity, non-conforming items, supplier name, inspection date, and others. Some documents include structured tables and fixed-format clauses.
Constraints imposed by these characteristics on document parsing and chunking
Document types and formats vary widely. Standardized sampling plan documents and non-standardized handwritten exception contact sheets are both common. Parsing components must support multiple formats. Data updates happen frequently. Teams must process new documents for each batch of incoming materials, which demands strong parallel batch parsing capabilities. Fields including batch numbers, supplier information, and others serve as core retrieval bases. When creating chunks, retain the full context around these fields to prevent key information from being split across chunks. Data originates from external supplier portals and inspection equipment. Some documents are scanned files or poorly formatted PDFs, so custom parsing logic must be supported to extract complete content.
Configuration Settings
| Configuration Item | Recommended Setting | Rationale |
|---|---|---|
chunkSize | 800–1200 characters | Matches the information density of documents such as incoming material reports and 8D reports. Prevents excessive single chunk size from reducing retrieval accuracy, while avoiding overly small chunk sizes that increase retrieval overhead |
chunkSplitter | Split by paragraph + heading | Matches the chapter structure of documents such as inspection specifications and sampling plans. Retains logically coherent content chunks and avoids splitting key clauses |
customPdfParse | Enabled | Extracts tables and embedded text from non-standard PDFs such as supplier quality certificates and incoming material reports. Resolves content missing issues from default parsing logic |
PARSE_FILE_TIMEOUT_SECONDS | 300–600 seconds | Adapts to the parsing duration of batch incoming material files. Prevents timeout failures when parsing multiple files in a single batch |
UPLOAD_FILE_MAX_SIZE | 1000 MB | Supports batch upload of single-batch summary reports. Meets the requirement for merging and uploading multiple sub-files |
chunkSettingMode | Custom | Adjusts chunking parameters for different document types (such as inspection specifications and exception contact sheets) to adapt to diverse document formats |
Three Common Configuration Mistakes
- Phenomenon: Parsed documents lack table content or embedded text descriptions. Cause: The
customPdfParseconfiguration is not enabled. The default parsing logic cannot recognize tables and image-attached text in non-standard PDFs. - Phenomenon: Core fields such as batch numbers and supplier names are split into different content chunks in the chunking result. Cause: The
chunkSizesetting is too small, and thechunkSplitterstrategy of splitting by paragraph + heading is not configured. This leads to truncation of key information. - Phenomenon: Some files fail to parse when uploading multiple incoming material files in batch. Cause:
PARSE_FILE_TIMEOUT_SECONDSorUPLOAD_FILE_MAX_SIZEare not adjusted. Exceeding the configuration limits causes upload timeout or file interception.
How to Verify Correct Configuration
- Upload a standard incoming material report PDF, and check if the parsed result contains all table content and core fields such as batch number and non-conforming items.
- Adjust the
chunkSizeparameter, then check if the chunking result retains the complete chapter logic of the inspection specification, with no key clauses split. - Upload 3 to 5 documents of different formats in batch (such as exception contact sheets in docx format and supplier audit reports in pdf format), and confirm that all files can be parsed normally and chunking results are generated.
- Check the parsing logs of FastGPT v4.17.0, and confirm that there are no error messages such as timeout, file size limit exceeded, or parsing failure.
The values provided are common starting points and should be measured against the user's own samples.