What the data for this category looks like
Daily documents for this category include receiving slips, putaway slips, material issue and return slips, inventory transfer slips, inventory count sheets, lot ledgers, first-in first-out and expiration date records, and similar items. Data primarily comes from ERP/WMS systems, barcode or RFID collection devices, purchase work orders, storage location management data, AGV or conveyor system logs, and weighing data. Update frequencies fall into three categories: real-time, daily, and monthly inventory counts. Most documents are structured tables, with dedicated fields such as material code, lot number, storage location number, expiration date, quantity, and weight. Some documents include associated barcode or RFID information.
What constraints these characteristics impose on document parsing and chunking
Multi-source heterogeneous data, structured document formats, and varied update frequencies for this category create multiple constraints for document parsing and chunking. First, header structures vary significantly across different document types. The document type must be accurately identified before matching the corresponding chunking rules, to avoid splitting and mixing lot fields from receiving slips and expiration date fields from inventory count sheets. Second, real-time material flow data requires that key information after chunking is not split across multiple chunks, otherwise the context integrity for subsequent process Q&A will be compromised. Third, monthly inventory count sheets typically contain multi-page, multi-lot data. Chunking must balance the amount of information per chunk and retain cross-page data associations, to prevent splitting from breaking the complete lot traceability required for inventory reconciliation.
How to configure the settings
This configuration is based on FastGPT v4.17.0.
| Configuration Item | Recommended Setting | Rationale |
|---|---|---|
chunkSize | 800–1200 characters | Warehouse documents are mostly structured tables. This range avoids splitting key fields such as material code and lot number across chunks |
chunkSplitter | Paragraph + custom delimiter | Supports splitting via fixed delimiters for document headers and table bodies, adapting to different types of warehouse document formats |
PARSE_FILE_TIMEOUT_SECONDS | 300–600 seconds | Monthly inventory count sheets have large data volumes. This duration covers the complete parsing process for multi-page documents |
customPdfParse | Enabled | Allows designing extraction rules tailored to structured tables in warehouse documents, to accurately extract dedicated fields such as lot number and expiration date |
UPLOAD_FILE_MAX_SIZE | 1000 MB | Allows uploading large documents such as multi-page monthly inventory count sheets, to meet bulk import requirements |
chunkSettingMode | Manual mode | Allows adjusting chunking parameters separately for different document types such as receiving slips and inventory count sheets, to adapt to scenario needs |
Three common mistakes
- Scenario: After custom chunking, upload a lot ledger, and the knowledge base automatically deletes duplicate chunks, causing the index order to not match the original document. Cause: The global duplicate chunk filtering function was not disabled. The system merges chunks with completely identical content by default, which breaks the lot order association in the ledger.
- Scenario: When bulk uploading monthly inventory count sheets, the parsing task times out and fails. Cause: The
PARSE_FILE_TIMEOUT_SECONDSparameter was not adjusted. The default short duration cannot cover the parsing process for multi-page documents. - Scenario: When parsing material issue and return slips, the extracted AGV-associated storage location field is empty. Cause: The
customPdfParsecustom parsing was not enabled. The default parsing rules cannot recognize specialized business fields in the documents.
How to confirm the configuration is properly set
- Upload a single receiving slip, check the chunk preview interface, and confirm that key fields such as material code, lot number, and expiration date are not split across chunks.
- Call the knowledge base parsing status API, and verify that it correctly returns three states: parsing, ready, and failed.
- Upload multiple identical lot ledger documents, and check that the chunk results have no duplicates and match the original lot arrangement order in the documents.
- Upload a document close to the
UPLOAD_FILE_MAX_SIZElimit, and confirm that the upload and parsing processes have no abnormal error messages.
The values listed here are common starting points and should be measured against the reader's own samples.