Reference only; verify against the target FastGPT release and deployment documentation.
What this category of data looks like
Internal office data for commodity trading mainly comes from purchase and sales contracts, warehouse receipt ledgers, customs declaration documents, employee expense reports and attendance documents, among others. Data updates proceed in line with transaction batches or daily business rhythms. Document types include unstructured long-text contracts, structured Excel transaction logs with tens of thousands of rows, and mixed-format business reports. Core fields include commodity name, delivery tonnage, transaction unit price, warehouse receipt number, business handler and other relevant items. Units involve ton, yuan/ton, batch number and other exclusive trade terms.
What constraints these characteristics impose on document parsing and chunking
The presence of long-text contracts and oversized Excel files requires the parsing link to adapt to the high character count and row scale of single files, to avoid parsing timeouts or chunk truncation. Scenarios with mixed structured and unstructured data require chunking logic to support both paragraph semantic splitting and row/transaction unit splitting, to prevent splitting from breaking business logic associations. Frequently updated business data requires the parsing link to have stable timeout fault tolerance and batch processing capabilities. At the same time, chunking must retain context associations between fields to ensure the accuracy of subsequent vector retrieval, and avoid business information loss caused by improper chunking.
How to set the configurations
| Configuration Item | Recommended Setting | Rationale |
|---|---|---|
chunkSize | 800–1200 characters | Adapt to long contracts and transaction ledgers for commodity trading, retain business association context between adjacent fields |
chunkSplitter | Paragraph + semantic chunking | Differentiate between unstructured contract text and structured Excel rows, avoid splitting cross-transaction logic units |
PARSE_FILE_TIMEOUT_SECONDS | 600–900 seconds | Cover actual parsing time required when processing 100,000-character Word documents and 15,000-row Excel data |
UPLOAD_FILE_MAX_SIZE | 10000 MB | Adapt to upload requirements for batch oversized transaction documents, avoid file truncation or upload failures |
chunkSettingMode | Custom chunking mode | Configure chunking rules separately for mixed-format business documents, meet different parsing needs of structured and unstructured data |
Three common mistakes
- After uploading batch business documents larger than 10 MB, 1000+ chunks are generated and some chunks fail vectorization, with the interface displaying "vector generation exception". The cause is failure to adjust
PARSE_FILE_TIMEOUT_SECONDSandchunkSizefor oversized files, leading to some chunks failing to fully load source data. - JSON or JSON array results returned by DB nodes cannot be directly mapped to knowledge base chunk fields, requiring manual code writing for parsing. The cause is failure to configure automatic parsing rules for structured data, as the default parsing logic only adapts to unstructured documents.
- After uploading a single 100,000-character Chinese Word document, the parsing task times out and fails, with the interface displaying "task timeout". The cause is that
PARSE_FILE_TIMEOUT_SECONDSis set lower than the actual required parsing time, failing to adapt to the parsing time of long documents.
How to confirm the configuration is correct
- Upload a 100,000-character test Word document, check the parsing task's time consumption log, and confirm that the time consumption does not exceed the set value of
PARSE_FILE_TIMEOUT_SECONDS. - Import a 15,000-row test Excel transaction ledger, check whether the chunking results are split by transaction unit or row, and no cross-transaction logic chunks appear.
- Import structured JSON data from a DB node, verify whether the chunks automatically extract key business fields without additional parsing code writing.
- Upload batch business documents larger than 10 MB, confirm that the upload progress does not interrupt, and the number of chunks matches the actual scale of the business documents.
The values given are common starting points and should be measured against the reader's own samples.
Question material comes from public community discussions. Configuration values are common starting points and should be measured against your own samples. Verified on 2026-09-28.