What the data for this category looks like
Reference only; verify against the target FastGPT release and deployment documentation.
What constraints do these characteristics impose on the document parsing and chunking link
The mixed structured and unstructured content feature requires that the chunker cannot rely solely on generic text line breaks for splitting. It must adapt to the row and column structure of tables to avoid splitting that breaks table content. The standardized fields feature requires that chunking preserves the correspondence between fields and values, to avoid query failures caused by separated fields and values after splitting. The high update frequency feature requires that chunking supports incremental parsing logic to reduce repeated processing of already synchronized data. The large single document size feature requires that parsing timeout settings match large file processing needs, to avoid parsing failures caused by mid-process interruptions.
How to set the configurations
| Configuration Item | Recommended Value Range | Rationale |
|---|---|---|
chunkSize | 800–1200 characters | Adapts to content mixing structured tables and unstructured system text, avoids vectorization deviation caused by overly long single chunks |
chunkSplitter | Adaptive chunker | Automatically switches chunking logic for scenarios with both plain text system clauses and structured project data |
PARSE_FILE_TIMEOUT_SECONDS | 600 seconds | Addresses cases where single project documents have large size and many parsing steps, avoids parsing timeouts |
chunkSettingMode | Manual configuration | Adapts to scenarios where internal documents of state-owned and public sector enterprises have unified formats but significant detail differences, allowing flexible adjustment of chunking rules |
UPLOAD_FILE_MAX_SIZE | 100 MB | Matches the common size upper limit of internal system and project data documents, reduces the risk of chunking exceptions |
Three common mistakes
- Phenomenon: Some chunks fail vectorization, and the interface displays a "vectorization exception" prompt. Retrying multiple times restores normal functionality. Cause: The
chunkSizesetting exceeds the adaptive range, and the document contains nested table structures, leading to single chunk content complexity exceeding the processing limit of the vectorization model. - Phenomenon: JSON or JSON arrays returned by database queries cannot be directly used as knowledge base content. Manual parsing via code nodes is required. Cause: Chunking rules adapted to structured data are not enabled. The default chunker cannot recognize JSON-formatted project data, leading to missing fields after parsing.
- Phenomenon: After uploading project documents larger than 10 MB, the number of chunks exceeds 1000 and parsing timeouts occur. Cause: The
PARSE_FILE_TIMEOUT_SECONDSsetting is too low, and does not match the processing time required for large file parsing.
How to confirm the configuration is correct
- Upload a typical mixed document of internal project systems and ledgers. Check the chunk preview interface to confirm that chunks cover both table content and text clauses, with no obvious content truncation or incorrect field splitting.
- Run a database query node once to verify that the parsed knowledge base content contains complete project field information, with no missing or misaligned field correspondences.
- Adjust the
chunkSizeparameter, and compare chunking effects across different values. Confirm that chunk length matches the format characteristics of business documents, avoiding overly short chunks that cause context breaks or overly long chunks that cause vectorization deviation. - Upload the maximum size sample of a single internal document. Check that the parsing process has no timeout errors, and the number of chunks falls within a reasonable range.
The values provided here are common starting points, and should be measured against the reader's own samples.
Question material comes from public community discussions. Configuration values are common starting points and should be measured against your own samples. Verified on 2026-09-28.