What data in this category looks like
Daily documents for this category include configuration item lists, relationship topologies, change records, and asset inventory sheets. Data sources are CMDB, cloud resource lists, automatic discovery tools, and procurement ledgers. Document structures are mostly structured tables and relationship text, containing fields such as configuration item unique identifiers, asset specifications, dependency chains, and change times. Update frequency is daily or weekly. Some batch-exported summary files have large file sizes, and there are compliance documents with digital signatures.
What constraints do these characteristics impose on the "document parsing and chunking" link
The mixed document structure of structured tables and relational topology text requires the parsing engine to adapt to both table content extraction and unstructured text splitting. The high-frequency daily/weekly update rhythm requires controlling chunk processing time to adapt to fast iterative maintenance needs. Compliance documents with digital signatures and complex dependency tables require parsing tools to be compatible with special formats and complex layouts. Batch-exported summary files have large sizes, so reasonable timeout and size limits must be set for the parsing link to avoid task interruptions.
How to set configurations
The following configurations apply to FastGPT v4.17.0.
| Configuration Item | Recommended Setting | Rationale |
|---|---|---|
chunkSize | 800–1200 characters | CMDB-style documents mostly consist of structured short entries and relationship text. This range preserves complete context for configuration items and dependency relationships |
chunkSplitter | markdown+table mixed mode | Daily documents include configuration item list tables and relational topology text, which can adapt to chunking rules for structured and unstructured content respectively |
customPdfParse | Enabled | Need to process PDF procurement ledgers and asset documents with digital signatures to avoid missing content recognition |
PARSE_FILE_TIMEOUT_SECONDS | 300–600 seconds | Batch CMDB export files have large sizes, so sufficient parsing processing time must be reserved |
UPLOAD_FILE_MAX_SIZE | 1000 MB | Adapt to the needs of single batch uploads of asset inventory sheets or change record summary files |
chunkSettingMode | manual | Need to adjust chunking rules separately for structured tables and topology diagrams to adapt to the specific document structure of CMDB maintenance |
Three common mistakes
- Uploading a PDF document with a digital signature fails to recognize complete content. The cause is that the
customPdfParseconfiguration is not enabled. The default parsing engine does not support signed PDF document structures. - Calling the parsing tool returns a file read error. The cause is that the external parsing service is not correctly associated in the FastGPT container, or the locally deployed parsing tool has not opened the corresponding access port.
- The table content of the configuration item list is incorrectly split, resulting in separation of configuration items and their associated relationships. The cause is that
chunkSizeis set too small, or the dedicated chunking rule for tables is not enabled.
How to confirm configurations are set correctly
- Upload a PDF document with a digital signature, check if the parsed text contains complete configuration items and ledger fields, with no missing content.
- Upload a batch CMDB export file, wait for the parsing task to complete and check the status, with no timeout or read failure errors.
- View the chunked results, confirm that the content of complex tables is not split, and that configuration items and their associated relationships are retained in the same chunk.
- Adjust the
chunkSizeparameter, compare retrieval matching effects under different chunk lengths, and confirm that they meet the needs of the business scenario.
The values provided are common starting points and should be measured against the reader's own samples.
Question material comes from public community discussions. Configuration values are common starting points and should be measured against your own samples. Verified on 2026-10-19.