This document is written based on the configuration logic of FastGPT v4.17.0.
What data in this category looks like
This data typically serves DBAs, data platform operations teams, and backend teams. Daily processed documents include database change tickets, slow SQL reports, backup records, and permission application forms. Data sources are mostly database monitoring, audit logs, SQL execution plans, and ticketing systems. Update frequency is daily or triggered by a single change. Document structures mix structured tables and free text, including fields such as instance identifiers, operation times, execution durations, and permission scopes. Units include milliseconds, count, GB, and similar. Some documents involve content related to sensitive permissions.
What constraints do these characteristics impose on document parsing and chunking
Since documents use a mixed format of structured tables and free text, parsing and chunking must support both table content extraction and context retention logic. This avoids splitting that loses complete associated information for a single slow SQL or single change, which reduces the efficiency of slow query troubleshooting and change review. Since updates occur daily or when a single change is triggered, the chunking process must support incremental processing based on file update time to reduce repeated parsing overhead. Since documents link to precise fields such as database instances and operation types, chunking must retain contextual association between adjacent document chunks. This ensures that retrieval can fully match the context of the corresponding operations scenario, avoiding broken links between permission application forms and their associated instances.
How to set the configurations
| Configuration Item | Recommended Value | Rationale |
|---|---|---|
chunkSize | 800–1200 characters | Adapts to the text length of database change tickets and slow SQL reports, retains complete context for a single operation or slow query |
chunkSplitter | Split by paragraph + table row | Matches the mixed structured and free text format of daily operations documents, prevents loss of table associations after splitting |
customPdfParse | Enabled | For uploaded PDF format change tickets and backup records, enable custom parsing rules to extract structured fields |
UPLOAD_FILE_MAX_SIZE | 1000 MB | Adapts to the file size of batch-imported backup records and audit logs, meets daily batch upload requirements for operations documents |
PARSE_FILE_TIMEOUT_SECONDS | 600 seconds | Matches the parsing time of large backup records, prevents parsing failure due to timeout |
chunkSettingMode | Manual configuration | Adjust chunking parameters for different types of operations documents such as slow SQL reports and permission application forms, to adapt to specific scenario needs |
Three common mistakes
- Phenomenon: Uploaded audio and video files fail to parse, returning an unsupported file type prompt. Cause: The corresponding media parsing plugin is not configured, and the default parsing rules only support text documents.
- Phenomenon: Uploaded PDF documents only extract plain text after parsing, losing table data. Cause: The
customPdfParseconfiguration is not enabled, and the default parsing logic does not adapt to the structured table format in operations documents. - Phenomenon: Actively pushed files do not appear in knowledge base search results. Cause: The chunking process is not completed, or the
chunkSizesetting is too large, causing a mismatch with the search recall threshold.
How to confirm the configuration is properly set
- Upload a PDF database change ticket, check if the parsed text includes core fields such as change time and instance ID, to verify that the
customPdfParseconfiguration is effective. - Upload a backup record file of around 100 MB, wait for parsing to complete and check the task status, to confirm that the
PARSE_FILE_TIMEOUT_SECONDSsetting is sufficient to cover parsing time. - Initiate a search for a slow SQL report, check if the returned document chunks include complete context information such as execution duration and scanned row count, to verify that the
chunkSizeandchunkSplitterconfigurations are reasonable. - Test pushing a permission application form through an external system, confirm that the corresponding content can be retrieved in the knowledge base after pushing, to verify that the chunking configuration for the active push process is effective.
The values provided are common starting points and should be measured against the reader's own samples.
Question material comes from public community discussions. Configuration values are common starting points and should be measured against your own samples. Verified on 2026-10-19.