What this category of data looks like
This category of data primarily comes from product help centers, customer organizational structures, training recording videos, and ticket systems. The update frequency is per order or per version. Daily documents include training courseware, user manuals, training attendance sheets, and question and answer lists. Document structures vary widely, including text and image operation instructions, as well as structured attendance data and short-text question and answer content. The data is primarily produced by customer success managers, training instructors, and product operations, and is intended to meet the capability building needs of customers during their initial use.
What constraints do these characteristics impose on the "document parsing and chunking" link
The multi-source and multi-structure nature of documents in this category requires the parsing process to adapt to multiple formats including PDF, Word, plain text, and transcribed recording videos. The high-frequency update feature requires chunking rules to be flexibly adjusted to adapt to document changes after version iterations. Structured attendance sheets and short-text question and answer content coexist with long-text operation manuals, so the chunking strategy needs to balance the integrity of short text and the semantic coherence of long text. At the same time, since the data is used for customer initial training, the parsing results must retain core information such as role permissions and operation steps, and avoid breaking critical contextual associations through chunking.
How to configure settings
The following configurations apply to FastGPT v4.17.0.
| Configuration Item | Recommended Value Range | Rationale |
|---|---|---|
chunkSize | 800–1200 characters | Documents in this category mostly include operation steps and text and image instructions. This range balances semantic integrity and chunk granularity, avoiding excessive length that causes context loss or excessive shortness that breaks semantic associations |
chunkSplitter | Split by paragraph + title hierarchy | Training and usage manuals have clear chapter structures. Splitting by title preserves chapter context, while splitting by paragraph ensures individual chunks have independent semantic meaning |
PARSE_FILE_TIMEOUT_SECONDS | 300–600 seconds | This category includes large-volume training courseware collections and transcribed recording video documents. This duration covers the parsing needs of most complex documents |
customPdfParse | Enabled | Most documents in this category are formatted PDF user manuals. Enabling this option preserves layout and image association information, avoiding loss of critical text and image content after parsing |
UPLOAD_FILE_MAX_SIZE | 1000 MB | Single uploads of training material collections may include multiple courseware and recording files. This upper limit meets the upload needs of most scenarios |
chunkSettingMode | Manual configuration | Documents in this category have large structural differences. Manual configuration allows adjusting chunking rules for different document types to adapt to diverse data characteristics |
Three common mistakes
- Phenomenon: Parsing the Playwright MCP service fails, but Gaode MCP can be parsed and called normally, and the service can be called normally via the local VSCode plugin. Cause: No exclusive parsing rules or access permissions are configured for Playwright MCP, resulting in service adaptation failure.
- Phenomenon: When uploading a Word document with image links to the knowledge base, image-related information cannot be retrieved during question and answer testing. Cause: The
customPdfParseconfiguration is not enabled, or the image parser is not associated with the knowledge base, resulting in image links not being correctly identified and associated. - Phenomenon: When clicking to copy parsing results, the prompt "Unable to use browser automatic copy, please manually copy the content below" is displayed. Cause: Cross-origin restrictions exist in the deployment environment, and the source domain name allowed for copy operations is not configured, resulting in the failure of the browser automatic copy function.
How to confirm that the configuration is correct
- Upload a typical user manual PDF, and check whether the parsed chunked content retains the contextual association between chapter titles and corresponding operation steps.
- View the parsing task log records to confirm that no timeout errors related to
PARSE_FILE_TIMEOUT_SECONDSoccur. - Upload a training courseware with images, and check whether the knowledge base successfully identifies the image links and can retrieve and associate image information during question and answer testing.
- Upload a training material collection with a volume close to 1000 MB, and confirm that the
UPLOAD_FILE_MAX_SIZEupload limit is not triggered.
Note: The configuration values listed above are common starting points and should be evaluated against the reader's own samples.
Question material comes from public community discussions. Configuration values are common starting points and should be measured against your own samples. Verified on 2026-10-19.