What this category of data looks like
Data for this category originates from project document libraries, configuration lists, meeting minutes, and test reports. Updates occur at the end of each project phase or upon full project delivery. Commonly processed documents include completion documentation, configuration manuals, training records, and acceptance certificates. Documents typically include fields such as project number, version number, and signing date, with no unified fixed layout. Some documents contain multi-page charts, encrypted attachments, or scanned copies.
What constraints do these characteristics impose on document parsing and chunking
The characteristics of documents in this category create multiple constraints for parsing and chunking. First, document types cover completion documentation, configuration manuals, acceptance certificates, and more, with significant layout differences. Some include multi-page tables and signed scanned pages, so general parsing rules cannot accurately extract structured content. Second, data sources include multiple types of business documents, and multiple versions of documents may exist for a single project. Version association information must be retained during chunking to avoid confusion. Third, updates are triggered by project phases. Chunked content must be linked to the corresponding project phase identifier to enable subsequent retrieval by project.
How to configure this
This configuration is based on FastGPT v4.17.0. Specific configuration items and their rationales are listed below:
| Configuration Item | Recommended Setting | Rationale |
|---|---|---|
chunkSize | 800–1200 characters | Documents in this category mostly contain technical parameters and project node descriptions. 800–1200 characters can fully cover a single technical parameter or a single paragraph of project process description, avoiding information fragmentation within the chunk |
chunkSplitter | Split by paragraph + hierarchical title | Documents in this category include hierarchical project phases and chapter titles. Splitting by title preserves the logical structure of the document, facilitating subsequent retrieval by chapter |
customPdfParse | Enabled | PDF documents in this category mostly include scanned acceptance certificates and signed completion documents. Enabling custom parsing adapts to non-standard PDF layouts and improves OCR recognition accuracy |
PARSE_FILE_TIMEOUT_SECONDS | 300–600 seconds | Parsing large completion documents or multi-page scanned copies takes significant time. 300–600 seconds covers the parsing duration of most documents, avoiding mid-process timeout failures |
UPLOAD_FILE_MAX_SIZE | 1000 MB | Document packages handed over for a single project may include multiple large PDF or PPT files. 1000 MB meets the batch document requirements for a single upload |
chunkSettingMode | Manual configuration | Documents in this category have large structural differences. Manual configuration allows adjusting chunking rules for different document types to adapt to project-specific document formats |
Three Common Misconfigurations
- Issue: PPT and PDF documents cannot be recognized after Feishu knowledge base synchronization. Cause: The
customPdfParseconfiguration is not enabled, and no parsing adapter for the corresponding document format is configured, resulting in non-standard documents failing to be recognized by the system. - Issue: Logs show OCR error after deploying Marker. Cause: The resolution of scanned documents is lower than 300 dpi, or the
PARSE_FILE_TIMEOUT_SECONDSconfiguration value is too low, causing the OCR recognition process to interrupt mid-process. - Issue: An error is returned when parsing JSON via online search. Cause: The
chunkSizeconfiguration value is too large, causing chunked content to exceed the model context limit and triggering a format error during parsing.
How to Confirm the Configuration Is Correct
- Upload a single 100 MB scanned acceptance certificate, check whether key information such as signatures and signing dates is extracted after parsing, to confirm that the
customPdfParseconfiguration is effective. - Batch upload multiple project documents of different formats, check whether the upload progress is completed within the duration set by
PARSE_FILE_TIMEOUT_SECONDS, to confirm that the timeout configuration is reasonable. - View the chunked document list, confirm that each chunk contains complete paragraphs or chapter content, and is associated with the version number of the corresponding project, to confirm that the
chunkSizeandchunkSplitterconfigurations match the document structure. - Trigger Feishu knowledge base synchronization, check whether PPT and PDF documents appear normally in the knowledge base list, to confirm that the document format adaptation configuration is correct.
The values provided are common starting points and should be measured against the reader's own samples.
Question material comes from public community discussions. Configuration values are common starting points and should be measured against your own samples. Verified on 2026-10-19.