This document is compatible with FastGPT v4.17.0.
What data looks like for this category
Typically, daily documents for this category fall into four types: personnel access rules, training materials, performance sheets, and settlement statements. Data sources include crowdsourcing platforms, annotation quality data, training exam records, and financial systems. Update frequencies are mostly daily or monthly. Document structures usually include hierarchical headings, structured tables, and scattered text. Fields include personnel ID, annotation volume, pass rate, settlement amount, batch number, etc. There is no unified fixed format, and it is adjusted according to project requirements.
Constraints on document parsing and chunking
The differences between the four document types impose multiple constraints on parsing and chunking. Personnel access rules and training materials often contain images and hierarchical headings. The logical structure of the documents must be retained. Performance sheets and settlement statements contain structured fields. Associated data across cells must not be broken during chunking. Data sources are scattered and update frequencies are flexible. Batch import and custom link parsing must be supported. Additionally, some documents contain complex tables and multi-page PDFs. Enhanced parsing capabilities must be adapted to avoid issues such as missing table recognition and disconnected text.
How to set configurations
| Configuration Item | Recommended Setting | Rationale |
|---|---|---|
chunkSize | 800–1200 characters | This category of documents contains associated structured fields. Excessively long chunks will destroy field associations, while excessively short chunks will lose contextual logic |
chunkSplitter | Split by paragraph + heading | Daily documents have hierarchical headings and independent paragraphs. Following this splitting rule preserves the business logical structure of the documents |
customPdfParse | Enabled | This category of documents often contains training material screenshots and tables embedded in performance sheets. Enabling this enhances image and table recognition effects |
UPLOAD_FILE_MAX_SIZE | 500 MB | Historical performance sheets and settlement statements imported in batches have large individual file sizes, so batch upload requirements must be accommodated |
PARSE_FILE_TIMEOUT_SECONDS | 600 seconds | Large multi-page PDF training materials take a long time to parse, so parsing tasks are prevented from being interrupted by timeout |
chunkSettingMode | Manual configuration | This category of documents has significant format differences. Manual configuration allows flexible adjustment of chunking rules to adapt to different business documents |
Three common configuration errors
- After configuring custom URL parsing, no data is returned. This occurs when the file format of the custom link is not confirmed to be supported by FastGPT, or when the link does not have public access permissions enabled.
- After importing a Word document containing a large number of tables, recognition performance is poor. This occurs when the
customPdfParseconfiguration is not enabled, so the enhanced table parsing function is not activated. - The model cannot read the uploaded file. This occurs when the
UPLOAD_FILE_MAX_SIZEconfiguration value is smaller than the uploaded file size, or whenPARSE_FILE_TIMEOUT_SECONDSis set too short, causing parsing to time out before completion.
How to verify correct configuration
- Upload a single personnel access rules PDF, and check if parsed chunk results are split by chapter, retaining the association between headings and corresponding paragraphs.
- Upload a Word performance sheet containing embedded tables, and verify that table content in parsed results is fully extracted, with no missing fields or disconnected text.
- Upload a settlement statement file of approximately 100 MB, confirm that the parsing task does not trigger a timeout interruption, and that the file size does not exceed the
UPLOAD_FILE_MAX_SIZEconfiguration value. - Test importing training materials via a custom link, confirm that extracted text and image content can be viewed normally after the parsing task completes.
The values provided are common starting points and should be measured against the reader's own samples.
Question material comes from public community discussions. Configuration values are common starting points and should be measured against your own samples. Verified on 2026-10-19.