Reference only; verify against the target FastGPT release and deployment documentation.
What this category of data looks like
Data sources for this category include government portal websites, document management systems, government information disclosure directories, policy document repositories, and press conference materials. The data update frequency is daily. Daily documents cover policy documents, policy interpretation drafts, public disclosure directories, information release approval forms, press releases, and website column update records. Documents include fixed fields such as public attribution, document number, release date, and column affiliation. Some documents have structured elements such as tables, headers, and footers. Supported formats include common office formats such as Word, PDF, and Excel.
What constraints do these characteristics impose on the "document parsing and chunking" link
Daily batch document updates require the parsing process to support efficient batch processing, to avoid delaying publication schedules due to parsing delays. Diverse document formats and structures require chunking logic to adapt to different file types. For example, Excel public directories must be split by single row. Policy PDFs must retain chapter associations. Also, avoid merging chunks across public attribution or security review nodes, to prevent issues where content that should be public is not disclosed, or sensitive information is accidentally disclosed. Large Word files also impose higher requirements on parsing performance, so sufficient processing time and resources must be reserved.
Configuration Settings
| Configuration Item | Recommended Value | Rationale |
|---|---|---|
chunkSize | 800–1200 characters | Adapt to the standard paragraph length of policy documents, avoid unreasonable splitting across chapters or key fields |
chunkSplitter | Split by paragraph + title | Match the chapter structure of policy documents, retain context association, suitable for policy interpretation drafts, press releases and other documents |
UPLOAD_FILE_MAX_SIZE | 10–20 MB | Cover common large files of around 10 MB in this category, avoid import failure due to exceeding size limits |
PARSE_FILE_TIMEOUT_SECONDS | 600 seconds | Reserve sufficient time for parsing large files, prevent timeout interruptions during processing |
customPdfParse | Enabled | Adapt to structured elements such as tables, headers and footers in policy documents, improve the accuracy of key information extraction |
chunkSettingMode | Manual configuration | Adjust parameters for different document types, for example, set chunking rules for Excel public directories to split by single row |
Three Common Mistakes
- Large file parsing timeout, with parsing timeout prompt displayed in the interface | The
PARSE_FILE_TIMEOUT_SECONDSparameter was not adjusted. The default value is insufficient for processing policy files around 10 MB. - Multiple records in Excel format public directories are merged into one chunk after import | The
chunkSplitterwas not set to split by row. The default chunking logic does not adapt to the single-row entry requirement of Excel. - Specific chunks cannot be recalled, and normal recall works after creating a knowledge base with the same configuration |
customPdfParsewas not enabled during the first import, causing key document number and public attribution fields in policy documents to be missed. Normal functionality resumed after reconfiguring later.
How to Confirm the Configuration is Correct
- Upload a single 10–20 MB policy Word file, check whether the parsing progress completes within the preset
PARSE_FILE_TIMEOUT_SECONDSrange. - Import an Excel format public directory, verify whether each entry is displayed as an independent chunk.
- Parse a policy PDF file with tables, check whether table content is fully extracted and correctly assigned to corresponding chunks.
- Randomly sample chunked content, confirm that key fields such as public attribution and document number are not split across different chunks.
The values listed here are common starting points, and should be evaluated against your own sample data.
Question material comes from public community discussions. Configuration values are common starting points and should be measured against your own samples. Verified on 2026-09-28.