What this category of data looks like
Data sources include internal service guide manuals, policy documents issued by state-owned assets supervision departments, historical question-and-answer archives from service windows, and annual operation public notification materials. Update cycles trigger on demand in line with policy adjustments. Internal service guides are reviewed and updated every quarter. Historical Q&A archives receive incremental weekly supplements. Most documents use a mixed structured and semi-structured format, with fields such as item code, processing time limit, required number of materials, and consultation response templates. Units include workdays, copies, person-times, and similar units.
What constraints these characteristics impose on document parsing and chunking
State-owned assets policy documents have fixed document numbers and structured service workflows. During chunking, retain the full integrity of core fields such as item code and processing time limit. Do not split content that spans key process nodes. Semi-structured historical Q&A archives contain contextually linked questions and responses. Retain the complete logic of each individual Q&A pair during chunking. Do not separate a question from its matching response. On-demand updates and incremental supplements require chunking tools to support incremental parsing. Only process newly added or modified document content. Multiple unit fields require accurate unit identifier recognition during parsing. Avoid mixing up processing time limit units across different items.
Configuration settings
Reference only; verify against the target FastGPT release and deployment documentation.
| Configuration Item | Recommended Setting | Rationale |
|---|---|---|
chunkSize | 800–1200 characters | Aligns with the length of individual items in semi-structured Q&A and policy documents, retains complete service logic |
chunkSplitter | By paragraph + semantic boundary | Avoids splitting consecutive steps in policy workflows, retains contextual links for Q&A pairs |
customPdfParse | Enabled | Accurately extracts structured content such as document numbers and tables from policy files, prevents format loss |
UPLOAD_FILE_MAX_SIZE | 1000 MB | Supports upload requirements for large annual operation reports and batch policy documents |
PARSE_FILE_TIMEOUT_SECONDS | 600 seconds | Meets parsing time requirements for large documents, prevents timeout interruptions |
chunkSettingMode | Manual configuration | Allows separate adjustment of chunking rules for policy documents and Q&A archives, adapts to multiple document types |
Three common configuration mistakes
- After uploading a policy link from a state-owned assets supervision department, the parsing result only returns the page title, with no specific service process content. Cause: Web structured parsing adaptation for
customPdfParseis not enabled, and parsing rules for dynamically loaded content on government service pages are not configured. - After storing custom-split service Q&A documents in the knowledge base, duplicate blocks are automatically deleted, leading to disordered order of pre-built custom indexes. Cause: The automatic deduplication switch for the knowledge base is not turned off, or the chunk content generated by the chunking rules highly matches existing blocks in the library.
- When parsing a single large document, the task terminates early with an error indicating file size limit exceeded. Cause: The value set for
UPLOAD_FILE_MAX_SIZEis lower than the actual size of the uploaded document, or the value set forPARSE_FILE_TIMEOUT_SECONDSis insufficient to complete parsing of the long document.
How to confirm correct configuration
- Upload a single policy PDF document, verify that extracted fields such as document number and processing time limit are complete, with no format loss.
- Upload a batch of historical Q&A documents, check that each chunk after splitting contains complete question and response content, with no separations between matching pairs.
- View the knowledge base upload logs, confirm that no timeout or file size limit exceeded errors appear in parsing tasks, and that task status is successful.
- Try uploading an external government service link, verify that the parsing result contains complete service guide content, and avoid only returning page metadata.
The configuration values listed here are common starting points. They should be evaluated against sample documents and operational requirements tailored to specific use cases.
Question material comes from public community discussions. Configuration values are common starting points and should be measured against your own samples. Verified on 2026-09-28.