What This Category of Data Looks Like
Data for policy and regulatory basis Q&A for new energy development comes primarily from announcements, normative documents, and project application guidelines issued by energy regulatory authorities such as the National Energy Administration and local development and reform commissions. Update frequency aligns with policy release cycles, with no fixed schedule. New policies are archived and updated immediately upon release. Most documents are in PDF or Word format. They include fields such as document number, issuing authority, effective date, applicable project type, and subsidy standard. Some files include table attachments. They follow standard official document structural hierarchy, with divisions for chapters, sections, articles, and clauses.
Constraints Imposed on Document Parsing and Chunking
Official policy documents have varied formats. They include standardized tables, headers, footers, document number markers, and other redundant elements. These must be filtered accurately to retain core policy information. Documents have clear hierarchical structures. Clauses have strong contextual links. Chunking must avoid breaking clause integrity. Otherwise, policy basis cannot be matched during Q&A. Some policy documents include large attachments. Parsing must support large file uploads and extended parsing time. Policy fields include energy industry-specific units such as yuan per kilowatt-hour. Unit information must be retained during parsing. This prevents separation of units and numerical values during chunking.
Configuration Settings
| Configuration Item | Recommended Value | Rationale |
|---|---|---|
chunkSize | 800–1200 characters | Matches the clause length of new energy policy documents. Each chunk can hold 1 to 2 complete policy clauses, avoiding context breaks |
chunkSplitter | By title + paragraph | Adapts to the table of contents structure of policy documents. Splitting by title preserves clause hierarchy, preventing cross-title chunking |
customPdfParse | Enabled | Supports filtering redundant elements such as headers, footers, and document number markers in policy documents. Also enables OCR parsing of policy text within images |
UPLOAD_FILE_MAX_SIZE | 1000 MB | Adapts to multi-page large attachments included in some local policies, preventing large file uploads from being blocked |
PARSE_FILE_TIMEOUT_SECONDS | 600 seconds | Adapts to parsing time for large policy compilation files, preventing parsing interruptions due to timeout |
chunkSettingMode | Manual configuration | Allows adjustment of chunking rules for different policy document formats, adapting to diverse policy text structures |
Three Common Mistakes
- Phenomenon: After directly importing an Excel package into an existing knowledge base, Q&A matching results do not meet expectations. Cause: FastGPT's official knowledge base migration tool was not used. The exported Excel only contains raw text, and does not retain chunking configurations and vector indexes.
- Phenomenon: After uploading images or PDFs with embedded images, the parsed results do not extract policy text from the images. Cause: The OCR parsing option of
customPdfParsewas not enabled, or the uploaded image format does not meet system support requirements. - Phenomenon: The table dataset option cannot be selected when creating a knowledge base. Cause: The table extraction function of
customPdfParsewas not enabled, or table dataset parsing support is not enabled in the current version.
How to Verify Proper Configuration
- Upload a single typical new energy policy PDF. Check if the parsed text retains core fields such as document number and issuing authority, with no redundant header or footer content.
- Set
chunkSizeto 800–1200 characters. Check chunking results after parsing. Confirm each chunk contains 1 to 2 complete policy clauses, with no cross-title splitting. - Upload a policy attachment containing a subsidy standard table. Confirm the parsed result retains the table's row and column structure, numerical values and units, with no content loss.
- Use FastGPT's official knowledge base export and import tool to complete migration. Verify that the chunking configuration of the knowledge base on the new server is completely consistent with the original server.
The configurations on this page apply to FastGPT v4.17.0. The values listed here are common starting points. Adjust them based on your own sample data.
Question material comes from public community discussions. Configuration values are common starting points and should be measured against your own samples. Verified on 2026-09-28.