What data in this category looks like
The primary users of data in this category are data governance teams, security consultants, and legal and compliance personnel. Daily used data assets include data catalogs, classification and grading rules, important data identification tables, and review minutes, sourced from data asset inventories, database metadata, business system descriptions, and regulatory documents. Data update frequency is quarterly or triggered when corresponding business systems change. Documents include three structural types: structured table lists, rule texts, and annotated review records. Fields include data asset ID, sensitivity level, owning department, compliance requirements, etc. There is no unified fixed unit; extraction must be adapted to the document type.
What constraints these characteristics impose on the document parsing and chunking link
The multi-source nature of data in this category requires the parsing module to adapt to multiple formats including structured tables, long rule texts, and annotated review minutes, to avoid losing metadata fields or details from regulatory texts. The quarterly or ad-hoc update frequency requires the chunking logic to be flexibly adjustable to accommodate newly added or revised rule documents. Multi-field tables in data catalogs and identification tables require accurate extraction of row and column associated content, to avoid breaking field correspondence after chunking. Precise chunking of compliance-related content can prevent important data identification rules from being split apart, reducing the risk of missed oversight.
How to configure settings
This document adapts to the knowledge base parsing function of FastGPT v4.17.0.
| Configuration Item | Recommended Approach | Rationale |
|---|---|---|
chunkSize | 800–1200 characters | Adapts to the length of classification and grading rule texts, avoids breaking rule logic after chunking |
chunkSplitter | Split by paragraph + title | Matches the title hierarchy structure of data catalogs and review minutes, preserves field association relationships |
customPdfParse | Enable OCR mode | Adapts to PDF documents containing scanned data asset inventories, fully extracts table and annotation content |
UPLOAD_FILE_MAX_SIZE | 1000 MB | Supports batch upload of full data asset inventory files updated quarterly |
PARSE_FILE_TIMEOUT_SECONDS | 600 seconds | Adapts to parsing time required for large regulatory documents and complex review minutes |
chunkSettingMode | Manual configuration | Allows adjustment of chunking parameters based on different document types, such as rule tables and review minutes |
Three common configuration errors
- Parsing results are empty after uploading images or PDFs containing images. The cause is that the OCR mode of
customPdfParseis not enabled, so embedded image content cannot be recognized. - Table datasets cannot be selected in the knowledge base after parsing. The cause is that the table extraction function of
chunkSplitteris not enabled, resulting in table content not being correctly identified as indexable entries. - Tasks time out when parsing large regulatory documents or review minutes. The cause is that
PARSE_FILE_TIMEOUT_SECONDSis not adjusted to a value adapted to the document size, causing the parsing process to be forcibly interrupted.
How to confirm the configuration is complete
- Upload a single test document containing structured tables, check whether the row and column associated content of the table is fully extracted in the parsing result.
- Upload a PDF document containing embedded images, confirm that the parsing result includes OCR-extracted text or annotation content corresponding to the images.
- Adjust the
chunkSizeparameter, view the chunking preview interface, confirm that classification and grading rule documents are not split into overly short segments. - Upload a review minute file that meets the conventional volume limit, confirm that the parsing task can complete normally without abnormal interruption.
The values provided are common starting points and should be measured against the reader's own samples.
Question material comes from public community discussions. Configuration values are common starting points and should be measured against your own samples. Verified on 2026-10-19.