This document is compatible with FastGPT v4.17.0.
What this category of data looks like
This category of data mainly comes from databases, data dictionaries, masking tools, and access approval systems. Updates align with project batch or cycle timelines. Commonly processed documents include masking plans, field mapping tables, masking verification reports, and approval records. Most document structures include sensitive field definitions, rule configuration items, verification results, approval process nodes, and similar content. Fields are mostly text or enumeration types, with no unified fixed units. Some fields are associated with unique identifiers of business systems.
What constraints do these characteristics impose on the document parsing and chunking workflow
Decentralized data sources create diverse document formats. Parsing logic must support structured and semi-structured documents exported from different systems. Updates align with project batch or cycle timelines, so chunking logic must flexibly adapt to scale differences of single-batch documents. Documents include sensitive content such as sensitive fields and rule configurations. Chunking processes must avoid sensitive information leaking across chunks. Most fields are text or enumeration types. Adjust chunk granularity separately for short fields and long rule texts, to prevent field splitting from breaking mapping relationships and disrupting subsequent matching and verification of masking rules.
How to set configurations
| Configuration Item | Recommended Value | Rationale |
|---|---|---|
chunkSize | 800–1200 characters | Adapts to the text density of documents such as masking plans and field mapping tables, avoids including too many sensitive fields in a single chunk to reduce leakage risk |
chunkSplitter | Separate by paragraph + heading | Matches the structured chapter structure of masking documents, retains contextual integrity of field mappings and rule descriptions |
customPdfParse | Enabled | Adapts to PDF format documents such as masking verification reports, retains original typesetting and field position information, and supports PDF files with digital signatures |
UPLOAD_FILE_MAX_SIZE | 1000 MB | Matches the overall upload scale of single project batch documents, prevents large files from being blocked during parsing |
PARSE_FILE_TIMEOUT_SECONDS | 600 seconds | Adapts to parsing time of complex masking documents, prevents parsing interruptions for long rule texts and multi-chapter documents |
chunkSettingMode | Manual configuration | Adjust chunking parameters for different types of masking documents (such as plan-type, report-type) to adapt to specific business scenarios |
Three common mistakes
- Uploading a PDF document with a digital signature results in failed content recognition. The default parsing logic cannot support encrypted PDF format with digital signatures, because the
customPdfParseconfiguration is not enabled. - The Doc2x tool returns a file read error. The
UPLOAD_FILE_MAX_SIZEconfiguration value is smaller than the actual uploaded file size, so the system blocks the file upload. - Sensitive fields are scattered across multiple chunks after document parsing. The
chunkSplitterconfiguration is set to separate by characters, so the document's chapter structure is not retained, breaking the contextual association of masking rules.
How to confirm the configuration is correct
- Upload a single batch of masking documents, check if the number of parsed chunks matches the number of document chapters. Adjust
chunkSizeandchunkSplitterconfigurations to fit the business scenario. - Upload a PDF masking report with a digital signature, verify that text content can be extracted normally, and confirm the
customPdfParseconfiguration status. - Submit a test file, check if the parsing time is within the expected range, adjust
PARSE_FILE_TIMEOUT_SECONDSconfiguration to fit the current document scale. - Check the size of the uploaded file, confirm it does not exceed the value set in
UPLOAD_FILE_MAX_SIZEto avoid upload being blocked by the system.
The values provided are common starting points and should be measured against the reader's own samples.
Question material comes from public community discussions. Configuration values are common starting points and should be measured against your own samples. Verified on 2026-10-19.