What this category of data looks like
Data for this category comes from private vulnerability reports, code repositories, CVE numbering bodies, and official release channels. Update cadence follows one or multiple documents per single event, with no fixed cycle. Daily processed documents fall into four categories: security advisories, vulnerability reports, fix patches, and CVE application materials. Vulnerability reports include fields such as CVE numbers, affected versions, and reproduction steps. Fix patches include version numbers and change logs. CVE application materials require submission of project information and vulnerability details. The core fields of all documents use string or version number formats, with no unified multi-dimensional statistical attributes.
What constraints do these characteristics impose on the document parsing and chunking link
The four types of documents have large structural differences. Some documents contain sensitive private information. Parsing must avoid leaking undisclosed vulnerability details. The size of documents involved in a single event varies widely, ranging from security advisories of hundreds of characters to patch packages of dozens of megabytes. Chunk processing must adapt to documents of different sizes. Core fields such as CVE numbers and affected scopes must be fully retained within a single chunk, and must not be split across multiple chunks. Otherwise, subsequent vulnerability impact assessment will be affected. Additionally, chunks of private data must ensure only authorized personnel can access them. Parsing workflows must comply with security and compliance requirements.
How to set the configurations
| Configuration Item | Recommended Setting | Rationale |
|---|---|---|
chunkSize | 800–1200 characters | Adapts to the information density of vulnerability reports and security advisories, avoids splitting critical context containing CVE numbers and affected scopes |
chunkSplitter | Split by paragraph + heading | Preserves the integrity of paragraphs such as repair recommendations and reproduction steps, adapts to compliant documents with multiple structures |
customPdfParse | Enabled | Processes PDF documents for fix patches, retains metadata and embedded text content, adapts to security advisories containing images |
PARSE_FILE_TIMEOUT_SECONDS | 300 seconds | Adapts to the parsing duration of large CVE application materials, avoids timeout interruptions caused by large document sizes |
UPLOAD_FILE_MAX_SIZE | 50 MB | Matches the common maximum size of vulnerability reports and patch packages submitted in a single batch |
chunkSettingMode | Manual configuration | Adjusts chunking rules for different document types, meets differentiated processing needs for private documents and public advisories |
Three common mistakes
- After uploading a document, the copy function prompts "Unable to use browser automatic copy, please manually copy the content below". The cause is that parsed document chunk data is stored in a non-homologous storage service, and the frontend cannot directly call the browser's native copy API.
- Playwright MCP service cannot be parsed, but Amap MCP can be called normally, and local VSCode plugin verification passes. The cause is that the parsing configuration of Playwright MCP does not adapt to FastGPT's
customPdfParserules, or the service port does not open corresponding access permissions. - After uploading a Word document containing images to the knowledge base, question and answer testing cannot recognize image content. The cause is that only image link fields are extracted during chunking, and the image parser is not enabled to extract text descriptions or alt information from images. As a result, chunks do not include image-related content usable for question and answer.
How to confirm the configuration is correct
- Upload a typical vulnerability report document under FastGPT v4.17.0, check if the parsed chunk list fully retains core fields such as CVE numbers and affected versions, with no split of fields across multiple chunks.
- Submit a patch PDF document larger than 20 MB, confirm that the parsing process does not trigger timeout errors, and that chunk content is complete.
- After enabling the
customPdfParseconfiguration, upload a security advisory document containing embedded images, check if chunks include text extracted from images. - Test the PDF download function, confirm that the generated PDF file contains all core content of the original document, with no missing content or formatting errors.
The values provided are common starting points, and should be evaluated against the reader's own samples.
Question material comes from public community discussions. Configuration values are common starting points and should be measured against your own samples. Verified on 2026-10-19.