What the Data for This Category Looks Like
The data for this category primarily comes from scanning plans and vulnerability lists generated by vulnerability scanners, enterprise asset lists, public CVE databases, and patch announcements. Daily processed documents include standardized scanning configurations, vulnerability entries grouped by assets, progress records of rectification work orders, and result comparison reports from post-rescan checks. Data updates occur on a weekly or monthly cycle. Each document contains fields such as asset IP/domain name, vulnerability CVE number, harm level, and repair status. There are no complex nested formats, but the number of entries per document varies based on the enterprise’s asset scale.
Constraints Imposed by These Characteristics on Document Parsing and Chunking
Data sources include both native text documents and image-scanned reports. This requires support for non-native text PDF parsing. Weekly or monthly update cycles require chunked data to be easy to synchronize and update. Overly long chunks increase redundant processing costs during subsequent synchronization updates. Each document contains a large number of standardized vulnerability entries. Chunking must split by entry or paragraph to avoid cross-vulnerability context confusion. The asset range confirmation link relies on accurate field association. Chunking must retain the binding relationship between assets and corresponding vulnerabilities to prevent loss of key corresponding information after splitting. High-risk vulnerabilities carry significant error consequences. Parsing and chunking must be complete to avoid missing key entries.
Configuration Settings
| Configuration Item | Recommended Value | Rationale |
|---|---|---|
chunkSize | 800–1200 characters | Adapts to the length of single vulnerability descriptions and associated fields in vulnerability lists, avoiding loss of context after splitting |
chunkSplitter | Split by paragraph + fixed character truncation | Matches the structured paragraph format of vulnerability documents, while preventing ultra-long entries from being unable to be fully parsed |
customPdfParse | Enable image OCR mode | Adapts to rescan reports in scanned document format, extracting text content embedded in images |
PARSE_FILE_TIMEOUT_SECONDS | 700 seconds | Covers the batch parsing time of large asset lists, avoiding timeout errors |
UPLOAD_FILE_MAX_SIZE | 1000 MB | Adapts to batch-imported scanning plans and asset list documents |
chunkSettingMode | Manual configuration | Adjust chunking parameters for different document types (such as vulnerability lists, rectification work orders) to meet field association requirements |
This setup is compatible with FastGPT v4.17.0.
Three Common Misconfigurations
- After uploading a rescan report in scanned document format, the parsing result is empty or only extracts a small amount of text. The cause is that the image OCR mode of
customPdfParseis not enabled, and only native text content in the document is recognized. - When uploading a large asset list document, the platform returns a timeout error. The cause is that
PARSE_FILE_TIMEOUT_SECONDSis not adjusted to a duration matching the document scale, and the default timeout limit cannot cover the parsing process of batch assets. - When searching for vulnerability data under specified conditions, the number of returned entries is less than the actual number of entries in the source Excel file. The cause is that the association relationship between asset fields and vulnerability entries is not retained during chunking, and some entries cannot be fully indexed due to context truncation after splitting.
How to Verify Correct Configuration
- Upload a rescan report in scanned document format, and check whether the parsed text contains vulnerability entries and asset information from the image.
- Upload an asset list document exceeding 500 MB, and verify that the parsing process does not trigger a timeout error.
- Search for vulnerability entries of a specified harm level, and confirm that the number of returned results matches the number of entries in the source document.
- Adjust
chunkSizeto different ranges, and test whether the chunked text retains the complete associated information of a single vulnerability.
The values listed are common starting points and should be evaluated against relevant samples.
Question material comes from public community discussions. Configuration values are common starting points and should be measured against your own samples. Verified on 2026-10-19.