What the data for this category looks like
This category of data primarily originates from target systems, public vulnerability databases, automated scanner results, and manual verification records. Daily produced documents fall into four categories: test authorization letters, penetration test reports, vulnerability reproduction records, and rectification suggestions. The length of individual documents varies widely. Data updates follow individual project or major version iteration cycles, with no fixed global update rhythm. Fields for individual data entries include authorization scope, vulnerability ID, reproduction steps, risk level, and others, with no unified mandatory field format requirements.
What constraints do these characteristics impose on the knowledge base retrieval and recall link
The data characteristics of this category create multiple constraints for the retrieval and recall process. Multi-source heterogeneous and inconsistently formatted data sources require the retrieval system to support flexible parsing and adaptation logic, compatible with content in different formats such as scanner outputs and manual records. The wide variation in individual document length means long reports may exceed the context carrying limit of basic retrieval, requiring targeted chunking and truncation strategies. No fixed global update cycle requires the retrieval system to support incremental updates to avoid repeated processing of historical project data. Test authorization letters involve compliance boundaries, so recall results must strictly match the authorization scope to prevent information beyond permitted bounds from being returned. Vulnerability reproduction records include attack path details, so the sorting logic must prioritize matching the verification needs of the current test scenario.
How to configure the settings
These configurations are formulated based on FastGPT v4.17.0.
| Configuration Item | Recommended Value Range | Rationale |
|---|---|---|
similarity | 0.72–0.85 | Matches the semantic similarity requirements for vulnerability descriptions and reproduction steps in penetration testing scenarios. Prevents excessive irrelevant results from low values, and avoids missing relevant vulnerability records from high values. |
limit | Top 8–12 entries | Adapts the number of single-round retrieval results to the manual review efficiency of red team engineers, avoiding excessive results that increase review burden. |
searchMode | vector | Prioritizes vector retrieval for semantic matching of semi-structured data such as vulnerability reproduction records and scan results. |
usingReRank | Enabled, re-rank top 5 entries | Filters irrelevant chunks from recall results of long document chunks via a re-ranking model, improving the recall accuracy of core vulnerability information. |
datasetSearchUsingExtensionQuery | Enabled | Automatically expands synonyms related to testing, adapting to the professional terminology habits of penetration testing scenarios. |
Three common mistakes
- Phenomenon: After deploying the re-ranking model, the interface displays successful deployment, but the
usingReRankfield in retrieval interface call results remains always false. Cause: TheusingReRank=trueparameter was not explicitly passed in the retrieval interface request parameters, or the binding relationship between the knowledge base and the re-ranking model was not configured. - Phenomenon: After uploading a penetration test report, it is stored in chunks, but the actual chunk size deviates significantly from the preset value. Cause: No reasonable chunking boundary was set for long paragraphs of vulnerability reproduction steps, causing the system's automatic splitting to deviate from the preset parameters.
- Phenomenon: When the total length of recalled chunks exceeds the system's default threshold, the re-ranking model is not triggered. Cause: The re-ranking trigger logic after long context truncation was not configured. The system defaults to triggering re-ranking only when the number of recall results is below the threshold, which does not adapt to the scenario of long document chunks.
How to confirm the configuration is properly set
- Call the retrieval interface, pass penetration testing-related keywords, and check if the similarity parameters of the returned results match the preset configuration.
- Explicitly pass the
usingReRank=trueparameter to initiate a retrieval, and confirm that the sorting of the returned results differs from the original vector recall results. - Upload a single long document and initiate a retrieval, confirming that the system triggers the re-ranking model and returns sorted results.
- Create a knowledge base via the interface and pass associated parameters, then call the chat interface to verify that the retrieval content of this knowledge base can be accessed normally.
The values provided are common starting points and should be evaluated against samples relevant to the specific deployment.
Question material comes from public community discussions. Configuration values are common starting points and should be measured against your own samples. Verified on 2026-10-19.