What This Category of Data Looks Like
The data for this category primarily comes from target business systems, public vulnerability databases, output results from automated scanners, and manual verification records from red team engineers. Daily produced documents include test authorization letters, penetration test reports, vulnerability reproduction records, and rectification suggestions. Fields cover authorization scope, vulnerability IDs, reproduction steps, risk ratings, and more. Data updates follow individual project cycles, with synchronized updates when major vulnerability versions are released. The content of individual documents is gradually improved as testing progresses.
What Constraints Do These Characteristics Impose on Reranking and Hybrid Retrieval
Documents for penetration testing services have strong project correlation. Each piece of data only corresponds to a specific test target, and update frequency fluctuates with project progress. In customer technical support and pre-sales Q&A scenarios, users need to quickly locate authorization and vulnerability information for the corresponding project. Hybrid retrieval must not return irrelevant data across projects. The reranking link must adjust priority for long texts of vulnerability reproduction steps. It must also filter test documents without authorization scopes to prevent compliance risks. Additionally, data volume per project grows with testing depth. Retrieval result range must be limited to avoid context overload and ensure information accuracy.
Configuration Settings
| Configuration Item | Recommended Value | Rationale |
|---|---|---|
similarity | 0.65–0.75 | Filter low-relevance scanner raw results, retain vulnerability information directly related to customer questions |
limit | Top 8 entries | Adapt to the scale of data per project, avoid context overflow caused by excessive retrieval results |
usingReRank | Enabled | Adjust sorting priority for long-text vulnerability reproduction records, highlight reproducible high-risk vulnerabilities |
searchMode | Hybrid retrieval | Combine vector retrieval and keyword retrieval, balance keyword matching for authorization documents and semantic relevance for technical documents |
| Rerank model call timeout | 30 seconds | Adapt to the long text length of manual verification records from red team engineers, avoid retrieval failure due to model response timeout |
Three Common Misconfigurations
- Symptom: Reranking results show false or no sorted content. Cause: The specified reranking model is not bound in the knowledge base configuration, or the interface for the deployed reranking model has not been granted access permissions.
- Symptom: Retrieval results only include scanner raw data, with no vulnerability reproduction records. Cause: The
similaritythreshold is not set, causing low-relevance scan results to occupy retrieval result slots. - Symptom: Non-reranking language models cannot be called in the channel model. Cause: In the FastGPT commercial edition model configuration, the reranking model is mistakenly set as a mandatory enabled item, and the call entry for general language models is not retained.
How to Confirm Correct Configuration
- Enter the knowledge base retrieval test page of FastGPT v4.17.0, input a penetration testing-related question, and check if returned results include mixed content of vulnerability reproduction records and authorization documents.
- After enabling the
usingReRankfunction, manually trigger a retrieval, and check if the result list is reordered by risk rating or reproduction difficulty. - Check the deployment status of the reranking model, confirm that the sorting results returned by the interface are not empty, and differ from the order of the original retrieval results.
- View the model call logs in the FastGPT backend, confirm that the call values of parameters such as
similarityandlimitmatch the preset configuration.
The values provided here are common starting points and should be evaluated against the reader's own samples.
Question material comes from public community discussions. Configuration values are common starting points and should be measured against your own samples. Verified on 2026-10-19.