Test Cases and Defect Analysis for Security Vulnerability Disclosure Handling: Citation Sources and Traceability

This category’s data is generated across the full lifecycle of vulnerabilities in open-source projects. Daily documentation includes security

What Data for This Category Looks Like

This category’s data is generated across the full lifecycle of vulnerabilities in open-source projects. Daily documentation includes security advisories, vulnerability reports, fix patches, and CVE application materials. Data sources include private vulnerability report submission channels, project code repositories, official CVE numbering bodies, and official project release channels. Updates follow individual vulnerability event cycles. Each entry includes fields such as vulnerability ID, affected version range, patch submission time, and disclosure coordination progress. There is no fixed batch update cycle.

Constraints Imposed by These Characteristics on Citation Sources and Traceability

Because data is organized per individual vulnerability event and includes sensitive private vulnerability information, citation and traceability must strictly distinguish between public and non-public data sources. Private vulnerability reports may only be cited within the authorized scope of the project, with clear access permission markers retained. Patch commit records from code repositories must be linked to specific commit hashes, avoiding references to vague code snippets. Public data from CVE numbering bodies must be cross-checked against official release times to ensure the timeliness of traceability information. Additionally, the length of data related to a single vulnerability varies widely. Some patch documentation may exceed standard chunking limits, requiring adaptation of long-text citation rules.

How to Configure Settings

Configuration ItemRecommended ValueRationale
quoteMaxToken1200–1800 tokenAdapts to the text length of most vulnerability reports and patch descriptions, avoiding truncation of critical vulnerability impact details
Single search limitTop 3 resultsAssociated data sources for a single vulnerability event typically do not exceed 3 categories, preventing irrelevant information from interfering with traceability
responseDataReturn only public data source fieldsStrictly separates private and public data, only including disclosable information in citation scope
detailEnable full field displayRetains critical traceability identifiers such as vulnerability IDs and commit hashes to facilitate subsequent verification
Maximum context3000–4000 charactersSupports the stitching needs of multi-source data, ensuring complete context for the vulnerability event

Three Common Mistakes

  • Symptom: After setting quoteMaxToken to 1500 tokens, knowledge base chunks exceeding the length are still cited. Cause: The chunk length threshold was not adjusted synchronously. After long text is chunked, the individual chunk length still exceeds the quoteMaxToken limit, and the system automatically matches without enforcing the length restriction.
  • Symptom: Vulnerability report data is retrieved from the knowledge base, but only citation markers are returned without specific content. Cause: The configuration to return public data source fields for responseData was not enabled. Content from private vulnerability reports is filtered, leaving only traceability identifiers.
  • Symptom: An input error occurs when running code after referencing a plugin in a workflow. Cause: The traceability identifier parameter for the detail field was not passed as required, preventing the AI chat step from matching valid citation sources.

How to Confirm Successful Configuration

  • In the FastGPT v4.17.0 test environment, upload a standard vulnerability report document, and check whether the citation sources in the retrieval results include fields such as vulnerability IDs and commit hashes.
  • Set quoteMaxToken to 1200 tokens, upload a patch description document with a length of 1500 characters, and confirm that critical descriptions in the retrieval results are not truncated.
  • Enable the public data source filter for responseData, upload a private vulnerability report, and verify that full content is only visible in the authorized internal environment.
  • Run a workflow that includes citation traceability, and confirm that the citation markers returned by the AI chat correspond one-to-one with the actually retrieved data sources.

The values provided here are common starting points and should be measured against the reader's own samples.

Question material comes from public community discussions. Configuration values are common starting points and should be measured against your own samples. Verified on 2026-10-19.