Test Cases and Defect Analysis for Code Security Auditing: Citation Sources and Traceability

Data for this category originates from source code repositories, dependency lists, static application security testing (SAST) tools and private

This document is compatible with FastGPT v4.17.0.

What Data for This Category Looks Like

Data for this category originates from source code repositories, dependency lists, static application security testing (SAST) tools and private vulnerability databases, as well as public vulnerability databases. Daily produced documents include code audit reports, vulnerability proof documents, repair recommendation documents and retest records. Data updates follow project version iterations or the completion of a single audit task. Each data entry corresponds to code snippets, vulnerability details and repair actions for a specific project. Fields include code paths, vulnerability severity levels, attack path descriptions, associated dependencies and more. There is no unified fixed unit, with adjustments made based on the dimensions of individual audit projects.

The scattered nature of data sources requires the traceability link to simultaneously associate source code, static scanning tool results and vulnerability database information, to avoid bias from single data sources. The update frequency tied to project iterations requires the traceability process to bind code snapshots of the corresponding version, ensuring cited content matches the current audit version. The diverse document types require the traceability system to adapt to citation sources in different formats such as reports and retest records, and uniformly extract key identifiers including vulnerability IDs and code paths. The multi-field structure requires precise matching of fields such as code paths and vulnerability severity levels during traceability, to avoid citation confusion across projects.

How to Configure Settings

Configuration ItemRecommended SettingRationale
quoteMaxToken800–1200 tokenMatches the average snippet length of code audit reports, to avoid truncating critical vulnerability descriptions
detailEnabledUsed to extract traceability fields from vulnerability proof documents and retest records, ensuring complete cited content
Single search limitTop 6 entriesMatches the typical number of vulnerabilities in a single audit task, covering core citation sources
Maximum contextCalibrated via actual testingAdapts to the stitching needs of multi-source data, to avoid exceeding model context limits
Citation source format validationEnabledFilters irrelevant content unrelated to audits, ensuring traceability only links code and vulnerability data

Three Common Configuration Errors

  • Setting quoteMaxToken too high prevents any content from being recalled and cited. This occurs because the model context capacity limit is exceeded, making citation sources unable to be effectively integrated into responses.
  • After enabling citation source format validation, only content from code audit reports is recognized in the knowledge base, and other vulnerability proof documents are not cited. This occurs because field mapping rules for corresponding document types are not configured, meaning key identifiers from non-report documents cannot be extracted.
  • Failing to bind project version identifiers in the traceability link causes code paths cited in responses to not match the currently deployed project version. This occurs because version snapshot binding is not enabled in the configuration, leading to a mismatch between traceability data and actual code.

How to Confirm Correct Configuration

  • Submit a test audit task, check whether the response includes key traceability fields such as code paths and vulnerability IDs, to confirm citation sources are correctly extracted.
  • Adjust quoteMaxToken to different ranges, verify that the length of cited content in responses changes with the configuration, with no truncation or overflow.
  • Call an external API to initiate a query, check whether the returned responseData field includes associated audit report links and code snippets, to confirm traceability information is returned completely.
  • Simulate a version iteration and re-run the query, confirm that code paths cited in responses match the content of the new version code repository, to verify version binding is effective.

The values provided here are common starting points, and should be evaluated against the reader's own samples.

Question material comes from public community discussions. Configuration values are common starting points and should be measured against your own samples. Verified on 2026-10-19.