Test Cases and Defect Analysis for Security Product Rule Operations: Tool Calling and Plugins

Data for this category primarily comes from WAF, IDS, firewalls, and threat intelligence platforms. Updates occur daily or when a protection event is

What the data for this category looks like

Data for this category primarily comes from WAF, IDS, firewalls, and threat intelligence platforms. Updates occur daily or when a protection event is triggered. Daily produced documents include rule change tickets, intercept logs, false positive analysis reports, and policy rollback records. Core fields include rule ID, intercept timestamp, business domain name, attack signature string, and false positive trigger frequency. Common units are timestamp, count, and times.

What constraints these characteristics impose on tool calling and plugin workflows

Heterogeneous multi-source data requires tool calling to support integration with protection device APIs from different vendors. This avoids the limitations of single-source calls. High-frequency updates require plugin configurations to include scheduled pull tasks. This ensures analysis samples use the latest intercept events. The rich set of document fields requires specifying filter parameters during tool calling. Only extract fields relevant to test cases to reduce invalid data loading. Consequences of errors such as false positives and missed protections require adding a pre-validation step to tool calling. This prevents incorrect rule triggers.

How to configure settings

This page applies to FastGPT v4.17.0 tool calling and plugin configurations.

Configuration ItemRecommended SettingRationale
toolChoiceautoFits rule testing scenarios for multiple types of protection devices, automatically selects matching tool calling logic
functionCallSet based on actual testingFits parsing of intercept log fields of varying lengths, avoids parsing failures caused by overly strict parameters
Single tool call timeout300 secondsMatches the time required for batch intercept log pulls, avoids timeouts caused by excessive data volume
Tool call return field filterOnly retain rule ID, intercept time, business domain nameReduces invalid data transmission, fits the core analysis needs of test cases
Maximum concurrent call countFirst 3Avoids triggering interface rate limits by pulling too much protection device data simultaneously

Three common mistakes

  • The program stalls after executing a tool call node, with no return results. The cause is missing interface authentication parameters for multi-source protection devices, leading to blocked or timed-out call requests.
  • No associated intercept log details are included in tool call return results. The cause is not enabling the raw data association switch in the plugin configuration, so only rule analysis content generated by the model is returned.
  • Tool call execution process information is continuously displayed in the chat window. The cause is not disabling the real-time log output configuration for tool calls, causing intermediate plugin operation steps to be synchronized to the conversation interface.

How to confirm the configuration is complete

  • Send a simulated rule test request, check if the tool call return results include the preset intercept log fields.
  • View the tool call execution logs, confirm successful integration with the protection device API, with no authentication, timeout, or format errors.
  • Disable the real-time output configuration for tool calls, send a test request, confirm the conversation window does not display intermediate steps of plugin execution.
  • Adjust the toolChoice parameter to none, verify that the tool calling logic is disabled, which fits non-tool calling test scenario requirements.

The values provided are common starting points and should be measured against the reader's own samples.

Question material comes from public community discussions. Configuration values are common starting points and should be measured against your own samples. Verified on 2026-10-19.