What this type of data looks like
Data sources for this data type include SIEM, EDR, firewall logs, and host images. Updates occur in real time or are generated per individual incident. Daily output documents include incident response reports, forensics records, disposal plans, and notification materials. Data fields include attack source IP, disposal time, impact scope, evidence hash value, and others. Units include timestamps, byte counts, IP address ranges, and others. Each data entry is linked to a unique event ID, used for full-process traceability and compliance checks.
Constraints imposed by these characteristics on the database and operations link
Decentralized, real-time data sources require databases to support high-concurrency writes and low-latency queries, to avoid delaying incident response. Single-incident data volume varies with attack scale, so elastic storage capacity must be reserved to prevent storage overflow and evidence loss. Multiple document types need unified structured storage to support subsequent evidence chain organization and compliance traceability. The system must also support quick retrieval by event ID to accommodate time-consuming attack path restoration steps. Real-time data writes cannot be interrupted, so fault-tolerant mechanisms must be configured to prevent regulatory report delays or expanded data leaks caused by database failures.
How to configure settings
The following configurations apply to FastGPT v4.17.0.
| Configuration Item | Recommended Setting | Rationale |
|---|---|---|
| MongoDB connection address | Configure according to the actual deployed MongoDB instance address, add the retryWrites=true&w=majority parameter | Ensures retry mechanism and read consistency for real-time log writes, adapts to high-concurrency write demands from multi-source logs |
| PostgreSQL connection address | Configure according to the actual deployed PostgreSQL instance address, enable SSL encrypted transmission | Encrypts storage of incident response reports and forensics records, complies with security data regulatory requirements |
| Maximum database connections | 200–300 connections | Adapts to multi-source concurrent writes from SIEM, EDR, and firewall logs, prevents critical log loss caused by connection exhaustion |
| Redis connection address | Configure according to the actual deployed Redis cache instance address | Caches commonly used event IDs and temporary forensics metadata, reduces query pressure on core databases |
| Write timeout threshold | 60 seconds | Adapts to real-time data write timelines, prevents log write failures caused by network fluctuations |
| Log archiving cycle | 7 days | Balances storage capacity and evidence retention needs, facilitates subsequent traceability and compliance checks |
Three common misconfigurations
- Phenomenon: When concurrency increases, database writes or queries experience excessive latency, returning 429 status codes or timeout errors. Cause: No reasonable maximum database connection count is configured, leading to connection exhaustion and failure to process multi-source log write requests in a timely manner.
- Phenomenon: Database workflows return empty results after executing SQL queries, making forensics data unavailable. Cause: Database connection access permissions are not configured correctly, or SQL statements do not match field indexes associated with events, leading to failed query hits.
- Phenomenon: An error occurs when uploading PPT-format forensics materials to the storage link associated with the database. Cause: No file parsing and storage rules supporting PPT format are configured, and only general document format upload permissions are enabled.
How to verify configurations are correct
- Access the database monitoring dashboard, confirm that the current connection count does not exceed the configured maximum database connection threshold. This can be validated by simulating multi-source log write requests.
- Run an SQL query linked to a preset event ID, confirm that the returned results match the simulated forensics data, verifying that the query rule configuration is active.
- Upload PPT-format forensics materials, confirm that the upload and storage process has no errors, verifying that the format support rule configuration is correct.
- Review the generation logs for incident response reports, confirm that data write and read latency meets the preset timeline requirements.
The values provided are common starting points and should be measured against the reader's own samples.
Question material comes from public community discussions. Configuration values are common starting points and should be measured against your own samples. Verified on 2026-10-19.