What the data for this category looks like
The data sources for this category are SIEM, EDR, NDR, and threat intelligence feeds, with update cycles of real-time or daily. Commonly processed documents include alert disposition sheets, duty logs, threat intelligence bulletins, and daily/weekly security reports. Alert disposition sheets contain fields such as alert ID, trigger time, associated assets, and disposition steps. Duty logs include duty periods, abnormal records, and handover items. Threat intelligence bulletins include threat type, impact scope, and disposition suggestions. Daily/weekly reports include daily/weekly alert statistics and disposition summaries. Most document structures are structured tables or segmented narratives, with no complex nested formatting.
What constraints these characteristics impose on the "document parsing and chunking" link
Real-time or daily updated data requires the document parsing process to have fast batch processing capabilities, to avoid delays impacting alert analysis and escalation responses. Multiple types of business documents require adaptation to differentiated structures, to prevent incorrect splitting of structured fields. Temporal disposition steps in alert disposition sheets must retain contextual associations, and chunking must not disrupt operational logic. Key fields such as impact scope and disposition suggestions in threat intelligence bulletins must be accurately extracted, to prevent information loss from chunking. Aggregated statistical data in daily/weekly reports must be fully retained, to avoid inability to restore overall conclusions after splitting.
How to set configurations
This configuration is based on FastGPT v4.17.0.
| Configuration Item | Recommended Setting | Rationale |
|---|---|---|
chunkSize | 800–1200 characters | Adapts to the field length of alert disposition sheets and duty logs, retains complete operational logic within a single chunk |
chunkSplitter | Split by semantic segments | Adapts to the narrative structure of threat intelligence bulletins and daily/weekly reports, avoids disrupting paragraph semantics |
customPdfParse | Enable structured parsing mode | Adapts to PDF alert sheets exported from SIEM, accurately extracts table and field information |
UPLOAD_FILE_MAX_SIZE | 1000 MB | Supports batch import of daily threat intelligence bulletin and duty log files |
PARSE_FILE_TIMEOUT_SECONDS | 600 seconds | Adapts to the parsing duration of large-volume daily/weekly reports, avoids timeout interruptions |
chunkSettingMode | Manual configuration | Adjust chunking parameters for different document types, meets parsing needs for multiple business document categories |
Three Common Errors
- Symptom: Some PDF files fail to be recognized, and core fields are empty after parsing. Cause: The structured parsing mode of
customPdfParseis not enabled, so table fields in PDF alert sheets exported from SIEM cannot be extracted. - Symptom: After uploading large-volume daily/weekly reports, the parsing task times out and fails. Cause: The value set for
PARSE_FILE_TIMEOUT_SECONDSis lower than the actual required parsing duration, causing the task to be forcibly terminated. - Symptom: Alert disposition sheets are split into overly short chunks, and the context of disposition steps is lost. Cause: The value set for
chunkSizeis too small, disrupting the temporalized disposition operation logic.
How to Verify Proper Configuration
- Upload a single alert disposition sheet PDF, check whether parsed chunks contain complete disposition steps and associated asset information.
- Upload a large-volume daily/weekly report, confirm that parsing task running duration is lower than the value set for
PARSE_FILE_TIMEOUT_SECONDS. - Batch import multiple types of business documents, confirm that parsing results for different document types retain core fields and semantic integrity.
- Check the file upload limit, confirm that imported file size does not exceed the value set for
UPLOAD_FILE_MAX_SIZE.
The values provided are common starting points and should be measured against the reader's own samples.
Question material comes from public community discussions. Configuration values are common starting points and should be measured against your own samples. Verified on 2026-10-19.