What data looks like for this category
Data for this category mainly comes from cloud audit logs, IAM permission systems, CMDB resource inventories, and security baseline scan results. Regularly produced core documents include compliance checklists, permission audit reports, resource inventories, and rectification tracking forms. Most document structures combine structured tables and sectioned descriptions. Fields include resource ID, permission scope, compliance status, rectification deadline, and similar items. There are no additional custom units. The update frequency is monthly or quarterly.
What constraints these characteristics impose on the document parsing and chunking link
Documents in this category mostly combine structured tables and explanatory text. When performing chunking, avoid splitting table cells or core compliance fields. Ensure individual chunks can be used independently for compliance verification. The monthly or quarterly update cycle causes fluctuations in document data volume. Parsing and chunking duration must adapt to documents of different sizes. Documents from multiple sources have varying formats. The system must support formats exported from different platforms. Ensure complete extraction of key fields such as resource ID and compliance status. Avoid field loss caused by format incompatibility.
How to set the configuration
These configurations are based on FastGPT v4.17.0.
| Configuration Item | Recommended Value | Rationale |
|---|---|---|
chunkSize | 800–1200 characters | This category of documents contains structured tables and text descriptions. This range balances the completeness of single-chunk content and retrieval granularity |
chunkSplitter | Split by paragraph + table block | Fits the structured format of compliance checklists and rectification tracking forms. Avoid splitting tables or paragraph logic |
customPdfParse | Enabled | Most documents in this category are cloud platform export reports. Enabling custom parsing preserves table and field association relationships |
PARSE_FILE_TIMEOUT_SECONDS | 300–600 seconds | Monthly updated resource inventory documents have large data volumes. This duration covers the complete parsing process |
UPLOAD_FILE_MAX_SIZE | 1000 MB | Quarterly resource inventory documents in this category have large file sizes. This upper limit covers standard document dimensions |
chunkSettingMode | Manual configuration | Adjust chunking rules based on the proportion of tables in compliance reports. Adapt to different document structures |
Three common mistakes
- Symptom: After custom chunking, upload a compliance checklist. The knowledge base automatically deletes duplicate document chunks. This causes the preset index order to not match the actual stored chunks. Cause: The
chunkSettingModeconfiguration is not set to manual chunking mode, or the system's default duplicate block filtering rule is not disabled. - Symptom: Large-volume quarterly resource inventory documents fail to parse. The interface displays a parsing timeout status. Cause: The
PARSE_FILE_TIMEOUT_SECONDSvalue is not adjusted to the duration range suitable for large documents. This causes the parsing process to terminate early. - Symptom: Table content in imported permission audit reports is split into scattered text chunks. The permission scope field cannot be read completely. Cause:
customPdfParseis not enabled, or a chunking rule suitable for structured tables is not selected. This causes the table structure to be damaged.
How to confirm the configuration is correct
- Upload a single compliance checklist. Review the chunking results. Confirm that table content is not split into multiple independent chunks.
- Upload a quarterly resource inventory document. Verify that the parsing process does not trigger a timeout error, and the final number of generated chunks matches expectations.
- Import multiple rectification tracking forms with similar structures. Confirm that the custom chunking rule does not automatically delete duplicate chunks, and the preset index order is retained.
- Call the document parsing API. Check whether the returned parsing status field covers the three standard statuses: parsing in progress, ready, and failed.
The values provided are common starting points and should be measured against the reader's own samples.
Question material comes from public community discussions. Configuration values are common starting points and should be measured against your own samples. Verified on 2026-10-19.