This document adapts to the vector model and index configuration logic of FastGPT v4.17.0.
What the data for this category looks like
Documents covered under this category mainly include internal control self-assessment reports, audit notices, audit working papers, issue lists, rectification ledgers, and rectification closing certificates. Data sources cover enterprise internal control systems, ERP systems, financial sharing systems, audit management systems, and procurement contract project systems. Data update frequency is annual. Most document structures combine structured fields and unstructured paragraphs, including fields such as problem description, rectification requirements, supporting material numbers. Some documents come with exclusive identifiers such as voucher numbers and project numbers. There is no unified fixed unit standard.
What constraints do these characteristics impose on the vector model and index link
The data characteristics of this category bring multiple constraints to the vector model and index link. Decentralized data sources across systems require indexes to support unified parsing of multi-format documents, avoiding information loss of structured fields and unstructured paragraphs. The annual low-frequency update feature means that index construction mainly uses full updates. It is necessary to guarantee the stability and execution efficiency of full indexes. Documents contain strongly related content such as problem descriptions, rectification requirements, and supporting numbers. Vector models need to focus on aligning the semantic association between problems and rectification measures, while adapting to the vectorization effect of short text fields. In addition, differences in field names across systems need to have basic mapping completed in advance to ensure unified semantic dimensions during vectorization.
How to set the configurations
| Configuration Item | Recommended Value Range | Rationale |
|---|---|---|
| Chunk Length | 800–1200 characters | This category of documents contains long paragraphs of problem descriptions and rectification requirements. This range balances semantic completeness and index density |
| Overlap Length | 100–150 characters | Avoids semantic breakage across chunks, and adapts to associated content across paragraphs in rectification ledgers |
| Similarity Threshold | 0.72–0.78 | Needs to distinguish the similarity and difference of audit issues, avoiding misjudgment of similar issues or omission of associated rectification items |
| Single Recall Limit | Top 8 entries | Associated items in audit documents are mostly concentrated in a small number of similar issues and rectification records. Excessive recall will increase context redundancy |
defaultToken | 4096 | Adapts to the average length of documents in this category, ensuring content completeness of single-batch vectorization |
maxToken | 8192 | Supports contextual input for integrating multiple associated documents, such as audit working papers and rectification ledgers |
Three common mistakes
- Index construction takes too long. Cause: Failed to adjust chunk parameters for annual low-frequency update datasets, setting an excessively small chunk length leading to a sharp increase in the number of chunks, or not enabling batch index optimization.
- No optional models appear in the text understanding model dropdown menu when creating a knowledge base. Cause: Only embedding models and language models are configured, but the permission binding of the corresponding models is not completed in system settings, or the model deployment path is not correctly mapped to the FastGPT container.
- Empty fields appear in the vectorized dataset. Cause: Failed to complete cross-system field format verification and mapping in advance, and exclusive fields of some documents such as voucher numbers were not correctly parsed, leading to loss of corresponding content during vectorization.
How to confirm the configuration is complete
- Upload a single audit working paper, and check whether the chunked text fragments cover the complete problem description paragraph without obvious semantic truncation.
- Initiate an associated retrieval, verify whether the number and similarity of recall results fall within the preset threshold range, with no excessive redundancy or too few matching items.
- Check system logs to confirm that the execution duration of the full index task meets expectations, and no continuous timeout errors occur.
- Enter the model management page, confirm that all configured embedding models and language models are displayed as available, with no configuration errors.
The values provided are common starting points and should be measured against the reader's own samples.
Question material comes from public community discussions. Configuration values are common starting points and should be measured against your own samples. Verified on 2026-09-28.