This configuration applies to FastGPT v4.17.0.
What this category of data looks like
Data sources for this category include raw datasets, ETL logs, annotation platform outputs, and sampling results. The update frequency is daily or after each batch task is completed. Daily produced documents include cleaning rules, quality inspection sampling tables, abnormal data lists, and delivery reports. Most document fields include data identifiers, anomaly types, processing rules, sampling ratios, delivery status, and other items. There are no unified mandatory units, and some fields need to match the field definitions of the original dataset.
What constraints these characteristics impose on knowledge base retrieval and recall
The scale of the raw dataset and the timeliness of ETL logs require that retrieval must support filtering recall results by data batch or time range. Documents such as cleaning rules and abnormal lists have high update frequencies, so a scheduled refresh or real-time synchronization mechanism for retrieval indexes must be ensured. Most fields of quality inspection sampling tables and delivery reports are associated with original data identifiers, so retrieval needs to balance precise field matching and semantic association. The time-consuming links of duplicate data identification and abnormal value processing require that retrieved recall results must accurately cover frequently queried rules and anomaly types, to avoid invalid recall slowing down the overall process.
How to configure the settings
| Configuration Item | Recommended Value | Rationale |
|---|---|---|
similarity | 0.75–0.85 | Matches the semantic similarity of professional terms such as cleaning rules and abnormal data, to avoid recalling irrelevant sampling table content |
limit | Top 8–12 entries | Covers high-frequency query scenarios such as duplicate data identification and abnormal value processing, while controlling the context length of a single round of retrieval |
searchMode | Hybrid mode of semantic retrieval + full-text retrieval | Balances semantic matching of professional terms and precise matching of field identifiers, adapting to multi-type query needs within documents |
usingReRank | Enabled | Sorts recall results by relevance, prioritizing returning the most matching cleaning rules and abnormal handling solutions |
datasetSearchUsingExtensionQuery | Enabled | Optimizes semantic expansion of query terms, adapting to professional term abbreviations or industry jargon commonly used by data engineers |
collectionIds | Specify the corresponding knowledge base collection according to the business scenario | Isolates different batches of datasets and quality inspection documents, avoiding invalid recall across businesses |
Three common misconfigurations
- Specifying a query for an existing document in the workspace returns no results. The cause is that the target document has not been added to the
collectionIdscollection configured for the current setup, or thesimilaritythreshold is set too high, filtering matching results. - The semantic retrieval tool returns "Connection error". The cause is that the configuration for the connection between FastGPT and the vector database is incorrect, or the vector database service is not running properly.
- Knowledge base queries exceed the preset upper limit. The cause is that the knowledge base collection is not split according to business batches, resulting in the data volume stored in a single collection exceeding the system's default limit.
How to confirm the configuration is correct
- Upload a test cleaning rule document, enter the core rule keywords of the document in the workspace, and confirm that the retrieval results include this document.
- View the vector database connection logs, and confirm that there are no frequent connection timeouts or error logs.
- Adjust the
similaritythreshold, and verify that the relevance of the recall results changes as expected with the threshold. - Check the knowledge base collection configuration, and confirm that the document being queried has been correctly associated with the collection specified by
collectionIds.
The values given are common starting points and should be measured against the reader's own samples.
Question material comes from public community discussions. Configuration values are common starting points and should be measured against your own samples. Verified on 2026-10-19.