Document and Material Drafting for Public Data Catalogs and Open Data: Knowledge Base Retrieval and Recall

Document and Material Drafting for Public Data Catalogs and Open Data: Knowledge Base Retrieval and Recall

What data in this category looks like

This category of data originates from local public data open platforms, integrated big data platforms, departmental business databases, and electronic certificate repositories. The update frequency is monthly. Daily produced documents include public data resource catalogs, data open lists, data item descriptions, desensitization rules, interface documents, and update frequency descriptions. Document structures include both structured fields and unstructured main text. Fields cover data item names, open scope, desensitization methods, interface addresses, update cycles, authorization conditions, and other relevant details.

Monthly updated data sources require knowledge base synchronization frequencies to match update cycles, preventing retrieval results from including outdated data. Multi-source, multi-type document mixed scenarios require retrieval to cover both structured fields and unstructured main text, avoiding missed key information. The strong binding between data items and desensitization rules requires retrieval results to accurately associate authorization and desensitization requirements for corresponding data, preventing data leakage risks. Additionally, single document drafts require association with a limited number of content entries, so retrieval result counts must be controlled to avoid context overload that harms generation quality.

How to set configurations

These configurations apply to FastGPT v4.17.0.

Configuration ItemRecommended SettingRationale
similarity0.75–0.85Match the field rigor of public data catalogs, avoid low-relevance data items from being included in drafted documents
limitTop 10 entriesAdapt to the small number of data items that need association for a single document draft, control context length
searchModeFull-text retrieval + field retrievalCover document main text and structured fields, adapt to multi-type catalog content
usingReRankEnabledOptimize sorting accuracy for professional content such as interface documents and desensitization rules
datasetSearchUsingExtensionQueryEnabledAutomatically expand government synonyms related to data items, adapt to industry term diversity
collectionIdsFill in according to the bound public data catalog datasetLimit retrieval scope, avoid interference from irrelevant data in retrieval results

Three common errors

  • Phenomenon: Retrieval results return an "access denied to dataset" error, or return empty results. Cause: The dataset of the corresponding department is not bound to the current knowledge base, or the collectionIds parameter is configured incorrectly.
  • Phenomenon: Clicking a retrieval result fails to open the original file preview before chunking. Cause: The knowledge base's original file storage configuration is not enabled, or the source document and chunked content are not correctly associated.
  • Phenomenon: Uploading PDF-format interface documents fails to produce valid retrieval chunks. Cause: The pdf-marker service is not deployed correctly, or the access port of this service is not specified in the FastGPT configuration.

How to confirm the configuration is correct

  • Manually enter a common data item name, check whether retrieval results include relevant content such as data item descriptions and interface documents.
  • View the metadata of retrieval results, confirm that the correct departmental dataset is associated, to avoid mixing irrelevant cross-department data.
  • After enabling usingReRank, check whether the sorting of professional term content conforms to business logic, with highly relevant desensitization rules or interface descriptions displayed first.
  • Click the associated preview link of a retrieval result, confirm that the original document before chunking can be opened, to verify that the original file preview function operates properly.

The values provided are common starting points and should be measured against the reader's own samples.

Question material comes from public community discussions. Configuration values are common starting points and should be measured against your own samples. Verified on 2026-09-28.