Knowledge Base Retrieval and Recall for Infectious Disease

Data Characteristics in This Category

Infectious disease clinical trial pre-screening data primarily originates from global clinical trial registries (e.g., ClinicalTrials.gov, WHO ICTRP), academic journals, medical conference abstracts, disease control and prevention agency reports, and pharmaceutical company internal R&D documents. This data updates frequently; new outbreaks or drug development advancements can generate a large volume of new information within weeks. Document structures are diverse, including structured trial protocols, unstructured full-text research papers, semi-structured patient enrollment criteria descriptions, and laboratory test reports containing medical abbreviations and specialized units (e.g., CFU/mL, IU/mL, ng/dL). Patient inclusion and exclusion criteria are typically described in natural language, involving complex logical relationships and medical background knowledge.

Constraints Imposed by These Characteristics on Knowledge Base Retrieval and Recall

The high update frequency of infectious disease data requires the knowledge base to support rapid updates and incremental indexing to ensure timely retrieval results. Document diversity necessitates flexible chunking strategies, effectively handling various information formats from tabular data to long text descriptions. Complex logic and medical terminology in patient inclusion/exclusion criteria demand high precision in recall; simple keyword matching may not capture deeper meanings. Additionally, specialized units and abbreviations require preprocessing or semantic understanding at the model level to prevent recall failures due to literal mismatches. The broad range of data sources means the knowledge base must integrate multi-source heterogeneous information, performing deduplication and conflict resolution.

Configuration Settings

Configuration Item	Recommended Value	Rationale
`Chunk size` (Chunk Length)	`500–800 characters` (characters)	Balances contextual completeness and retrieval efficiency, avoiding noise from overly long chunks or loss of critical information from overly short ones.
`Chunk Overlap Rate` (Chunk Overlap)	`100 characters` (characters)	Ensures contextual continuity at chunk boundaries, reducing semantic fragmentation caused by chunking.
`Recall count` (Recall Count)	`Top 8–12 entries` (top 8–12 items)	Given the complexity of infectious disease inclusion/exclusion criteria, increasing the recall count covers more potentially relevant information.
`Similarity threshold` (Similarity Threshold)	`0.75–0.85`	Allows for slightly broader recall while maintaining relevance, retrieving documents that may contain synonyms or similar expressions.
`Rerank result count` (Reranked Return Count)	`Top 5 entries` (top 5 items)	After reranking model optimization, focuses on the most relevant items, reducing the large language model's processing burden.
`MaxTokens`	`2000–3000`	Reserves sufficient context window for the large language model to process complex patient inclusion/exclusion criteria and trial descriptions.

Three Common Mistakes

The number of retrieved results is significantly lower than expected, or critical information is missing. This usually occurs when the Similarity threshold (Similarity Threshold) is set too high, filtering out semantically similar but not perfectly matching documents.
The large language model states "no relevant information found" in its response, but manual inspection reveals the information exists in the knowledge base. This may be due to an insufficient MaxTokens configuration, resulting in an incomplete context sent to the large language model.
New data is not retrieved promptly after a knowledge base update. This indicates that the knowledge base indexing update strategy or frequency is inadequate for the high update pace of infectious disease data.

How to Confirm Proper Configuration

Select multiple representative infectious disease clinical trial pre-screening queries. Check if the recalled documents contain all key patient inclusion/exclusion criteria and trial details.
For query results, manually evaluate the Similarity score distribution of recalled documents. Adjust the Similarity threshold (Similarity Threshold) accordingly to ensure highly relevant documents are recalled.
Immediately after a knowledge base update, execute queries targeting the new data. Verify that new information can be accurately retrieved and assess the timeliness of retrieval results.
Monitor the large language model's response time for queries. Compare MaxTokens with the actual context length to confirm context transmission is not truncated.

The values provided are common starting points. Measure against your own samples.

Question material comes from public community discussions. Configuration values are common starting points and should be measured against your own samples. Verified on 2026-09-21.