Knowledge Base Retrieval and Recall for Real-World Evidence

Data Characteristics

Real-World Evidence (RWE) quality documents originate from research protocols, data management plans, statistical analysis plans, ethics approval documents, study reports, and related Standard Operating Procedures (SOPs). These documents exist as PDFs, Word files, or structured data tables. Core research protocols and ethics documents are relatively stable. Data management plans and statistical analysis plans are revised quarterly or semi-annually as research progresses and data accumulates. RWE quality documents often contain extensive descriptive text, charts, statistical results, methodological details, and specific terminology and units, such as disease classification codes (ICD-10), drug dosage units (mg/kg), and follow-up periods (months/years). Documents have complex cross-references and dependencies.

Constraints on Knowledge Base Retrieval and Recall

The complexity and diversity of RWE quality documents pose multiple challenges for knowledge base retrieval and recall. First, long texts and charts require robust text parsing and semantic understanding capabilities to extract key information from unstructured data. Second, strong inter-document relationships mean a single document's retrieval results may not fully answer a question, necessitating multi-document associated recall. Third, accurate identification of specialized terminology and units is crucial for retrieval quality; incorrect unit identification can lead to severe misunderstandings. Frequent revisions and updates demand incremental update and version management capabilities to avoid recalling outdated information. Finally, due to varied data sources and formats, a unified cleaning and standardization process is essential to improve retrieval accuracy. Without it, data fragmentation and information silos can occur, affecting the completeness and accuracy of recall.

Configuration Settings

Configuration Item	Recommended Value	Rationale
`Chunk size` (Segment Length)	`800–1200 characters`	Balances semantic completeness of long texts with recall efficiency, preventing excessive fragmentation.
`Recall count` (Recall Count)	`Top 5 entries`	Covers core documents while avoiding interference from irrelevant information.
`Similarity threshold` (Similarity Threshold)	`0.75–0.85`	Ensures high relevance between recall results and the query, filtering out low-quality matches.
`Rerank result count` (Rerank Return Count)	`Top 3 entries`	Focuses on the most relevant few documents, improving the precision of the final answer.
`PARSE_FILE_TIMEOUT_SECONDS`	`600 seconds`	Accommodates the parsing time for large PDF or Word documents, preventing timeout failures.
`maxContext`	`4000 characters`	Ensures the large language model receives sufficient context to understand complex queries.

Common Pitfalls

Semantic retrieval scores are abnormally high or low (e.g., 4000 or 0), leading to unstable recall quality. This usually results from an inappropriate vector model selection or incorrect text segmentation strategy during document preprocessing, which affects vectorization quality.
API calls do not return a list of specific files referenced by the knowledge base. This typically occurs because the corresponding parameters were not enabled or configured in the API request, leading to missing file reference metadata in the returned data.
Some queries return no results, resulting in an empty search. This can be due to a Similarity threshold (Similarity Threshold) set too high, a lack of relevant content in the index, or defects in the search service (e.g., searchXNG) configuration that prevent correct query transmission.

Verification Steps

Execute simulated queries for core business questions. Check if recall results include expected documents and evaluate the coverage of key information in the recalled documents.
Inspect the document status in the knowledge base management interface. Confirm all RWE quality documents are successfully parsed and indexed, with no parsing failures or timeouts recorded.
Test via API interface. Verify if recall results include file_id and other file reference information. Check if the returned Recall count (Recall Count) and Rerank result count (Rerank Return Count) align with the configuration.
Regularly track knowledge base update logs. Confirm new RWE document revisions are promptly indexed and included in searches. Check if older document versions are managed as expected.

The values provided are common starting points. Measure against your own samples.

Question material comes from public community discussions. Configuration values are common starting points and should be measured against your own samples. Verified on 2026-09-21.