Data Characteristics
Rare disease clinical trial pre-screening data comes from diverse sources. These include global clinical trial registries (e.g., ClinicalTrials.gov, EU Clinical Trials Register), rare disease databases (e.g., Orphanet, OMIM), medical literature (PubMed, Embase), and patient registries. Update frequencies vary. Clinical trial registration information might update weekly or monthly. Rare disease gene and phenotype data are relatively stable, but new discoveries trigger irregular updates. Document structures are complex. They cover structured trial protocols, inclusion/exclusion criteria, disease classification codes (ICD-10, Orphanet code), and unstructured medical reports and patient histories. Field specificity is high. Examples include gene mutation sites, specific biomarker concentrations, and rare disease-specific symptom scores. Units involve genomic coordinates, protein expression levels, and disease severity scale scores.
Constraints from "Workflow Orchestration"
Rare disease data sources are dispersed with uneven update frequencies. This requires the workflow's data collection module to support multi-source fetching and incremental updates, and to handle heterogeneous data formats. Structured and unstructured data coexist. This means the workflow needs to integrate Information Extraction (IE) and Natural Language Processing (NLP) modules. These modules identify and standardize key entities like genes, symptoms, and drugs from text. The presence of specific fields and units demands higher requirements for data cleaning and validation within the workflow. This requires customized validation for rare disease-specific terminology and numerical ranges to prevent data distortion. Furthermore, rapid advancements in rare disease knowledge mean the workflow's knowledge base retrieval and reasoning modules must support dynamic knowledge graph construction and continuous learning. This adapts to new genes and therapies, ensuring the accuracy and timeliness of pre-screening results.
Configuration Guidelines
| Configuration Item | Recommended Value | Rationale for Recommendation |
|---|---|---|
maxContext | 3000 Tokens | Rare disease clinical trial protocols and medical reports are lengthy, requiring a larger context window for complete understanding. |
Recall count | 10–15 entries | The number of rare disease-related literature and trials is relatively small. Increasing the recall count improves coverage of highly relevant documents. |
Similarity threshold | 0.75–0.85 | Rare disease phenotypes and gene descriptions may have subtle differences. A higher threshold helps precise matching and filters out noise. |
Chunk size | 800–1200 characters | Medical literature paragraphs have high information density. This length helps retain complete semantics and reduces information fragmentation. |
PARSE_FILE_TIMEOUT_SECONDS | 300 seconds | Rare disease trial documents often contain many charts and complex structures. Parsing takes longer, requiring an extended timeout. |
Rerank result count | 5 entries | After retrieval and re-ranking, the final output to the user should be a small number of key pieces of information that are most relevant and highly matched. |
Common Pitfalls
- Workflow execution times out or returns empty results. Logs show
TaskTimeoutErrororNoMatchingDocuments. This happens when the complexity and specificity of rare disease literature are not fully considered. File parsing and knowledge base retrieval take too long, or retrieval parameters are set too strictly. - The AI conversation fails to cite knowledge base content, instead generating generic answers. This occurs when rare disease-specific terms and acronyms are not correctly identified and embedded during knowledge base indexing, preventing retrieval matching.
- Pre-screening results include a large number of irrelevant or low-relevance clinical trials. This is due to a similarity threshold set too low, failing to effectively filter out trials that do not match the target rare disease or specific inclusion/exclusion criteria.
Validation Steps
- Select multiple known clinical trial datasets. Simulate user queries. Check if the workflow accurately retrieves and cites corresponding trial information, inclusion/exclusion criteria, and disease characteristics.
- Select a batch of medical reports containing rare disease-specific terms and genetic information. Upload them to the system. Verify if the information extraction and entity recognition modules correctly extract key fields.
- Construct diverse pre-screening queries for different rare disease types and patient characteristics. Cross-reference the listed clinical trials in the output with expected results. Compare against manual assessment to determine a reasonable range for the similarity threshold.
Note: The values provided are common starting points. Measure them against specific samples.
Question material comes from public community discussions. Configuration values are common starting points and should be measured against your own samples. Verified on 2026-09-21.