Data Characteristics
Rare disease regulation data comes primarily from official documents, guidelines, directories, and provincial implementation details. These are published by national health commissions, medical product administrations, and healthcare security administrations. Documents update infrequently, usually annually, but may adjust for specific events like new drug approvals or healthcare reimbursement inclusions. Document structures include laws, policy documents, technical guidelines, and expert consensuses. Most are PDF or Word formats, lengthy, and contain dense technical jargon. Fields often include disease name, drug name, indications, treatment plans, reimbursement ratios, approval processes, and designated hospitals. Units involve monetary amounts, percentages, timeframes, and drug dosages, showing high standardization and normalization.
Constraints on Knowledge Base Retrieval and Recall
The technical and lengthy nature of rare disease regulation documents requires semantic integrity during document chunking to prevent truncation of critical information. Low update frequency means initial knowledge base construction can focus on detailed processing, with manageable subsequent maintenance costs. Diverse official file formats demand robust document parsing to accurately extract text from various PDFs and Word files. Standardized fields provide a foundation for structured information extraction and precise matching. However, the system must handle conditional statements and exceptions within policy clauses. For example, different regions may have subtle variations in healthcare reimbursement policies for the same rare disease, requiring the retrieval system to identify and differentiate these regional regulations.
Configuration Settings
| Configuration Item | Recommended Value | Rationale |
|---|---|---|
Chunk size (Chunk Length) | 800–1200 characters | Rare disease policy documents often have long paragraphs with multiple conditions and explanations. Longer chunk lengths help maintain semantic integrity. |
Recall count (Recall Count) | 8–12 items | Policy clauses are highly interconnected. Increasing recall count helps cover more comprehensive related regulations, improving answer accuracy. |
Similarity threshold (Similarity Threshold) | 0.75–0.85 | Rare disease terminology is precise. A high threshold effectively filters out irrelevant fuzzy matches, improving retrieval accuracy. |
embeddingModel | text-embedding-ada-002 or higher version | Handling technical terms and complex sentences requires a high-performance embedding model to capture deep semantic meaning. |
maxContext | 30000 characters | Ensures sufficient recalled text can be accommodated when generating answers, especially when dealing with cross-references across multiple related policies. |
PARSE_FILE_TIMEOUT_SECONDS | 600 seconds | For large policy documents, extending the file parsing timeout prevents parsing failures due to excessive file size. |
Common Pitfalls
- Retrieval results contain many irrelevant general medical policies. This occurs because the knowledge base chunking strategy is too coarse and does not effectively distinguish rare disease-specific clauses.
- After a user query, there is a long delay or an empty result. This may be because
PARSE_FILE_TIMEOUT_SECONDSis set too short, causing large PDF files to fail parsing. - The knowledge base fails to correctly answer specific rare disease healthcare reimbursement details. The returned policy clauses do not match actual situations, or critical numbers are missing. This often happens because document parsing fails to accurately extract numerical fields from tables or complex sentences.
How to Verify Configuration
- Select 5-10 typical rare disease policy query questions. Verify if recall results include all relevant key policy clauses and numerical information. Record the recall count and match score.
- Upload a rare disease policy PDF file containing complex tables and multi-level headings. Check if the knowledge base can fully parse and generate high-quality chunks.
- For a specific rare disease, query its diagnosis and treatment guidelines, healthcare reimbursement policies, and drug directory separately. Verify the completeness and accuracy of retrieval results, especially for cross-references involving different source documents.
Note: The values provided are common starting points. Measure against specific samples.
Question material comes from public community discussions. Configuration values are common starting points and should be measured against your own samples. Verified on 2026-09-21.