Data Characteristics
Rare disease regulation data comes from official documents. These include policies and regulations, clinical guidelines, and medical insurance catalogs published by national health commissions and drug administration agencies. It also includes internal Standard Operating Procedures (SOPs) from medical institutions. Documents are typically in PDF, Word, or Excel formats. Updates are infrequent, usually quarterly or annually, with ad-hoc revisions for major policy changes. Document structures are often chapter-based, containing extensive medical terminology, legal clauses, and procedural descriptions. Fields and units include disease codes (e.g., ICD-10), generic drug names, specifications, dosage units (mg, g, ml), cost units (Yuan), and time units (days, months, years). Some documents also contain complex tabular data.
Constraints on Deployment and Upgrades
Infrequent updates of rare disease regulation data mean less frequent knowledge base rebuilds or incremental updates. However, each update requires complete data integrity and version consistency. Documents are largely unstructured text, requiring robust text extraction and comprehension, especially for nested tables and figures. Specialized medical terminology and legal clauses demand high semantic understanding from text embedding models to avoid over-generalization. Complex fields and units, particularly in SOP question-answering, require the system to accurately identify and associate values across different units, such as drug dosage calculations or reimbursement ratios. Data involves policy compliance and clinical safety, so accuracy and traceability of question-answering results are critical. Post-deployment validation is essential.
Configuration Settings
| Configuration Item | Recommended Value | Rationale |
|---|---|---|
UPLOAD_FILE_MAX_SIZE | 200 MB | Policy documents and SOPs often contain many charts and large file sizes. |
PARSE_FILE_TIMEOUT_SECONDS | 600 seconds | Large PDF or Word documents take longer to parse; this avoids timeouts. |
Chunk size | 800–1200 characters | Ensures each text chunk has enough context for complex medical terms and legal clauses. |
Recall count | Top 8 entries | Increases recall to cover multiple relevant information sources like policy terms and treatment pathways. |
Similarity threshold | Calibrate by actual measurement | Ensures high relevance of recall results to specialized rare disease questions, avoiding over-generalization. |
Rerank result count | Top 3 entries | Selects the most relevant few items from a high recall set to improve question-answering precision. |
Common Pitfalls
- "Failed to create post presigned url" when uploading large PDF files indicates that
UPLOAD_FILE_MAX_SIZEis too low for the file upload. - "worker terminated due to reaching memory limit" during knowledge base creation indicates insufficient memory allocation when processing complex documents, especially PDFs with many tables.
- Incorrect numerical calculations for drug dosages or reimbursement ratios in question-answering results indicate an improper text segmentation strategy, where critical numerical or unit information was split, affecting model comprehension.
Verification Steps
- Upload a rare disease clinical guideline PDF containing complex tables and multiple chapters. Observe smooth file upload and parsing without timeouts or memory overflow errors.
- Ask questions about specific rare disease treatment processes or medical insurance reimbursement policies. Verify that answers accurately cite original passages and do not omit critical information.
- Test SOP detail questions involving different drug dosage unit conversions and cost calculations. Evaluate if the system correctly understands and provides logical answers.
- Check the parsing status of all documents in the knowledge base. Ensure the number of document blocks matches expectations, with no abnormal zero-block or excessively large blocks.
Note: The values provided are common starting points. Measure against your own samples for optimal performance.
Question material comes from public community discussions. Configuration values are common starting points and should be measured against your own samples. Verified on 2026-09-21.