Data Characteristics
Data for DTP pharmacies preparing registration and declaration documents primarily originates from pharmaceutical manufacturers' raw files, regulatory bodies' policies and regulations, and the pharmacy's own compliance records. Raw files include drug inserts, registration approvals, inspection reports, and clinical trial data, often in PDF, Word, or scanned image formats. Policies and regulations are mainly PDFs or government website pages, with frequent updates requiring close attention. Internal compliance records cover qualification certificates, personnel training records, and quality management system documents, typically stored in internal document management systems or spreadsheets. Data update cycles depend on drug launches, policy adjustments, and the pharmacy's internal management. Core documents like drug approvals are relatively stable, but regulations and supplementary application materials may update quarterly or annually. Document structures vary widely, including standardized tabular data, unstructured text descriptions, and charts. Fields and units vary; drug approvals involve generic names, brand names, dosages, specifications, and registration numbers, while inspection reports include content, purity, and expiry dates. Unit differences across drugs and testing methods require careful handling.
Constraints Imposed by Data Characteristics on Vector Models and Indexing
The heterogeneous data sources and diverse document formats in DTP pharmacy registration and declaration documents demand robust file parsing capabilities from vector models, especially accurate optical character recognition (OCR) for scanned PDFs. Frequent updates to regulatory documents necessitate efficient incremental update mechanisms for the index to ensure timely retrieval results. Drug inserts and clinical trial reports contain extensive medical terminology and complex medical expressions, requiring vector models with strong semantic understanding to prevent recall failures due to vocabulary mismatch. Furthermore, subtle differences may exist in descriptions of the same drug or regulatory clause across various documents; the index needs to identify and integrate this information. The mix of structured data and unstructured text in drug approvals requires vector models to handle both precise matching of structured fields and semantic retrieval of unstructured text. For specific fields and units, the vectorization process must preserve numerical information and support unit conversion or range queries during retrieval.
Configuration Guidelines
| Configuration Item | Suggested Value | Rationale |
|---|---|---|
UPLOAD_FILE_MAX_SIZE | 500 MB | Accommodates large drug inserts or inspection reports |
PARSE_FILE_TIMEOUT_SECONDS | 600 seconds | Ensures completion of text recognition and parsing for complex PDFs or scanned images |
Chunk size (Segment Length) | 800–1200 characters | Balances contextual completeness with vector model processing efficiency |
Recall count (Recall Count) | Top 10 entries (Top 10) | Covers more potentially relevant document segments, improving retrieval accuracy |
Similarity threshold (Similarity Threshold) | Calibrate empirically | Requires trade-off between precision and recall based on specific data and retrieval needs |
Rerank result count (Reranked Return Count) | Top 5 entries (Top 5) | Refines final results, reducing manual screening effort |
Common Pitfalls
- A knowledge base getting stuck during the final stage of index creation is often due to a file parsing timeout or the presence of unprocessable special characters.
- After a new Embedding model is successfully indexed, search tests failing may indicate incompatibility between the Embedding model and the FastGPT version, or the model service not responding correctly.
- The knowledge base failing to correctly vectorize text content within images, leading to missing retrieval results, occurs when OCR capability is not enabled or the OCR engine's recognition accuracy is insufficient.
Verification Steps
- Upload typical drug inserts and regulatory documents. Check the knowledge base document status to confirm all files show "Indexing Completed" (indexing completed).
- Perform searches using drug generic names, registration numbers, and regulatory clause numbers. Verify that the returned results include relevant approvals and original regulations, and check the number of returned results.
- Query for specialized medical terms such as adverse drug reactions and contraindications. Evaluate whether the recalled segments are precise and contextually complete, and verify the reasonableness of the
Similarity threshold(Similarity Threshold).
Note: The values provided are common starting points and should be measured against specific samples.
Question material comes from public community discussions. Configuration values are common starting points and should be measured against your own samples. Verified on 2026-09-21.