Data Characteristics in this Domain
Stability study data primarily originates from laboratory test reports, batch production records, quality standard documents, and relevant regulatory attachments. This data typically exists as structured tables, unstructured text reports, and scanned images. Structured data includes physicochemical indicators (e.g., content, dissolution, pH, moisture), microbial limits, and appearance traits at various time points (e.g., 0, 3, 6, 9, 12, 18, 24, 36, 48, 60 months). Unstructured text reports contain batch information, test conditions, analytical method descriptions, and result determinations. Data update frequency is usually low, aligning with product batch production and long-term stability observation plans, generally updated quarterly or annually. Field naming conventions vary across documents. Units include milligrams, grams, percentages, and pH values.
Constraints Imposed by these Characteristics on "Citation and Traceability"
The diverse sources of stability study data require the knowledge base to handle multiple file formats, especially structured tables and scanned documents, to ensure information completeness. A low data update frequency means that knowledge base index rebuilding or incremental updates do not need to be overly frequent; the focus is on accurate archiving and efficient retrieval of historical data. The presence of unstructured descriptions in document structures demands higher requirements for text segmentation and embedding models, necessitating effective identification of key information paragraphs. The complexity of fields and units requires accurate matching of values and units during citation to avoid misjudgments due to unit confusion. Since the data involves regulatory compliance, precise traceability of citation sources is crucial to ensure every judgment has clear supporting evidence.
Configuration Settings
| Configuration Item | Recommended Value | Rationale for this Value |
|---|---|---|
UPLOAD_FILE_MAX_SIZE | 200 MB | Stability study reports often contain numerous charts, graphs, and scanned images, leading to large file sizes. |
maxContext | 3000 Tokens | Ensures that key data tables and relevant descriptions can be accommodated, preventing information truncation. |
Chunk size | 800–1200 characters | Balances contextual completeness and retrieval efficiency, accommodating both tabular and descriptive text. |
Recall count | Top 8 entries | Increases the probability of recalling critical stability data from multiple relevant documents. |
Similarity threshold | 0.75 | Ensures high relevance of recall results to stability study queries, reducing noise. |
Rerank result count | Top 5 entries | Re-ranks recall results to prioritize the most critical stability data points. |
Three Common Pitfalls
- Output contains many irrelevant citations:
Similarity threshold(similarity threshold) is set too low, leading to irrelevant documents being recalled and cited. - Key numerical values are missing or units are incorrect: Document parsing fails to correctly identify numerical values and corresponding units in tables, or the model does not retain unit information when citing.
- Inability to locate specific data source pages or paragraphs: Knowledge base segmentation granularity is too large, or file types are not effectively parsed, leading to imprecise traceability information.
How to Confirm Proper Configuration
- For typical stability queries (e.g., "dissolution of a certain batch at 24 months"), check if the cited sources in the response include the correct batch number, test time point, and specific values.
- Upload stability reports containing complex tables and scanned documents to verify if the system can correctly parse and index key data fields and image descriptions.
- Test the number and relevance of cited sources for query results at different similarity thresholds and recall counts, and set acceptable thresholds based on actual business needs.
- Check if the hyperlinks or location information in the cited sources can accurately navigate to the specific page number or paragraph in the original document.
The values provided are common starting points and should be measured against your own samples.
Question material comes from public community discussions. Configuration values are common starting points and should be measured against your own samples. Verified on 2026-09-21.