Data Characteristics
Pharmacoeconomics data originates from clinical trial reports, real-world evidence (RWE) studies, health technology assessment (HTA) reports, national medical insurance catalog adjustment documents, and post-market drug surveillance reports. Document updates occur quarterly or annually, driven by policy changes, new drug approvals, and research advancements. Documents have complex structures, including numerous charts, tables, appendices, and references. The main content is often organized into chapters. Key fields include Quality-Adjusted Life Years (QALY), Incremental Cost-Effectiveness Ratio (ICER), disease burden, drug costs, treatment efficacy, and adverse event incidence rates. Units vary, including USD/QALY, years, percentages, and individuals. Currency and exchange rate conversions for different countries and regions are common.
Constraints on Document Parsing and Chunking
The complex structure of pharmacoeconomics documents, particularly the dense distribution of charts and tables, challenges traditional text parsing methods. This can lead to critical data loss or fragmented context. Diverse data sources require chunking to consider source differences, ensuring related data is effectively linked. The variety of fields and units demands parsers identify and retain the correspondence between values and units to avoid confusion. Update frequency necessitates knowledge base support for incremental updates and version management to ensure timely retrieval results. Furthermore, currency and exchange rate conversions across different countries and regions impose additional requirements for accurate numerical extraction and standardization. Chunking must preserve or associate relevant conversion logic.
Configuration Settings
| Configuration Item | Recommended Value | Rationale |
|---|---|---|
Chunk size | 500–800 characters | Balances contextual completeness with retrieval efficiency. Avoids excessive redundancy from overly long chunks and semantic incompleteness from overly short chunks. |
Chunk overlap | 50–100 characters | Ensures sufficient contextual overlap between chunks, improving RAG recall accuracy. |
ENABLE_TABLE_PARSING | true | Documents contain substantial critical tabular data. Enabling table parsing effectively extracts structured information. |
IMAGE_OCR_ENABLED | true | Many reports embed charts as images. OCR extracts text information from these images. |
PARSE_FILE_TIMEOUT_SECONDS | 600 seconds | Large reports require significant parsing time. Extending the timeout prevents parsing interruptions. |
CHUNK_STRATEGY | By chapter And Title | Pharmacoeconomics reports typically have a rigorous structure. Chunking by chapter helps maintain logical content integrity. |
Common Pitfalls
- Missing or misaligned table data in parsing results. Retrieval returns incomplete numerical values or values inconsistent with the original text. This occurs when table parsing is not enabled or the parser fails to correctly identify complex table structures.
- Inaccurate currency or unit conversion information in retrieval results. Returned values are not converted according to the exchange rates of the corresponding country or region. This happens when conversion rules or relevant reference data are not chunked as valid context during parsing.
- New data not reflected in retrieval results after knowledge base updates. User queries for the latest policies or research advancements return outdated information. This is due to a lack of incremental update mechanism configuration or unsynchronized update frequency with data sources.
Validation Steps
- Select a typical pharmacoeconomics report with complex tables and charts. Upload it and examine the parsed chunks. Confirm that critical data from tables and charts is accurately extracted and included in the chunks.
- For documents involving multi-country currency and unit conversions, perform RAG retrieval tests. Verify that the units and currency types of the numerical values in the returned results match the original text. Validate that relevant conversion logic is effectively recalled.
- Simulate a data update scenario. Upload a new version of a report or a revised document. Perform an incremental update. Then, retrieve answers to questions related to the updated content. Confirm the knowledge base reflects the latest information.
- Check parsing logs for a high volume of parsing failures or timeout errors. If present, adjust configurations or check file formats based on error messages.
The values provided are common starting points and should be measured against your own samples.
Question material comes from public community discussions. Configuration values are common starting points and should be measured against your own samples. Verified on 2026-09-21.