Data Characteristics
Pharmacovigilance data for bispecific antibodies primarily comes from clinical trial reports, real-world evidence (RWE) studies, case reports, adverse drug reaction (ADR) reports, and post-market regulatory documents. These documents are typically in PDF, DOCX, or plain text format. Update frequency varies: clinical trial and post-market study reports may update annually or at key research milestones, while ADR reports can be continuous. Document structures are complex, often containing charts, tables, intricate medical terminology, and abbreviations. Common fields include patient demographics, medical history, adverse reaction event descriptions, severity, outcome, causality assessment, drug dosage, and administration route. Units involve dosage (e.g., mg/kg), time (e.g., days, weeks), and laboratory indicators (e.g., μg/L, mmol/L). Unit inconsistencies may exist across different document sources.
Constraints on Document Parsing and Chunking
The complexity of bispecific antibody pharmacovigilance data imposes specific requirements on document parsing and chunking. First, documents contain extensive medical terminology and abbreviations. The parser must correctly identify and process these to avoid chunking errors due to vocabulary misunderstandings. Second, clinical and ADR reports often feature nested tables and charts. Traditional text-based parsing struggles to extract structured information accurately, requiring intelligent table recognition and content association capabilities. Third, adverse reaction event descriptions are often scattered across different paragraphs or even pages. Chunking must semantically link these to ensure a complete context for a single adverse reaction event is included in one chunk. Additionally, inconsistent units across different source documents require standardization after parsing to ensure accuracy in subsequent retrieval and generation. These constraints dictate that the chunking strategy must balance semantic completeness with information density.
Configuration Settings
| Configuration Item | Recommended Value | Rationale |
|---|---|---|
chunk_overlap_ratio | 0.1 | Ensures a small contextual overlap between adjacent chunks to maintain semantic coherence while avoiding redundancy. |
chunk_size | 800–1200 characters | Balances the completeness of adverse reaction event descriptions with the information density of a single chunk, avoiding overly large or small chunks. |
parser_mode | SEMANTIC_SPLITTER | Prioritizes semantic splitting to better identify and preserve the context of adverse reaction events. |
table_parsing_enabled | true | Ensures accurate extraction of adverse reaction-related tabular data from clinical trial reports. |
image_ocr_enabled | true | Recognizes text information within images, such as charts or text presented as images in some reports. |
PARSE_FILE_TIMEOUT_SECONDS | 600 seconds | Provides sufficient time for parsing large clinical study reports or multi-page ADR summary reports. |
Common Pitfalls
- In context references, document content does not render as Markdown as expected. This often happens when the parser fails to correctly convert specific characters or structures, leading to original content loss or format errors.
- After uploading a document, the system displays "Parsing failed" or "Document content is empty." The
PARSE_FILE_TIMEOUT_SECONDSsetting may be too low, causing a timeout when parsing large or complex documents, preventing content extraction. - Knowledge base chunks appear disjointed or semantically interrupted. This may result from a
chunk_sizesetting that is too small, causing a complete adverse reaction event description to be split into multiple unrelated chunks.
Verification Steps
- Upload a bispecific antibody clinical report containing complex tables and multiple pages of text. Check if the parsed knowledge chunks completely include tabular data and its context.
- Parse multiple documents from different sources (e.g., clinical reports, ADR reports). Verify that the descriptions of adverse reaction events in the parsed results are semantically complete and without obvious truncation.
- Use the knowledge base preview function to confirm that the parsed knowledge chunks retain key medical terminology and abbreviations from the original document and are readable.
The values provided are common starting points and should be measured against the reader's own samples.
Question material comes from public community discussions. Configuration values are common starting points and should be measured against your own samples. Verified on 2026-09-21.