Document Parsing and Chunking for Antibody-Drug Conjugate

Data Characteristics of ADC Quality Documents

Antibody-Drug Conjugate (ADC) quality documents are highly structured and specialized. The data originates from various stages of drug research and development, manufacturing, quality control, and regulatory submission. This includes batch production records, analytical reports, stability study reports, quality standards, and validation protocols and reports. Document update frequency depends on the drug's development phase and production batches. For example, batch production records update with each product batch, and quality standards may revise after process changes. Document structures typically follow ICH Q7, Q8, Q9, and Q10 guidelines, featuring clear chapter divisions, figures, tables, and appendices. Key quality attributes include physicochemical properties, purity, content, impurities, conjugation rate, and DAR values for antibodies, linkers, and small molecule toxins. Units are precise, such as micrograms/milliliter, millimoles/liter, percentage, and OD values.

Constraints Imposed by These Characteristics on Document Parsing and Chunking

The highly structured and specialized nature of ADC quality documents presents specific requirements for document parsing and chunking. First, documents contain numerous tables and graphical data. The parser must accurately identify table boundaries and content, and extract key information from graphs. Second, specialized terminology and abbreviations are dense, such as "DAR," "HPLC," and "MS." Chunking must maintain the integrity of these terms to prevent semantic loss due to word breaks. Third, critical quality attribute data (e.g., purity, content) often appears as numerical values with units. Chunking must ensure the association between values and units. Finally, regulatory compliance requires extensive cross-references and version control information within documents. Parsing must preserve these associations for subsequent traceability and verification. Traditional text-based chunking methods may not effectively handle these complex structures and specialized content.

Configuration Guidelines

Configuration Item	Recommended Value	Rationale
`Chunk size` (Chunk Length)	`500–800 characters`	Balances the completeness of individual knowledge blocks with query efficiency, preventing dilution of key information by overly long texts.
`Chunk overlap` (Chunk Overlap)	`100–150 characters`	Ensures contextual continuity between chunks, especially when tables or critical descriptions span across pages.
`Parsing Mode`	`Advanced mode（Table Enhancement）` (Advanced Mode (Table Enhancement))	ADC quality documents contain extensive tabular data, requiring specialized table parsing capabilities.
`PARSE_FILE_TIMEOUT_SECONDS`	`600 seconds`	Provides sufficient parsing time for large PDF documents, such as batch production records, preventing timeout interruptions.
`chunk_strategy`	`By Title and Paragraph`	Quality documents typically have clear chapter titles; this strategy maintains the logical integrity of content.

Three Common Pitfalls

When uploading large PDF files, the system log displays File Parsing Timeout (file parsing timeout). Reason: The PARSE_FILE_TIMEOUT_SECONDS parameter is set too low, failing to accommodate the parsing duration for large batch production records or stability reports.
In search results, a critical quality attribute (e.g., "conjugation rate") for an ADC drug shows its numerical value separated from its unit, or table data is not effectively extracted. Reason: The document parser did not enable or correctly configure table enhancement mode, leading to structured data parsing failure.
When uploading files via API, specified PDF parsing parameters are not applied; the file is still processed using default methods. Reason: The file_parser or related parsing strategy parameters were not correctly passed or parameter names did not match in the API call, preventing the system from recognizing custom parsing instructions.

How to Verify Correct Configuration

Upload an ADC quality document containing complex tables and graphs. Review its preview in the knowledge base to confirm that table structures and key data are fully preserved.
Perform retrieval tests using core specialized terms and key quality attributes from the document. Verify that the retrieved knowledge snippets include complete term definitions or value-unit pairs.
Upload a typical ADC batch production record PDF. Observe the file upload and parsing process for smoothness. Check system logs for any parsing-related errors or warnings.
Edit an ADC document in the knowledge base. Examine its chunking to determine if it meets expectations, for instance, that critical paragraphs are not unreasonably truncated.

Note: The values provided are common starting points. Measure them against your own samples for optimal results.

Question material comes from public community discussions. Configuration values are common starting points and should be measured against your own samples. Verified on 2026-09-21.