Data Characteristics in This Category
Data in the lead optimization phase primarily originates from high-throughput screening reports, compound synthesis records, in vitro activity test data, ADMET (Absorption, Distribution, Metabolism, Excretion, Toxicity) prediction reports, and crystal structure analysis reports. These documents typically exist as PDFs, Word documents, Excel files, or chemical structure files (e.g., SDF, MOL2). Data updates occur frequently, especially in compound synthesis and activity testing, where experimental results can be generated daily. Document structures are diverse; experimental reports usually include sections such as objectives, methods, results, and conclusions. The results section often contains tabular data, spectra, and chemical structures. Fields involved include compound ID, CAS number, molecular weight, activity values (e.g., IC50, Ki), ADMET parameters (e.g., LogP, TPSA), cell line names, and test batches. Units encompass nM, μM, mg/kg, log units, and others, and unit representation may lack uniformity.
Constraints Imposed by These Characteristics on "Reference Source and Traceability"
The diversity, high update frequency, and complex structure of lead optimization data impose specific requirements on reference sourcing and traceability. First, the large amount of tabular data and chemical structures in documents necessitates structured parsing capabilities. This ensures correct extraction and indexing of critical information, preventing information loss that would occur if only plain text were extracted. Second, high update frequency means the knowledge base content requires frequent synchronization or incremental updates to guarantee the timeliness of references. The diversity of document structures demands that the parser possesses flexible template matching or intelligent recognition capabilities to adapt to various report formats. The inconsistency of fields and units increases the difficulty of semantic retrieval, requiring standardization during data preprocessing or multimodal matching during retrieval to accurately link to references containing specific activity values or ADMET parameters. Reference traceability must precisely point to specific sections, tables, or even individual cells within the original document, meeting the high data reliability requirements of R&D personnel.
Configuration Guidelines
| Configuration Item | Recommended Value | Rationale for Recommendation |
|---|---|---|
Chunk size (Segment Length) | 500–800 characters | Balances contextual completeness and retrieval efficiency, preventing overly long segments from diluting key information. |
Recall count (Recall Count) | 8–12 items | Balances recall rate with large language model context length, covering multi-dimensional information. |
Similarity threshold (Similarity Threshold) | 0.75–0.85 | Filters out highly semantically relevant document snippets, reducing interference from irrelevant references. Calibrate based on actual measurements. |
Rerank result count (Reranked Return Count) | 4–6 items | Further refines the most relevant content, improving the quality of large language model responses. |
PARSE_FILE_TIMEOUT_SECONDS | 300 seconds | Accommodates the time required to parse large experimental reports or PDFs containing complex structural diagrams. |
maxContext | 6000–8000 Tokens | Ensures sufficient capacity for multiple recalled snippets and their context, providing enough information for support. |
Three Common Pitfalls
- Semantic retrieval returns too few results or results that do not align with the query intent. The reason may be that the
Similarity threshold(Similarity Threshold) is set too high, leading to overly strict recall and excluding some relevant but slightly less similar content. - After uploading a large experimental report file, the knowledge base training status remains "processing" for an extended period or directly reports an error. The reason is that the
PARSE_FILE_TIMEOUT_SECONDSparameter is set too low, failing to provide sufficient processing time for document parsing. - In knowledge base query results, references to compound activity values or ADMET parameters are inaccurate or lack units. The reason may be that the document parsing stage failed to effectively identify and extract numerical values and unit pairs from tables.
How to Verify Proper Configuration
- Upload typical high-throughput screening reports and ADMET prediction reports. Observe if knowledge base training is successful and check if the parsed segments contain key compound information, activity data, and units.
- Perform retrieval for specific compound IDs, activity value ranges, or ADMET parameters. Verify that the returned references accurately point to the corresponding data points and context in the original documents.
- Simulate actual R&D scenarios by posing complex queries involving multiple entities and attributes. Check the number of recalled references and the quality of reranked content to assess if it effectively supports the large language model's answers.
Note: The values provided are common starting points. It is recommended to measure against your own samples to determine optimal settings.
Question material comes from public community discussions. Configuration values are common starting points and should be measured against your own samples. Verified on 2026-09-21.