Data Characteristics
Quality documents for lead compound screening typically include high-throughput screening reports, activity verification data, physicochemical property reports, toxicity prediction reports, and batch analysis certificates. Data sources are diverse, encompassing internal laboratory LIMS systems, CRO analysis reports, and external databases. Document update frequency is relatively low, primarily occurring at key project milestones, such as after compound library screening, during activity confirmation, or before and after lead compound optimization. Documents are primarily in PDF, Word, or Excel formats. They often contain chemical structure images, spectra (e.g., NMR, MS), tabular data (e.g., IC50, LogP, solubility, ADME parameters), and detailed experimental method descriptions. Fields and units are highly specialized, for example, concentration in micromolar (µM), activity inhibition percentage (%), molecular weight (Da), pH value, and retention time (min). Documents from different sources may have naming discrepancies or inconsistent units.
Constraints on Deployment and Upgrades
The data characteristics of lead compound screening quality documents impose specific requirements on FastGPT deployment and upgrades. First, chemical structure images and spectra within documents challenge text extraction capabilities. OCR processing of non-standard image formats can lead to critical information loss or recognition errors. Second, diverse document formats and complex internal table structures require more robust file parsers to ensure accurate data chunking, preventing important data from being truncated or conflated. Third, low update frequency allows for more flexible scheduling of knowledge base rebuilding or incremental updates. However, each update must ensure data integrity to prevent loss of historical records. Finally, specialized fields and inconsistent units necessitate strict standardization and cleaning during data preprocessing. Failure to do so can affect recall accuracy and the model's understanding of specific parameters. During upgrades, compatibility with older file parsing logic and data indexing structures is crucial to prevent documents processed correctly by older versions from failing to parse in newer versions.
Configuration Guidelines
| Configuration Item | Recommended Value | Rationale |
|---|---|---|
UPLOAD_FILE_MAX_SIZE | 500 MB | Lead compound screening reports can contain numerous spectra and high-resolution images, resulting in large file sizes. |
PARSE_FILE_TIMEOUT_SECONDS | 600 seconds | Processing complex PDF documents and large Excel spreadsheets, especially during OCR, requires longer parsing times. |
Chunk size | 800–1200 characters | Ensures logical units like experimental methods and results analysis are not excessively split, maintaining contextual integrity. |
Recall count | Top 10 entries | Increases recall coverage to handle queries with many specialized terms and strong contextual relevance, reducing missed recalls. |
Similarity threshold | 0.75 | Lead compound data demands high precision. A higher threshold filters for more relevant results. |
Rerank result count | Top 5 entries | Re-ranking improves the order of results most relevant to the query intent while maintaining recall quantity. |
Common Pitfalls
- After an upgrade, table data in some uploaded PDF documents may not chunk or extract correctly. This appears as table content being recognized as a single block of text or missing fields. This usually occurs when the new file parser has insufficient compatibility with specific layouts or embedded fonts.
- After deploying the FastGPT container group, a container remains in a restarting state for an extended period, with logs showing "Reached the max retries." This can stem from misconfigured Zilliz or other dependent services, such as incorrect connection parameters or insufficient resource allocation.
- When users query for a specific compound's IC50 value, relevant documents may sometimes fail to be recalled, even if the document explicitly contains the information. This appears as empty or irrelevant recall results. This can happen if document content is incorrectly segmented during indexing, separating critical numerical values from their context.
Verification Steps
- Upload a lead compound screening report PDF containing complex tables and chemical structure images. Check if the document is reasonably segmented in the knowledge base, ensuring table content is correctly identified and chunked.
- Query for specific compound names, IC50 values, or experimental conditions. Verify that recall results include the expected relevant documents and that the number of recalled items and their ranking meet expectations.
- Check system logs for file parsing task completion status. Ensure there are no excessive timeout errors or parsing failures, especially when processing large or complexly formatted documents.
- Use the API to upload and query small text snippets containing specialized terms and units. Verify that the system's indexing and recall capabilities for these special fields function correctly.
Note: The values provided are common starting points. Measure them against your own samples for optimal performance.
Question material comes from public community discussions. Configuration values are common starting points and should be measured against your own samples. Verified on 2026-09-21.