Reference and Traceability for Structured Analysis of CMC

Data Characteristics

Data for Pharmaceutical CMC (Chemistry, Manufacturing, and Controls) research documents primarily originate from lab records, manufacturing batch records, quality control reports, and regulatory submission documents. These documents are typically structured PDF reports or scanned images. They contain extensive technical details, experimental data, analysis results, and manufacturing process descriptions. Updates are usually stable, occurring at different project development stages or after manufacturing batches are complete. Document internal structures are rigorous, often using titles, subtitles, tables, charts, and appendices to organize content. Fields and units are highly specialized, such as "Active Ingredient Content (%)", "Impurity Profile (ppm)", "pH Value", and "Dissolution Rate (mg/mL)". Numerical precision and unit consistency requirements are extremely high.

Constraints on Reference and Traceability

The highly structured and specialized nature of CMC documents imposes specific requirements on reference and traceability. First, documents contain numerous tables and charts. Extracting only plain text can lead to a loss of critical information, resulting in incomplete or inaccurate citations. Second, the precision required for specialized terminology and units means that segmentation granularity cannot be too coarse. Otherwise, it risks breaking context and affecting the model's understanding of professional content. Third, due to periodic data updates, there is a requirement for citation timeliness. Referenced data must be the latest version. Finally, the rigor of CMC research demands that every cited piece of content must be precisely traceable to its specific location in the original document, such as page number, section, or table row. This meets compliance and audit requirements, preventing issues like "Why is only the text dataset cited in the knowledge base?"

Configuration Strategy

Configuration Item	Recommended Value	Rationale
`Chunk size` (Segment Length)	`300–500 characters`	Balances the completeness of specialized terminology context with retrieval efficiency, avoiding segments that are too long or too short.
`Chunk Overlap Length` (Segment Overlap Length)	`50 characters`	Ensures contextual continuity at segment boundaries, reducing the risk of critical information being truncated.
`Recall count` (Recall Count)	`8–12 entries`	Increases the coverage of relevant document fragments, improving the probability of recalling critical information from complex documents.
`Similarity threshold` (Similarity Threshold)	`0.75–0.85`	Filters out low-relevance results, improving recall quality and reducing irrelevant citations.
`Rerank result count` (Rerank Return Count)	`5 entries`	Selects the most relevant fragments from the recall results, improving the accuracy of final citations.
`Document Parsing Strategy`	`Smart Segmentation + OCR`	Handles complex CMC documents containing tables, charts, and scanned images, ensuring comprehensive content extraction.

Common Pitfalls

The AI answer fails to cite critical data from tables or charts. This happens because the document parsing strategy does not enable OCR or lacks a dedicated table parsing module, leading to the non-extraction of non-text content.
Cited content is vague and cannot be pinpointed to a specific location in the original document. This occurs when Chunk size (segment length) is set too large or when original page numbers and section information are not recorded in the segment metadata.
The model's answer cites outdated data. This happens when the knowledge base lacks version control or when older document versions are not updated promptly, leading to the retrieval of non-latest information.

Verification Steps

Select a CMC R&D document containing complex tables and charts. Submit it to the knowledge base. Check if the parsed segments include critical data from the tables and charts.
Ask specific questions. Observe if the AI's citations precisely point to the original document's page number, section, or table location.
Upload both old and new versions of a document. Ask questions about the document's evolution. Verify if the AI can cite data from the latest version and explain the updates.
Simulate user questions. Check if the AI's answers avoid issues like "How to configure FastGPT iframe embedding to not display citation content," ensuring the display of citation sources meets expectations.

The values provided are common starting points. Measure them against your own samples.

Question material comes from public community discussions. Configuration values are common starting points and should be measured against your own samples. Verified on 2026-09-21.