Data Characteristics
Supplier audit data in pharmacovigilance primarily originates from pharmaceutical companies' audit reports, deficiency records, corrective action plans, and related regulatory compliance documents. These documents are typically in PDF, Word, or structured database formats. Audit reports detail findings, risk assessments, recommended actions, and supplier responses. Data update frequency varies, usually annually or as needed based on risk assessments.
Document structure for audit reports often includes a cover page, table of contents, executive summary, detailed findings, corrective action plans, and attachments. Key fields include supplier name, audit date, auditor information, deficiency ID, deficiency description, risk level, regulatory clause citation, corrective actions, and completion date. Risk levels may be high/medium/low or numerical scores. Date fields require a clear format.
Constraints on Citation and Traceability
The mixed structured and semi-structured nature of supplier audit data requires both precise matching and contextual understanding for citation and traceability. Specific deficiency descriptions or regulatory citations in audit reports must trace back to exact paragraphs or clauses in the original text. This ensures the compliance and rigor of pharmacovigilance decisions.
Report updates are infrequent, but a single update can involve extensive revisions. Therefore, managing and citing historical versions is critical. Audit reports are often lengthy and contain much non-core information. Effective chunking strategies are necessary to prevent irrelevant content from affecting citation accuracy. Reports may also include multi-level headings and numbering, requiring the citation mechanism to identify and leverage this structural information to improve traceability granularity.
Configuration Settings
| Configuration Item | Recommended Value | Rationale |
|---|---|---|
Chunk size (Chunk Size) | 800–1200 characters | Balances paragraph integrity in audit reports with retrieval efficiency, preventing context loss from over-segmentation. |
Recall count (Recall Count) | Top 5–8 items | Ensures coverage of potentially relevant audit findings and corrective action records while controlling LLM input length. |
Similarity threshold (Similarity Threshold) | 0.75–0.85 | Guarantees high relevance between recall results and query intent, reducing low-quality citations. |
Rerank result count (Reranked Return Count) | Top 3 items | Further improves the ranking of the most relevant content for model citation, based on initial recall. |
Citation limit (Citation Limit) | 1500 tokens | Restricts the total length of cited text, preventing exceeding the model's context window and balancing information volume with model processing capability. |
quote_type | text | Audit report content is primarily text; direct citation of text segments best meets traceability requirements. |
Common Pitfalls
quote type errorduring citation, preventing normal generation of cited content. This occurs when the knowledge base's configured citation type does not match the actual retrieved content format, or data type conversion fails during processing.- Replies lack specific page numbers or paragraph locations from the cited document, resulting in vague citation information and difficulty locating the original text. This happens when the knowledge base does not retain or correctly extract source document location metadata during chunking.
- Retrieved chunk length significantly exceeds the
Citation limit(Citation Limit) setting, causing model context overflow. This indicates a mismatch between the knowledge base's chunking strategy and the citation limit, where long chunks are cited directly after retrieval without secondary truncation.
Verification
- Ask questions about typical audit findings. Check if the response includes clear citation sources, such as document names and specific paragraphs.
- Simulate an auditor querying compliance with specific regulatory clauses in a supplier report. Verify the system accurately links to corresponding compliance statements or deficiency descriptions in the report.
- Input a question related to a supplier's corrective action plan known to have undergone multiple revisions. Verify the system prioritizes citing the latest or most relevant version information.
The values provided are common starting points and should be measured against the reader's own samples.
Question material comes from public community discussions. Configuration values are common starting points and should be measured against your own samples. Verified on 2026-09-21.