Knowledge Base Retrieval and Recall for Supplier Audit

Data Characteristics

Supplier audit data in the biopharmaceutical sector primarily originates from documents such as quality management system files, manufacturing process files, inspection reports, qualification certificates, change records, and Corrective and Preventive Action (CAPA) reports. These documents are typically in PDF, Word, or Excel formats. Some data may reside in internal Quality Management Systems (QMS) and are obtained via export. Update frequency varies based on audit type and supplier performance. Regular audits might occur annually or biennially, while retrospective audits or unannounced inspections have no fixed cycle. Document structures are standardized, often including fixed chapters and templates for supplier basic information, audit scope, audit findings, non-conformities, and rectification plans. Fields and units involve batch numbers, expiration dates, production dates, testing indicators (e.g., mg/mL, IU/mg, cfu/g for content, purity, microbial limits), demanding high accuracy and consistency.

Constraints Imposed on Knowledge Base Retrieval and Recall

The standardized and structured nature of supplier audit data imposes specific requirements on knowledge base construction and retrieval. First, documents contain numerous technical terms, abbreviations, and specific regulatory references, requiring tokenizers and embedding models to accurately understand this domain knowledge. Second, audit report non-conformities and rectification plans are often presented in lists or tables, necessitating semantic integrity during text segmentation to avoid splitting critical information. Third, the need for precise matching of fields like batch numbers and expiration dates means that purely semantic retrieval may be insufficient, requiring a combination of keyword matching or structured queries. Fourth, due to varying audit document update frequencies, the knowledge base must support incremental updates and version management to ensure timely and accurate retrieval results. Finally, audit findings often involve cross-references across multiple documents, requiring consideration of linked documents during recall.

Configuration Settings

Configuration Item	Recommended Value	Rationale
`Chunk size`	`800–1200 characters`	Ensures a complete non-conformity or rectification measure from an audit report is contained within a single segment, maintaining semantic integrity.
`Chunk Overlap Length`	`150 characters`	Provides sufficient contextual information to handle cross-paragraph references and descriptions, improving retrieval recall.
`Recall count`	`Top 8–12 entries`	Considers the complexity and interconnectedness of audit reports, recalling more potentially relevant snippets to cover a broader range of information.
`Similarity threshold`	`0.78–0.85`	Balances accuracy and recall, avoiding the retrieval of irrelevant general terms while not missing critical audit findings.
`Rerank result count`	`Top 5 entries`	Re-ranks initially recalled snippets to prioritize the most relevant audit findings or key information.
`PARSE_FILE_TIMEOUT_SECONDS`	`600 seconds`	Allots sufficient parsing time for large audit documents (e.g., quality management system files with hundreds of pages), preventing timeouts.

Common Pitfalls

Importing WeChat official account links results in empty content because of dynamic loading mechanisms or anti-crawler strategies preventing proper content fetching.
A query requires meeting conditions A, B, and C, but the returned results only satisfy some. This occurs due to an improper knowledge base segmentation strategy, where critical information is split or dispersed across different segments, preventing a single recall from covering all conditions.
A knowledge base configuration results in an HTTP 418 error. This might be due to parameters like UPLOAD_FILE_MAX_SIZE or PARSE_FILE_TIMEOUT_SECONDS being set too low, failing to handle the upload or parsing of large audit documents.

Verification of Configuration

Upload an audit report containing complex tables and technical terms. Check if the knowledge base correctly parses and creates effective segments, and if segment content is semantically complete.
Query for non-conformity descriptions within an audit report. Verify if the recall results include complete information about the non-conformity and related rectification suggestions.
Use queries containing batch numbers or specific testing indicators. Validate if the knowledge base accurately recalls corresponding product inspection reports or production records.
Through multiple tests with varying query complexities, observe if the combination of Similarity threshold and Recall count consistently provides highly relevant results. Adjust the threshold range based on actual business needs.

The values provided are common starting points and should be measured against the reader's own samples.

Question material comes from public community discussions. Configuration values are common starting points and should be measured against your own samples. Verified on 2026-09-21.