Knowledge Base Retrieval and Recall for Supplier Audit

Data Characteristics

Supplier audit regulation data in the biopharmaceutical sector originates from internal quality management system documents, supplier qualification certifications, audit reports, non-conformance rectification records, and regulatory texts. These documents are typically stored as PDFs, Word documents, or Excel files. Data update frequency is stable, primarily occurring during annual audits, new supplier onboarding, or regulatory updates. Document structures are standardized; audit reports often include fixed sections such as audit scope, audit findings, and rectification recommendations. Fields and units are industry-specific. For example, equipment calibration reports include calibration date, expiration date, and measurement deviation (μm). Material batch records contain batch number, production date, expiration date, and purity (%).

Constraints Imposed by These Characteristics on Knowledge Base Retrieval and Recall

The standardized and structured nature of supplier audit data allows for high accuracy in semantic knowledge base retrieval. The strictness of regulations and SOP texts demands precise and unambiguous retrieval results. Complex tabular data within audit reports challenges knowledge base chunking strategies, requiring complete and readable table content. The specificity of fields and units requires the retrieval model to understand and differentiate between units like "mg/L" and "g/kg" to avoid misjudgments due to unit confusion. The relatively low data update frequency means a comprehensive initial import is necessary during knowledge base construction, with daily maintenance focused on incremental updates.

Configuration Settings

Configuration Item	Recommended Value	Rationale
`Chunk size` (Chunk Length)	`500–800 characters` (characters)	Logical paragraphs in audit regulations and SOP documents typically fall within this length, balancing semantic completeness and retrieval granularity.
`Recall count` (Recall Count)	`Top 5–8 entries` (top 5–8 items)	Supplier audit questions demand high accuracy; increasing recall count appropriately covers more potentially relevant information.
`Similarity threshold` (Similarity Threshold)	`0.75`	Ensures retrieved results are highly relevant to the query intent, filtering out vague matches.
`Rerank result count` (Reranked Return Count)	`Top 3 entries` (top 3 items)	Further optimizes results using a reranking model based on high-similarity recall, focusing on the most relevant content.
`PARSE_FILE_TIMEOUT_SECONDS`	`600 seconds` (seconds)	Processing large PDF audit reports or regulatory texts can be time-consuming; this provides sufficient parsing time.
`maxContext`	`3000 Tokens`	Ensures the LLM has enough context to understand complex audit clauses and cross-references from multiple documents.

Common Pitfalls

After uploading numerous documents, the reranking model consistently returns false, leading to retrieval results not being sorted as expected. This occurs when the reranking model is not correctly deployed or not enabled in the knowledge base configuration.
When knowledge base chunks are stored, tabular content appears fragmented during retrieval. Table rows are split across different knowledge chunks, resulting in incomplete data. This happens when the default chunking strategy is not optimized for table structures and does not recognize table boundaries.
After creating a knowledge base and uploading files via API, the chat interface cannot retrieve content when linked to that knowledge base. Answers are empty or irrelevant. This occurs when the knowledge base has not been indexed or indexing failed after creation, making the content unretrievable.

Verification Steps

Upload a supplier audit report containing complex tables. Use the knowledge base preview function to check if table content is chunked completely without fragmentation.
Select several representative audit questions and perform simulated retrievals. Observe if the Recall count (Recall Count) and Rerank result count (Reranked Return Count) meet the expected numbers, and check the semantic relevance of the retrieved content.
Use queries containing specific regulatory clauses or measurement units. Verify that the retrieval results accurately include these key pieces of information and check if vague matches are effectively filtered out under the set Similarity threshold (Similarity Threshold).
Upload and index a new document via API, then immediately query it through the chat interface to confirm that the newly uploaded content is retrievable.

Note: The values provided are common starting points. Measure them against your own samples.

Question material comes from public community discussions. Configuration values are common starting points and should be measured against your own samples. Verified on 2026-09-21.