Vector Model and Indexing for Deviation and CAPA

Data Characteristics for This Category

Deviation and Corrective and Preventive Action (CAPA) registration and declaration documents primarily originate from an organization's internal quality management system. This includes deviation investigation reports, root cause analyses, CAPA plans, implementation records, and effectiveness verification reports. These data update frequently, especially when new deviations occur during production or when CAPA implementation cycles are long. Document structures typically include fixed fields such as title, date, deviation description, impact assessment, investigation results, root cause, CAPA measures, responsible person, completion date, and verification results. Documents are often in PDF or Word format, frequently containing non-textual information like flowcharts, images, and signature pages. Field content involves specific production batches, equipment numbers, reagent lot numbers, deviation levels, and risk levels. Units are often standard measurement units or internally defined codes.

Constraints Imposed by These Characteristics on Vector Models and Indexing

The high update frequency of deviation and CAPA data requires vector indexes to support efficient incremental updates, ensuring retrieval results are timely. The presence of flowcharts and images in documents limits purely text-based vectorization models. This necessitates considering multimodal vector models or preprocessing non-textual information. Fixed fields enable hybrid retrieval of structured and unstructured information. For example, initial filtering can occur based on "deviation level" or "responsible person," followed by fine-grained matching using vector similarity. Additionally, as these documents may contain sensitive information, the indexing process must ensure data isolation and access control. Documents contain specific terminology and internal codes, requiring vector models to understand domain-specific vocabulary well to avoid semantic deviations.

Configuration Settings

Configuration Item	Recommended Value	Rationale
`Chunk size` (Segment Length)	`300–500 characters`	Balances semantic completeness with indexing granularity, suitable for deviation report paragraph structures.
`Overlap Length`	`50–80 characters`	Ensures contextual continuity and handles semantic dependencies across paragraphs.
`Recall count` (Recall Count)	`Top 10`	Covers a sufficient number of potentially relevant documents, improving recall.
`Similarity threshold` (Similarity Threshold)	`Calibrated by actual measurement`	Requires adjustment based on actual retrieval effectiveness and business needs to avoid excessive noise.
`Vector Model` (Vector Model)	`text-embedding-ada-002` or `bge-large-zh`	Balances generality with performance in the Chinese domain, supporting specific terminology understanding.
`UPDATE_INDEX_INTERVAL`	`120 minutes`	Accommodates the update frequency of deviation and CAPA data, maintaining data timeliness.

Common Pitfalls

Retrieval results contain a large amount of irrelevant information or miss critical information: This may be due to Segment Length being set too large, causing individual vectors to include too much noise, or Recall Count being insufficient to cover all relevant context.
Chart content in some deviation reports cannot be retrieved: This occurs because the current vector model only supports text embedding and does not effectively process non-textual information like images and flowcharts.
Timeouts or processing failures occur when uploading large CAPA plan documents: This may be because the PARSE_FILE_TIMEOUT_SECONDS parameter is set too short, not allowing enough time for file parsing and vectorization.

How to Verify Configuration

Select a batch of typical deviation and CAPA documents. Test with different query statements to check if the ranking of relevant documents in the retrieval results is reasonable.
Regularly track newly added or updated documents. Use specific queries to verify if this new data can be retrieved promptly and accurately, evaluating the effectiveness of index updates.
For documents containing charts or special formats, attempt to extract key textual content for querying. Confirm the accuracy of text parsing and vectorization.
Monitor system logs for indications of file parsing failures, abnormal vectorization service calls, or index update errors. Ensure stable system operation.

The values provided are common starting points and should be measured against your own samples.

Question material comes from public community discussions. Configuration values are common starting points and should be measured against your own samples. Verified on 2026-09-21.