What Data Looks Like for This Category
Data for pharmaceutical dosage form manufacturing originates from approved formulation processes, material batches, MES electronic batch records, equipment and environmental monitoring data, LIMS test records, quality management systems, and similar sources. Daily generated documents include master production files, batch production records, batch packaging records, cleaning records, test records, deviation/CAPA/change documents, batch release audit forms, and other related materials. Document fields include material batch numbers, weighing values, process temperatures, time parameters, with corresponding units of g, kg, ℃, and min. Data is updated each time a batch is produced or a key process step is executed. Equipment and environmental data is updated per fixed monitoring schedules.
Constraints for Vector Models and Indexes
The batch-based update pattern requires indexes to support incremental updates. This eliminates extra time spent on full index reconstruction. The mixed multi-document scenario demands vector models to split both structured fields and unstructured text. Models must retain field and unit information to preserve complete business semantics. Under GMP compliance rules, data traceability must be ensured. Indexes must record the source document and corresponding production batch information for every chunk. The specificity of fields and units requires splitting rules to retain full business associations. This prevents damage to the contextual logic of process parameters.
Configuration Settings
| Configuration Item | Recommended Value | Rationale |
|---|---|---|
| Chunk Length | 800–1200 characters | Adapts to the text length of a single process step in batch production records, avoiding splitting that breaks complete business logic |
| Overlap Length | 100–150 characters | Retains contextual associations between adjacent process steps, ensuring complete process deviation correlation information can be matched during retrieval |
| Similarity Threshold | 0.75–0.85 | Meets GMP data matching rigor requirements, filtering irrelevant records with low matching scores |
maxToken | 8192 | Adapts to the input upper limit of mainstream vector models, meeting long-text processing needs for batch record summarization |
defaultToken | 512 | Used as the basic unit for index chunks, balancing splitting accuracy and index construction efficiency |
| Single Recall Limit | Top 20 entries | Covers all record types related to a single batch production, avoiding missing key deviation or test data |
Three Common Configuration Mistakes
- Issue: A single batch record file remains in the indexing state for an extended period, with progress stuck on the interface. Cause: The file contains an extremely long complete batch production record that was not split per the set chunk length. This results in a single chunk exceeding the
maxTokenlimit. - Issue: Cross-knowledge-base retrieval cannot call content per preset priority. Returned results mix entries from two knowledge bases. Cause: Knowledge base priority binding was not enabled in the vector index configuration. Or the permission scope of the corresponding knowledge base was not associated with the production batch.
- Issue: A vector model that works normally in local testing fails during server-side retrieval, returning empty results. Cause: The embedding dimensions of the vector model used locally and on the server were not confirmed to be consistent. Document text was not processed using the same set of tokenization rules.
How to Verify Correct Configuration
This verification process is based on FastGPT v4.17.0.
- Upload a single complete batch production record, check the chunking log, and confirm each chunk’s length falls within the set range.
- Input keywords from deviation records to initiate retrieval, confirm returned results’ similarity scores fall within the set threshold range.
- Upload documents from multiple batches, verify the index supports incremental updates by batch, without requiring full index reconstruction each time.
- Configure cross-knowledge-base retrieval rules, verify retrieval results preferentially return content from the specified knowledge base.
The values provided are common starting points and should be measured against the reader's own samples.