What data looks like for this category
This category of data comes from registered process documents, raw material batch ledgers, DCS systems, weighing systems, LIMS, chromatography data systems, and QMS. Daily generated documents include batch production records, material balance sheets, reaction and purification process records, cleaning validation records, impurity profile reports, deviation and change documents. Data is updated on a per-production batch, per-process-step trigger basis. Documents contain fields such as process parameters, batch numbers, impurity content, solvent residue levels, yield data, and more. Units mostly involve professional industrial parameters including weight, concentration, reaction temperature, and time.
What constraints these characteristics impose on the knowledge base retrieval and recall link
Multi-source heterogeneous data sources lead to inconsistent data formats. Match field mappings across different systems to retrieve valid content. High-frequency batch updates require the retrieval system to support incremental recall and real-time synchronization. This prevents data lag. Precise matching for professional fields such as impurity profiles and yields requires the retrieval model to adapt to chemical and pharmaceutical professional terminology and numerical association logic. Cross-department document coverage requires support for filtering recall results by document category. This lets quality and regulatory departments quickly locate compliance-related content.
How to set up configurations
The following configurations are based on FastGPT v4.17.0, adapted to the ledger retrieval needs of active pharmaceutical ingredient and pharmaceutical intermediate manufacturing scenarios.
| Configuration Item | Recommended Setting | Rationale |
|---|---|---|
similarity | 0.75–0.85 | Matches professional terminology for active pharmaceutical ingredient process parameters and impurity profiles. It avoids irrelevant recall caused by too low a value, and prevents missing valid batch records caused by too high a value |
limit | Top 8 entries | Covers the number of ledger documents for multiple steps in a single batch, meets the basic data requirements for production summary reports |
searchMode | mixedRecall | Combines semantic retrieval and keyword matching to accurately retrieve key values such as reaction temperature and yield in batch production records |
usingReRank | Enabled | Rearranges retrieved multi-source documents by professional relevance, prioritizes displaying core reports such as impurity profiles and material balances |
datasetSearchUsingExtensionQuery | Enabled | Automatically expands synonyms related to processes, such as matching "fermentation" with "biotransformation", to cover terminology differences in registered processes |
collectionIds | Bind corresponding knowledge bases by department | Separates exclusive documents for synthesis, quality, and regulatory departments, avoids interference from cross-department irrelevant data in retrieval |
Three common configuration errors
- Only retrieval results are displayed in the interface, with no knowledge base citation sources shown. This happens because the knowledge base citation display switch is not enabled. Citation information is hidden in some scenarios under default configurations.
- Content cannot be retrieved from a specific knowledge base, but retrieval works normally for newly created knowledge bases. The symptom is zero retrieval result entries, with logs showing no matching vectors. The cause is that the document chunking of the original knowledge base does not match the length of pharmaceutical professional documents, or the embedding model does not adapt to the vector space of chemical terminology.
- New ledger information cannot be actively added to the knowledge base during conversations. The symptom is that no new data is retrieved after calling the knowledge base update interface. The cause is that the incremental synchronization trigger mechanism is not configured, or the updated documents are not uploaded to the knowledge base collection bound to the corresponding
collectionIds.
How to confirm the configuration is correctly set
- Enter the process parameter keywords for a single batch, verify whether the document type and fields of the retrieval results match the target ledger content.
- View the citation sources of the retrieval results, confirm whether the document path and batch number of the corresponding knowledge base are displayed.
- Upload a new impurity profile report, trigger knowledge base synchronization, then perform retrieval, confirm that the new document is successfully retrieved.
- Adjust the value of the
similarityparameter, verify whether the number and relevance of retrieval results meet expectations.
The values provided are common starting points and should be measured against the reader's own samples.