What the data for this category looks like
This category of data comes from registered process documents, raw material batch ledgers, DCS production data, weighing system records, LIMS systems, chromatography data systems, and QMS quality management documents. Daily documents include batch production records, material balance sheets, reaction and purification process records, cleaning validation records, impurity profile reports, deviation and change documents. Data is updated per production batch and per process step. Fields include reaction parameters, impurity content, solvent residue, batch number, operator, and timestamp. Units include molar ratio, percentage, milligrams per milliliter, hours, and others.
Constraints on knowledge base retrieval and recall
Data sources are scattered and mostly output by industrial systems. Multi-source heterogeneous data access and incremental synchronization must be supported to ensure that documents updated after each production batch can be retrieved in a timely manner. Documents contain professional chemical parameters and compliance content. Retrieval must match professional terminology and field precision to avoid generalized recall of irrelevant content. Single document content is lengthy and includes critical numerical values. Long text chunking and recall logic must be adapted. Exception reports and change documents are associated with historical batch data. Cross-document associated queries must be supported for recall to ensure the accuracy of process anomaly traceability.
How to set configurations
The following configuration settings are based on FastGPT v4.17.0.
| Configuration Item | Recommended Value | Rationale |
|---|---|---|
similarity | 0.72–0.82 | Adapt to the professional matching precision of active pharmaceutical ingredient process parameters, filter low-relevance general documents, and retain highly relevant batch records and impurity profile data |
limit | Top 6–10 entries | Balance context window usage and recall coverage, avoid model context overflow caused by excessive long text |
searchMode | mixedRecall | Combine vector retrieval and keyword retrieval to match structured and unstructured content such as process terminology and batch numbers |
usingReRank | Enabled | Perform secondary sorting on initial recall results to improve the sorting priority of key information such as impurity content and reaction conditions |
datasetSearchUsingExtensionQuery | Enabled | Automatically expand professional terminology queries based on user questions to adapt to industry-specific terminology scenarios |
collectionIds | Bind corresponding knowledge base collections by department | Only recall exclusive documents of synthesis, quality, and registration support departments to avoid interference from cross-department irrelevant data |
Three common mistakes
- Phenomenon: After configuring the knowledge base, a 418 status code is returned, and the function works normally when the knowledge base is not configured. Cause: The
collectionIdsparameter is not configured correctly, or the bound knowledge base collection is not authorized to the current application, so the retrieval process cannot read the target dataset content. - Phenomenon: Attempting to directly import WeChat Official Account articles into the knowledge base results in a large number of non-process-related content in retrieval results. Cause: No industry scenario filtering is performed on imported documents, and official account articles do not have marked professional fields, leading to generalized recall of irrelevant information.
- Phenomenon: Storage space insufficient or errors failing to load a large number of documents occur during retrieval. Cause: No attention is paid to the knowledge base's file count and storage space limits, and expired non-essential historical data is not cleaned according to the per-batch update frequency.
How to confirm the configuration is complete
- Input process-related questions, such as "the cause of impurity exceeding the standard in a certain batch", and check whether the recall results only include batch records, impurity profile reports and other documents exclusive to this category.
- Check the number of retrieval returned results, and confirm that it matches the preset value of the
limitparameter. - Verify the sorting of key information such as impurity content and reaction parameters in the results, and confirm that the
usingReRankfunction is enabled. - Test input simplified terminology, such as "yield", and check whether relevant documents containing "reaction yield" and "intermediate yield" can be recalled, confirming that the
datasetSearchUsingExtensionQueryfunction works normally.
The values given are common starting points and should be measured against the reader's own samples.