What this category’s data looks like
Daily documents for this category include batch production records, material balance ledgers, reaction and purification step logs, cleaning validation records, impurity profile reports, deviation and change request documents. Data sources cover registered process documents, raw material batch ledgers, DCS production control systems, weighing system data, LIMS laboratory information management systems, chromatography data systems, and QMS quality management systems. Data updates trigger per production batch and each process step. Document fields include process parameters, yield data, impurity content, solvent residue levels, change approval status, and more. Units include mole percent, milligrams/liter, batch numbers, and others.
Constraints on vector models and indexing for this category
Data for this category mostly consists of structured process records and semi-structured reports, with high-frequency updates per batch and step. Fine-grained process details must be retained to support accurate tendering and quotation preparation. Fields that combine numerical and text content, such as impurity profiles and yields, require indexes to support associated recall of multiple content types. Each batch has an independent data structure, so vector mixing across batches must be avoided. Fast incremental index updates must also be supported. Tendering documents and quotation preparation require association with historical process compliance data. Indexes must accurately recall process parameters and approval records for specific batches, which imposes clear constraints on recall precision and chunking granularity.
Configuration settings
| Configuration Item | Recommended Setting | Rationale |
|---|---|---|
| Chunk Length | 800–1200 characters | Documents for this category mostly consist of single-step process descriptions and numerical records. 800–1200 characters can fully cover single-step process logic and avoid splitting critical parameters |
| Overlap Length | 100–150 characters | Reduces the chance that consecutive process steps are split across chunks, and retains parameter association information across chunks |
| Maximum Paragraph Depth | 3 | Matches the three-level process step hierarchy of business documents, and prevents merging content across different hierarchies into a single chunk |
defaultToken | 4096 token | Adapts to the input text limit of mainstream embedding models, and meets the embedding requirements for single-chunk process documents |
maxToken | 8192 token | Adapts to the batch indexing processing limit of FastGPT v4.17.0, and prevents batch tasks from failing due to exceeded parameter limits |
| Single Recall Count | Top 3–5 entries | Accurately matches the specific batch process data required for tendering and quotation preparation, and avoids excessive irrelevant content interfering with recall results |
Three common mistakes
- After enabling the embedding model, entering a custom request address and API key results in a direct test error. This occurs when the embedding model’s request format does not match FastGPT’s interface specifications, or when the API key does not have embedding call permissions enabled.
- In FastGPT knowledge base settings, multi-vector support is not enabled for tabular impurity profile reports, leading to incorrect indexing of table data. This occurs when multi-vector association rules for table data are not configured, causing table content to be chunked as plain text.
- A prompt indicates no available indexing models, but service and conversation agent tests run normally. This occurs when the currently used embedding model is not bound in knowledge base settings, or when model configurations have not been synchronized to the system cache.
How to confirm correct configuration
- Navigate to the knowledge base settings page in FastGPT v4.17.0, and confirm that the enabled embedding model matches the currently configured model.
- Upload a typical batch production record document, run a chunking preview test, and check if the chunking results match the set chunk length and overlap length.
- Initiate a recall test targeting process parameters for a specific batch, and confirm that the recalled content includes yield data and impurity profile information for the target batch.
- Check the
defaultTokenandmaxTokenconfiguration values to ensure they do not exceed the input limit of the currently used embedding model.
The values provided are common starting points and should be measured against the reader's own samples.