What this category of data looks like
This category of data primarily comes from purchase orders, drawing specifications, supplier quality certificates, data collected by inspection equipment, historical quality ledgers, and review materials submitted via supplier portals. Daily processed documents include inspection specifications, sampling plans, incoming material inspection reports, annual supplier review reports, exception liaison sheets, 8D corrective action reports, and concession approval forms. Data updates are generated in real time with each batch of incoming materials. Supplier quality evaluations are completed monthly or quarterly. Special reviews are conducted irregularly. Document fields include batch number, supplier code, inspection items, nonconformity types, corrective measures, and more. There are no unified fixed units. Measurement identifiers such as pieces, batches, and items are matched based on the specific document type.
What constraints these characteristics impose on vector models and indexes
This category of data has high update frequency and is generated dynamically per batch. It requires vector indexes to support incremental updates, to avoid resource consumption from full reconstruction. Data sources include multiple document types, ranging from structured inspection equipment data to unstructured 8D corrective action reports. This requires indexes to adapt to text splitting for different formats. Documents are associated with metadata such as batch numbers and supplier codes. This requires the retrieval link to support precise filtering based on metadata, to narrow the recall scope. Some documents have large length differences, from short incoming material reports to complete review reports. This requires chunking rules to adapt to text of different lengths, to avoid semantic fragmentation.
Configuration Settings
Configurations on this page are based on FastGPT v4.17.0, and are adapted to the business data characteristics of this category:
| Configuration Item | Recommended Setting | Rationale |
|---|---|---|
| Chunk Length | 800–1200 characters | Adapts to the common text lengths of inspection reports and corrective action reports in this category, avoiding semantic segmentation |
| Overlap Length | 100–150 characters | Retains contextual association between chunks, ensuring semantic coherence during retrieval |
| Similarity Threshold | 0.75–0.85 | Distinguishes feature descriptions of qualified and nonconforming incoming materials, reducing the probability of false or missed recalls |
| Single Search Limit | Top 10 results | Retrieval for this category is often limited to supplier or batch scopes; a small number of results covers associated requirements |
defaultToken | 4096 | Adapts to the input upper limit of most embedding models, avoiding truncation of critical supplier qualification and corrective action content |
maxToken | Set to the selected embedding model/service maximum supported input tokens | maxToken is the input-token maximum supported by the selected embedding model/service; verify it in that provider configuration. See the v4.17.2 model-field reference. It is not retrieval/LLM context or a FastGPT-wide fixed cap. |
Three Common Misconfigurations
- Symptom: Task progress stalls after switching the knowledge base vector model, and the original model cannot be reselected. Root cause: Switching the model directly without clearing the current unfinished index task first triggers a system resource lock.
- Symptom: A 404 error is returned after testing the configured embedding model. Root cause: The API gateway address or interface path for model access is not configured correctly, so requests cannot reach the target service.
- Symptom: Retrieval results do not filter by batch number after uploading new incoming material documents. Root cause: Metadata fields are not bound in the retrieval configuration, so filtering conditions unique to this category such as batch number and supplier code cannot be associated.
How to Verify Successful Configuration
- Upload a single batch of incoming material inspection reports, run a retrieval test, confirm that returned results include historical documents from the same supplier or batch, and check that the number of retrieval results matches the single search limit configuration.
- View index construction logs, confirm that incremental update tasks trigger normally with no error messages, and verify that
defaultTokenandmaxTokenconfigurations do not trigger truncation warnings. - Attempt to switch the embedding model, confirm that the system allows reselecting the model and starting a new index task with no locked status.
- Upload a document containing metadata, add supplier code as a filtering condition during retrieval, and confirm that the retrieval scope is correctly restricted.
The values provided here are common starting points, and should be evaluated against the user’s own samples.