This configuration is based on FastGPT v4.17.0.
What data looks like for this category
The core data for this category comes from manufacturing enterprise purchase orders, drawing specifications, supplier quality certificates, data collected by inspection equipment, historical quality records, and supplier portal documents. Daily processed documents include inspection specifications, sampling plans, incoming material inspection reports, supplier audit reports, exception contact sheets, 8D correction reports, and concession acceptance approval forms. Data updates are triggered by batches: each incoming material batch generates corresponding inspection records. Supplier overall evaluations are conducted monthly or quarterly, with special audits carried out irregularly. Data fields include supplier code, batch number, inspection date, non-conforming item description, correction status, and other items, mixing structured and unstructured content.
What constraints these characteristics impose on the vector model and indexing link
Data for this category comes from scattered sources with high update frequency, mixes documents of different formats, and requires association with business identifiers such as suppliers and batch numbers. Long supplier audit reports and 8D reports need to be split for processing. High-frequency batch-level updates require indexes to support incremental synchronization. At the same time, retrieval results must be associated with business context to avoid invalid results unrelated to the current supplier or batch. Therefore, vector indexes must bind metadata fields, and key business information across chunks must be retained during chunking to ensure retrieval accuracy and relevance.
How to set the configurations
| Configuration Item | Recommended Setting | Rationale |
|---|---|---|
| Chunk Length | 800–1200 characters | Adapts to the typical paragraph length of incoming material inspection reports and 8D reports, avoiding overly scattered or redundant content in a single chunk |
| Overlap Length | 100–200 characters | Retains cross-chunk key metadata such as supplier code and batch number, preventing business-related information from being cut off by chunking |
defaultToken | 1024 | Matches the input limit of the specified vector model, adapting to the standard text length after chunking |
maxToken | 2048 | Reserves context expansion space for associating multiple related documents from the same batch |
| Similarity Threshold | 0.75–0.85, calibrated via actual testing | Distinguishes semantic differences between qualified and non-conforming inspection records, meeting business judgment requirements |
| Single Search Limit | Top 10 results | Covers historical inspection data from the same supplier and batch, meeting reference needs during audits |
Three common mistakes
- Issue: When uploading a supplier audit report larger than 10MB, it is split into 1000+ chunks, some chunks fail vectorization, the interface displays "vectorization exception", and some exceptions remain unresolved after repeated retries. Cause: The chunk length is not adjusted according to the
defaultTokenlimit of 1024, the text length of some chunks exceeds the model input limit, and the automatic chunk truncation function is not enabled. - Issue: After batch uploading multiple incoming material inspection reports, server resources are exhausted and a crash occurs. After restarting, the knowledge base status shows "not ready", and index tasks cannot be automatically recovered. Manual triggering of indexes for each knowledge base is required. Cause: Sharded full index tasks are not configured, and the setting to automatically resume suspended indexes after server restart is not enabled.
- Issue: When retrieving historical incoming material inspection records, returned results have very low relevance to the current batch, and some results lack corresponding supplier code or batch number fields. Cause: Business metadata of documents is not bound during chunking, so vector indexes are not associated with key business identifiers such as suppliers and batches, and irrelevant results cannot be filtered during retrieval.
How to confirm the configuration is correct
- Upload a typical incoming material inspection report, check the chunk details, confirm that the length of each chunk falls within the preset chunk length range, and the overlapping part contains key metadata such as supplier code and batch number.
- Initiate a vector retrieval, input "non-conforming incoming material inspection records for a certain supplier", check the number and relevance of returned results, and adjust the similarity threshold to meet business judgment requirements.
- Restart the FastGPT service, check the index status of the knowledge base, confirm that all uploaded documents have completed indexing automatically, and no manual triggering is required.
- Upload a supplier audit report larger than 10MB, confirm that the number of chunks is reasonable, and all chunks have completed vectorization with no exception prompts.
The values provided are common starting points and should be measured against the reader's own samples.