What this category's data looks like
Daily documents for this category include cotton blending or ingredient lists, process setup sheets, equipment operation records, yarn inspection reports, grey fabric inspection reports, and defect records. Data sources cover raw fiber inspection, ERP systems, real-time spinning and loom data, evenness tester data, fabric inspection machine collected data, and order specification files.
Data update rhythm shifts with production batches, work shifts, and fabric yardage. New business data is generated for each raw material batch, each work shift, and each piece of fabric. Document fields typically include fiber indicators, process parameters, defect types, and other related content. Some parameters use specialized industry-specific units.
Constraints for vector models and indexing
This category has high-frequency, scattered data updates. New data is created for every raw material batch, work shift, and piece of fabric. Indexes must support high-frequency incremental updates to avoid excessive time spent on full reconstruction.
Daily document types are diverse with significant format differences. The fiber indicator format on cotton blending sheets differs from the defect description format in defect records. This requires adapting chunking and vectorization workflows for multiple text types.
Real-time equipment-collected data and static ERP order data coexist. Indexes must support adaptation for both structured fields and unstructured text.
Additionally, defect records have a strong correlation with corresponding process parameters. Indexes must support cross-document associated retrieval, to ensure complete business chain data can be matched during Q&A.
Configuration Settings
This page’s configuration is based on FastGPT v4.17.0.
| Config Item | Recommended Value | Rationale |
|---|---|---|
| Chunk Length | 800–1200 characters | Adapts to the standard text length of process sheets and inspection records, avoids including too many unrelated parameters in a single chunk |
| Overlap Length | 100–150 characters | Covers cross-chunk process parameter associations, ensures retrieval continuity between defect records and corresponding setup sheets |
defaultToken | 2000 token | Matches the core field length of a single batch of cotton blending sheets, ensures context integrity for basic Q&A |
maxToken | Set to the selected embedding model/service maximum supported input tokens | maxToken is the input-token maximum supported by the selected embedding model/service; verify it in that provider configuration. See the v4.17.2 model-field reference. It is not retrieval/LLM context or a FastGPT-wide fixed cap. |
| Similarity Threshold | 0.75–0.85 | Differentiates matching accuracy between process parameters and defect descriptions, avoids irrelevant results being included in retrieval lists |
| Single Search Limit | Top 8 entries | Balances retrieval efficiency and result completeness, adapts to the retrieval needs of multi-document association for this category |
Three common configuration mistakes
- Phenomenon: Mixed retrieval requests take more than 10 seconds, considerably longer than the ~8 second runtime of pure vector retrieval. Cause: This category’s index includes multiple types of highly correlated business documents. Mixed retrieval must call multiple index branches simultaneously, lengthening the retrieval chain.
- Phenomenon: The system gets stuck during the indexing step for the final index set, and cannot complete knowledge base initialization. Cause: Incremental update mode is not enabled. Full traversal of the massive small file data generated by each raw material batch and work shift causes indexing tasks to block.
- Phenomenon: Some cotton blending sheets and defect records cannot be vectorized correctly, and retrieval results are empty. Cause: Chunking parameters are set improperly. Short text documents are split too finely or merged too extensively, leading to loss of vectorization features.
How to confirm proper configuration
- A typical process setup sheet and corresponding defect records are uploaded. A retrieval test is executed, and returned results are checked for matching process parameters and defect descriptions.
- New cotton blending data is uploaded in high-frequency incremental batches. The completion speed of indexing tasks is observed, and blockages are confirmed to be absent.
- The similarity threshold is adjusted. The matching accuracy of retrieval results is verified against business requirements, and the reasonableness of the threshold setting is confirmed.
- The time difference between mixed retrieval and pure vector retrieval is tested. The runtime of mixed retrieval is confirmed to stay within the business acceptable range.
The values provided here are common starting points, and should be evaluated against samples from the specific deployment.