What data looks like for this category
Data for this category comes from design BOMs, silicon steel sheet and insulating material ledgers, vacuum drying equipment operation logs, insulating oil test reports, and partial discharge and withstand voltage records from high-voltage test stations. The update rhythm follows that each batch of materials and each process for a single product generates independent documents. Documents include winding and core assembly records, vacuum drying records, insulating oil reports, partial discharge and withstand voltage test reports, nameplate parameter tables, and more. Most fields are measured values with clear units, such as insulation moisture content, withstand voltage level, and partial discharge amplitude.
What constraints these characteristics impose on vector models and indexes
Documents for a single product have high relevance. Indexes must be built by batch or single product to avoid mixing data across products. Measured fields with clear units require vector models to accurately capture the association between values and units, to avoid semantic deviation. Frequently updated single-product documents need to support incremental indexing to avoid extra time spent on full reindexing. Long-text test reports require reasonable chunking rules to retain contextual association of key parameters and prevent core information from being split and lost.
How to set configurations
| Configuration Item | Recommended Value | Rationale |
|---|---|---|
| Chunk Length | 800–1200 characters | Adapts to long text segments from insulating oil reports and withstand voltage test reports, retains association between parameters and context |
| Overlap Length | 50–80 characters | Prevents key parameters across chunks from being truncated, ensures semantic integrity of fields such as partial discharge and withstand voltage |
| Similarity Threshold | 0.72–0.85 | Filters low-match irrelevant documents for measured values with units, focuses on test data of the same category |
| Single Search Limit | Top 6 entries | Matches the number of associated documents for a single product, avoids introducing too many records from non-target batches |
| maxToken | 8192 | Adapts to embedding processing of long-text test reports, complies with input limits of mainstream open-source embedding models |
Three common mistakes
- Symptom: After connecting qwen3-embedding-8b in FastGPT v4.17.0, the file status continuously displays "Indexing". Cause: The input token limit of this embedding model does not match the currently configured
maxToken, causing the embedding task to fail to complete normally. - Symptom: Manually inserted knowledge base documents have their automatically generated default indexes disappear after several hours. Cause: Index persistence configuration is not enabled, and temporarily stored indexes are automatically cleared after service restart or cache refresh.
- Symptom: After replacing the embedding model, the recall rate of multilingual documents does not improve significantly. Cause: Re-embedding operation is not performed on historical documents. Original vectors are still generated based on the semantic space of the old embedding model, which is incompatible with the semantic space of the new model.
How to confirm configurations are set correctly
- Randomly select an insulating oil report, manually trigger the embedding task, and check whether there are token limit-related errors in the task log to confirm that the
maxTokenconfiguration meets the requirements of the current embedding model. - Upload a complete process document for a single transformer, check the storage path after index generation to confirm that the index has been persistently saved.
- Enter a query such as "XX model transformer partial discharge anomaly analysis" to check whether the returned results only associate test reports of the same batch and same model products, to confirm that the similarity threshold and search limit configurations are in effect.
- After replacing the embedding model, perform a full re-embedding operation to verify whether the vector update status of historical documents is normal.
Note: The values listed above are common starting points, and should be evaluated against specific samples relevant to the deployment context.