What This Category of Data Looks Like
Data for this category originates from design BOMs, silicon steel sheet and insulating material ledgers, vacuum drying equipment logs, oil test reports, and high-voltage test station records. Daily documents include winding and core assembly records, vacuum drying records, insulating oil test reports, partial discharge and withstand voltage test reports, and equipment nameplate parameter tables. Data updates follow a set rhythm: full record updates for each individual product, and synchronous updates of batch material parameter data for each raw material batch. Document fields include insulation moisture content, partial discharge peak value, withstand voltage rating, and some fields use exclusive units such as ppm and kV.
Constraints on Vector Models and Indexes
Full-link records for a single product must be bound to a unique serial number. Indexes must support precise recall of corresponding batch data using the product identifier, to avoid invalid cross-batch matching. Frequently updated raw material and test data requires indexes to support incremental updates, reducing full reindexing time to align with the update rhythm of each product or batch. Professional fields such as insulation parameters and partial discharge data require vector models to support high-precision semantics; general vector models may lose semantic features tied to professional contexts. Document lengths vary widely, from short parameter tables to thousands-word test records. Text chunking must support different lengths to avoid critical information truncation or redundancy.
Configuration Guidelines
This configuration applies to FastGPT v4.17.0.
| Configuration Item | Recommended Range | Rationale |
|---|---|---|
| Chunk Length | 800–1200 characters | Adapts to the length of a single test report segment for a single product, avoiding truncation of critical test data |
| Overlap Length | 100–150 characters | Covers association information for insulation parameters across adjacent chunks, preventing professional semantics from being split |
| Similarity Threshold | 0.75–0.85 | Matches semantic similarity of professional test data such as partial discharge and withstand voltage, filtering low-relevance redundant results |
maxToken | 4096 tokens | Adapts to long-text input of high-voltage test reports, meeting vector generation requirements for complete test data |
| Single Search Limit | Top 6 entries | Matches the number of associated records for a single product, avoiding recall of excessive historical data from non-target batches |
Three Common Misconfigurations
- Symptom: Knowledge base disk usage exceeds estimates after local deployment. Cause: Duplicate uploaded raw material batch documents are not cleaned up, and incremental storage for chunks and vectors is not enabled.
- Symptom: Search results include non-target batch insulating oil reports. Cause: Index filtering rules based on product serial numbers are not configured, and recall relies solely on global similarity.
- Symptom: Vector generation triggers timeout errors. Cause: A reasonable
maxTokenlimit is not set, causing ultra-long withstand voltage test reports to trigger model context overflow.
How to Verify Successful Configuration
- View the knowledge base storage directory, confirm that original files, chunked files, and embedded vectors are stored in separate subdirectories to verify that incremental storage configuration is active.
- Enter partial discharge test keywords for a single transformer, verify that search results only include associated test records for that product, with no cross-batch irrelevant data.
- Upload a complete high-voltage test report, view the vector generation log, confirm that no context overflow error is triggered, and the
maxTokenconfiguration meets requirements. - On the AIProxy configuration page, verify that the vector model channel is correctly bound, and embedded vectors for test data can be generated normally.
The values provided are common starting points and should be measured against the reader's own samples.