What the data for this category looks like
Technical documents for electric motor and generator manufacturing include winding process cards, insulation treatment records, dynamic balance reports, voltage withstand insulation test records, and type and factory test reports. Data sources cover electromagnetic design data, batch ledgers for copper wire and insulating materials, winding equipment operation logs, dynamic balancing machine detection data, test results from electrical test benches, and production records from ERP and MES systems. Data is updated once per unit or batch of products, and synchronized for each product type change. Documents include clear fields such as winding turns, impregnation temperature, dynamic balance deviation value, and voltage withstand value, with corresponding units attached.
What constraints these characteristics impose on the vector models and indexing link
The multi-source nature of category data requires the indexing link to support unified preprocessing across system formats, to avoid vector extraction deviations caused by differences in device logs and ledger formats. The update rhythm per unit or batch requires indexing to support incremental updates, to avoid time-consuming issues from full reconstruction. A large number of numeric fields with units are included in the documents. Vector models must adapt to the semantic association of such structured content, to ensure accurate recall of key information such as winding turns and voltage withstand values. At the same time, errors directly relate to product safety, so the recall accuracy of the index must meet the strict requirements of business verification. Reasonably set defaultToken to adapt to the vector extraction benchmark for short texts such as winding process cards, and set maxToken to adapt to complete extraction of long texts.
How to set the configurations
This configuration applies to FastGPT v4.17.0.
| Configuration Item | Recommended Value Range | Rationale |
|---|---|---|
| Chunk Length | 800–1200 characters | Matches the valid single-section content length of winding process cards and test reports, to avoid splitting key fields into different chunks |
| Overlap Length | 100–150 characters | Retains key cross-chunk context such as winding turns and voltage withstand values, to prevent loss of associated information after chunking |
| Similarity Threshold | 0.75–0.85 | Calibrated based on actual testing, filters low-relevance redundant chunks and focuses on core verification content such as dynamic balance and voltage withstand |
maxToken | 4096 | Adapts to the maximum length requirements of long texts such as voltage withstand test reports, to ensure complete extraction of core information from a single document |
defaultToken | 1024 | Adapts to the vector extraction benchmark length for short texts such as winding process cards, to ensure semantic integrity of key fields |
| Single Search Limit | Top 6 entries | Matches the typical number of test report entries for a single motor, to avoid recalling excessive irrelevant content that disrupts review |
Three Common Errors
- Phenomenon: The number of document chunks after indexing does not match the number displayed during upload, with duplicate chunks appearing. Cause: No chunking rule based on batch granularity was set for batch documents of a single motor, leading to incorrect merging or repeated splitting of process card content from adjacent batches.
- Phenomenon: Previously queryable document content cannot be recalled after upgrading FastGPT. Cause: A full index rebuild was not triggered after the upgrade, and the vector model configuration of the new version is incompatible with the vector features of the original index.
- Phenomenon: Batch adjustment of vector model training parameters and retraining cannot be performed. Cause: Only single-file upload was used to trigger training instead of the batch task function, making it impossible to uniformly adjust training parameters and rebuild indexes in batches.
How to Confirm Proper Configuration
- Upload a winding process card, check the chunk preview interface, and confirm that the chunk length falls within the preset chunk length range.
- Initiate a query for "winding turns verification", check that the number of recalled chunks matches the number of chunks split from the actual document, with no duplicate chunks.
- Upload a test report containing voltage withstand value fields, initiate a query for "voltage withstand anomaly analysis", confirm that the recalled results prioritize paragraphs related to the target values.
- Check the system logs of the indexing task, confirm there are no error prompts for duplicate indexing or chunking anomalies.
The values given are common starting points and should be measured against the reader's own samples.