This document is written for FastGPT v4.17.0.
What the data for this category looks like
Typical data for this category comes from ERP systems, WMS systems, barcode or RFID equipment, purchase work orders, storage location information, AGV transport systems, and weighing equipment. Daily documents include receiving slips, putaway slips, material picking and return slips, warehouse transfer slips, inventory checklists, lot ledgers, first-in first-out records, and expiration date records.
Data update frequencies include real-time (such as AGV positions and weighing data), daily (such as material picking and return records), and monthly inventory (such as inventory reconciliation data). Document fields include material codes, lot numbers, storage location numbers, expiration dates, quantities, inspection statuses, and other relevant fields. Units include pieces, kilograms, days, and additional standard units.
Constraints imposed on vector models and indexing by these characteristics
Multi-source heterogeneous data sources require vector models to support unified embedding of cross-system fields. This avoids embedding bias caused by data format differences.
Real-time updated material data requires indexes to support incremental synchronization. This reduces resource consumption from full index rebuilding.
Documents have many fields, including strongly correlated business fields such as lot numbers and expiration dates. Chunking and index construction must retain these key pieces of information. Otherwise, retrieval accuracy will be affected.
High-frequency verification and kitting delivery scenarios require retrieval responses to meet business timeliness requirements. This avoids process stalls caused by insufficient index efficiency.
How to set configurations
| Configuration Item | Recommended Value | Rationale |
|---|---|---|
| Chunk Length | 800–1200 characters | Matches the field density of documents for this category, prevents key information such as lot numbers and expiration dates from being split apart |
| Overlap Length | 100–150 characters | Preserves cross-chunk associated fields, ensures complete contextual information such as lot flow |
| Similarity Threshold | 0.75–0.85 | Adapts to the precise matching requirements of material codes and lot numbers, reduces mismatch probability |
| Single Search Limit | Top 10 entries | Covers multi-material matching scenarios required for work order kitting delivery |
maxToken | 8192 | Adapts to embedding processing of long documents, supports complete inventory checklist data |
defaultToken | Calibrated via actual testing | Adapts to custom vector channels, avoids errors related to token permissions |
Three common configuration errors
- A UI prompt displays "This token does not have permission to use the model", and logs show request failures. The cause is incorrect configuration of token permissions for custom vector channels, or the bound model is not enabled in the system.
- Knowledge base search returns no results or an insufficient number of results. The cause is that the chunk length is set too large, causing key fields such as lot numbers and storage location numbers to be truncated. This prevents matching with retrieval requests.
- Knowledge base search responses are slow, with latency exceeding business expectations. The cause is that incremental index updates are not enabled. Full index rebuilding consumes excessive server resources, and optimized indexes are not set up for frequently retrieved lot fields.
How to confirm correct configuration
- Upload a test receiving slip document, verify that embedded vector fields include preset key information such as lot numbers and expiration dates.
- Submit a retrieval request for a specific lot number, confirm that the similarity of returned results falls within the preset threshold range.
- Check system logs, confirm that no errors related to token permissions appear during vector model calls.
- Simulate a high-frequency retrieval scenario, verify that single retrieval response time meets timeliness requirements for the business scenario.
The values provided are common starting points, and should be evaluated against the reader's own samples.