What this category of data looks like
This category of data typically comes from manufacturing enterprise PMC, planning, shop scheduling, and material control departments. Daily covered documents include sales orders, master production schedules, material requirement plans, weekly and daily production schedules, work orders, material shortage lists, and delivery change orders. Data sources cover ERP orders and inventory, BOM and process routes, equipment capacity, staff shifts, work-in-progress, and supplier delivery information. Update cycles are daily, weekly, or triggered by orders or abnormal events. Data fields include order numbers, material codes, capacity values, scheduling time periods, delivery dates, and more. There are no unified fixed units, and values adjust based on document type.
What constraints do these characteristics impose on the vector models and indexes link
Data for this category is scattered across multiple sources with uneven update cycles. This requires vector indexes to support flexible configuration for incremental synchronization and on-demand rebuilding. Diverse document types and significant field differences demand chunking rules adapted to different document structures, to avoid breaking business association logic. Data has high relevance: for example, orders correspond to work orders, and schedules are bound to capacity information. Vector retrieval must retain field association relationships to prevent results from being disconnected from business context. Additionally, data volume fluctuates with orders and abnormal events, so dynamically adjusted index storage thresholds are needed to handle sudden read and write pressure.
How to set configurations
The following configurations are adapted for FastGPT v4.17.0.
| Configuration Item | Recommended Setting | Rationale |
|---|---|---|
| Chunk Length | 800–1200 characters | Adapt to the paragraph length of documents such as work orders and production schedules, and retain complete business semantics |
| Overlap Length | 100–200 characters | Prevent chunking from breaking cross-paragraph business associations, such as the link between orders and their corresponding processes |
| Similarity Threshold | 0.72–0.85 | Filter low-relevance results, meet the strict requirements of production planning data, and calibrate based on actual testing |
| Single Recall Limit | Top 10 entries | Control the scale of retrieval results to avoid exceeding context length limits |
maxToken | 8000 token | Match the total length of production planning documents, ensuring all associated information is included in the context |
defaultToken | 1024 token | Match the default length for basic semantic vector generation, ensuring vector integrity for core business fields |
| Maximum Context Length | 4000–6000 characters | Balance retrieval result richness and model processing capacity |
Three common configuration errors
- Upgrading to a new version and failing to complete vector database migration, with a prompt that the old index format cannot be recognized. This occurs because existing data was not migrated per the new version's incremental synchronization rules, and a full direct replacement caused format incompatibility.
- Attempting to rebuild all knowledge base indexes, which exhausts server read and write resources. This happens because no batch rebuilding threshold was set, and the full index task simultaneously occupies excessive memory and disk IO, exceeding the server's carrying limit.
- Calling the index creation interface and receiving a 400 status code with no response body. This is caused by incorrectly configuring vector model parameters associated with the index, leading to a request body format that does not meet interface requirements.
How to confirm the configuration is complete
- Vector index chunking logs are reviewed to confirm that chunk length and overlap length match the preset configuration, with no abnormally truncated business fields.
- A test retrieval is run, the number of recalled results and similarity matching degree are checked, and the threshold is adjusted to meet business strictness requirements.
- Index update task trigger rules are reviewed, confirming that incremental updates can be triggered for scenarios such as orders and abnormal events, and full updates are only executed when necessary.
- Server resource monitoring panels are checked to confirm that memory and disk IO usage during index task execution stays within a reasonable range.
The values provided are common starting points and should be measured against the reader's own samples.