What this category of data looks like
This category of data comes from manufacturing enterprise MES systems, online inspection equipment, laboratory systems, process parameter libraries, customer standard documents, and historical defect libraries. Daily documents include control plans, patrol inspection records, SPC reports, final product inspection reports, nonconforming product review records, release orders, and quality monthly reports. Data update cycles are per production batch, per shift rotation, or continuous collection. Document fields include inspection item number, defect code, release threshold, batch number, detection value unit, and more. Some documents need to be associated with production work orders and material batch information.
How these characteristics impact vector models and indexes
Dispersed data sources and uneven update frequencies require indexes to support incremental synchronization and batch update mechanisms, adapting to incremental access of multi-batch data. Diverse document types include professional inspection parameters and thresholds, so support for field splitting and structured vector generation is needed to avoid information loss from unstructured text. Inspection and release decisions rely on associations between historical defects and current parameters, so the relevance and timeliness of recall results must be ensured. At the same time, the total amount of information recalled in a single round must be controlled to adapt to the decision-making rhythm of quality reviews.
How to set configurations
The following configurations are based on FastGPT v4.17.0.
| Configuration Item | Recommended Range | Rationale |
|---|---|---|
| Chunk Length | 800–1200 characters | Adapts to the information density of single blocks in documents such as control plans and inspection reports, preventing key inspection items from being split |
| Overlap Length | 50–100 characters | Retains associations of process parameters across chunks, preventing context breaks in release rules |
| Similarity Threshold | 0.75–0.85 | Matches the strictness requirements of process standards, filtering low-relevance historical defects and patrol inspection records |
| Single Search Limit | Top 10 entries | Controls the total amount of recalled results, adapting to the information processing rhythm of quality reviews |
maxToken | Set to the selected embedding model/service maximum supported input tokens | maxToken is the input-token maximum supported by the selected embedding model/service; verify it in that provider configuration. See the v4.17.2 model-field reference. It is not retrieval/LLM context or a FastGPT-wide fixed cap. |
| Reranking | Enabled | Optimizes the relevance ranking of recalled results, prioritizing content directly related to current process issues |
Three common mistakes
- Symptom: A connection refused error is returned when calling the vector model, but curl tests show normal access. Cause: Access permissions for the vector service are not correctly configured, or FastGPT connection parameters do not match the server's verification rules.
- Symptom: Duplicate inspection records still exist after index merging. Cause: Index deduplication configuration is not enabled, or the deduplication rule does not associate unique identifier fields such as batch number and inspection item number.
- Symptom: Access permissions for the vector database cannot be assigned to different roles. Cause: Multi-user permission management is not enabled, or the access scope of the corresponding role is not bound through data source configuration.
How to confirm the configuration is correct
- Upload a standard control plan document, check whether the chunked text retains complete inspection item and threshold information.
- Submit a query containing specific inspection questions, verify whether the number and sorting of recall results match the configured single search limit and reranking rules.
- Add a new historical defect record, trigger incremental indexing, then submit the relevant query again to confirm that the new data has been included in the recall scope.
- Switch the configured vector model type, verify whether the query return results match the output logic of the corresponding model.
The values given are common starting points and should be measured against the reader's own samples.