What data for this category looks like
This category of data originates from slab or billet batches, heating furnaces, mill automation control systems, online thickness and width measuring equipment, surface inspection devices, and manufacturing execution systems. Daily documents include rolling plans, heating schedules, rolling regulations, plate shape and thickness records, surface quality reports, quality certificates, and similar materials. Data updates use single coils or single billets as the update unit. Updates run in real time or are completed in batches. Core fields include batch numbers, temperature parameters, reduction amounts, thickness and width values, surface defect levels, quality certificate numbers, and similar fields. Full coil-level end-to-end traceability must be supported.
Constraints on vector models and indexes
Coil-level data updated in real time or in batches requires vector indexes to support high-frequency writes and incremental updates. This avoids performance losses from full index rebuilding. Multi-dimensional process and quality fields require embedding models to capture features from mixed structured and semi-structured data, including numerical and text values. Coil-level traceability demands that indexes associate relationships across multi-source data such as batches, equipment, and quality reports. This prevents context loss during vector retrieval. Differences in document formats require chunking logic to adapt to mixed inputs of long-text regulations and short numerical records. This ensures embedding accuracy.
Configuration Settings
All following configurations are based on FastGPT v4.17.0.
| Configuration Item | Recommended Setting | Rationale |
|---|---|---|
| Chunk Length | 800–1200 characters | Adapts to semantic integrity for long-text documents such as rolling regulations and quality certificates, while preventing single-chunk token counts from exceeding model limits |
| Overlap Length | 50–100 characters | Preserves process-related information across chunks, avoiding context breaks for coil-level traceability |
maxToken | 8192 | Matches the maximum input token limit of the Qwen3-Embedding-8B model, adapting to stitching requirements for long document chunks |
| Single Search Limit | Top 10 results | Balances retrieval efficiency and recall completeness, covering multi-source data associations for the same coil batch |
| Reranking Model | dengcao/Qwen3-Rerank | Optimized for process text in the steel rolling domain, improving ranking accuracy for similar documents |
defaultToken | 4096 | Adapts to embedding tasks for short numerical record documents, balancing computing resource consumption and semantic accuracy |
Three Common Errors
- Symptom: The number of vector retrieval results does not match expectations, or data for the target coil batch is missing. Cause: No reasonable overlap length is configured. This leads to loss of cross-chunk process-related information, preventing matching of coil-level associated semantic features.
- Symptom: Vector database deployment fails, with container startup failures and logs indicating database connection exceptions. Cause: A Milvus Compose template with PostgreSQL dependencies is incorrectly used. No minimal deployment configuration adapted to the current environment is selected.
- Symptom: Embedding model connection tests return timeouts or format errors. Cause: The API port of the locally deployed Qwen3-Embedding-8B model is not opened to the network range accessible to FastGPT. Or model call parameters are not configured correctly.
How to Verify Correct Configuration
- Import the complete ledger document for a single coil billet. Run coil-level association retrieval, and check if returned results include the batch’s rolling plan, heating schedule, and quality report.
- Adjust chunk length and overlap length parameters. Compare retrieval semantic consistency across different configurations, and confirm alignment with business scenario recall requirements.
- Check vector database write logs. Confirm that real-time or batched ledger data can be normally written to the index, with no failed write records.
- Connect locally deployed embedding and reranking models. Run test calls, and confirm that returned embedding vectors and ranking results match expected formats.
The values provided are common starting points and should be measured against the reader's own samples.