What This Category of Data Looks Like
Data sources for this category include hot metal and scrap steel data, L2 process control system, laboratory test results, manufacturing execution system (MES), energy medium system, and equipment condition monitoring system. Daily documents cover heat plans, batching lists, converter smelting records, composition test reports, continuous casting process records, and slab judgment sheets. Data update frequencies follow per heat, real-time, and per shift cycles. Fields include heat number, hot metal temperature, smelting duration, continuous casting pulling speed, slab defect level, and others, with corresponding units: number, degrees Celsius, minutes, meters per minute, level, and others.
What Constraints These Characteristics Impose on the Vector Model and Index Link
The high-frequency update and strong coupling characteristics of this category's data impose multiple constraints on the vector model and index link. First, the per-heat and real-time update requirements demand that indexes support incremental writing, avoiding business process blockage caused by full reconstruction. Second, multi-source heterogeneous document structures and field characteristics require indexes to adapt to mixed retrieval needs for structured numerical values and process records, while supporting cross-heat and cross-process association matching. Third, strict timeliness requirements under high-temperature continuous processes mandate that index query delay matches the business response window, avoiding impacts to quality traceability efficiency.
Configuration Settings
| Configuration Item | Recommended Value | Rationale |
|---|---|---|
| Chunk Length | 800–1200 characters | Matches the average text length of single heat smelting records, avoids splitting cross-process information within a single chunk |
| Overlap Length | 100–150 characters | Ensures cross-chunk association between heat plans and subsequent process records is not lost |
| Similarity Threshold | Calibrated via actual measurements | Adapts to semantic matching precision requirements for composition test values and defect records |
| Maximum Search Results per Query | Top 20 entries | Balances recall scope and query performance for multi-heat data retrieval |
| Maximum Context | 6000 characters | Adapts to the default context capacity limit of FastGPT v4.17.0, avoids exceeding the model's processing range |
defaultToken | Set from representative sample documents | defaultToken is the default token count for direct text chunking; tune it on representative source documents. See the v4.17.2 model-field reference. It is not a vector dimension, embedding-model input cap, or retrieval/LLM context. |
Three Common Misconfigurations
- Symptom: Calling the FastGPT API fails to retrieve the original content of chunked indexes, with returned fields empty. Cause: The original content storage switch for chunked indexes was not enabled in the backend, causing the index to only save vectors without retaining original text fragments.
- Symptom: Multiple index entries are generated for the same document chunk, resulting in duplicate results during retrieval. Cause: No unified chunk boundary rules were set, causing overlapping parts of heat plans and batching lists to be chunked repeatedly.
- Symptom: Document indexing takes more than 30 minutes, with updates unable to complete for an extended period. Cause: Incremental indexing was not configured for real-time heat data, and full scans of all historical data were performed for each indexing operation, exceeding the business-allowed update window.
How to Verify Proper Configuration
- Manually trigger an indexing task for a single heat record, confirm whether the index status in the interface completes within the preset time.
- Submit a search query containing a specific heat number, verify whether the returned results include all associated document chunks for that heat.
- Import test slab judgment sheets and continuous casting process records, verify that the similarity ranking of search results conforms to business logic.
- Call the chunked index API interface, confirm that the returned results include original text fragments and vector information.
The values provided here are common starting points, and should be evaluated against the reader's own samples.