What the Data for This Category Looks Like
Data for this category comes from slab or billet batches, heating furnaces, mill automation control systems, online thickness and width measuring equipment, surface inspection devices, and MES systems. Daily documents include rolling plans, heating schedules, rolling specifications, shape and thickness records, surface quality reports, and quality certificates. Data updates are performed per coil or per piece. Some operating condition data is synced in real time. Fields include batch number, coil number, thickness (millimeters), surface defect level, temperature parameters, and others.
Constraints on Vector Models and Indexes
The high-frequency, coil-level update rhythm requires vector indexes to support incremental synchronization. This avoids computational resource usage caused by full index reconstruction. Multi-source heterogeneous data sources create differences in field formats. Unified alignment must be completed before vectorization to ensure consistent input for embedding models. The coil-level traceability business requirement demands that each vector segment be bound to clear identifying fields such as batch number and coil number. This enables accurate matching of audit results in subsequent steps. The length span of different documents varies widely. Long documents like rolling specifications and short documents like surface quality reports require flexible chunking rules. Strict quality audit judgment requirements also demand sufficient relevance from vector recall results. This prevents misjudgments that lead to steel coil downgrading or equipment damage.
Configuration Settings
This document is written based on the configuration logic of FastGPT v4.17.0.
| Configuration Item | Recommended Setting Range | Rationale |
|---|---|---|
| Chunk Length | 800–1200 characters | Adapts to the content density of long documents such as rolling specifications and quality certificates, avoiding information overload in a single chunk |
| Overlap Length | 100–200 characters | Retains contextual association between adjacent chunks, ensuring information coherence for short documents such as shape and thickness records |
| Similarity Threshold | 0.75–0.85 | Matches the strict judgment standards of steel quality inspection, filtering low-correlation historical audit data |
maxToken | 8192 token | Adapts to the input upper limit of the text-embedding-3-large model, avoiding truncation of key parameters during embedding |
| Single Recall Limit | Top 20 entries | Covers multi-dimensional audit records for a single steel batch, meeting the query requirements for process quality traceability |
defaultToken | Calibrated based on actual measurements | Adapts to token counting rules for different environments, ensuring consistency between chunking and embedding |
Three Common Misconfigurations
- Phenomenon: Indexes and data entries in the dataset automatically increase without manual operation. Cause: No deduplication rules for incremental synchronization are configured, causing repeated coil-level data to be embedded repeatedly and generate new indexes.
- Phenomenon: The service freezes when calling text-embedding-3-large, and does not recover after restarting with the model commented out. Cause: Configuration related to embedding model request timeouts is not set correctly, and residual unfinished task queues are not cleared, causing service blocking.
- Phenomenon: The associated language model cannot be called after creating an index model. Cause: The call link configuration between the index and the language model is not correctly bound, causing the system to fail to match the call parameters and permissions of the corresponding model.
How to Verify Proper Configuration
- Manually upload a rolling plan document, check that the number of generated vector chunks matches the set chunk length.
- Trigger an incremental synchronization task, verify that the number of newly added coil-level data indexes matches the actually uploaded data.
- Initiate a surface quality audit query, confirm that the returned result relevance conforms to the expected judgment logic.
- View system operation logs, confirm that no format errors or timeout exceptions occur in embedding model call requests.
The values provided are common starting points and should be measured against the reader's own samples.