What data looks like for this category
Typical entities include hot rolling, cold rolling, section steel, and steel pipe enterprises. Daily documents cover rolling plans, heating schedules, rolling specifications, plate shape and thickness records, surface quality reports, and quality certificates. Data sources include slab or billet batches, heating furnaces, mill automatic control systems, online thickness and width measurement tools, surface inspection systems, and MES systems. Update frequency is per coil or per piece, real-time. Data fields include batch number, heating temperature, rolling speed, thickness deviation, surface defect level, quality certificate number, and others. Units include degrees Celsius, meters per second, millimeters, square meters, and others. Coil-level traceability features are prominent, and single records often link to continuous data across multiple processes.
What constraints do these characteristics impose on knowledge base retrieval and recall
The real-time, coil-level update requirement means the knowledge base must support incremental synchronization and precise batch association, to avoid mixing data across batches. The multi-process linked document structure means retrieval must cover both exact fields such as batch number and semantic descriptions such as rolling specifications. A single retrieval mode cannot meet the demand. The large data volume per coil and frequent updates require limiting the number of results and context length per single retrieval, to avoid exceeding model processing limits. Some documents such as surface quality reports and quality certificates have long text lengths, so the system must adapt to long text chunking and recall logic.
Configuration settings
| Configuration Item | Recommended Setting | Rationale |
|---|---|---|
similarity | 0.75–0.85 | Matches professional documents such as rolling specifications and thickness records, ensuring recall accuracy while covering associated data from the same batch |
limit | Top 10 entries | The number of documents linked to a single coil ledger is large, limiting the number of entries avoids context splicing exceeding the model's capacity limit |
searchMode | embedding | Use semantic/vector retrieval. Exact-field, metadata, batch, and document-type constraints require separate filters and are not enforced by this mode. |
usingReRank | Enabled | Reorders initially recalled results by relevance, filtering irrelevant surface quality reports or non-same-batch data |
datasetSearchUsingExtensionQuery | Enabled | Automatically expands retrieval terms related to batch numbers, linking full process data including heating, rolling, and inspection for the same batch |
collectionIds | Knowledge base groups grouped by rolling process | Differentiates documents from different processes such as hot rolling, cold rolling, and section steel, narrowing the retrieval scope and improving recall efficiency |
Three common errors
- Issue: After setting the single text chunk limit to 1500 characters, rolling ledger document chunks exceeding this length are still recalled and included in context references. Cause: Chunking strategy is not configured correctly, or retrieval does not filter by chunk size, resulting in ultra-long chunks being directly included in context splicing.
- Issue: When importing multi-batch ledger data in bulk, the system freezes, and the Ollama log shows "try reducing the size of the batch". Cause: The single batch data volume for bulk import is too large, and bulk processing parameters are not adjusted, resulting in memory overflow.
- Issue: After passing knowledge base retrieval results to an HTTP interface for processing, AI conversations cannot correctly reference the retrieved document content. Cause: Core fields such as retrieval result document ID, original text fragment, and similarity score are not passed, and the format does not meet requirements.
How to confirm the configuration is correct
- Upload the complete rolling specification document for a single coil, enter the corresponding batch number for retrieval, and verify that associated documents such as heating schedules and plate shape and thickness records from the same batch can be recalled.
- Check the retrieval log to confirm that the number of recalled results does not exceed the configured
limitvalue, and the reordered results meet expected relevance standards. - Trigger a bulk import of ledger data for 10 or more coils, check whether bulk processing freezes occur, and confirm that bulk parameters adapt to the current data scale.
- Upload a surface quality report with a length exceeding 2000 characters, verify that retrieval can correctly chunk the content and include it in context splicing, without truncation errors.
The provided configuration values are common starting points and should be evaluated against your own specific samples. This configuration applies to FastGPT v4.17.0.