What this type of data looks like
Steel trade investment research data mainly comes from spot quotation platforms, factory ex-factory price ledgers, port inventory daily reports, industry association research reports, and trader purchase-sales-stock statements. Data update rhythms vary: spot transaction and port inventory data updates daily, industry research report data updates weekly or monthly, and internal enterprise purchase-sales-stock data syncs daily.
There are two types of document structures:
- Structured Excel ledgers, with headers including fields such as product name, specification model, origin, transaction price, inventory volume, transportation cost. Units are mostly yuan/ton, ten thousand tons, kilometers.
- Semi-structured industry analysis documents, with chapter divisions and data tables.
What constraints do these characteristics impose on the knowledge base retrieval and recall link
The multi-field attributes of structured ledgers require retrieval to prioritize matching core fields such as product name and specification, to avoid irrelevant results from full-text fuzzy retrieval. Data with different update frequencies needs to support incremental synchronization, otherwise real-time data such as recalled spot prices will become outdated. Product name text with specifications must retain complete specification information. Do not break the "product name + specification" association when splitting documents. Cross-document associated data such as transportation cost and port inventory needs to recall multiple associated texts. Only recalling single isolated data cannot cover associated requirements.
How to set the configurations
| Configuration Item | Recommended Value Range | Rationale |
|---|---|---|
chunk_size | 800–1200 characters | Steel trade data mostly consists of long-text transaction records with specifications and research report paragraphs. This range retains the association between specifications and context, avoiding loss of key information after splitting |
top_k | Top 8–12 results | Steel trade data has many fields and high correlation requirements. Too many recalled results increase context pressure, too few results miss associated data |
similarity_threshold | 0.72–0.85 | For product name retrieval with specifications, filter low-match irrelevant category data. This range balances recall accuracy and coverage |
rerank_top_k | Top 3–5 results | Focus on core transaction data and research report conclusions, avoiding redundant results interfering with investment research judgments |
enable_incremental_sync | Enabled | Spot data updates daily. Incremental sync reduces resource consumption of full reconstruction and ensures the timeliness of retrieved data |
field_matching_weight | Calibrated based on actual testing | Structured transaction data prioritizes matching product name and specification fields. This weight needs adjustment based on actual retrieval scenarios |
The parameter values provided on this page are all conventional recommendations used as a starting point for configuration. Actual values are affected by material form, data volume and business rules. Specific issues require specific analysis. It is recommended to test on your own samples before finalizing the configuration.
Three common mistakes
- When deploying version 4.9.12, retrieved results include the phrase "Reference Mark: [1]". Cause: The reference mark output switch for knowledge base retrieval results is not disabled, causing the large model call to carry the serial number mark of the retrieval source.
- The number of recalled results from knowledge base retrieval is far lower than expected. Cause: The set
similarity_thresholdis too high, filtering some qualified low-match associated data, or the retrieval term is not split by specification leading to insufficient match degree. - The text block of a single retrieval result is too large, causing increased retrieval time and context loading pressure. Cause: The
chunk_sizeparameter is not adjusted, the default value is directly used to split long-text transaction records of steel trade, and no splitting is done according to field granularity, leading to redundant single-segment content.
How to confirm the configuration is correct
- Upload a single steel trade transaction data entry with clear specifications, perform retrieval matching that specification, and verify whether the recalled results include this data.
- After enabling the incremental sync function, add one new latest industry data entry, perform retrieval and confirm that the new data is recalled.
- Check the format of the retrieved results, confirm that the reference mark can be disabled via the corresponding configuration item, avoiding output carrying irrelevant identifiers.
- Adjust the segment length parameter, compare the integrity of retrieval results and loading time across different values, and confirm that the value is adapted to the current data structure.
Question material comes from public community discussions. Configuration values are common starting points and should be measured against your own samples. Verified on 2026-09-14.