What data in this category looks like
Consulting and reply data for refining production mainly originates from production process specifications, equipment maintenance logs, post-shift inspection records, raw material quality inspection reports, safety and compliance documents, and similar sources. Update rhythms vary significantly: post-shift inspection records are updated per shift, equipment maintenance logs are updated with equipment overhaul cycles, and process specifications and compliance documents are revised quarterly or per policy requirements. Single documents contain fields such as equipment number, operating parameters, abnormal conditions, and corrective measures. Units include MPa, ℃, m³/h, batch number, runtime and other industrial standard units. Some long documents can reach tens of thousands of characters.
What constraints do these characteristics impose on vector models and indexing
The mixed, multi-type characteristics of refining production data impose multiple constraints on the vector model and indexing workflow. Mixed numeric fields with units and specialized industrial terms require vector models to adapt to industrial scenario semantics, to avoid misinterpreting specialized terms. Wide variation in data length requires indexes to support flexible chunking configurations, to accommodate vector generation for short shift inspection records and long process specifications. Incremental update requirements demand indexes support incremental index construction, to adapt to shift-updated inspection log data. QA-split document collections require indexes to optimize retrieval efficiency for QA pairs, to meet precise recall needs for consulting and reply services. Bulk-imported long documents increase index construction load, so configurations need adjustment to balance efficiency and recall accuracy.
How to set configurations
| Configuration Item | Recommended Setting | Rationale |
|---|---|---|
| Chunk Length | 800–1200 characters | Accommodates mixed lengths of long refining process specifications and short shift inspection records, avoids including unrelated cross-process information in a single chunk |
| Overlap Length | 50–100 characters | Preserves cross-chunk process association information, such as continuous operating steps for equipment start and stop |
| Similarity Threshold | 0.72–0.85, calibrated via actual testing | Adapts to semantic similarity of industrial terms, avoids false recalls or insufficient recall |
maxToken | 8192 token | Matches context limits of mainstream open-source embedding models, supports vector generation for long documents |
| Reranking | Enabled, top 3 results | Filters low-relevance shift inspection records, focuses on matching content for core processes and consulting services |
defaultToken | 1024 token | Unifies token quota for basic vector generation, balances vector quality per entry and indexing efficiency |
Three common misconfigurations
- Phenomenon: A 504 timeout status code is returned when calling the knowledge base interface to generate an index. Cause: The
maxTokenparameter was not adjusted for long refining process documents, causing single-entry vector generation time to exceed interface limits. - Phenomenon: A large number of empty field records with no matching business scenario appear in retrieval results. Cause: The similarity threshold was set too low, incorrectly recalling non-target data with weak semantic association.
- Phenomenon: Retrieval response time exceeds expectations for QA-split document collections created via OpenAPI. Cause: Chunk length was set too large, and reranking configuration was not enabled, leading to excessive redundant data recalled by the index.
How to confirm configurations are correctly set
- Upload a typical refining process specification document, check token consumption in FastGPT v4.17.0's vector generation log, confirm the
maxTokenvalue matches the configured setting. - Input a standard question about equipment abnormalities, verify that the number of retrieval results and the top 3 reranked records match business expectations.
- Call the OpenAPI interface to batch import 100 QA-split shift inspection records, check whether the index construction progress update speed meets expectations.
- Compare recall results from different embedding models, confirm that semantic matching accuracy for industrial terms meets business scenario requirements.
The values listed above are common starting points, and should be evaluated against the user's own sample data.
Question material comes from public community discussions. Configuration values are common starting points and should be measured against your own samples. Verified on 2026-09-28.