Data characteristics for this category
Oil and gas exploration and development internal office data mainly comes from exploration design reports, real-time drilling logs, trial production monitoring data, internal office process documents, and employee training materials. The data update rhythm varies significantly: drilling logs are updated daily or in real time, exploration reports are updated in stages, and office documents are mostly updated quarterly or annually. Document structures include long text paragraphs, structured tables with physical units such as well depth and formation pressure, and attached drawings. Core fields include well numbers, operation time, parameter values, and other key information.
Constraints imposed on vector models and indexing
The mixed characteristics of long text reports and structured logs require vector models and index configurations to balance the needs of long text chunking and structured field indexing. Data with different update rhythms requires flexible switching between incremental indexing and full indexing, to avoid frequent full index reconstruction. Fields with physical units require indexes to retain unit information, preventing matching deviations during semantic retrieval caused by missing units. Large-scale exploration data requires index configurations to adapt to efficient sharding and recall logic, balancing retrieval efficiency and result coverage.
Configuration settings
| Configuration Item | Recommended Approach | Basis for This Approach |
|---|---|---|
| Chunk Length | 800–1200 characters | Adapts to the typical paragraph length of oil and gas exploration reports, avoids exceeding the maxToken limit |
| Overlap Length | 100–200 characters | Retains contextual association between chunks, prevents information breakage of key exploration parameters |
| Similarity Threshold | 0.75–0.85 | Filters low-relevance log fragments, improves retrieval accuracy for internal office scenarios |
| Single Search Limit | Top 10 entries | Balances retrieval efficiency and result coverage, meets the conventional retrieval needs of internal employees |
maxToken | 8192 | Adapts to the token limit of most mainstream embedding models, such as the supported range of Alibaba text-embedding-v3 |
defaultToken | Calibrated based on actual measurements | Adapts to the general token requirements of internal office documents, compatible with process documents of different lengths |
Three common configuration errors
- After replacing the embedding model, abnormal errors occur in old knowledge base retrieval, or returned results do not meet expectations. The cause is that vector spaces generated by different embedding models differ, and vectors from the old index cannot adapt to the vector space of the new model. Indexes must be regenerated.
- The sorting of knowledge base search results does not follow semantic similarity, and the priority does not meet expectations. The cause is that reranking configuration is not enabled, or the reranking trigger conditions are set incorrectly, leading to retrieval results that have not undergone semantic reranking.
- When selecting an embedding model other than ada 002, the interface returns the error prompt "undefined model must match ^(text". The cause is that the model's call identifier is not filled in correctly, or the API call permission for the corresponding model is not activated.
How to verify successful configuration
- Upload a typical drilling log fragment, check whether the chunked text length falls within the configured chunk length range.
- After replacing the embedding model, select an existing knowledge base and perform a reindex operation, confirm that index generation is successful and there are no error logs.
- Perform a semantic retrieval, check whether the sorting of results matches semantic similarity, confirm that the reranking configuration is effective.
- View the detailed index logs, confirm that the unit information of structured fields has been correctly retained and no loss has occurred.
- Confirm that the currently used FastGPT version is v4.17.0 to ensure compatibility of configuration items.
The values provided are common starting points and should be measured against the reader's own samples.
Question material comes from public community discussions. Configuration values are common starting points and should be measured against your own samples. Verified on 2026-09-28.