What the data for this category looks like
Data sources for gas industry investment research knowledge bases include gas enterprise operation reports, pipeline inspection logs, upstream gas source quotes, industry policy documents, and professional association reports. Update frequencies vary significantly. Gas source quotes are updated daily. Pipeline inspection logs are submitted in real time alongside inspection tasks. Industry policies and association reports are released on an irregular schedule.
Documents fall into two structural categories. Structured reports contain fields such as total gas supply, pipeline pressure, and revenue breakdowns. Units include cubic meters, kilopascals, and yuan. Unstructured documents are mostly inspection records and policy interpretation texts. Some include free-form on-site descriptions. Document lengths vary widely. Short entries like quotes may be only a few hundred characters. Full policy documents can span tens of thousands of characters.
What constraints do these characteristics impose on vector models and indexing?
Data sources with varied update frequencies require indexes that support both incremental building and full reconstruction. This prevents excessive computational resource consumption from full reconstruction operations.
Structured data contains many numerical fields paired with units. Vector models must support mixed semantic representation of numerical and text data. Directly embedding raw numerical values will cause loss of semantic associations.
Wide variation in document lengths creates specific requirements. Short texts must not be too short to convey complete semantics. Long texts must be split appropriately to avoid separating critical information. This sets clear requirements for split parameter configuration.
Mixed retrieval across multiple data source types requires index structures that balance precise short-text recall and long-text semantic matching.
How to configure the settings
| Configuration Item | Recommended Setting | Rationale |
|---|---|---|
embedding_model | m3e-base | Supports embedding of text and structured numerical descriptions in Chinese scenarios, and adapts to unit-based field conversion requirements in gas industry data |
chunk_size | 800–1200 characters | Adapts to the typical length of gas pipeline inspection logs. Prevents excessive single-segment text from impacting embedding accuracy, while ensuring complete semantics per segment |
incremental_index | Enabled | Adapts to the high-frequency update requirements of gas source quotes and inspection logs. Only builds indexes for new data to reduce redundant computation |
recall_top_k | Top 10–15 results | Balances the need for multiple types of information (policies, operations, quotes) required for gas industry investment research, while balancing recall scope and context length limits |
similarity_threshold | 0.72–0.78 | Filters low-similarity irrelevant recall results, while retaining semantic associations for similar gas operation data |
PARSE_FILE_TIMEOUT_SECONDS | 300 seconds | Adapts to the parsing and embedding time required for large industry policy documents, preventing timeout during long document parsing |
The parameter values provided on this page are general recommendations for establishing initial configuration baselines. Actual values are affected by material format, data volume, and business rules. Specific issues require targeted analysis. It is recommended to test on your own samples before finalizing settings.
Three common configuration errors
- Symptom: The vector database returns excessively slow results when querying identical gas operation data repeatedly. Cause: No vector recall caching mechanism is configured. Every query re-runs embedding and index retrieval.
- Symptom: No corresponding index entry is generated in the knowledge base after calling the "Create Training Order" API. Cause: Vector database connection parameters are not configured correctly. The training order only completes text parsing without triggering the index building process.
- Symptom: Significant semantic deviation occurs in embedded gas pressure numerical fields. Cause: Raw numerical fields are not converted to text descriptions with units. Directly embedding raw values prevents the model from recognizing semantic associations.
How to confirm proper configuration
- Upload a gas pipeline inspection log document that includes pipeline pressure values. Review the parsed segment results. Confirm that segment lengths fall within the configured
chunk_sizerange. - Submit two identical queries for gas operation data. Compare the time taken for both queries. Confirm that the caching mechanism is active.
- Call the "Create Training Order" API to upload a gas source quote document. Check the knowledge base index list for the corresponding entry. Confirm that the index building process works correctly.
- Enter a search query including "total monthly gas supply". Review the recall results for numerical descriptions with units. Confirm that the embedding model correctly processes structured fields.
Question material comes from public community discussions. Configuration values are common starting points and should be measured against your own samples. Verified on 2026-09-14.