What this category's data looks like
Refinery and chemical production document data comes from unit operation procedures, maintenance records, quality inspection reports, production daily reports, material balance sheets, and safety compliance documents. Data updates follow two rhythms: daily production reports update daily, maintenance records update when events trigger, and compliance documents update in batches quarterly or annually. Document structures include long-text process descriptions and structured tables. Structured fields include unit ID, production shift, test values, and similar items. Supporting units include tons, cubic meters, degrees Celsius, pascals, and other professional industrial units.
What constraints these characteristics impose on the vector model and indexing link
Long-text process descriptions make up a large share of data. Splitting must avoid breaking the complete logic of individual process steps, otherwise vector retrieval loses key process associations. Special structured fields and professional units require indexes to support both semantic vector matching and exact field matching, to prevent mismatches between equipment numbers and process descriptions. Differentiated update frequencies require indexes to support incremental updates for daily new production data, while retaining full update capabilities for batch imports of compliance documents.
Configuration settings
| Configuration Item | Recommended Setting | Rationale |
|---|---|---|
| Chunk length | 800–1200 characters | Refinery and chemical production documents mostly include continuous process step descriptions. This length preserves complete individual process logic and avoids splitting that breaks process associations |
| Overlap length | 50–100 characters | Compensates for context gaps after long-text splitting, ensuring vector indexes cover front and rear connections of process steps |
| Similarity threshold | 0.72–0.85 | Process terms in refinery and chemical scenarios are highly specialized. This range filters low-relevance results and avoids mismatching procedures for non-target equipment |
| Single search limit | Top 6 results | Matches information needs for refinery and chemical production documents, avoiding excessive redundant results that interfere with drafting logic |
maxToken | 4000 | Adapts to context recall for long documents, ensuring complete process parameters and compliance requirements can be called during drafting |
defaultToken | 1500 | Adapts to vector encoding length of professional refinery and chemical terms, ensuring accurate term matching |
| Deduplication switch | Disable per scenario | For maintenance records with repeated structured blocks, disabling automatic deduplication preserves index order from custom splitting |
Three common mistakes
- Phenomenon: After custom-split documents are stored in the knowledge base, repeated blocks are automatically removed, causing index order to not match pre-planned splitting sequences. Cause: Default index configuration enables automatic deduplication, and deduplication rules are not adjusted for maintenance records with repeated structured blocks in refinery and chemical production.
- Phenomenon: After switching vector models, search returns similarity values outside the 0-1 range, and valid results cannot be filtered using interface filtering settings. Cause: Similarity normalization configuration for the vector model is not enabled, resulting in uncalibrated raw score output.
- Phenomenon: When batch importing long refinery and chemical production documents, the system prompts index construction timeout and cannot complete the import. Cause: Incremental index rules are not configured, full index reconstruction runs for each update, and no incremental processing is applied to daily updated production reports.
How to confirm configuration is correct
- Upload a single refinery and chemical production operation procedure, check that the number of split chunks matches custom splitting rules, and confirm no abnormal merges or splits occur.
- Initiate a search for a specific equipment number, check that recalled results include process documents for the corresponding equipment, and confirm field matching and vector retrieval work together.
- View index construction logs in FastGPT v4.17.0, confirm incremental updates only process newly added production daily report documents, and no full reconstruction records exist.
- Test the similarity filtering function, adjust the threshold and observe changes in returned result counts, confirm configuration parameters take effect normally.
The values provided are common starting points and should be measured against the reader's own samples.
Question material comes from public community discussions. Configuration values are common starting points and should be measured against your own samples. Verified on 2026-09-28.