What the data for this category looks like
Internal data for refined oil sales comes from the fuel card management system, gas station inventory ledger, employee scheduling system, customer service ticket platform, gas station inspection system, and internal office document library. Update frequencies vary: fuel card top-up and inventory data are updated daily, tickets and inspection reports are submitted in real time, and office operation manuals are updated quarterly. Documents include two categories: structured and unstructured. Structured data has fields such as fuel grade, fuel volume (liters), and top-up amount (yuan). Unstructured data includes pure text content such as ticket descriptions, inspection records, and operation guides.
What constraints do these features impose on vector models and indexing?
The mixed data structure requires vector models to adapt to both structured fields and long text passages. Differentiated chunking rules must be configured for different data types. Differences in update frequency require support for switching between incremental indexing and full indexing, to avoid re-vectorizing static data. Some fields have fixed units. Unit information must be retained during the preprocessing stage to ensure semantic consistency during vector matching. A large number of duplicate ticket templates and inventory entries require deduplication rules to reduce invalid vector storage and retrieval overhead.
How to set the configurations
| Configuration Item | Recommended Setting | Rationale |
|---|---|---|
| Chunk Length | 800–1200 characters | Matches the typical paragraph length of documents such as gas station inspection reports and ticket descriptions, avoids overly fragmented single-chunk content |
| Overlap Length | 100–200 characters | Covers cross-chunk context, such as ticket problem descriptions and solution steps |
| Similarity Threshold | 0.75–0.85 | Filters low-match redundant data, ensures semantic relevance between returned results and employee queries |
| Single Search Limit | Top 10 entries | Adapts to internal office information display habits, avoids excessive results interfering with quick retrieval |
| Default Token Count | 4000 | Adapts to the context length of long documents such as gas station annual operation manuals, meets vector matching requirements for complete paragraphs |
| Maximum Token Count | 8000 | Controls the total token volume of a single recall, avoids exceeding the model processing limit of FastGPT v4.17.0 |
Three common misconfigurations
- Phenomenon: The "undefined model must match" error occurs when using a vector model other than ada-002. Cause: The corresponding model's API key and interface address were not configured in FastGPT's vector model settings. Only the model name was selected, and the link was not fully established.
- Phenomenon: Search result ranking does not meet expectations. High-similarity inventory or ticket results do not appear at the top. Cause: Special field weights were not configured for structured data from refined oil sales. The general ranking logic cannot adapt to the priority requirements of the business scenario.
- Phenomenon: Auxiliary data is not correctly recalled and does not appear in search results. Cause: The vectorization switch for auxiliary data was not enabled, or the format of the auxiliary data does not meet the platform's plain text requirements.
How to confirm the configuration is complete
- Upload a typical refined oil sales internal document, such as a gas station inspection report, and check whether the chunked content matches the preset chunk length and overlap length.
- Initiate a query targeting inventory data, such as "92# gasoline current daily stock", and check whether the returned results are ranked in line with business expectations, with no obvious irrelevant content.
- Upload a piece of auxiliary data, such as employee attendance regulations, initiate a relevant query, and confirm that the auxiliary data is correctly recalled to the result list.
- View FastGPT's vector model call logs, confirm that the configured model parameters and interface links are normal, with no errors returned.
The values provided are common starting points and should be measured against the reader's own samples.
Question material comes from public community discussions. Configuration values are common starting points and should be measured against your own samples. Verified on 2026-09-28.