Data Overview
Data primarily comes from internal work procedure guide libraries, audio transcriptions of consultations from offline service windows, archived records from ticket systems, and policy documents issued by higher-level energy regulators. Policy documents are synchronized quarterly. Daily consultation records are added to the repository in real time. Each individual document typically includes these fields: consultation question, standard reply, handling process, required materials, responsible department, and handling time limit. Handling time limits are calculated in working days. Material lists are counted in individual items.
Constraints on Vector Models and Indexing
The spoken language in consultation records differs significantly from formal written policy documents. Adjust chunking logic for each text type. High-frequency real-time updates of tickets require support for incremental indexing. This avoids full reindexing that consumes excessive server resources. Among the multi-field structure, only the consultation question and standard reply fields need vector generation. Structured fields such as handling time limits do not need to be included in indexing. This reduces unnecessary computation. The length of individual documents varies widely. Dynamically adjust chunking thresholds. This avoids semantic fragmentation after splitting long documents, or loss of information from short documents.
Configuration Settings
| Configuration Item | Recommended Value | Rationale |
|---|---|---|
| Chunk Length | 800–1200 characters | Adapts to mixed text lengths of policy documents and consultation records, avoids semantic fragmentation |
| Overlap Length | 100–150 characters | Preserves contextual connections between chunks, supports semantic coherence of colloquial consultation content |
maxToken | 8192 token | Matches the input limit of mainstream vector models, accommodates splicing requirements after long document splitting |
defaultToken | 2048 token | Used for vector generation of short consultation questions, avoids exceeding model input limits |
| Single Index Batch | 500 entries | Balances indexing speed and server resource usage, adapts to incremental updates of real-time tickets |
| Timeout Threshold | 600 seconds | Prevents task interruptions due to timeout during long document indexing |
Three Common Misconfigurations
- Issue: CSV files that previously split chunks correctly throw errors after upgrading to FastGPT v4.17.0. Cause: The new version adjusts the field parsing rules for CSV files. The original configured field mapping relationship is not matched. This causes field reading failures during chunking.
- Issue: Knowledge base indexing tasks get stuck. Unfinished data cannot continue execution. Cause: Incremental indexing does not skip processed document IDs. Repeated submissions lead to task blocking. Or server resources are insufficient to handle the current indexing load.
- Issue: Knowledge base reference results do not call the intended vector model. Cause: The target model is not switched on the vector model configuration page. Or the configured model interface times out and fails to take effect. This causes the system to fall back to the default model.
How to Verify Proper Configuration
- A single typical document is uploaded. Chunking results are checked against expected length and overlap rules.
- An incremental indexing task is submitted. Task logs are checked for field parsing errors or timeout prompts.
- The vector model is switched. Consultation and question answering return results are tested against the new model’s generation logic.
- The indexing progress panel is viewed. Incremental update tasks for real-time tickets are confirmed to start and complete normally.
The values provided are common starting points. They should be evaluated against the reader’s own samples.
Question material comes from public community discussions. Configuration values are common starting points and should be measured against your own samples. Verified on 2026-09-28.