What the data for this category looks like
Data sources include project approval documents, completion inspection reports, daily maintenance logs, road network operation scheduling logs, and roadside facility inspection archives. Update cadence falls into two categories: project approval and completion documents update as projects advance, while daily operation and maintenance documents update on a regular inspection cycle. Document structure includes structured line parameter tables, section subdivision details, as well as unstructured inspection reports and inspection photo annotation text. Fields include line mileage, design speed, maintenance frequency, facility identification numbers, and more, with units mostly kilometers, hours, times per year, and similar units.
What constraints these characteristics impose on the vector models and indexing link
First, documents mix long unstructured text paragraphs and structured parameters. Vector models must support long-context embedding to avoid truncating critical line parameters and maintenance records. Second, fields carry specific units. The embedding process must preserve unit semantics to prevent confusion between similar parameters with different units. Third, update frequencies differ significantly. Static completion documents and dynamic operation and maintenance documents require separate index update strategies to avoid resource waste from full reindexing. Fourth, a single due diligence report contains multi-section, multi-dimensional data. The index must support grouped recall by section and line dimensions to improve retrieval accuracy.
How to set configurations
| Configuration Item | Recommended Value | Rationale |
|---|---|---|
chunk_size | 800–1200 characters | Adapts to long paragraph text in railway and highway due diligence reports, avoids truncation of critical information such as line parameters and maintenance records |
embedding_model | dengcao/Qwen3-Embedding-8B:F16 | Supports long-text semantic embedding, compatible with local Ollama deployment environments, meets associated matching requirements for multiple fields |
index_type | IVF_FLAT | Adapts to index storage for million-level road network data, balances query speed and recall accuracy, meets batch retrieval needs for due diligence reports |
incremental_index_threshold | 50 new documents | Sets update trigger rules to distinguish static completion documents and dynamic operation and maintenance documents, reduces resource overhead of full indexing |
rerank_top_n | Top 10 entries | Filters redundant recall results, focuses on line and section data related to due diligence questions, improves relevance of returned content |
token_limit_per_query | 4096 tokens | Matches the context window of the embedding model, avoids truncation of semantic information in retrieval requests |
The parameter values provided on this page are common recommendations used as starting points for configuration. Actual values are affected by material form, data volume, and business rules. Specific issues require individual analysis, and it is recommended to test on your own samples before finalizing settings.
Three common mistakes
- Phenomenon: Line section information is missing from vector recall results, with insufficient retrieval accuracy. Cause: No separate embedding rules are configured for structured fields, and parameters with units are embedded together with free text, leading to semantic matching deviations.
- Phenomenon: The embedding model deployed locally on Ollama fails to start, with logs showing insufficient memory. Cause: The F16 quantized version of the model is not specified, and pulling the full parameter version directly exceeds local video memory limits.
- Phenomenon: The Milvus container fails to start, returning a PostgreSQL connection timeout error. Cause: The pg mount configuration in the official compose file is not modified, and no switch is made to in-memory indexing or external pg mode, causing container startup dependencies to be unready.
How to confirm the configuration is correct
- Upload a railway or highway completion inspection report, check that the segmented results after embedding cover all long paragraphs with no obvious content truncation.
- Initiate a due diligence-related retrieval request, check the system's token consumption statistics, confirm that consumption data can be split by application dimension for different links.
- Start the Milvus container, check that there are no PostgreSQL connection errors in the running logs, confirm that the index type configuration matches the vector database operating mode.
- Check the embedding model's running logs, confirm that the specified F16 quantized version has been loaded, with no model format or loading errors.
Question material comes from public community discussions. Configuration values are common starting points and should be measured against your own samples. Verified on 2026-09-14.