What the data for this category looks like
Survey and design documents and materials mainly come from on-site project survey data, internal design documents of design institutes, change negotiation records, construction drawing approval documents, and similar sources. Update rhythm is triggered by project progress and technical changes, with no fixed cycle. Updates are made after each project milestone or adjustment. Document structure includes three parts: project metadata, professional technical content, and drawing index. Fields include project number, survey point coordinates, design load value, drawing version number, and others. Some documents include technical parameters with units.
What constraints these characteristics impose on the vector models and indexes link
Survey and design documents contain both structured parameters and unstructured descriptions. This requires vector chunking to retain field boundaries, and avoid breaking the association between professional terms and technical parameters. Document lengths vary widely, from a few pages of negotiation records to dozens of pages of general design descriptions. A flexible chunking strategy is needed to accommodate this range. Technical parameters with units require the vector model to adapt to professional technical text semantics. Non-fixed update frequency requires the index to support incremental synchronization, to avoid resource consumption caused by full reconstruction.
How to set configurations
Reference only; verify against the target FastGPT release and deployment documentation.
| Configuration Item | Recommended Value Range | Rationale |
|---|---|---|
| Chunk Length | 800–1200 characters | Adapts to the varying lengths of survey and design documents, balancing semantic completeness and index granularity |
| Overlap Length | 100–150 characters | Prevents chunking from cutting the contextual association of professional terms and technical parameters |
| Similarity Threshold | 0.72–0.85 | Distinguishes similar survey point descriptions and design parameters of different projects, calibrated based on actual testing |
| Single Search Limit | Top 8 results | Professional relevance of survey and design documents is high, and a small number of results can cover core query needs |
defaultToken | 4096 | Adapts to the input token limit of most open-source embedding models, matching conventional chunk lengths |
maxToken | 8192 | Handles general design description documents with large amounts of drawing annotations, avoiding truncation of key technical content |
Three common mistakes
- Symptom: The large language model chat function operates normally, but vector index tasks stall with no log updates. Cause:
maxTokenis not configured correctly, causing chunked text to exceed the input limit of the embedding model, blocking requests. - Symptom: Irrelevant survey project documents appear in search results, core fields are empty or parameter values do not match. Cause: Field boundaries are not retained during chunking, leading to vector semantic confusion and failure to accurately associate professional content.
- Symptom: Only partially updated content is included in incremental indexing, and old content is not correctly overwritten. Cause: Incremental synchronization is not triggered based on project changes, and full indexing mode is still used, causing update logic to fail.
How to confirm configurations are correct
- Upload a single survey negotiation record of around 1000 characters, check if the chunk preview retains core fields such as project number and change content, with no truncation.
- Initiate a professional query containing "survey data of a certain point", check if the similarity scores of returned results fall within the set threshold range, and no irrelevant documents are mixed in.
- Submit a new change document to trigger incremental indexing, wait 10 minutes before querying again, confirm that the new content has been correctly included.
- Check the configuration values of
defaultTokenandmaxTokento ensure they match the official input limit of the used embedding model, with no over-configuration or under-configuration.
The values provided are common starting points and should be measured against the reader's own samples.
Question material comes from public community discussions. Configuration values are common starting points and should be measured against your own samples. Verified on 2026-09-28.