What the data for this category looks like
Data sources include the national public resource trading platform, natural resource trading systems, property right exchange systems, and state-owned assets supervision systems. Daily processed documents include transfer announcements, trading notices, bid application forms, margin vouchers, transaction confirmation letters, listing result announcements, and property right transfer specifications. Data is updated daily. Documents include both structured fields and semi-structured text. Fields cover rights identification numbers, trading target types, margin amounts, bid qualification requirements, and more. Units include square meters, ten thousand yuan, parcels, and others.
What constraints these characteristics impose on the vector models and indexing link
Multi-source data access causes format inconsistencies. The indexing link must support document structures exported from different systems. The daily update frequency requires indexing to support incremental synchronization. This avoids performance losses from full reconstruction. Documents include both structured fields and long-form text. The system must balance field-level matching and semantic retrieval. Transactions center on key information such as rights and valuation. Vector retrieval must prioritize these high-weight content. Otherwise, retrieval results may deviate from business needs and lead to compliance risks.
How to set configurations
| Configuration Item | Recommended Setting | Rationale |
|---|---|---|
| Chunk Length | 800–1200 characters | Adapts to semantic integrity for long-form documents such as transfer announcements and property right transfer specifications |
| Overlap Length | 100–150 characters | Prevents key rights and trading condition information from being truncated after long text chunking |
| Similarity Threshold | 0.75–0.85, calibrated via actual testing | Meets retrieval accuracy requirements for core business information such as rights and trading qualifications |
| Single Search Limit | Top 10 results | Controls the volume of retrieval results, avoids redundant information interfering with business judgment |
maxToken | 4096 | Adapts to the total context length of most transaction documents, ensures the model can fully read associated information |
defaultToken | Calibrated against the average length of knowledge base documents | Unifies token quotas for vector generation, prevents abnormal vector generation for single documents |
Three common mistakes
- Symptom: After upgrading FastGPT v4.17.0, existing knowledge base retrieval returns no results, or prompts an index mismatch. Cause: The embedding dimension or preprocessing logic of the new version’s vector model has changed. Existing indexes were not regenerated, leading to inconsistent vector spaces between old and new versions.
- Symptom: When importing CSV files of the same format, older versions could chunk normally, but after upgrading, chunking failure logs appear or index creation is interrupted. Cause: The new version adjusted CSV column delimiters and text escaping rules. It does not adapt to the encoding or format details of existing files.
- Symptom: Searching for "bid qualification requirements for a certain plot" returns a large number of unrelated margin amount information. Core rights information is not prioritized in recall. Cause: Vector weights for core fields such as rights and trading conditions were not adjusted, or the similarity threshold was set too low. This recalls a large volume of low-relevance content.
How to confirm configurations are set correctly
- Import a single typical document (such as a state-owned property right transfer specification). Manually review the chunking results. Confirm that long text is split reasonably and no key information is truncated.
- Initiate a core business query (such as "rights restriction conditions for a certain mining right"). Check whether retrieval results cover the core content of the corresponding document.
- Perform an incremental synchronization operation. Confirm that newly updated daily documents can be properly indexed and included in the retrieval scope.
- View vector generation logs. Confirm that
maxTokenanddefaultTokenquotas do not trigger abnormal error reports.
The values provided are common starting points and should be measured against the reader's own samples.
Question material comes from public community discussions. Configuration values are common starting points and should be measured against your own samples. Verified on 2026-09-28.