General import declarations: document chunking for multilingual work

Review document chunking for multilingual work in General import declarations. Check input mappings, source evidence and failure handling before use.

What the documents look like

General trade import declaration work centres on dutiable value, freight and insurance, and tax payment, which separates it from export declaration priorities. Each declaration file brings a set of materials: the import declaration form, commercial invoice, packing list, bill of lading, and contract. These arrive and change on a per-declaration rhythm.

The materials commonly mix languages inside the same field. Product names, transaction terms, freight and insurance breakdowns, and reference numbers may appear in Chinese alongside English or another language. Semantic retrieval can match across languages because it works from vector distance rather than surface wording. Full-text retrieval suits reference numbers, model codes, and proper nouns where exact keyword hits matter.

Chunking decisions sit underneath both. A condition, its explanation, and the content it governs need to stay inside one data block. When a chunk is cut too short, a retrieved fragment can carry a conclusion without the condition that made it applicable. When a chunk grows too long, the semantic content of the block is diluted and similarity matching becomes less precise. The documented guidance states that complete, single-meaning segments improve precision, and that reducing the length of index content trades retrieval breadth for accuracy.

Constraints at the document chunking step

FastGPT uses multi-vector mapping. One set of source data can map to several vectors, and when the same source data is recalled through more than one route, the vector score taken is the highest score. The smallest search unit is the knowledge base; a collection only classifies data and does not affect search results.

Retrieval mode shapes what a chunk needs to carry. Semantic retrieval depends on vector distance and handles cross-language and multimodal matching, subject to model capability. Full-text retrieval depends on keyword hits and suits numbers, model codes, and proper nouns. Hybrid retrieval runs both and merges the two result sets through RRF. Rerank produces a 0-1 relevance score and is commonly applied after question optimisation when the question is complete and candidates are numerous.

Two filters apply. The minimum relevance value is a 0-1 threshold that works only under semantic retrieval or when reranking is used; it filters low-relevance results and does not act as a final ordering rule, and it does not separate businesses or restrict records by permission. The citation limit caps each search at a stated number of tokens rather than top k, because chunk lengths vary widely across mixed knowledge bases and top k results become unstable.

Documented parameter advice relevant here: when results are too broad or off-topic, raise the minimum relevance, reduce the citation limit, and improve chunking; when numbers, model codes, or proper nouns retrieve poorly, use full-text or hybrid retrieval and avoid over-applying question optimisation to exact identifiers.

Configuration and implementation checks

The values below are a hypothetical trial starting point for the paper sample in the next section. They are not official defaults, and each is adjusted by comparing it against another value under the same sample.

SettingHypothetical trial starting pointReason and comparison method
Search modeHybrid retrievalSample mixes Chinese product names with English reference numbers; compare against semantic retrieval alone
Minimum relevance0.5Compare against 0.3 and 0.7 for whether recalled fragments keep the full condition
Citation limit2000 tokensCompare against 1000 for whether the freight and insurance explanation is truncated
Question optimisationEnabledCompare against disabled for recall completeness across follow-up turns
RerankEnabledCompare against disabled for ordering stability across candidates
Chunking approachSegment by material structureCompare against fixed-length segmentation for whether conditions and their governed content stay in one block

Three common errors

  1. Treating a completed upload as an updated index. In the paper sample, a contract revision is uploaded on 10 March 2025 and the interface shows the upload complete, while training and index status are not checked, so retrieval may still return the earlier segmentation. Check: input the contract number, compare the returned content against the revised clauses, and observe whether earlier clause wording still appears.
  2. Treating the minimum relevance value as a permission filter. In the paper sample, a broker account searches the same knowledge base, and a low relevance threshold does not exclude records that account is not authorised to view. Check: input a document number belonging only to another party, compare the returned results against the access list, and observe whether fragments outside the authorised scope appear.
  3. Treating RRF as a switch that can be turned on. In the paper sample, hybrid retrieval merges multiple result routes through RRF, and this merging is performed by the retrieval scheme. Check: inspect the search parameter interface for a separate RRF switch, compare the documented description against the interface fields, and observe whether only a search mode option is present.

How to verify the setup

The verification below uses a hypothetical paper sample. Sample source text: "Contract number HYP-2025-0310, buyer is a domestic import enterprise, seller is an overseas supplier, transaction term CIF, freight and insurance borne by the seller, signing date 1 March 2025, contract valid until 31 December 2025." Draft sentence: "The buyer separately pays the freight and insurance under this contract."

  1. Input the draft sentence together with the sample source text, compare the field for the party bearing freight and insurance, and note that the source states the seller bears them while the draft states the buyer pays them. The same field holds mutually exclusive values, so the draft is judged contradictory.
  2. Input the handover request "please complete the payment authorisation according to the contract", compare the contract source text against authorisation records, and note that the source text contains no payment authorisation clause. The request sits in a pending-verification state and is judged unsupported.
  3. Input the empty-result paper input [] and compare the downstream design for handling empty references, observing whether it returns an empty reference. No claim is made that a fictional rule search necessarily returns [].

Knowledge bases built on different vector models cannot be associated together. Verify identifiers, exclude those that do not match, and record what is actually observed. Image retrieval requires image input and a model capable of it; reranking requires a rerank model enabled in settings. A pending request is not an approval, an offered option is not execution, and an unsigned termination is not a completed termination.