B2B marketplaces: document chunking for multilingual work

Review document chunking for multilingual work in B2B marketplaces. Check document scope, failure paths and source evidence through controlled tests.

What the documents look like

B2B marketplace matching relies on online certification and electronic contracts rather than physical stall transactions, so cross-border materials and multilingual retrieval are routine. Enterprise certification materials, inquiry and quotation sheets, electronic contracts, and platform settlement statements form the main document sources. Platform operations, suppliers, and enterprise buyers use these materials for review, matching, and fulfillment tracking. Such documents commonly contain entity names, identifiers, amounts, validity periods, and multilingual versions, so parsing and chunking need to preserve the semantic boundaries of these fields. Data updates occur daily or per order. After a material changes, the training and index state needs re-checking; a successful upload is not the same as an updated retrieval index.

Constraints at the document chunking step

A FastGPT knowledge base has three levels: library, collection, and data. The smallest search unit is the library; collections only classify data and do not affect search results. Vector storage uses the PostgresSQL PG Vector plugin with an HNSW index. The MongoDB dataset.datas table stores original data information, and its indexes field records the corresponding vector IDs, so one data group can map to multiple vectors. Multi-vector mapping balances content length and semantic richness; when any one vector is recalled, the whole data group is recalled. During retrieval, vectors are recalled first, then the original data is located in MongoDB by vector ID. When vectors correspond to the same data group, they are merged and the highest vector score is kept. Semantic retrieval supports cross-language understanding. Hybrid retrieval fuses vector and full-text results through RRF. Full-text retrieval suits exact queries such as identifiers, models, and proper nouns. The reference limit is measured in tokens. The minimum relevance score takes effect only in semantic retrieval or with result reranking, and it is a filter threshold rather than a final ranking rule. Image retrieval requires an index built at ingestion time and depends on a vision model or a vector model that supports images. Retrieval and similarity do not perform permission or entity isolation, and libraries using different vector models cannot be jointly associated.

Configuration and implementation checks

The following items relate to the document parsing and chunking task on this page, illustrated with a hypothetical paper exercise sample.

Configuration itemSetting or check on this pageExplanation and basis
Search modeHybrid retrievalUses vector and full-text recall together, fused through RRF; suits inquiry sheets where identifiers and natural language coexist
Reference limitHypothetical exercise starting point of 3000 tokensA hypothetical starting point to observe whether cited content covers complete clauses; compare against the actual token volume of results before adjusting
Minimum relevance scoreHypothetical exercise starting point of 0.35A hypothetical starting point effective only in semantic retrieval or result reranking; compare against the rerank score distribution before adjusting
Question optimizationEnabled as neededUseful for multi-turn dialogue or short questions; adds one model call and latency
Vector indexCheck whether the indexes field records multiple vector IDsOne data group can map to multiple vectors; confirm whether the corresponding index was generated at ingestion
Full-text retrieval pathCheck the current vector library typeMilvus uses BM25; other vector libraries use MongoDB $text with jieba segmentation

Hypothetical paper exercise sample: a supplier submits a cross-border electronic contract whose source text reads "This contract takes effect on the date both parties sign it, remains valid until December 31, 2026, and covers 500 units of model A-102 fittings." A draft sentence awaits judgment: "The contract has taken effect, and the supply period runs until December 31, 2026." The task is to decide whether the draft sentence matches the source. The source states only the contract's effectiveness and validity period; it does not state a supply period. The draft treats the validity period as the supply period, which is unsupported.

Three common errors

  1. Setting the reference limit as top k. The documentation states the reference limit is measured in tokens, and top k produces unstable results in mixed knowledge bases because chunk lengths vary. Check: run the same query with a token-based limit and a top k setting, then compare whether returned items fluctuate with chunk length.
  2. Treating the minimum relevance score as a ranking rule or permission filter. The documentation states it takes effect only in semantic retrieval or result reranking, and it is a filter threshold that does not perform entity isolation. Check: run a cross-entity query and observe whether results still include other entities' materials, confirming the threshold changes neither ranking nor permissions.
  3. Treating a successful upload as an updated index. The documentation requires confirming whether data training has completed; upload and training are different states. Check: after a material change, search by the source identifier and compare returned content before and after; if old content still returns, the training or index state has not updated.

How to verify the setup

The following verifications use the hypothetical paper exercise sample above and do not claim any live test.

  1. Segmentation completeness check. Input the source text "This contract takes effect on the date both parties sign it, remains valid until December 31, 2026, and covers 500 units of model A-102 fittings," then compare whether the returned chunk completely contains the three elements: effectiveness condition, validity period, and subject matter. If the returned chunk contains only the validity period and omits the effectiveness condition or the subject matter, segmentation does not meet the condition.
  2. Draft sentence consistency check. Input the draft sentence "The contract has taken effect, and the supply period runs until December 31, 2026," and compare it against the source. The source does not state a supply period; the draft equates the validity period with the supply period, so the judgment is unsupported. If the draft instead reads "The contract remains valid until December 31, 2026," it matches the source.
  3. Post-update index state check. After a material change, search by the source identifier A-102 and compare whether the returned content is the updated version. If the pre-change content still returns, training or indexing has not updated, and the actual training state needs checking. For an empty-result scenario, a paper input of [] can be used to check downstream design, without claiming that a fictional rule search necessarily returns empty.

This page addresses only B2B marketplace matching, cross-border materials and multilingual retrieval, and document parsing and chunking. During operations handover, the authorization state of materials, actions, and requests cannot be inferred from retrieval or forms; retrieval is not authorization execution.