Store price and display checks: document chunking for multilingual work

What the documents look like

Price and display auditing happens during ordinary trading periods, so its records accumulate on a weekly or monthly rhythm rather than around a single opening event. The sources named in the brief are the supervision system, POS price lists, store cameras and head office promotion calendars. The working documents are store visit photographs, price tag lists, display standards and rectification notices.

Two features of this material shape the chunking task. First, the same promotion calendar can carry a Chinese price tag instruction and a foreign-language display requirement in one file, so a single document may be multilingual before any retrieval begins. Second, documents are uploaded by teams in different regions, which means the document identity and version need to be confirmed before the parsed result is trusted. The brief marks its own confidence as broadly indicative, so the update rhythm is treated as context rather than as a fixed schedule.

Constraints at the document chunking step

At the parsing stage, the original structure and the parsed output are checked against each other, and the document identity and version that allow a chunk to be traced back to the source are retained. At the chunking stage, paragraphs are examined against the structure of that specific document type. A price tag list carries a store range and an effective date, and those conditions need to stay with the tag entries they govern. When a condition, an explanatory note and the content it constrains are split apart, retrieval can return a tag entry without the scope that makes it meaningful.

Data and index are separate concerns. One group of source data can map to several indexes, and the indexes field records an array of vector IDs, so a single record may be reached through more than one vector. During retrieval, vectors are recalled first, then the vector ID is used to locate the source content in the MongoDB dataset.datas table, and results that resolve to the same source record are merged with the highest vector score kept. Libraries built on different vector models cannot be associated with one another; the identifiers are checked first and non-matching libraries are excluded.

Semantic retrieval supports cross-language understanding and suits natural-language questions. Full-text retrieval suits precise lookups such as numbers and model designations. Hybrid retrieval combines semantic and full-text recall through RRF, and Rerank performs a second ordering of candidate text with a score in the range 0 to 1. The reference cap is expressed in tokens. The minimum relevance value takes effect only under semantic retrieval or when result reranking is used, and it acts as a filter threshold rather than a final ordering rule.

Configuration and implementation checks

The table below describes a hypothetical exercise sample. Each starting value is a choice made for this exercise, with its reason and the comparison object stated.

Three common errors

1. Treating a successful upload as evidence that retrieval has been updated. After source material changes, the actual training and index state is checked; a completed upload does not by itself show that the index has been rebuilt.
2. Treating semantic recall or a relevance value as a permission filter. Minimum relevance is a filter threshold and does not exclude unrelated business records or unauthorised records, and a retrieval result does not constitute authorisation.
3. Treating RRF after hybrid retrieval as a switch that can be turned on. RRF is a fusion process; after hybrid retrieval the search scope is large and similarity filtering cannot be applied directly, so filtering relies on the rerank score.

How to verify the setup

Hypothetical exercise sample: a regional supervisor at a chain brand submits a cross-border promotion document. The source text reads: "Store numbers CN-021 to CN-035, summer price tag standard effective from 1 March 2025; the foreign-language display requirement is on page 2 of the attachment." The draft sentence to be assessed is: "Price tag replacement has been completed for this group of stores." The material states an effective start date and the content of the standard, and does not state a completion status for replacement.

• Verification one: input the source text above together with the draft sentence, and compare whether the draft sentence is supported by the source. The source gives an effective start date and a standard requirement, and gives no completion record, so the draft sentence is unsupported and the evidence gap is recorded.
• Verification two: input the store number "CN-021" and the natural-language question "which price tag standard applies to this group of stores", then compare whether the passages recalled by full-text retrieval and by semantic retrieval contain both the store range and the effective date. If a recalled passage is missing the store range or the effective date, the chunking has separated the condition from the content it constrains.
• Verification three: input an empty reference list [] and check how the downstream design handles an empty result. No claim is made that a fictional rule search returns an empty list. The handling of an empty result needs to be defined by the downstream implementation, and the retrieval step records only what is observed.

Across all three checks, the comparison stays on the same subject, the same period and the same field. A field is contradictory only when the same field carries explicitly mutually exclusive values. Material that is not stated, or that shows only an intermediate stage, is recorded as unsupported or pending confirmation. A pending review does not show that an operating permission is absent.

Related reading

• guide/dataset/dataset_engine
• FastGPT document chunking