Brand member operations: document chunking for document comparison

What the documents look like

Brand member private-domain operations centre on customer data and repeat-purchase outreach. The working materials come from a CRM system, enterprise messaging, POS consumption records and mini-programme behaviour data. The day-to-day documents are member tag tables, outreach plans, coupon rules and privacy authorisation records. This category does not handle store shift scheduling. When something goes wrong, the visible consequences are outreach harassment complaints, uncontrolled coupon cost and member churn.

For contract and transaction material comparison, the task is to place member authorisation, coupon rules, outreach arrangements and transaction evidence into one group of materials and check whether they agree on subject, object, status and valid scope. A tag table, an outreach plan, a coupon rule and an authorisation record each carry their own identifiers and their own effective range. Comparison depends on reading the original wording, so the chunking step decides whether the conditions, notes and the content they constrain stay together.

Materials update daily and per order in the source description. That cadence does not change the chunking or comparison method described here.

Constraints at the document chunking step

Chunking directly affects whether materials can be checked against their original structure. Segmentation needs to preserve the subordinate relationship between a condition, its explanation and the content it constrains. A valid scope attached to one authorisation record should not be split across separate chunks.

Original data and retrieval index need to be kept distinct. In the documented storage structure, one group of original data can correspond to multiple vectors. The indexes field in the MongoDB dataset.datas table records the vector IDs belonging to that data as an array. During retrieval, vectors are recalled first, then the vector ID is used to find the original content in MongoDB. If several vectors map to the same group of original data, the results are merged and the vector score takes the highest value. Chunking that breaks this mapping splits one subject into separate records.

Retrieval is a search mechanism. It does not perform authorisation judgement. A relevance score is a filtering threshold, not a category, entity or permission filter, and it does not exclude unrelated businesses or unauthorised records. The citation limit is measured in tokens, and the minimum relevance value takes effect only under semantic retrieval or result re-ranking. Neither replaces manual checking of authorisation status.

Hypothetical paper exercise sample. Material one reads: "Member A signed the Member Privacy Authorisation on 1 March 2025. The authorised scope is limited to member tag analysis and coupon outreach, valid until 28 February 2026." Material two reads: "Member A's outreach plan is scheduled to send a beauty category coupon to Member A on 1 April 2025." The draft sentence to be judged reads: "Member A's coupon outreach has obtained valid authorisation." The check compares the subject, object, status and valid scope stated in the authorisation against the subject, object and time range in the outreach plan, and asks whether both fall inside the same authorised scope.

Configuration and implementation checks

Three common errors

1. Treating a retrieval hit as authorisation confirmation. The input is the draft sentence "Member A's coupon outreach has obtained valid authorisation". The comparison object is the subject, object, status and valid scope in the authorisation text. The observable result is whether the retrieved chunk contains the authorised scope and validity period. If only the outreach plan is hit and the authorisation is not, the authorisation status remains unverified.
2. Treating a successful upload as a completed retrieval update. The input is the upload operation record after a material change. The comparison object is the actual training and index state. The observable result is whether retrieval can hit the changed content. If it cannot, the update state is not supported.
3. Splitting multiple indexes of one group of original data into separate subjects. The input is the multiple indexes corresponding to one group of original data. The comparison object is the original data ownership recorded in the indexes field in MongoDB. The observable result is whether merging returns to the same group of original data. If it is split into separate subjects, the chunk ownership has drifted.

How to verify the setup

1. Retrieve the authorisation text and the outreach plan from the hypothetical sample separately. The inputs are "Member A authorisation scope" and "Member A outreach plan". Compare whether the retrieval results return to the same group of original data. The observable result is whether the subject, object and valid scope of the two materials correspond within the same chunk or associated chunks. If the authorised scope is split away, chunk completeness is not met.
2. Check the draft sentence "Member A's coupon outreach has obtained valid authorisation". The inputs are this draft sentence and the authorisation text. Compare whether the validity period stated in the authorisation and the outreach plan time fall within the same period. The observable result is whether 1 April 2025 falls inside 1 March 2025 to 28 February 2026. If it does not, the authorisation status is not supported.
3. Check the downstream design with an empty-result case. The input is the explicit paper input []. Compare whether the downstream handling records the empty result as a gap. The observable result is whether the system distinguishes an empty result from an absence of authorisation. If an empty result is directly judged as no authorisation, the judgement goes beyond the evidence in the materials.

Related reading

• guide/dataset/dataset_engine
• FastGPT document chunking