Near-expiry markdowns: document chunking for document comparison

Review document chunking for document comparison in Near-expiry markdowns. Check sources, input boundaries, error handling and repeatable test evidence.

What the documents look like

Near-expiry markdown work concerns the remaining sellable time of shelf stock, not item-by-item weighing and sorting. Each day or shift produces near-expiry lists, markdown approvals, damage reports and promotional price tags. These materials enter contract and transaction comparison together. A near-expiry list records batch and expiry date. A markdown approval records the application and its approval status. A damage report records isolation and disposal quantities. A promotional price tag records the discounted price and its effective range. When field conventions differ across these four material types, the same batch can appear in multiple versions within search results, so a reviewer first confirms material identity and version, then judges whether field values are mutually exclusive.

Two facts shape this page. Materials update daily or per shift, so version identity matters each time comparison runs. The four material types carry different field conventions, so chunking must keep the document identity and version that allow a return to the source text.

Constraints at the document chunking step

Chunking must preserve document identity and version that can point back to the original text. A condition statement and the item it constrains must not be separated. One set of original data can map to multiple retrieval indexes, and original data and indexes must be checked separately. Retrieval locates materials and does not perform authorization judgment. Whether a markdown approval has been authorized, and whether a damage report has been executed, require returning to the approval record and the execution record themselves.

Hypothetical drill sample: a store's morning-shift near-expiry list dated 12 March 2026 states "Batch A yogurt, 40 units, shelf life to 15 March 2026, markdown to 60% applied for, applicant Li, approval status is pending review." A promotional price tag draft of the same day states "Batch A yogurt 60%, effective from 12 March 2026." The draft sentence to be judged is "The Batch A yogurt markdown has been approved and has taken effect." Compared with the approval status field on the list, this sentence is unsupported.

Configuration and implementation checks

Configuration itemHypothetical starting point for this trialReason and comparison method
Search modeHybrid searchMaterials contain batch numbers and natural-language descriptions, so exact hits and semantically similar content both need coverage
Reference limitHypothetical starting point 3000 tokensControls the length returned in one search; compare with average chunk length after chunking before adjusting
Minimum relevanceHypothetical starting point 0.4Applies only during semantic search or result reranking; compare with manual review of hits
Question optimizationOff for nowFor a single-turn, explicit query, observe unoptimized results first, then decide whether to enable
Chunking methodSegment by material typeList, approval, damage report and price tag each form their own segment, so conditions and items stay together
Post-update reviewManual check of training and index statusUpload success and retrieval being updated are different facts and are recorded separately

Three common errors

  1. Treating pending review as approved. The list field reads "pending review" while the draft sentence reads "has been approved." These are mutually exclusive values on the same approval status field, so this is a contradiction. The check is to input the list source text and the draft sentence, compare the approval status field, and mark a contradiction when the draft value differs from the source value.
  2. Treating a price tag draft as an effective fact. The price tag draft states an effective date and does not state a publication or execution record. The check is to input the price tag source text and the draft sentence, compare whether a publication record field exists, and mark unsupported when the source text has no such field.
  3. Treating a retrieval hit as authorization confirmation. A retrieval returning a markdown approval entry does not mean the approval passed. The check is to input the retrieval result and the approval record field, compare the approval status value, and refrain from judging authorization when the retrieval result does not contain an approved value.

How to verify the setup

  1. Input the list source text and the draft sentence above, and compare the approval status field. Expected observation: the draft sentence and the source text differ in value, judged as a contradiction; if the source text does not state that field, record it as unsupported.
  2. Input the price tag source text and the draft sentence above, and compare whether a publication or execution record exists. Expected observation: the source text does not state a publication record, judged as unsupported; if the source text states a publication record matching the draft sentence, judge it as supported.
  3. Input the Batch A identifier and a natural-language question separately, and compare whether the returned content contains original data for the same batch. Expected observation: both query forms locate that batch's materials; if one query form returns an empty result, record that query form as not hit, and do not judge the material absent on that basis.

A retrieval hit does not confirm authorization. A pending request is not approval, an offered option is not execution, and an unsigned termination is not a completed termination. Where a material field is not provided, the value remains unknown.