What the documents look like
Near-expiry markdown handling follows the shelf-stock lifecycle rather than per-item weighing at order fulfilment. The working records are near-expiry lists, markdown approvals, damage reports and promotional shelf labels, refreshed by day or by shift. Supporting data arrives from POS stock age, store inspections, ERP inventory and quality-inspection records. Product question answering over these materials has to parse batch identifiers and expiry dates from near-expiry lists, the authorization state recorded on markdown approvals, the segregation conclusion on damage reports, and the price plus applicable scope on promotional labels. Structures differ widely between these document types, so chunking must keep a condition attached to the content it constrains.
A near-expiry list may state a batch, a date and a quantity in one line. A markdown approval may state a requested discount and leave a signature field blank. A damage report may state that goods were moved to a non-sale area. A promotional label may state a price and a scope of stores or dates. These are separate documents with separate identities, and a chunk that merges a condition from one with a conclusion from another loses the ability to trace back to source.
Constraints at the document chunking step
Source verification keeps a document identity and version that can point back to the original text. Segmentation integrity requires that a condition, its explanation and the content it constrains are not split apart. Data and index are distinct: in MongoDB the dataset.datas table stores vector source data, and its indexes field records the corresponding vector IDs, so one data group can map to multiple vectors. In PostgresSQL a vector field stores vectors. During retrieval, vectors are recalled first, then the original data is located in MongoDB by vector ID; entries corresponding to the same source data are merged and the highest vector score is kept.
Sample retrieval should test terminology, identifiers and natural language separately. After source material changes, the actual training and index state must be checked; a successful upload does not mean retrieval has updated. The documented search chain combines problem optimization, semantic recall, full-text recall, image handling where applicable, RRF fusion, ReRank and finally similarity filtering with a citation token limit. Not every path runs on every request: image retrieval needs image input and matching model capability, and ReRank needs a model enabled by configuration.
Hypothetical paper test case: a store inspection record dated 12 March 2026 states "Batch A yogurt, shelf life to 15 March 2026, current stock 24 boxes, status awaiting markdown approval." A markdown approval form of the same date states "Batch A, request 30 percent discount, approver signature blank." A draft sentence reads "Batch A has completed markdown approval and can be sold at 30 percent off." Judgement requires the same subject, same period and same field: the approval form signature field is empty, while the draft sentence claims completed approval, so the two are mutually exclusive on that field. The inspection record states only an awaiting-approval status and does not state approval. The draft sentence is unsupported.
Configuration and implementation checks
| Configuration item | Hypothetical starting point for this page | Reason and comparison method |
|---|---|---|
| Search mode | Hybrid retrieval | Near-expiry lists carry batch identifiers and natural-language description; hybrid retrieval uses semantic recall and full-text recall combined through RRF |
| Citation limit | Hypothetical 3000 tokens | Chunk lengths differ widely, so a token limit is steadier than top k; compare against recalled content and adjust |
| Minimum relevance | Hypothetical 0.3 | This value applies only under semantic retrieval or with result reranking; compare against the score distribution of recalled results |
| Problem optimization | Hypothetical enabled | Short follow-up questions need coreference resolution; compare retrieval results with the setting disabled |
| Result reranking | Hypothetical enabled | With many candidates, ReRank performs a second ordering and yields a 0 to 1 score; compare against results without reranking |
| Index verification | Check training state after upload | Confirm the indexes field in dataset.datas records vector IDs, then run sample retrieval |
Three common errors
- Treating an inspection record's "awaiting markdown approval" as completed approval. Input is the inspection record text; compare the approval form signature field; observe whether the draft sentence matches the fact that the signature field is empty. When the signature field is empty, the draft sentence claiming completed approval is unsupported.
- Treating a successful upload as updated retrieval. Input is the upload operation record; compare the training state and the index field; observe whether sample retrieval can recall the new content. When training is incomplete, retrieval results do not include the new material.
- Treating minimum relevance as a permission or entity filter. Input is the relevance threshold setting; compare recalled results against the authorization scope; observe whether unrelated business records are excluded. Relevance filters low-scoring results only and does not perform authorization isolation.
How to verify the setup
- Verification one: input the inspection record text and the draft sentence from the hypothetical sample; compare the approval form signature field against the approval-completion state claimed by the draft sentence. When the signature field is empty and the draft sentence claims completed approval, the judgement is unsupported. The draft sentence is supported only if the signature field records an approval with a date inside the valid range.
- Verification two: input batch A's identifier "A" and the natural-language question "can batch A yogurt still be sold," running full-text retrieval and semantic retrieval respectively. Compare whether the recalled content includes the inspection record text. When the recalled content contains the batch status and expiry date, segmentation meets the retrieval requirement. When it is not recalled, record the gap and do not claim the rule necessarily returns empty.
- Verification three: input an empty citation list [] to check the downstream question-answering design. Compare whether the downstream treats an empty result as no cited content, and observe whether an unsupported answer is triggered. When the downstream design does not state how empty results are handled, record this as pending verification and do not claim retrieval necessarily returns [].
Authorization boundary: retrieval and form collection do not constitute approval, damage reporting or price execution. Approval requires a signature, damage reporting requires segregation confirmation, and price execution requires downstream system implementation. An awaiting-review state does not demonstrate absence of operating permission.