What the documents look like
A Shopify standalone store leaves payment, shipping and fulfillment configuration to the merchant, so the operating documents behind procurement and supplier records sit with the merchant rather than arriving in a platform-generated format. Order details, shipping templates, shipping labels and refund records live across Shopify Admin, payment gateways, carrier systems and inventory location data. Procurement and supplier records are merchant-held documents: quotation sheets, purchase requests, supply-scope notes and approval states. These materials update per order and per day, and retrieval has to separate the currently effective version from historical versions. The subject, object, status and validity range carried inside each record determine whether a retrieved passage can be quoted directly.
For this page the working task is knowledge-base retrieval and recall over procurement and supplier documents. The dataset search node can attach one or more knowledge bases that use the same vector model. Search modes cover semantic retrieval, full-text retrieval and hybrid retrieval, with hybrid results merged through RRF. The reference cap is measured in tokens. Minimum relevance takes effect only under semantic retrieval or when result reranking is used. Question optimization performs coreference removal and question expansion and adds one model call before retrieval. Reference content is returned as an array whose length can be zero, and the empty-result path still completes. Retrieval is not authorization, and a similarity score does not act as a permission or entity filter.
Constraints at the retrieval step
A hypothetical paper sample is used below. It is a constructed exercise, not a real record, and nothing has been tested or executed.
Hypothetical sample source text: "Supplier A, quotation number A-001, quotation valid until 31 March 2025, supply scope cotton T-shirts, minimum order quantity 200 pieces. Purchase request number P-007, requested quantity 200 pieces, requester operations group, approval status pending review."
Draft sentence to be assessed: "Supplier A has confirmed supply of 200 cotton T-shirts."
The draft sentence turns a purchase request still pending review into a confirmed supply commitment. The quotation validity date is also not a supply deadline. Both points matter when a retrieved passage is handed to a downstream drafting or handover step.
Several constraints follow. Attached knowledge bases must share the same vector model; the model identifier is checked before attachment, and bases that do not match are excluded rather than compared across models. RRF is a fusion step for multiple recall paths, not a switch that is turned on. Image retrieval requires an image input and matching model capability, and reranking requires a rerank model that is enabled; neither path runs on every request. The reference cap is a token quantity, not a top-k item count. Minimum relevance is a filter threshold, not a final ordering rule, and after question optimization, hybrid retrieval, image retrieval or reranking, the final result may reflect multi-path fusion rather than a single vector similarity order. A pending approval is not an approval, an offered option is not execution, and an unsigned termination is not a completed termination. Where a field is not present in the source, the value remains unknown.
Configuration and implementation checks
| Configuration item | Hypothetical starting value for this exercise | Reason and comparison method |
|---|---|---|
| Attached knowledge bases | Supplier records base and order fulfillment base | Both bases are checked for an identical vector model identifier before attachment; non-matching bases are excluded |
| Search mode | Hybrid retrieval | Quotation numbers and purchase request numbers are exact identifiers, so semantic and full-text paths are both used and merged through RRF |
| Reference cap | 800 tokens for this exercise | Supplier records carry quotation, scope and minimum quantity in several passages; passages of different lengths are compared for content completeness |
| Minimum relevance | 0.35 for this exercise | Relevant and irrelevant samples are labelled first, and the filter effect is compared only under semantic retrieval or reranking |
| Question optimization | Enabled | In multi-turn follow-ups, the optimized question is compared against the original to check whether the subject was altered |
| Result reranking | Enabled | When candidates are numerous, text results are reordered and compared by rerank score |
Three common errors
- Treating a retrieval hit as authorized execution. Input: the query "Supplier A supply". The quotation and the purchase request are both recalled. Compare the approval status field inside the recalled content; while it reads pending review, the observable result is that the draft sentence "has confirmed supply" is not supported. The error is reading a pending review as a confirmation.
- Treating quotation validity as a supply deadline. Input: the quotation source text. Compare "quotation valid until 31 March 2025" against the supply commitment in the draft sentence. The observable result is that these are different fields, no supply deadline is stated, and an evidence gap remains. The error is deriving a supply deadline from a validity date.
- Using a similarity score to exclude unrelated records. Input: minimum relevance 0.35. Compare the filtered reference content against the supply-scope field in the supplier records. The observable result is that the score does not distinguish supplier entities, and records for unrelated suppliers may still be recalled. The error is treating relevance as an entity or permission filter.
How to verify the setup
- Input the hypothetical sample source text and the draft sentence "Supplier A has confirmed supply of 200 cotton T-shirts." Compare the approval status field of purchase request P-007 against the supply wording in the draft sentence. If the approval status reads pending review, the draft sentence is not supported and the evidence gap is retained.
- Input quotation number A-001. Compare the quotation validity field against the supply deadline field. If the record contains only a validity date and no supply deadline, the supply deadline remains to be verified and is not recorded as a contradiction.
- Input an explicit paper input of an empty array. Check how the downstream design handles an empty reference array and observe whether the path completes, without asserting that a given rule always returns an empty array. For bases using different vector models, check the model identifier first and exclude non-matching bases, then record what is actually observed rather than running a cross-model retrieval to compare missed recalls.