What the documents look like
Shopify storefront fulfillment differs from TikTok Shop US operations in one structural way: the merchant configures payment, shipping and fulfillment services, and the platform does not run unified automatic returns. Store service and internal training knowledge bases therefore have to cover merchant-defined shipping templates, shipping labels and refund records rather than a single platform return flow. The main data sources are Shopify Admin, payment gateways, carrier systems and inventory location data. Daily documents are order details, shipping templates, shipping labels and refund records, with updates occurring per order and daily. Staff use these documents to answer fulfillment questions, and retrieval output supplies quoted content only. That output does not authorize a refund, an address change or a replacement shipment.
A hypothetical drill sample is used throughout this page. The source text for a store service training file reads: "Return requests for this store must be submitted within seven days of signed delivery; requests past that period are manually reviewed by the store supervisor. Refunds are issued only to the original payment channel; shipping fees are not refunded. The shipping template 'East China Standard' applies to Jiangsu, Zhejiang and Shanghai, with a first-weight limit of one kilogram." A draft sentence under review reads: "A return request submitted on the tenth day after signed delivery may be refunded by the store directly to the original payment channel." The draft conflicts with the source on the seven-day period and on the supervisor review subject, so it is unsupported.
Constraints at the retrieval step
The knowledge base search node can associate one or more knowledge bases that use the same vector model. Retrieval operates on the knowledge base as the smallest search unit; collections serve classification management only and do not affect search results. One retrieval combines question optimization, semantic retrieval, full-text retrieval, hybrid retrieval, image retrieval and reranking according to the input and search parameters, then returns quoted content after similarity filtering and a reference cap. The output is an array whose length can be zero, and the output chain still completes when nothing is found. The minimum relevance value applies only under semantic retrieval or when result reranking is used; it is a filter threshold and does not perform permission or entity isolation. The reference cap is measured in tokens. Question optimization handles coreference resolution and question expansion and adds a model call. Image retrieval requires image input and the corresponding model capability, and reranking requires a model and must be enabled by configuration.
For this task, the retrieval scope must contain the material needed for the check, or that material must be supplied as an explicit manual input. When it is absent, the gap is recorded rather than filled.
Configuration and implementation checks
| Configuration item | Hypothetical starting point for this drill | Reason and comparison object |
|---|---|---|
| Associated knowledge bases | Store service base and fulfillment base, with matching vector model identifiers | Check the identifiers and exclude bases with a different vector model |
| Search mode | Semantic retrieval first; full-text retrieval tested separately for numbered queries | Compare quoted content for the same question under both modes |
| Reference cap | A starting value sized to hold one complete shipping template text | Compare against actual materials of differing length for completeness |
| Minimum relevance | A starting value in the middle of the 0 to 1 range | Label relevant and irrelevant samples first, then compare against the threshold |
| Question optimization | Enabled for multi-turn follow-up; disabled for single clear questions | Compare the original and optimized question to check whether the subject was altered |
| Result reranking | Enabled when candidates are numerous and the question is complete | Compare ordering for the same question with and without reranking |
These values are hypothetical starting points for this drill, not official defaults. RRF is the fusion process for hybrid retrieval results and is not configured as a standalone switch. The optional paths in the overall retrieval flow are not all executed on every request.
Three common errors
- Treating a retrieval hit as authorization for a refund or address change. The input is the quoted content array; compare whether it contains any authorization field; the observable result is that quoted content does not change business state, and a refund still requires downstream implementation.
- Associating knowledge bases that use different vector models. The input is the vector model identifier of each base; compare whether the identifiers match; the observable result is that non-matching bases are excluded first, and a cross-model retrieval is not run to compare missed recalls.
- Treating minimum relevance as a permission or entity filter. The input is a similarity score and a target record; compare the score field and the access permission field; the observable result is that the score does not change whether the record belongs to the current store.
How to verify the setup
- The input is the hypothetical source text and the draft sentence "A return request submitted on the tenth day after signed delivery may be refunded by the store directly to the original payment channel." Compare the draft against the source period field "within seven days" and the handling subject "manually reviewed by the store supervisor." The observable result is that both the period and the subject differ, so the draft is unsupported.
- The input is the same source text and the retrieval question "Which regions does the East China Standard shipping template apply to?" Compare whether the quoted content includes "Jiangsu, Zhejiang and Shanghai" and "first-weight limit of one kilogram." The observable result is that coverage of that scope satisfies the check; missing coverage is recorded as a gap, and a first-position hit is not guaranteed.
- The input is a paper empty result array [] and the downstream handoff design. Compare whether the downstream continues through the chain on empty quotes and records the unsupported outcome. The observable result is that the downstream does not automatically trigger a refund or replacement because of an empty result. If that handoff is not configured, it is recorded as pending implementation, and an empty return is not asserted.