What the documents look like
Tmall flagship store daily operations centre on brand authorisation and flagship store identity, so after-sales answers frequently touch on delivery commitments, late-delivery compensation and after-sales tickets. The working materials come from the Tmall merchant backend, the Tmall rules centre, ERP order tables and customer service chat logs. The documents handled day to day include product listing pages, brand authorisation letters, delivery commitment records and after-sales tickets. Retrieval and recall here serve order fulfilment and after-sales question answering, and the materials update daily and per order. Retrieved output is quotation content only; it does not constitute authorisation, approval or compensation execution.
The brand authorisation letter and the delivery commitment record carry different kinds of value. One states who may operate the store and until when; the other states what was promised for a defined order window. A retrieval hit on either document returns text, and the text still has to be read against the actual order, ticket status and any downstream approval. The update rhythm matters because a commitment record written for one month does not extend to later orders, and a ticket status can change between the moment it is indexed and the moment a question is asked.
Constraints at the retrieval step
The knowledge base search node can associate one or more knowledge bases that use the same vector model. Knowledge bases built on different vector models cannot be associated together. The vector model identifier is checked first, and any base with a different identifier is excluded; cross-model retrieval is not run in order to compare missed recalls. Quotation content is output as an array and its length can be zero, which means the downstream chain still proceeds when nothing is found. Retrieval is not authorisation. Forms collect values, and business verification, approval, database writes and external synchronisation each need their own downstream implementation.
Hypothetical drill sample, for paper comparison only. Source text one: "For orders placed between 00:00 on 1 March 2025 and 23:59 on 31 March 2025, this store's delivery commitment is dispatch within 48 hours of payment; the commitment record number is A-2025-03." Source text two: "The brand authorisation letter names a certain brand management company as the authorising party, the authorised store as a certain Tmall flagship store, and the authorisation as valid until 31 December 2025." Source text three: "After-sales ticket T-2025-0312 records that the buyer applied for a refund on 12 March 2025 and that the ticket status is pending review." Draft sentence to be assessed: "This order has received late-delivery compensation, and the compensation amount follows platform rules." The sentence lacks a compensation arrival record and a basis for the amount, so it is unsupported.
Configuration and implementation checks
| Configuration item | Hypothetical starting point for this drill | Reason and comparison method |
|---|---|---|
| Associated knowledge bases | Associate only the delivery commitment record base and the after-sales ticket base whose vector model identifiers match | The official description requires the same vector model; check the identifiers first, compare against a base with a different identifier, and exclude it |
| Search mode | Start with hybrid retrieval | Numbered questions suit full-text retrieval and natural-language questions suit semantic retrieval; compare how each mode recalls the sample |
| Quotation limit | Set to 3000 tokens | A hypothetical starting point for this drill because the sample contains numbers and dates; compare against 1500 and 6000 for content completeness |
| Minimum relevance | Set to 0.5 | A hypothetical starting point for this drill, effective only under semantic retrieval or result reranking; label relevant and irrelevant samples before adjusting the threshold |
| Question optimisation | Enable for multi-turn follow-ups | The official description covers coreference resolution and question expansion; compare the original question with the optimised question and check whether the subject was altered |
| Result reranking | Enable when a ReRank model is available | The official description gives reranked scores in the range 0-1; compare ordering before and after reranking without assuming the first item must match |
Three common errors
- Treating a retrieval hit as authorisation or compensation execution. The input is the ticket source text and the draft sentence; the comparison is the status field against the claim of payment; the observable result is that a pending-review status does not equal completed compensation.
- Associating knowledge bases that use different vector models. The input is the vector model identifiers of the two bases; the comparison is whether the identifiers match; the observable result is that a mismatch leads to exclusion, with no cross-model retrieval run.
- Treating minimum relevance as a permission or entity filter. The input is the relevance threshold and the sample set; the comparison is the quotation content before and after filtering; the observable result is that the value only filters low-relevance results and does not exclude unauthorised records.
How to verify the setup
- Input source text one together with the draft sentence "this order has received late-delivery compensation". Compare the commitment scope in the delivery commitment record against the existence of a compensation record. The observable result is that, when evidence of compensation arrival is missing, the sentence is judged unsupported.
- Input source text two together with the handover request "please execute compensation according to the authorisation letter". Compare the authorising party, the authorised store and the validity period in the authorisation letter. The observable result is that the letter states no compensation authorisation, so the request exceeds the authorisation boundary.
- Input source text three together with the draft sentence "the ticket has passed review". Compare the ticket status field. The observable result is that pending review and passed review are mutually exclusive values of the same field, so an unstated pass is judged pending verification.
For the empty-result branch, a paper input of [] can be used to check the downstream design; this page does not claim that a fabricated rule query returns [].