What the documents look like
Tmall flagship store daily operations centre on brand authorization, flagship store identity, and Tmall points compensation rules, which differ from JD POP store operations. The main data sources are the Tmall merchant backend, the Tmall rule centre, ERP order tables, and customer service chat records. Routine documents include product publishing pages, brand authorization letters, delivery commitment records, and after-sales tickets. Operations permission and handover management must confirm the operating subject, authorization status, and handover point across these documents. Knowledge base retrieval and recall locates relevant entries from existing materials; the retrieval result itself provides quoted content only and does not constitute authorization execution.
The daily update frequency and per-order update rhythm shape how quickly these documents change, but they do not alter the retrieval step described here. What matters for this page is that handover materials carry explicit status fields, and that a retrieval result is evidence of what was found, not evidence of what was approved.
Constraints at the retrieval step
The knowledge base search node can associate one or more knowledge bases that use the same vector model. Knowledge bases using different vector models cannot be associated together. Search modes include semantic retrieval, full-text retrieval, and hybrid retrieval; hybrid retrieval merges multiple result paths through RRF, and result reranking uses a ReRank model for secondary ordering. The reference limit is measured in tokens. The minimum relevance score takes effect only under semantic retrieval or when result reranking is used. Question optimization performs coreference resolution and question expansion. Quoted content is output as an array and its length can be zero; when nothing is found, the output path still completes.
A hypothetical paper exercise sample: a Tmall flagship store brand e-commerce department hands over a batch of after-sales tickets on 1 March 2025, with the retrieval scope set to the "after-sales tickets" knowledge base. The material text supplied for checking reads: "Ticket A-031: the buyer applied for delayed delivery compensation on 20 February 2025; the store has confirmed the delivery commitment record as 48 hours; the compensation process awaits store manager review." The draft sentence to be assessed reads: "Ticket A-031 compensation has been reviewed and approved by the store manager, and handover is complete." Comparing the material text with the draft sentence, the original writes "awaits store manager review" while the draft writes "has been reviewed and approved by the store manager." The two values for the same field are mutually exclusive, and the draft sentence is not supported by the material. The retrieval scope must include the knowledge base holding this ticket. If retrieval returns an empty array, record the evidence gap rather than inferring the ticket does not exist.
Configuration and implementation checks
| Configuration item | Hypothetical starting point for this exercise | Note and comparison method |
|---|---|---|
| Associated knowledge bases | Only the "after-sales tickets" knowledge base | Check that this knowledge base matches the vector model identifier used for retrieval; exclude knowledge bases with different vector models |
| Search mode | Hybrid retrieval | For the same handover question, compare quoted content from semantic, full-text, and hybrid retrieval separately; do not assume one mode suits all materials |
| Reference limit | Set to 3000 tokens as the hypothetical starting point for this exercise | This starting point is chosen because handover materials contain ticket text and compensation records; test with materials of different lengths to see whether content is complete, then compare with adjacent values |
| Minimum relevance score | Set to 0.5 as the hypothetical starting point for this exercise | First label relevant and irrelevant samples, then adjust the threshold; it takes effect only under semantic retrieval or result reranking |
| Question optimization | Enabled | Compare the original question with the optimized question, checking whether the subject and ticket number were altered |
| Result reranking | Enabled as needed for this exercise | Requires a ReRank model and enabling through settings; the rerank score is used for filtering and is not written as a toggle switch |
Three common errors
- Treating a retrieval result as authorization execution. When retrieval returns quoted content for ticket A-031, this only shows the content was recalled; it does not show the compensation was reviewed and approved or that handover is complete. During checking, compare the quoted content with the material text and observe whether states such as "reviewed and approved" appear that the original text does not record.
- Writing a pending review as an approved review. The material text writes "awaits store manager review" while the draft sentence writes "has been reviewed and approved by the store manager"; the two values for the same field are mutually exclusive. During checking, compare status words character by character and observe whether an unsupported conclusion appears.
- Associating knowledge bases with different vector models together. Check the vector model identifier of each knowledge base, exclude those that do not match, and record the actual observation. Do not infer runtime results of associating them together.
How to verify the setup
- Input the hypothetical sample material text and the draft sentence, and compare "awaits store manager review" with "has been reviewed and approved by the store manager." If the two values for the same field are mutually exclusive, the draft sentence is not supported and the evidence gap must be retained.
- Input the handover request with retrieval scope limited to the "after-sales tickets" knowledge base, and compare whether the returned quoted content includes the original text of ticket A-031. If an empty array is returned, record that this ticket was not recalled within the retrieval scope of this run; do not infer that it does not exist.
- Input the same handover question under semantic retrieval, full-text retrieval, and hybrid retrieval separately, and compare how the quoted content returned by each mode corresponds to the material text. Observe whether the quoted content after RRF merging in hybrid retrieval includes key fields such as the ticket number. Do not guarantee a first-position hit or complete recall.