What the documents look like
Operating reviews and exception reviews for a Tmall flagship store draw on four document types: product listing pages, brand authorization letters, delivery commitment records, and after-sales tickets. The parties named in these materials are the brand owner, the store entity, order numbers, and ticket numbers. Status fields include authorization validity period, committed delivery time, and ticket handling stage. For retrieval, identifiers such as item numbers, model numbers, order numbers, and ticket numbers are exact-query objects, while authorization scope and compensation conditions are semantic-query objects. The materials update daily and per order. Retrieved content reflects the text stored at ingestion time and does not replace the current state in the merchant backend.
A brand authorization letter and an after-sales ticket may sit in the same knowledge base or in different ones. The knowledge base search node can associate one or more knowledge bases that use the same vector model. Knowledge bases built on different vector models cannot be associated together. When checking, verify the vector model identifier first and exclude non-matching knowledge bases; do not run cross-model retrieval to compare missed recalls.
Constraints at the retrieval step
The knowledge base search node outputs cited content as an array, and the length can be 0. When no content is found, the output path still completes. Search modes are semantic retrieval, full-text retrieval, and hybrid retrieval. Hybrid retrieval merges multiple result paths through RRF; RRF is a fusion process, not a switch. The minimum relevance score takes effect only under semantic retrieval or when result reranking is used. It is a filter threshold, not a final sorting rule, and it does not perform permission or entity isolation. Retrieval is not authorization.
Question optimization performs coreference resolution and question expansion before retrieval. Image retrieval requires an image input and the relevant model capability; reranking requires a ReRank model and must be enabled in settings. These optional paths do not run on every request.
Hypothetical drill sample: a brand authorization letter states "Authorizing party A, authorized party B, authorized product category is home appliances, authorization validity period from January 1, 2024 to December 31, 2024." An after-sales ticket states "Ticket number A-1001, order number B-2001, buyer reports delayed shipment, ticket status is pending review." A draft sentence to be assessed reads "B still holds the brand authorization in January 2025 and can continue to use the authorization letter to process category qualification." The materials needed for the assessment must be within an accessible retrieval scope or an explicit manual input. If missing, record the gap.
Configuration and implementation checks
| Configuration item | Setting or check for this page | Relation to the sample |
|---|---|---|
| Associated knowledge bases | Select one or more knowledge bases using the same vector model | If the authorization letter and the after-sales ticket use different vector models, they cannot be associated together |
| Search mode | Compare semantic, full-text, and hybrid retrieval against the test question | Ticket number A-1001 suits full-text retrieval; authorization scope suits semantic retrieval |
| Citation limit | Unit is tokens; this hypothetical trial starting point is 3000 | Chosen to accommodate the full authorization sentence and ticket fields; compare with 1500 to see whether content is truncated |
| Minimum relevance | Takes effect only under semantic retrieval or when result reranking is used | Label relevant and irrelevant samples before adjusting the threshold; not used to exclude unauthorized records |
| Question optimization | Used for coreference resolution and question expansion | Check whether the optimized question changes party B to another party |
| Result reranking | Requires a ReRank model and must be enabled in settings | Compare the ranking position of the authorization letter candidate before and after reranking |
Three common errors
- Treating a retrieval hit as valid authorization. The input is the draft sentence "B still holds the brand authorization in January 2025." The comparison object is the authorization letter's validity end date of December 31, 2024. The observed result is that the period stated in the draft sentence exceeds the original text's range, so the claim is unsupported.
- Treating a pending-review ticket as completed handling. The input is the ticket text "ticket status is pending review." The comparison object is any statement claiming the ticket has been handled to completion. The observed result is that the original text does not state a handling outcome, so the matter is pending verification.
- Treating minimum relevance as a permission filter. The input is an after-sales ticket unrelated to the authorization. The comparison object is the cited content returned by retrieval. The observed result is that the relevance threshold affects only sorting and filtering; it does not prevent unauthorized records from being recalled. Permission requires downstream implementation.
How to verify the setup
- Input the hypothetical authorization letter text and the draft sentence. Compare the authorization period stated in the draft sentence with the validity period field in the original text. Observe whether the draft sentence is marked as unsupported. If retrieval does not return the authorization letter, record this as a gap.
- Input ticket number A-1001 for full-text retrieval. Compare whether the returned content contains the ticket number and status field. Observe whether the status shows pending review. If an empty array is returned, use the explicit paper input [] to check the downstream design.
- Input the authorization scope question for semantic retrieval. Compare the authorization letter candidate's position and score before and after reranking. Observe whether adjusting the minimum relevance changes only the sorting. Whether reranking is enabled depends on the model and settings; when not enabled, record that this path was not executed.