What the documents look like
Seller ratings and experience score maintenance happens after fulfillment. The working materials are negative-review handling records, experience score breakdowns, service diagnostic reports and remediation ledgers. These documents update daily or weekly, and they are used by customer service supervisors and quality inspection staff to attribute negative reviews, request supporting evidence and track post-remediation indicators.
The comparison task requires checking order facts and after-sales status recorded in a negative-review ticket against contract or transaction text, field by field. The retrieval step locates usable source text. A retrieval hit is not an authorization, and it does not show that a contract has taken effect or that remediation has been completed.
Constraints at the retrieval step
The knowledge base search node can associate one or more knowledge bases that use the same vector model. Semantic retrieval, full-text retrieval and hybrid retrieval are compared per test question; no single mode is assumed to suit every document. Hybrid retrieval merges multiple result paths through RRF, and RRF is a fusion process rather than a switch that can be turned on. The reference cap is measured in tokens. The minimum relevance value applies only during semantic retrieval or when result reranking is used. Question optimization performs coreference resolution and question expansion and adds one model call. Image retrieval requires image input and matching model capability. Reranking requires a rerank model and must be enabled according to settings. A similarity score does not carry permission or entity isolation duties. Knowledge bases with different vector models cannot be associated together; the identifiers are checked and non-matching bases are excluded.
Hypothetical paper test case: On 10 June 2025, a store quality inspection role checks a negative-review ticket, order number A-20250601-017. The buyer applied for a return and refund on 3 June. The after-sales status record reads "return signed for, refund in process." The contract excerpt reads: "This order is eligible for seven-day no-reason return; refund review is completed within three working days after the return is signed for." The draft sentence to be judged: "The refund for this order was completed on 5 June." The retrieval input is the two source passages above. The comparison object is "was completed" in the draft sentence against "in process" and "completed refund review" in the source text. The source text does not state when the refund arrived. The draft sentence is unsupported, and the evidence gap is recorded.
Configuration and implementation checks
| Configuration item | Value or check item in this sample | Basis and comparison method |
|---|---|---|
| Associated knowledge bases | Only the ratings maintenance base with the same vector model | Check that the vector model identifier matches; exclude bases with different models |
| Search mode | Full-text retrieval for the order number first, then semantic retrieval for the status description | The order number is an exact identifier and the status description is natural language; compare hits separately |
| Reference cap | 800 tokens as a hypothetical starting point for this trial | The source contains a contract clause and a ticket passage; compare longer and shorter material for completeness |
| Minimum relevance | 0.3 as a hypothetical starting point for this trial | Label relevant and irrelevant samples first, then compare whether the clause text survives filtering |
| Question optimization | Not enabled in this sample because the question contains an explicit order number | Compare the original question with the optimized question and check whether the subject was altered |
| Result reranking | Enabled according to the rerank model actually available | The rerank score can be used for filtering and is compared with the semantic score |
Three common errors
- Treating a retrieval hit as evidence that the refund was completed. The input is the draft sentence "the refund was completed on 5 June." The comparison object is "refund in process" and "completed refund review" in the source text. The observed result is that the source does not state the arrival time, so the sentence is judged unsupported.
- Treating minimum relevance as a permission or entity filter. The input is a negative-review record from an unrelated store. The comparison object is the relevance threshold and the record's ownership. The observed result is that the threshold affects ranking and filtering only; it does not exclude unauthorized records, and access rights are checked separately.
- Associating knowledge bases with different vector models. The input is two bases with different vector model identifiers. The comparison object is the node association requirement. The observed result is that a mismatched identifier leads to exclusion; no cross-model retrieval is performed to compare missed recalls.
How to verify the setup
- Input the order number and contract text from the hypothetical sample. Compare the returns from full-text retrieval and semantic retrieval. Observe whether both the clause passage and the ticket passage are located. If only one is found, record the gap and adjust the reference cap.
- Input the draft sentence "the refund for this order was completed on 5 June." Compare its field values against "in process" and "completed refund review" in the source text. Observe whether mutually exclusive values appear for the same subject, same period and same field. When the arrival time is not stated, judge the sentence as pending verification.
- Input a clearly labelled paper empty result []. Compare how the downstream node handles a reference length of zero. Observe whether the output path runs through. Do not claim that a fictional rule search necessarily returns an empty array.