What the documents look like
Platform store ratings and experience score maintenance deals with consumer feedback after after-sales fulfilment. The working materials are negative-review handling records, experience score detail sheets, service diagnostic reports and rectification ledgers. These materials update daily or weekly and originate from platform experience scores, marketplace service indicators, after-sales review systems and customer service systems. Store operations staff, customer service supervisors and quality inspection specialists compare contract clauses against transaction records item by item, confirming whether the order, service commitment and after-sales handling cited by a given review match the original materials.
Three industry facts shape this page's task. First, the materials are feedback records produced after fulfilment, so a citation must point to a specific order and a specific handling stage rather than to a general service promise. Second, the same review may be supported by several materials at once, which means the comparison must track which document supplies which field. Third, the handling status recorded in these materials moves through stages, and a citation that collapses two stages into one cannot be verified against the source.
Constraints at the citation tracing step
Contract and transaction material comparison requires every citation to trace back to a specific material. The knowledge base search node selects one or more knowledge bases that share the same vector model, and outputs cited content as an array whose length may be zero. The retrieval result is passed to the answering step, where each answer is checked line by line against the cited original text. The minimum relevance value is a filtering threshold that takes effect only in semantic retrieval or when result reranking is used; it does not perform permission or entity isolation. Hybrid retrieval merges multiple result paths through RRF, and reranking requires a model and is enabled according to configuration. Knowledge bases built on different vector models cannot be associated together; the identifiers should be checked first and non-matching bases excluded.
Hypothetical drill sample: the source material reads, "The experience score detail sheet for a certain store covering 1 March 2025 to 31 March 2025 shows that the negative-review handling record for an order dated 15 March states 'consumer contacted to supply additional evidence, pending quality inspection review'; the service diagnostic report for that order is marked 'after-sales fulfilment complete, review attribution pending confirmation'." The draft sentence to be assessed is, "The negative review for this order has been fully handled and the experience score has been restored." In checking, the draft sentence is compared with the the source material reads the source states only pending quality inspection review and pending attribution confirmation, and does not state that handling is complete or that the experience score has been restored. The draft sentence is not supported.
Configuration and implementation checks
| Configuration item | Hypothetical starting point for this drill | Reason for the choice and comparison method |
|---|---|---|
| Associated knowledge bases | One knowledge base sharing the same vector model | Check the selected knowledge base identifier and material version; exclude bases on different vector models |
| Search mode | Hybrid retrieval | Uses semantic recall and full-text recall together, merging results through RRF |
| Minimum relevance | Hypothetical starting point of 0.5 for this drill | Compare against the 0-1 scores returned by retrieval and observe whether the cited content changes after filtering |
| Citation limit | Hypothetical starting point of 2000 tokens for this drill | Compare against the tokens actually cited per search and observe the length of cited content |
| Result reranking | Enabled for this drill | Requires a ReRank model; compare candidate ordering before and after reranking |
| Cited content output | Check the length of the output array | Compare against the input of the downstream answering step; when the array is empty, confirm the path still completes |
Three common errors
- Treating a retrieval hit as authorized execution. Knowledge base search returns an array of cited content; it does not indicate that negative-review handling has been approved or that a contract has taken effect. In checking, compare the cited original text with the draft sentence and observe whether the draft sentence goes beyond the status stated in the original.
- Treating minimum relevance as a permission or entity filter. Minimum relevance filters only low-relevance results and does not exclude unrelated business or unauthorized records. In checking, compare cited content before and after filtering and observe whether filtered items were removed solely because of a low score.
- Writing a pending-review status as completed. The source material states "pending quality inspection review" while the draft sentence states "fully handled." In checking, compare the same field value and observe whether the two are mutually exclusive.
How to verify the setup
- Input the hypothetical source material and draft sentence above, and compare the subject, object, status and validity scope of the draft sentence against the original. If the draft sentence contains "handling complete" or "experience score restored" that the original does not state, judge it unsupported and record the evidence gap.
- Input the hypothetical source material above and check whether the cited content output array contains that original text. If the array length is zero, confirm whether the downstream answering step is designed to accept an empty result; do not claim that retrieval necessarily returns an empty array.
- Input the hypothetical source material above and compare the order date, handling status and field in the cited content against the source material for the same subject, same period and same field. If the status field values are mutually exclusive, judge a contradiction; if the original does not state the field, judge it pending verification.