What the documents look like
Platform seller ratings and experience score maintenance deals with consumer feedback after post-sale fulfillment. The primary data sources named in the brief are Douyin Shop experience scores, Tmall service indicators, JD after-sales reviews and the customer service system. Day-to-day documents cluster around negative review handling records, experience score breakdowns, service diagnostic reports and rectification ledgers. In cross-border material and multilingual retrieval, a single negative review handling record may exist as both a Chinese original and a translation, and a retrieval result has to return to a specific source before it can be checked. The brief gives a daily and weekly update frequency, and material version changes affect whether a citation remains traceable.
Constraints at the citation tracing step
The knowledge base search node can select one or more knowledge bases that share the same vector model, and it outputs citations as an array whose length may be zero. That output is handed to the answering step, where each answer has to be checked against the cited source text. Retrieval itself does not perform authorization decisions, and a similarity score is not a permission or entity isolation basis.
Hypothetical drill sample: On 10 March 2025 a shop operator organizes a negative review handling record. The original text reads: "Order A-2031, the buyer reported on 2 March 2025 that the outer packaging was damaged, customer service registered a reshipment on 3 March 2025, reshipment tracking number SF-7788, and as of 9 March 2025 the buyer had not confirmed receipt." A separate experience score breakdown states: "From 2 March to 8 March 2025, this shop's logistics experience score fell from 4.6 to 4.4." The draft sentence to be judged reads: "This negative review has completed rectification and the experience score has recovered." The draft asserts two facts, completed rectification and score recovery, while the source text shows only a reshipment registration and a score decline. The draft and the source text do not agree.
Configuration and implementation checks
| Configuration item | Hypothetical starting point for this drill | Reason and comparison object |
|---|---|---|
| Associated knowledge bases | Select two knowledge bases with the same vector model | Compare against a knowledge base with a different vector model identifier, and exclude those that cannot be associated together |
| Search mode | Hybrid search | Compare against semantic search alone for hit behavior on identifier-type queries |
| Citation limit | Set to 800 tokens | Compare against the length of a single negative review record to avoid truncating source text |
| Minimum relevance | Set to 0.35 | Compare against the default value and observe whether source text is filtered out |
| Result rerank | Enabled | Compare against the disabled state to see whether candidate ordering moves closer to the source |
| Question optimization | On | Compare against the off state for retrieval completeness in multi-turn follow-ups |
Three common errors
- Treating retrieval output as an already authorized action. A returned citation only means a match was found. A reshipment registration in a negative review handling record is not completed rectification, and a score decline in an experience score breakdown is not recovery. The check compares the draft sentence against the source text on subject, period and field, and confirms whether the rectification status field is stated.
- Treating minimum relevance as a permission filter. Minimum relevance is a filtering threshold and does not exclude unrelated businesses or unauthorized records. The check confirms that the retrieval scope is limited to knowledge bases the operator may access, and verifies whether the returned citation belongs to the target shop.
- Treating an empty citation output as a workflow failure. Citation content length may be zero, and an empty array still passes through the output path. The check uses an explicit paper input of [] against the downstream design and observes whether the downstream handles it as an empty result, without inferring a specific failure cause.
How to verify the setup
- Input the hypothetical source text above and the draft sentence, then compare "completed rectification" in the draft against "registered reshipment" in the source on the same field and the same period. The source does not state that rectification was completed, so the draft is unsupported and should be recorded as pending verification.
- Input the experience score breakdown text and "the experience score has recovered" from the draft, then compare the field values for 2 March to 8 March 2025. The source shows a fall from 4.6 to 4.4, so the draft contradicts the source and should be judged inconsistent.
- Input a retrieval request containing only a Chinese question and observe whether the returned citations include the corresponding Chinese source text. If the returned citation is an empty array, check whether the downstream handles it as an empty result and record the actual observation, without asserting that retrieval will necessarily hit or necessarily return empty.