What the documents look like
Platform refunds and returns occur continuously per order. A refund is not necessarily tied to a promise made in a single livestream session, so the material set for one order can span several stages that are recorded in different systems. The after-sales backend, customer service chat, WMS receiving records and payment refund records each capture a different part of the process. The main documents are the refund application, the return logistics waybill, quality inspection photos and the refund approval. One order can sit at different nodes across these materials at the same time.
Product information and selection Q&A must extract citable original text from these materials. Citation tracing requires that an answer can be returned to a specific material, a specific field and a specific time range. The knowledge base search node outputs a citation array whose length can be zero; an empty array still lets the downstream output path run. The citation upper limit is controlled in tokens. The minimum relevance value takes effect only under semantic retrieval or result reranking. Hybrid retrieval merges multiple result paths through RRF, and the rerank score can be used for filtering. Linked knowledge bases must use the same vector model; bases with different vector models cannot be linked together. Whether a retrieval result is adopted still depends on review of the valid version. Retrieval itself does not constitute authorization.
Hypothetical drill sample: the refund application R-2024-0312 states: order number A-88231, application date 12 March 2024, return reason entered as "item does not match description", application status "pending review". The return logistics waybill L-2024-0315 states: order number A-88231, signed date 15 March 2024, signed status "signed". The draft sentence to be judged is: "Order A-88231 has completed the return and refund."
Constraints at the citation tracing step
The search node returns citations as an array, and a zero-length array is a normal output rather than a workflow failure. The citation upper limit is expressed in tokens, so a long chunk can consume the budget faster than a short one. The minimum relevance value filters low-relevance results only when semantic retrieval or result reranking is in use. When hybrid retrieval is active, RRF combines the semantic and full-text paths, and reranking can produce a score between 0 and 1 that supports filtering. A high similarity or rerank score does not decide whether the cited record belongs to the right order, the right stage or the right authorization state.
The linked knowledge base must use the same vector model. Checking the vector model identifier and excluding bases that do not match is a configuration check, not a retrieval experiment. Retrieval is not authorization. A status such as "pending review" in the source material does not become an approval because a citation was returned. Whether a draft sentence is supported depends on comparing the same subject, the same period and the same field. A field that is not stated, or evidence that only shows a related stage, remains unsupported or pending verification.
Configuration and implementation checks
| Configuration item | Hypothetical starting point for this drill | Reason and comparison object |
|---|---|---|
| Linked knowledge bases | Select only the same-vector-model bases related to product information and refund applications | Compare with a base using a different vector model to verify whether joint linking is possible |
| Search mode | Hybrid retrieval | Compare with pure semantic retrieval to observe hits on numbered queries |
| Citation upper limit | Set to 3000 tokens | Compare with 1500 tokens to observe whether citation content is truncated |
| Minimum relevance | Set to 0.3 | Compare with 0.5 to observe whether low-relevance fragments are filtered |
| Question optimization | Enabled | Compare with disabled to observe retrieval on multi-turn follow-up questions |
| Result reranking | Enabled | Compare with disabled to observe ordering changes when candidates are numerous |
These values are starting points for this paper drill, not official defaults.
Three common errors
- Treating retrieval output as business authorization. Error: reading the "pending review" status in a citation as an approved refund. Check: input the hypothetical sample text, compare the draft sentence with the status field in the original, and observe whether a jump from "pending review" to "completed" exists.
- Linking knowledge bases across vector models. Error: linking bases with different vector models and then running retrieval. Check: verify the vector model identifier of each base, exclude those that do not match, and record the actual observation. Do not run a cross-model retrieval to compare missed recalls.
- Using a similarity score in place of source verification. Error: treating a high rerank score as proof that the citation supports the draft sentence. Check: compare the draft sentence with the cited original field by field, and observe whether subject, object and status agree.
How to verify the setup
- Input the refund application and return logistics waybill text from the hypothetical sample. Compare the draft sentence "Order A-88231 has completed the return and refund" with the status fields in the original. Observe whether the original states "completed" or an equivalent expression. Expected: the original shows only "pending review" and "signed", so the draft sentence is unsupported.
- Input an explicit paper empty array []. Compare whether the downstream answer step still runs, and observe whether the output path is interrupted. Expected: an empty citation does not cause the workflow to error out.
- Input the refund application and return logistics waybill for the same order. Compare the order number and date fields in the two materials, and observe whether they share subject, period and field. Expected: the order numbers match, but the refund status and logistics status belong to different fields and cannot support each other.
For citation input handling, check whether retrieval output is passed to the answer step and verify each answer against the cited original. For citation display verification, use a traceable sample to check the displayed original; do not substitute the thinking process or answer fluency for source evidence. For material timeliness, test sources with the current version and a historical version; whether a citation is adopted still depends on review of the valid version.