What the documents look like
Complaints and official replies are addressed to platforms or regulators, so the material must carry formal evidence and remediation wording. The main data sources are the 12315 platform, platform complaint centres, customer service chats and the order system. The documents handled day to day are complaint tickets, regulatory inquiry letters, order evidence and remediation reports. These materials update irregularly and case by case, which means knowledge-base retrieval for operating reviews has to be organised around individual cases rather than a fixed synchronisation cycle.
The most time-consuming steps are reconstructing the transaction, preparing the evidence pack and submitting the remediation reply. Operating reports and anomaly reviews need to locate comparable handling precedent in historical tickets and replies, and the retrieved content directly supports the review conclusion. A knowledge-base search node can be linked to one or more knowledge bases that use the same vector model; this is a precondition for retrieval.
Constraints at the retrieval step
Complaint tickets, regulatory inquiry letters, order evidence and remediation reports are maintained by different roles and differ substantially in structure. Semantic retrieval relies on vector similarity and suits natural-language questions. Full-text retrieval relies on keyword hits and suits identifiers, model numbers and proper nouns. Hybrid retrieval uses both recall paths and merges the results through RRF. The reference limit is measured in tokens. The minimum relevance value takes effect only under semantic retrieval or when result reranking is used. Question optimisation performs coreference resolution and question expansion, and adds one model call.
Retrieval is not authorisation. Knowledge-base search returns reference content only; it does not judge whether material has been approved or may be submitted externally. When nothing is retrieved, the output path still completes, and the reference array may have a length of zero.
Hypothetical paper exercise sample: a shop receives a regulatory inquiry letter asking it to explain the refund handling basis for one order. The knowledge base holds a complaint ticket whose original text reads: "Ticket number TS-2024-0312. The complainant claims the refund for order A has not arrived. Customer service replied on 15 March 2024 that the refund had been initiated and that bank processing takes 3 to 5 working days." It also holds a draft remediation report whose original text reads: "For TS-2024-0312, this shop has verified the refund record and intends to submit the bank statement voucher before 20 March 2024." A draft sentence now awaits checking: "This complaint has been fully handled and the refund has arrived." The draft sentence must be checked against the material above.
Configuration and implementation checks
| Configuration item | Setting or check relevant to this page | Explanation |
|---|---|---|
| Linked knowledge bases | Select the knowledge bases holding complaint tickets, regulatory inquiry letters, order evidence and remediation reports | Multiple knowledge bases must use the same vector model; otherwise they cannot be linked together |
| Search mode | Compare semantic, full-text and hybrid retrieval against the test question | Identifier queries favour full-text or hybrid; natural-language questions favour semantic or hybrid |
| Reference limit | Measured in tokens; test content completeness with real material of differing lengths | This hypothetical trial starting value is 2000 tokens, chosen because a single remediation report is usually long; compare with a 500-token starting value to see whether content is truncated |
| Minimum relevance | Takes effect only under semantic retrieval or when result reranking is used | Label relevant and irrelevant samples first, then adjust the threshold; this hypothetical starting value is 0.4, compared with 0.6 for filtering effect |
| Question optimisation | Used for coreference resolution and question expansion | Compare the original question with the optimised question and check whether the subject was altered incorrectly |
| Result reranking | Requires a ReRank model enabled under the settings | Reranking produces a score from 0 to 1, which can be used for filtering |
Three common errors
- Treating a retrieval hit as authorised execution. Input: "TS-2024-0312 refund arrived". Retrieval returns the complaint ticket original text, which records only the customer service reply that the refund "had been initiated" and does not record arrival. Compare the draft sentence "the refund has arrived" with the ticket original text: the two states do not match. The observable result is that "has arrived" in the draft sentence has no corresponding record in the ticket and remains unsupported.
- Linking knowledge bases that use different vector models. Input: two knowledge base identifiers, one of which uses a different vector model. Checking the identifiers shows the same-vector-model condition is not met. Exclude the non-conforming one and do not run a cross-model retrieval to compare missed recalls.
- Treating minimum relevance as a permission or entity filter. Input: a query unrelated to the complaint, with minimum relevance set to 0.4. That value filters low-relevance results only; it does not exclude unrelated business lines or unauthorised records. The observable result is that retrieval may still return similar text from another business line, and a person must check whether the reference content falls within this case.
How to verify the setup
- Input the ticket original text from the hypothetical sample and the draft sentence "This complaint has been fully handled and the refund has arrived." Compare the draft sentence with the ticket record "the refund had been initiated and bank processing takes 3 to 5 working days." The condition is met when retrieval returns the ticket original text and "has arrived" in the draft sentence is identified as inconsistent with the recorded state and logged as pending verification.
- Input the draft remediation report original text from the hypothetical sample and the query "TS-2024-0312 remediation submission time". Compare whether the retrieval result contains "intends to submit the bank statement voucher before 20 March 2024." The condition is met when full-text or hybrid retrieval hits that sentence and the reference content retains the date and object completely. If an empty array is returned, check the downstream design against that empty result and do not claim that an invented rule necessarily returns empty.
- Input two knowledge base identifiers, one of which uses a different vector model. Compare whether the vector models corresponding to the two identifiers are the same. The condition is met when, after confirming they differ, the non-conforming one is excluded and the actual observation is recorded, without running a cross-model retrieval and without inferring missed recalls or redundant results.