What the documents look like
Channel diversion review work concerns regional and price order. It does not calculate rebate amounts. The working materials are diversion complaint forms, anti-counterfeit code records, penalty notices and investigation photographs. These arrive from anti-counterfeit code systems, e-commerce platform prices, distributor shipment records and market inspections. Updates are weekly or irregular.
The forms differ in shape. A complaint form carries the complaining party, the party complained against, the region involved and a date. An anti-counterfeit code record carries a code range, a query time and a query location. A penalty notice carries the penalized party, the clause relied on and an execution status. An investigation photograph carries a capture time, a location and the visible content.
At handover these materials sit in different states: submitted, pending review, signed, effective. The retrieval step returns cited content for a question. It does not decide whether a material is authorized or whether an action has been carried out. Operations permission and handover management depend on reading those states correctly, and retrieval supplies only the cited passages that a person then compares against the state fields.
Constraints at the retrieval step
The knowledge base search node can associate one or more knowledge bases that use the same vector model. Knowledge bases with different vector models cannot be associated together. The vector model identifier is checked first, and non-matching bases are excluded. No cross-model retrieval is run in order to compare missed recall.
Search mode is compared across test questions: semantic retrieval, full-text retrieval and hybrid retrieval. No single mode is assumed to suit every material type. Full-text retrieval suits exact queries such as identifiers, models, proper nouns and error codes. Semantic retrieval suits natural language questions and semantically close content. Hybrid retrieval uses both and merges them through RRF.
The citation limit is measured in tokens. It is tested with materials of different lengths to see whether the cited content stays complete, and then adjusted. The minimum relevance value is a 0-1 number and applies only under semantic retrieval or when result re-ranking is used. Relevant and irrelevant samples are labelled first, then a threshold is compared against filtered results. Question optimization performs coreference resolution and question expansion; the original question is compared with the optimized question to check whether the subject was altered.
Semantic recall and relevance scores do not perform permission or entity isolation. The same-vector-model requirement for associated knowledge bases does not extend to business registration sites or other external systems. RRF is a fusion process for mixed results. Empty output is returned as an array and its length can be zero.
Hypothetical paper exercise sample. The material text to be checked reads: "Complaint form CX-2024-0317 states: the complaining party is East China distributor A, the party complained against is South China distributor B, the region involved is South China, the complaint date is 17 March 2024, and the status is pending review. Penalty notice CF-2024-0402 states: the penalized party is distributor B, the clause relied on is regional diversion clause 3, and the execution status is unsigned." The draft sentence to be assessed reads: "The penalty against distributor B has been fully executed and its regional authorization may be released." This sample is used to check whether the cited retrieval content supports that assessment. No real customer, company or lawful identity number is used.
Configuration and implementation checks
| Configuration item | Assumed trial starting point for this exercise | Reason and comparison method |
|---|---|---|
| Associated knowledge bases | Associate one knowledge base containing complaint forms and penalty notices | Check that the vector model identifiers match; bases with different vector models cannot be associated together |
| Search mode | Test semantic retrieval first on natural language complaint-form questions | Compare with full-text and hybrid retrieval results, observing differences in hits for identifier-type questions |
| Citation limit | Set a token amount sufficient to hold one full complaint form | Test with complaint forms of different lengths to see whether citations stay complete, then adjust |
| Minimum relevance | Label relevant and irrelevant samples first, then set a 0-1 threshold | Applies only under semantic retrieval or result re-ranking; compare results before and after filtering |
| Question optimization | Enable for short questions and multi-turn follow-ups | Compare the original question with the optimized question and check whether the subject was altered |
| Result re-ranking | Used when a re-ranking model is available and enabled by configuration | Compare ordering before and after re-ranking and observe changes in relevance scores |
Three common errors
- Treating a retrieval hit as authorization confirmation. Input: a complaint form with status pending review and a penalty notice with execution status unsigned. If a draft is written from the retrieved penalty notice stating that the penalty has been executed, compare the notice's execution status field "unsigned" with the draft sentence "fully executed." The two are inconsistent. Retrieval returns cited content only; it does not determine execution status.
- Associating knowledge bases with different vector models. Input: two knowledge bases whose vector model identifiers differ. If they are associated together, compare the two vector model identifiers and observe that they do not match. The correct action is to check the identifiers and exclude non-matching bases. No cross-model retrieval is run to compare missed recall.
- Treating minimum relevance as a permission filter. Input: a knowledge base containing unrelated business records. If the minimum relevance is set high in the expectation of excluding unrelated records, compare results before and after filtering and observe that the relevance threshold does not filter by business entity or permission. Semantic recall and relevance scores do not perform permission or entity isolation.
How to verify the setup
- Using the complaint form text and the draft sentence from the hypothetical sample as input, compare the draft sentence "the penalty has been fully executed" with the penalty notice execution status field "unsigned," and observe whether the cited retrieval content contains that field. If the cited content contains only the pending-review complaint form, the draft sentence is unsupported and the evidence gap is recorded.
- Using two knowledge bases whose vector model identifiers differ as input, compare their vector model identifiers and observe whether they match. If they do not match, exclude the joint association and record the actual observation without inferring a runtime result.
- Using one full complaint form as input, compare the citation limit token amount with the actual length of the complaint form and observe whether the cited content is complete. If the cited content is truncated, adjust the citation limit and compare again. Empty output is returned as an array and its length can be zero; an explicit paper input of [] can be used to check the downstream design.