What the documents look like
Permit handling happens before customs declaration and decides whether a shipment may lawfully cross the border. A retrieval result that cites the wrong permit type can therefore distort an order-fulfilment judgement. Materials come from commerce-authority systems, customs commodity codes, supplier files and customer end-use statements, and each shipment is checked again on its own. Everyday documents include automatic import licences, dual-use item licences, contracts and product descriptions. Within one shipment, the parties, goods scope and validity period across these materials need to stay consistent. The knowledge-base search node can associate one or more knowledge bases that use the same vector model; bases on different vector models cannot be associated together. When checking, confirm the vector model identifier first, then exclude those that do not match.
Constraints at the retrieval step
Permit materials carry licence numbers, commodity codes and end-use wording. Full-text retrieval suits numbers and proper nouns, semantic retrieval suits natural-language questions, and hybrid retrieval merges multiple result paths through RRF. Minimum relevance takes effect only in semantic retrieval or when result re-ranking is used; it is a filter threshold and does not perform permission or entity isolation. A retrieval hit does not mean a licence has been authorised or a contract has taken effect. The citation limit is measured in tokens, and materials with large length differences need actual testing before the limit is adjusted. Question optimisation performs coreference removal and question expansion, adds one model call, and may change the subject of the question, so the original question and the optimised question need to be compared.
Hypothetical paper exercise: a foreign-trade compliance role receives a customer follow-up during order-fulfilment Q&A. One product description in the knowledge base reads: "This product is used for industrial equipment lubrication and does not fall within the scope of dual-use item control." One contract reads: "The seller shall arrange shipment within ten working days after receiving the automatic import licence." A draft sentence to be judged reads: "The contract has taken effect and the goods can be shipped immediately." When checking, compare the contract text with the draft sentence: the contract text only states a shipping condition tied to the licence, and does not state the contract's effective status or that the licence has been obtained. The subject, status and condition in the draft sentence all lack support in the source text, and the item should be recorded as pending verification.
Configuration and implementation checks
| Configuration item | Hypothetical starting point for this exercise | Reason and comparison method |
|---|---|---|
| Associated knowledge bases | Associate only the licence and contract bases that share one vector model | Check the vector model identifier first, exclude bases on other models, and do not run cross-model retrieval comparisons |
| Search mode | Start with hybrid retrieval | Compare semantic, full-text and hybrid results for the same test question, and choose according to what is hit |
| Citation limit | Set a token amount that can cover one complete product description | Test with materials of different lengths to see whether content is truncated, then adjust the limit |
| Minimum relevance | Label relevant and irrelevant samples first, then set a threshold | Takes effect only in semantic retrieval or result re-ranking; compare filtering against the labelled samples |
| Question optimisation | Enable for multi-turn follow-ups, disable for number-type questions | Compare the original question with the optimised question and check whether the subject has been changed incorrectly |
| Result re-ranking | Enable when there are many candidates | Re-ranking requires a model and must be enabled in settings; compare the re-rank score with the vector score |
Three common errors
- Treating a retrieval hit as an authorisation status. The input is the draft sentence "The contract has taken effect and the goods can be shipped immediately"; the comparison object is the contract text; the observable result is that the draft sentence lacks support in the source text and should be recorded as pending verification. It must not be written as the contract not having taken effect.
- Treating minimum relevance as a permission or entity filter. The input is an unrelated query; the comparison object is the retrieval setting; the observable result is that the setting only filters low-relevance content and does not exclude unauthorised records. Retrieved results still require manual checking of the authorisation boundary.
- Treating the RRF of hybrid retrieval as a separate switch that can be turned on. The input is the search-mode setting; the comparison object is the documented description; the observable result is that RRF is the fusion process for hybrid results, and the configuration item is the search mode. It must not be written as RRF being enabled on its own.
How to verify the setup
- Input the draft sentence and the contract text from the hypothetical exercise, and compare whether subject, status and condition match. The observable result is that a draft sentence lacking support in the source text is recorded as pending verification; only when the same field takes mutually exclusive values in the source text is it recorded as a contradiction.
- Input the same test question and use semantic retrieval, full-text retrieval and hybrid retrieval separately, then compare the hit content of the three result sets. The observable result is that number-type questions are hit in full-text retrieval and natural-language questions are hit in semantic retrieval; when hybrid retrieval returns richer results, record it as actually observed.
- Input an empty-reference result and compare whether the downstream design accepts a reference array of length zero. The observable result is that the output path runs through; it must not be claimed that a fabricated rule always makes retrieval return an empty array.