What the documents look like
Permit handling happens before declaration and determines whether goods may be imported or exported lawfully. Store service and internal training therefore build a searchable knowledge base around four material types: automatic import licences, dual-use item licences, contracts, and product descriptions. These materials arrive from commerce authority systems, customs commodity codes, supplier documents, and customer statements of use. Each shipment can produce a new licence type or a new statement of use, and updates are irregular.
When a customer asks which certificate a given item requires, the retrieved reference content shapes what the store prepares next. A retrieval deviation carries into customs clearance and contract performance. The knowledge base search node can associate one or more knowledge bases that use the same vector model, and that condition is a hard constraint when a store runs several business lines in parallel.
Constraints at the retrieval step
Licence names, commodity codes, and regulatory certificate numbers are exact-query objects. Semantic retrieval can return a similar but different certificate entry, while full-text retrieval is more direct for numbers and proper nouns. Hybrid retrieval merges multiple result paths through RRF. A ReRank model reorders candidate text and returns a score between 0 and 1. The minimum relevance value takes effect only under semantic retrieval or when result reranking is used. The reference cap is measured in tokens, and licence materials vary widely in length, so a low cap can cut off a key passage of a statement of use. Question optimisation handles coreference resolution and question expansion; in a store's multi-turn follow-up setting it can complete the subject, and it adds a model call and extra latency.
The retrieval step returns reference content only. It does not perform permission checks or entity isolation, and it does not decide whether a given employee may view a given licence record. An empty result is output as an array whose length can be zero, and the downstream path still runs.
Configuration and implementation checks
The settings below apply to the hypothetical drill sample on this page. The values are a hypothetical starting point for this exercise, used to compare retrieval behaviour before and after adjustment, and are not official defaults.
| Configuration item | Hypothetical starting point for this exercise | Reason and comparison method |
|---|---|---|
| Associated knowledge bases | Associate only the store permit knowledge base that uses the same vector model | Check the vector model identifier of each candidate base, exclude those with a different identifier, then compare single-base and multi-base retrieval results |
| Search mode | Run one round of semantic retrieval and one of full-text retrieval, then test hybrid retrieval | Use the same set of licence-number questions and natural-language questions to compare the returned content of the three paths |
| Reference cap | Hypothetically set to 3000 tokens | Test one round with a long statement of use and one with a short certificate entry, then compare whether key passages are truncated |
| Minimum relevance | Hypothetically set to 0.4 | Label relevant and irrelevant samples first, then compare the share of irrelevant fragments entering the references at different thresholds |
| Question optimisation | Off first, then on for multi-turn follow-up | Compare whether the subject of the original question and the optimised question is changed incorrectly, and record the latency difference |
| Result reranking | Off first, then on for questions with many candidates | Compare the ranking position and the 0-1 score distribution for the same question before and after reranking |
Three common errors
- Associating knowledge bases that use different vector models. The knowledge base search node requires associated knowledge bases to use the same vector model, and cross-model association has no documentary support. Executable check: inspect the vector model identifier of each base to be associated, exclude those whose identifiers do not match, record the actual association scope, and do not infer the runtime result of cross-model retrieval.
- Treating minimum relevance as a permission or entity filter. This parameter is a 0-1 filtering threshold that takes effect only under semantic retrieval or when result reranking is used. Executable check: run the same query at different thresholds, observe how the returned reference content changes, and confirm that a threshold change affects relevance filtering only and does not change the range of accessible knowledge bases.
- Treating retrieved reference content as business authorisation. Retrieval outputs reference content only; it does not perform approval, write to a database, or synchronise with an external system. Executable check: inspect whether downstream nodes include a separate form collection, manual review, or system integration design, and record any path that has retrieval results only as having a pending authorisation step.
How to verify the setup
Hypothetical drill sample: a store knowledge base holds a product description whose source text reads, "This product is an industrial centrifugal pump, 7.5 kilowatts, intended for export to Southeast Asia, with the customer's stated use being factory cooling circulation." A separate automatic import licence entry records in its certificate status field, "Application submitted, pending review." A draft sentence for internal training reads, "The product has obtained the automatic import licence and shipment can be arranged." The request to be judged is, "Please confirm whether the product's licence is in effect and arrange shipment."
- Input the draft sentence and the source product description, and compare the certificate status field "Application submitted, pending review" with the draft sentence "has obtained the automatic import licence." The condition is met when retrieval returns the source text of that certificate entry and the draft sentence is marked as inconsistent with the source status, since pending review does not equal in effect.
- Input the request to be judged and the licence entry, and compare "arrange shipment" in the request with the retrieved content. The condition is met when the retrieval result contains no shipment authorisation clause and the request is recorded as beyond what the retrieval step can confirm, requiring separate verification of the approval status.
- Input a short question that lacks subject information, and compare the retrieval terms with question optimisation off and on. The condition is met when the optimised retrieval terms complete the product and certificate type, the returned reference content corresponds to the product description, and the subject is not replaced with a different item.
Empty-result check: use an explicit paper input of an empty array to check the downstream design, observe whether the output path still runs, and record the downstream node's actual behaviour when the reference length is zero, without presuming that retrieval necessarily returns an empty array.