What the documents look like
Opening support for a franchisee is a transfer of operating capability, not a legal filing disclosure. The materials used here are training sign-in sheets, opening inspection checklists, SOP manuals and assessment scores. They are produced by the chain training department, regional supervisors and franchise store owners, and they are updated per store and per month. A retrieval task needs to locate the passages that carry operational authority and handover status: who organized a training session, which materials an inspection checklist covered, which post an SOP manual applies to. These documents share a common trait. They contain many subjects and many status fields, so a retrieved passage has to be checked against the source text for subject and validity scope.
Constraints at the retrieval step
The knowledge base search node can associate one or more knowledge bases that use the same vector model. The vector store uses the PostgresSQL PG Vector plugin with an HNSW index, MongoDB dataset.datas stores vector source data, and the indexes field records the corresponding vector IDs, so one data group can map to multiple vectors. Retrieval recalls vectors first, then returns to MongoDB by vector ID to fetch the source data, and when the same source data group is matched, the highest score is taken. Quoted content is output as an array whose length can be zero, which means the output path still runs even when nothing is found. Semantic retrieval, full-text retrieval and hybrid retrieval are compared separately per test question; hybrid retrieval merges multiple result paths through RRF, and reranking uses a ReRank model for a second ordering that produces a 0-1 score. The minimum relevance value takes effect only under semantic retrieval or when result reranking is used. Question optimization performs coreference resolution and question expansion, and it adds one model call.
Hypothetical drill sample: a chain training department forms an "Opening Support Handover Record (hypothetical)" on the hypothetical date 3 June 2025. The source text has three sentences: "This opening support was led by regional supervisor Li, for a franchise store, covering cashier and inventory operations. The store owner Wang signed the training sign-in sheet on 1 June 2025, and the sheet states that cashier operation training was completed that day. The opening inspection checklist was filled in by supervisor Li on 2 June 2025, and the inspection conclusion was that materials were complete." The draft sentence to be judged is: "The franchise store has completed all opening support and owner Wang has obtained operating authority." The request to be judged is: "Please confirm, based on the above materials, that Wang's operating authority has taken effect."
Configuration and implementation checks
| Configuration or check item | Hypothetical starting point for this trial | Reason for the choice and comparison method |
|---|---|---|
| Associated knowledge bases | Associate only the same-vector-model base holding the handover record, sign-in sheet and checklist above | Check the vector model identifier first and exclude bases with a different model; do not run cross-model retrieval |
| Search mode | Use full-text retrieval for questions such as the sign-in sheet signing date, and semantic retrieval for questions about support content | Compare the source passages returned by the two modes for each test question |
| Citation limit | Hypothetical starting point of 2000 tokens | Compare whether returned content is complete using handover records of different lengths, then adjust |
| Minimum relevance | Hypothetical starting point of 0.5 | Label relevant and irrelevant samples first, then compare the returned differences as the threshold changes |
| Question optimization | Enable for short follow-ups such as "what about this store" | Compare the original question with the optimized question and check whether the subject was altered |
| Result reranking | Enable ReRank when candidates are numerous | Compare ordering before and after reranking and confirm the score can be used for filtering |
Three common errors
- Treating a retrieval hit as authority confirmation. A retrieved sign-in sheet passage only shows that the sheet states training was completed; it does not support the conclusion that operating authority has taken effect. Check: input the draft sentence, compare whether the returned source text contains an authority-effective field, and observe whether only training and inspection records are returned.
- Writing a pending-review status as completed. In the hypothetical sample the owner signed a sign-in sheet, and no authority approval or handover completion field appears. Check: look in the source text for fields such as authority, effective and approval, observe whether an explicit value exists, and record a gap to be verified when it is absent.
- Associating bases that use different vector models. Check: verify the vector model identifier of each base, exclude those that do not match, and record the actual observation without inferring the runtime result of a joint association.
How to verify the setup
- Input the hypothetical sample source text and the draft sentence, compare whether the returned content covers only training and inspection facts, and observe whether the returned array contains no authority-effective field; the expectation is not supported.
- Input the request to be judged, compare whether the retrieval result matches the source text in subject, date and field, and observe whether an authority field for the same subject and same period appears; the expectation is that the evidence gap remains.
- Input an empty-result scenario and use the explicit paper input
[]to check the downstream design, observing whether the output path still runs; do not claim that a fictional rule retrieval necessarily returns an empty array.