What the documents look like
Franchisee training and opening coaching materials come from the training system, the supervisor app, store photographs and the headquarters SOP library. The everyday documents are training sign-in sheets, opening checklists, SOP manuals and assessment scores. This category differs from franchise filing administration because opening coaching is a transfer of operating capability, covering factual states such as training completion, materials in place and corrective action on SOP steps. Document comparison for contracts and transaction materials must align subject, object, state and validity scope across sign-in sheets, checklists, SOP manuals and assessment scores. A sign-in sheet records people and dates, an opening checklist records a store and inspection items, an SOP manual gives the action standard, and an assessment score records a person and a score. These materials update per store and per month. After an update, actual training state and index state need separate confirmation; a successful upload does not by itself mean retrieval reflects the update.
Constraints at the document chunking step
Chunking must inspect paragraph completeness against the structure of each specific material, so that conditions, explanations and the content they constrain are not split apart. One group of source data may map to multiple indexes. At retrieval time the system recalls vectors first, then uses the vector ID to find the source data content in MongoDB; entries corresponding to the same source data group are merged, and the vector score takes the highest value. Semantic retrieval depends on vector distance, while full-text retrieval depends on keyword hits. Numbers, models and proper nouns suit full-text or hybrid retrieval. The minimum relevance value is a filter threshold and applies only in semantic retrieval or when result reranking is used; it does not enforce permissions or entity isolation. A requirement that libraries sharing one vector model stay together does not extend to external systems needing the same model. RRF is a fusion process for mixed results, not a toggle.
Hypothetical paper exercise sample: the source text reads "From 1 March 2025 to 5 March 2025, store manager B of franchisee A's store attended cashier and inventory training. The sign-in sheet records B signing in on 1 March, 2 March and 3 March. The assessment score records B scoring 82. The opening checklist records that on 5 March the cashier counter materials at store A were in place, signed by inspector C." The draft sentence to be judged is "B has completed all training and meets opening conditions." Judging requires the same subject, the same period and the same field. The sign-in sheet records three sign-in days, the assessment score records 82, and the checklist records materials in place. None of these states that all training is complete or that opening conditions are authorized. The draft sentence is unsupported, and the evidence gap must be preserved.
Configuration and implementation checks
| Configuration item | Hypothetical starting value for this exercise | Reason for the choice and comparison method |
|---|---|---|
| Search mode | Hybrid retrieval | Materials contain numbers and natural language, so semantic recall and full-text recall are both used. Compare candidate content for the same question under the two modes. |
| Minimum relevance | Hypothetical starting value 0.3 | This value is the starting point for this exercise because materials contain precise fields such as sign-in dates. Compare against 0.5 and observe the change in candidate count after filtering. |
| Reference limit | Hypothetical starting value 2000 tokens | This value is the starting point for this exercise because a single checklist and sign-in sheet are short. Compare against 4000 tokens and observe the coverage of referenced content. |
| Question optimization | Enabled | In multi-turn follow-up scenarios, references are completed. Compare recalled content for the same follow-up with the setting on and off. |
| Result reranking | Enabled | When candidates are numerous, a second ordering is applied. Compare the ranking position of the same question with reranking on and off. |
| Paragraph completeness check | Check paragraph by paragraph against material structure | Inspect whether conditions, explanations and the content they constrain are split apart. Compare recalled content for the same material before and after segmentation. |
Three common errors
- Treating a successful upload as retrieval already updated. Input is the upload operation record for one SOP manual. The comparison object is the actual training state and index state. The observable result is whether retrieving the manual's new clause returns the updated content. Without checking training state, retrieval cannot be judged updated.
- Treating minimum relevance as a permission filter. Input is a retrieval request from one franchisee account. The comparison object is the minimum relevance setting and the returned content. The observable result is whether unrelated business records are excluded by the similarity threshold. This setting does not enforce entity isolation, so permission cannot be judged effective on this basis.
- Treating a pending review request as approved and executed. Input is an opening materials supplement request submitted by one store. The comparison object is the request state and the approval record. The observable result is whether a signature or approval field exists. A pending review does not show that operating permission is absent, and it does not equal execution.
How to verify the setup
- Retrieve "B training sign-in 1 March" using the hypothetical sample source text. Input is the sign-in sheet source text. The comparison object is whether the returned content contains B, 1 March and the sign-in field. The observable result is that the returned content can be traced back to the source text and does not present the conclusion "all training complete." If the returned content lacks the subject or the date, record the gap.
- Check the hypothetical sample draft sentence "B has completed all training and meets opening conditions." Input is the draft sentence together with the sign-in sheet, assessment score and checklist source texts. The comparison object is whether the source text states completion of all training and authorization of opening conditions. The observable result is that the source text contains only three sign-in days, a score of 82 and a record of materials in place, so the draft sentence is judged unsupported.
- Retrieve "store A cashier counter materials 5 March" using the hypothetical sample checklist. Input is the checklist source text. The comparison object is whether the returned content contains store A, cashier counter, materials and 5 March. The observable result is that the returned content can be traced back to the checklist and does not present a signature-in-force conclusion. If the returned content is empty, use the explicit paper input [] to check the downstream design and record the actual observation.