What the documents look like
A brand-run livestream carries its own scripts, inventory responsibility and after-sales handling, so the working documents are livestream scripts, product explanation cards, shift schedules and livestream reviews. The data sources named in the brief are Douyin Shop Compass, Juliang Qianchuan, livestream comments and transaction details. For store service and internal training, these materials are placed into a knowledge base so that the floor controller, host and ad buyer can retrieve them before a session or during a handover. Materials update frequently, and a retrieved passage has to line up with a specific session and a specific product; otherwise the training wording and the store-level execution drift apart. The knowledge base search node can associate one or more knowledge bases that use the same vector model. The search unit is the knowledge base; a collection only classifies data and does not change the search result.
Constraints at the retrieval step
Retrieval returns reference content. It does not grant authorization, approve anything or verify inventory. Reference content is output as an array and its length can be 0, so the output path still runs when nothing is found. Semantic search, full-text search, hybrid search, result reranking, question optimization and image retrieval are available paths; whether each one executes depends on the input type, model capability and parameter settings. The minimum relevance value takes effect only under semantic search or when result reranking is used, and it is a filter threshold. Hybrid search merges multiple result paths through RRF, which is a fusion process rather than a separate switch. Knowledge bases built on different vector models cannot be associated together; during checking, confirm the vector model identifier first, exclude those that do not match, and do not run a cross-model search to compare missed recalls.
Configuration and implementation checks
The table below is built around one hypothetical drill sample. Hypothetical drill sample: a paper training handover document from a brand livestream team states, "Session of 8 March 2025, host A used product explanation card V3 to explain the skincare set; the explanation card marks the set gift as a three-piece travel kit; the shift schedule records host A on duty from 20:00 to 23:00 that day." A separate draft sentence awaits judgment: "Host A completed gift-wording training for the 8 March session according to explanation card V3." The materials do not state a training completion record or a signature.
| Configuration item | Hypothetical starting point for this drill | Reason for the choice and comparison method |
|---|---|---|
| Associated knowledge bases | Associate only the store service and internal training knowledge base | Check the vector model identifier first; association is possible only when the identifiers match |
| Search mode | Run semantic search and full-text search separately | Use the same question to compare the two recall paths; do not assume one mode suits all materials |
| Reference limit | Start at 2000 tokens | Test whether content stays complete given the length difference between explanation cards and shift schedules, then compare with a smaller value |
| Minimum relevance | Start at 0.4 | Label relevant and irrelevant samples first, then compare filtering results at different values |
| Question optimization | Enable for multi-turn follow-up scenarios | Compare the original question with the optimized question and check whether the subject was changed incorrectly |
| Result reranking | Enable when a rerank model is available | Compare ordering before and after reranking, and check whether the score is used for filtering |
Three common errors
- Treating a retrieval result as an authorization credential. The draft sentence says training was completed, while the materials record only the explanation card version and the shift. No training completion record is stated, and a retrieved explanation card does not support the claim that training was completed. For the check, input the draft sentence together with the source material, compare subject, object, period and field, and note that the material states no training completion field, so the claim is unsupported.
- Treating the reference limit as a fixed number of items. The unit of the reference limit is tokens, and results become unstable when long and short chunks are mixed. For the check, input the same question and retrieve with 2000 tokens and with a smaller value, then observe whether key sentences from the explanation card are missing.
- Treating minimum relevance as a permission or entity filter. That value takes effect only under semantic search or result reranking, and it does not exclude unrelated businesses or unauthorized records. For the check, input an unrelated question, observe whether low-relevance fragments are still returned, and record the actual observation without assuming the result must be empty.
How to verify the setup
- For the draft sentence "Host A completed gift-wording training for the 8 March session according to explanation card V3," input the retrieval question "8 March host A gift-wording training record" and compare whether the returned content contains a training completion field. The source material contains only the explanation card version and the shift hours, so the expected return is explanation card and shift content, the training completion field is absent, and the claim is unsupported.
- For the same material, input the question "8 March host A duty hours" and compare whether the returned content matches the shift schedule entry "20:00 to 23:00." The expected hit is the shift schedule fragment; if a time from another session is returned, record it as pending verification.
- For the empty-result path, use the paper input
[]to check the downstream design and confirm the path still runs when reference content length is 0, without claiming that a given rule must return an empty array. Knowledge bases on different vector models are checked by identifier and excluded first, without running a cross-model retrieval comparison. Image retrieval and reranking require the corresponding input and model capability; when these are absent, record the step as not executed and keep the evidence gap open.