What the documents look like
Short-video affiliate distribution relies on long-term content exposure and product-link conversion rather than live explanation. The same script, linked-product list, commission plan and violation feedback are reused across language sites. Multilingual retrieval places a Chinese script and an English product description inside the same knowledge base for comparison. Material updates occur daily and weekly. During handover, operations must distinguish three states: material uploaded, material parsed, and index generated.
A script may carry a product number, a model designation and a commission rate. Semantic retrieval suits natural-language questions; full-text retrieval suits numbers and proper nouns. A commission plan may state an eligibility period, a site scope and a settlement condition in one sentence. Chunking must keep a condition, its explanation and the content it constrains in the same segment.
The FastGPT knowledge base consists of library, collection and data. The smallest search unit is the library; collections classify data and do not affect search results. In the MongoDB dataset.datas table, one data record holds an indexes field, an array of vector IDs, so one data record can map to multiple vectors. PostgreSQL with the PG Vector plugin serves as the vector retriever with an HNSW index; PostgreSQL handles vector retrieval only, and MongoDB stores other data. A group of data can be mapped to multiple vectors, and if any one vector is retrieved, that data is recalled. Indexes and authorization status are separate; retrieval does not perform permission filtering.
Constraints at the document chunking step
Multilingual scripts and product lists often contain numbers, model designations and commission ratios. Semantic retrieval handles natural-language phrasing, including cross-language matching such as a Chinese question against an English knowledge point. Full-text retrieval handles numbers, model designations, proper nouns and error codes. Hybrid retrieval uses both and merges results through RRF. ReRank performs a second ordering and produces a 0–1 score that can be used for filtering. The minimum relevance value applies only under semantic retrieval or when result reranking is used. It is a filter threshold, not a final ordering rule.
Chunking must keep a condition, its explanation and the content it constrains in one segment so that a commission effective condition is not separated from its ratio. The same data record can correspond to multiple vector indexes. Retrieval does not verify authorization, approval or business eligibility.
Hypothetical paper exercise sample: a material text to be checked reads, "This plan applies to creators on Southeast Asia sites who signed between 1 March 2025 and 31 March 2025, commission rate 12%, settlement based on platform-confirmed orders." A draft sentence to be judged reads, "This plan applies to creators on all sites, commission 12%, settlement upon signing." The draft expands the subject scope and rewrites the settlement condition. The two must be compared item by item.
Configuration and implementation checks
| Configuration or check item | Hypothetical starting value for this exercise | Reason and comparison method |
|---|---|---|
| Search mode | Hybrid retrieval | The sample contains both numbers and natural language; compare recall against semantic-only retrieval |
| Minimum relevance | 0.4 (hypothetical starting value) | Compare with 0.2 and 0.6; observe whether irrelevant fragments remain after filtering |
| Reference limit | 2000 tokens (hypothetical starting value) | Compare with 1000 and 4000 for completeness of returned content |
| Question optimization | Enabled | Compare with disabled on short questions; record the change in elapsed time |
| Result reranking | Enabled | Compare with disabled on candidate ordering; the rerank score is used for filtering |
| Segment check | Manual execution | Confirm segment by segment that a condition and its ratio are not separated |
The reference limit is expressed in tokens, not top k, because chunk lengths differ greatly in mixed knowledge bases. Question optimization adds one model call before formal retrieval and typically increases elapsed time. RRF is a fusion process for combining multiple retrieval channels and is not an on/off switch. Image retrieval requires an image input and a model that supports it; reranking requires a model and must be enabled by configuration. These optional paths are not executed on every request.
Three common errors
- Treating a successful upload as an updated index. Input: a changed commission plan. Compare the upload time with the index generation state. Observe whether retrieval still returns the old ratio.
- Using semantic retrieval for an exact number. Input: a product number. Compare semantic and full-text retrieval hits. Observe whether the number is covered by similar text.
- Treating a retrieval result as an authorization credential. Input: a script for an unauthorized site. Compare the retrieval hit with the authorization scope. Observe whether recalled content is used for placement.
How to verify the setup
- Input the hypothetical source text and the draft sentence given above. Compare the subject scope and the settlement condition. The check fails when the draft sentence does not match the source text. The source text names Southeast Asia sites, a signing window of 1 March 2025 to 31 March 2025, and settlement based on platform-confirmed orders; the draft sentence names all sites and settlement upon signing, so the two differ in subject scope and settlement condition.
- Input the product number. Compare full-text and semantic retrieval results. The condition is met only when the number is matched exactly and no similar number is mixed in. A first-position hit is not guaranteed.
- Input an empty reference list []. Check whether the downstream design handles an empty result. Do not presume that a fictional rule necessarily returns an empty list. Whether the returned list is empty depends on the actual data and settings, and remains unknown until observed.
All three checks must record actual observations. Fields not stated in the supplied documents remain unknown. A pending request is not an approval, an offered option is not an execution, and an unsigned termination is not a completed termination. Retrieval is not authorization. Forms collect values; business verification, approval, database writes and external synchronization require their stated downstream implementation.