What the documents look like
A brand-run livestream carries its own scripts, stock, and after-sales work. Procurement and supplier materials decide whether a product explanation card can state sellable quantity and arrival timing clearly. The day-to-day material set is livestream scripts, product explanation cards, shift schedules, and livestream reviews, sourced from Douyin Shop Compass, Juliang Qianchuan, livestream comments, and transaction details, with updates per session or per hour. These materials are frequently stored alongside supplier quotations and purchase confirmation records. Parsing and chunking must keep the identity and version of each document so later retrieval can point back to the original text.
Within FastGPT, a knowledge base is made of library, collection, and data. A collection works like a file, one library can hold several collections, and one collection can hold several groups of data. The smallest search unit is the library, so a search runs against the whole library; collections only classify data and do not affect search results. In the MongoDB dataset.datas table, vector source data is stored with an indexes field that records the corresponding vector IDs as an array, meaning one group of data can map to several vectors. In PostgresSQL, a vector field stores the vector, the index is HNSW, and PostgresSQL is used only for vector retrieval while MongoDB handles other data access. During retrieval, vectors are recalled first, then the vector ID is used to find the source content in MongoDB; entries that map to the same source group are merged and the highest score is taken.
Constraints at the document chunking step
Content length and semantic richness pull against each other inside one vector. FastGPT uses multi-vector mapping so a longer text can carry several index groups, and a data group is recalled when any one of its vectors is matched. Splitting a condition away from the content it constrains lets both semantic retrieval and full-text retrieval hit a fragment that is missing its qualifier. Hybrid retrieval combines semantic and full-text recall through the RRF formula, and because the merged scope is large and cannot be filtered by similarity directly, a rerank model is normally used to reorder once and filter by its score. A rerank score is a 0-1 value representing relevance between the searched content and the question. The minimum relevance value applies only under semantic retrieval or when result reranking is used; it is a filter threshold, not a final ranking rule. The citation cap limits how many tokens of content each search may cite. Retrieval does not constitute authorization, and a form collecting a value is not the same as approval or a database write.
Hypothetical drill sample. The material text for review reads: "Supplier A's March 2025 quotation states: Model A thermos, 500 ml, unit price 18 yuan, quotation valid until 30 April 2025; a note on the same page says the first shipment can release 300 units, with the remainder subject to the purchase confirmation form." The draft sentence to be judged reads: "Supplier A is confirmed to supply 300 units of the Model A 500 ml thermos in April, and the purchase has taken effect." This sentence treats the quotation validity period as a supply period and treats a noted quantity as an effective purchase. The purchase confirmation form and its signature status are absent, so the sentence is unsupported.
Configuration and implementation checks
| Configuration or check item | Use on this page | Implementation |
|---|---|---|
| Dataset and collection split | Group materials by supplier and procurement stage | Build collections according to the dataset.datas structure; search operates at library level |
| Vector model and index | Generate text vectors and write them into indexes | Use one vector model within a library and record the model identifier |
| Chunking and multi-vector | Preserve quotation conditions and confirmation status | Add several index groups to long passages so conditions are not split away |
| Full-text and hybrid retrieval | Match model numbers, unit prices, and identifiers | Use full-text or hybrid retrieval for identifier queries, merged through RRF |
| Result rerank and minimum relevance | Filter low-relevance results once reranking is enabled by setting | Rerank score is 0-1; minimum relevance applies only under semantic retrieval or reranking |
| Citation cap | Control the tokens cited per search | This hypothetical trial start is set at 3000 tokens, chosen to cover quotation and confirmation passages; compare against 1500 tokens to observe omissions |
Three common errors
- Treating a quotation validity period as a supply period. The source text states only that the quotation is valid until 30 April 2025 and states no supply period, so the judgment is unsupported.
- Treating a purchase confirmation form awaiting review as already effective. Pending review does not show a lack of operating permission and does not equal a completed signature; the signature and effective status must be checked.
- Treating a retrieval hit as a basis for authorization. Retrieval returns cited content only; procurement execution requires a separate business system implementation.
How to verify the setup
- Input the hypothetical source text and the draft sentence above, compare whether "quotation validity period" and "supply period" are the same field over the same interval, and observe the draft sentence being marked unsupported.
- Search with the model number "Model A 500 ml" and the unit price "18 yuan" separately, compare whether the full-text and hybrid retrieval hits contain the complete condition, and observe whether the chunking keeps the note attached.
- Input the paper empty result
[], check whether the downstream design handles it as an empty citation, and observe that the empty result is not read as no stock or no authorization. Across libraries using different vector models, check the model identifier first and exclude those that do not match, then record the actual observation.