What the documents look like
Channel diversion review concerns regional and price order. It does not concern rebate amounts. Materials arrive from anti-counterfeit code systems, e-commerce platform prices, distributor shipment records and market inspections. Everyday documents include diversion complaint forms, anti-counterfeit code records, penalty notices and investigation photographs. These materials carry fields such as goods-code source, region, batch, store and time. A goods-code source on one complaint form needs to cross-reference shipment records and low-price store screenshots. Updates are weekly or irregular. A successful upload only means the file entered the library; the actual training and indexing state requires separate confirmation.
For document parsing and chunking, the complaint form number, subject, status and effective scope must stay in the same data block. Conditions and the content they constrain must not be split apart.
Constraints at the document chunking step
A FastGPT knowledge base consists of library, collection and data. The smallest search unit is the library. Collections only classify and manage data; they do not affect search results. Vector retrieval uses the PostgresSQL PG Vector plugin with an HNSW index. The MongoDB dataset.datas table stores original data, and the indexes field records the corresponding vector IDs. One group of data can correspond to multiple vectors. Retrieval first recalls vectors, then returns to MongoDB by ID to fetch original data. When vectors map to the same group of original data, results merge and the vector score takes the highest value.
Segmentation needs to keep semantics complete and single. A concise index can raise precision while sacrificing some retrieval range. A richer set of index entries raises the chance of a hit. Full-text retrieval on PG and similar vector stores uses MongoDB $text with jieba tokenization. Hybrid retrieval merges multiple result paths through RRF. Result reranking uses a ReRank model to produce a 0-1 score. The minimum relevance value takes effect only during semantic retrieval or result reranking; it is a filter threshold and does not perform permission or entity isolation. The reference limit is controlled by tokens, and its difference from top k lies in the large chunk length variation within mixed knowledge bases.
Hypothetical paper exercise sample: a brand channel control team receives a diversion complaint form. The material text reads: "Complaint form number CX-2024-018, complained store is East China authorized distributor A, goods-code source range A-3301 to A-3350, batch involved 2024-07, complaint reason is cross-region low-price sales, registration date 12 August 2024, status pending verification." A draft handover sentence reads: "This distributor has completed the cross-region diversion penalty." The subject required for judgment is distributor A, the objects are the goods-code source range and batch, the status is pending verification, and the effective scope is East China. No penalty notice appears in the material. Penalty execution is a business action that requires a separate system implementation.
Configuration and implementation checks
The table below lists documented settings or implementation check items around this page's sample. The values are hypothetical starting points for this paper comparison, used only for paper-based comparison and recorded after comparison with the reference value.
| Configuration item | Hypothetical starting point for this exercise | Reason for selection and comparison method |
|---|---|---|
| Search mode | Hybrid retrieval | Complaint forms contain both numbers and natural-language reasons; compare differences between full-text and semantic recall paths |
| Minimum relevance | Hypothetical starting point 0.3 | Compare with 0.5 and observe whether the pending-verification status sentence is filtered |
| Reference limit | Hypothetical starting point 2000 tokens | Compare with 4000 and check whether number and status return in the same block |
| Question optimization | Enabled | Compare with disabled and check completion effect on goods-code source range follow-up |
| Result reranking | Enabled per setting | Requires a ReRank model; compare rerank score with vector score |
| Segmentation check | Manual inspection | Confirm number, subject, status and effective scope are in the same data block |
Three common errors
- Treating a successful upload as retrieval already updated. Input is the completed upload of the complaint form file. The comparison object is the actual training and indexing state. The observable result is whether retrieval can recall that complaint form number. A successful upload and completed indexing are two different facts.
- Treating minimum relevance as permission filtering. Input is the complaint form in pending-verification status. The comparison object is the authorized scope and the retrieval result. The observable result is whether unrelated stores are excluded by the threshold. A relevance score does not perform entity isolation; permissions require a separate system implementation.
- Reading pending-verification status as completed penalty. Input is the draft sentence "This distributor has completed the cross-region diversion penalty." The comparison object is the material's original status field "pending verification." The observable result is whether the two are mutually exclusive on the same subject, same period and same field. Pending review does not demonstrate that a penalty was executed.
How to verify the setup
- Verification one: Input the hypothetical sample text and the draft sentence. Compare the draft sentence status "completed penalty" with the original text "pending verification." The observable result is that when the two take mutually exclusive values on the same subject, same period and same field, the draft sentence is judged unsupported. The expectation is to record the evidence gap, without fabricating a penalty notice.
- Verification two: Input goods-code source range A-3301 to A-3350 and batch 2024-07. Compare whether the content returned after chunking simultaneously contains the number, subject, status and effective scope. The observable result is that the condition is met when all four are present within the same data block. If status and number are split into different blocks, record the segmentation as incomplete.
- Verification three: Input an explicit paper-based empty result []. Compare how the downstream design handles an empty reference. The observable result is that the condition is met when the downstream executes the empty-reference branch. The retrieval check records actual observations and failure branches; it does not guarantee a first hit or that unrelated queries must be empty. Libraries across different vector models cannot be associated together. Verify the identifier first and exclude those that do not conform; do not perform cross-model retrieval to compare missed recalls.