What the documents look like
Channel diversion review work covers regional and price order. It does not cover rebate amounts. The working materials are diversion complaint forms, anti-counterfeit code records, penalty notices, and investigation photographs. Sources include anti-counterfeit code systems, e-commerce platform prices, distributor shipment records, and market inspections. Brand channel control teams, anti-counterfeit teams, and regional managers track goods-source codes, compare low-price shops, and carry out penalties. Material updates arrive weekly or irregularly.
For document parsing and chunking, three of these features matter directly. Complaint forms and anti-counterfeit records may be written in Chinese or English, and the same case may combine both. Anti-counterfeit codes and shipment record numbers must be matched exactly, so keyword-oriented retrieval paths carry real weight. A complaint form carries a subject, a region, a date, and a status, and those fields need to stay inside one chunk. When a chunk separates a condition from the content it constrains, the retrieved text no longer carries a checkable original meaning.
Constraints at the document chunking step
A knowledge base in FastGPT has three parts: library, collection, and data. The smallest search unit is the library; a collection only classifies data and does not affect search results. Vectors are stored in a PostgresSQL vector field with an HNSW index. The MongoDB dataset.datas table stores the original data plus an indexes field, and one data item can map to multiple vectors. Semantic retrieval supports cross-language understanding. Full-text retrieval suits exact queries such as numbers, models, and proper nouns. Hybrid retrieval merges both paths through RRF. ReRank performs a second ordering of candidate text and returns a 0-1 score. The reference limit is measured in tokens. The minimum relevance value takes effect only in semantic retrieval or when result reranking is used.
For cross-border materials and multilingual retrieval, one data item can carry several indexes, so chunking should preserve document identity and version. Retrieval is a recall action and does not perform authorization checks. Image retrieval requires an image input and the corresponding model capability. Reranking requires a model and must be enabled through settings. Libraries built on different vector models cannot be associated together.
Hypothetical drill sample: the source text is "Complaint form CX-2024-017, complaint subject is East China distributor A, anti-counterfeit code range A1001 to A1050, complaint date 12 March 2024, status pending verification." The draft sentence under review is "Distributor A has completed cross-region diversion and a penalty should be carried out." The subject, object, status, and valid range needed for judgment all appear in the source text. The draft sentence turns a pending-verification status into a completed fact and lacks a basis for penalty.
Configuration and implementation checks
| Configuration item | Hypothetical starting point for this drill | Reason and comparison object |
|---|---|---|
| Search mode | Hybrid retrieval | Complaint forms contain both codes and natural-language description, so both query types need coverage |
| Reference limit | Hypothetical starting point 2000 tokens | Compared against the length of a single chunked item, to avoid cutting a conditional sentence |
| Minimum relevance | Hypothetical starting point 0.3 | Compared against the number of recalled items with filtering off, to observe noise changes |
| Question optimization | Enabled | Completes references in follow-up turns; compared against retrieval results with it off |
| Result reranking | ReRank enabled | Second ordering when candidates are numerous; compared against ordering with RRF alone |
| Chunk review | Manual item-by-item review | Checks whether condition, statement, and constrained content sit in the same chunk |
Three common errors
- Writing a pending-verification status as a completed fact. The draft sentence "Distributor A has completed cross-region diversion and a penalty should be carried out" and the source text "status pending verification" give mutually exclusive values on the same subject, same object, and same field. The check is to input the source text and the draft sentence, compare the status field, and observe whether the draft introduces a completion conclusion the source does not carry.
- Treating a retrieval hit as an authorization basis. The input is a retrieved complaint-form chunk. Compare whether it contains an approval or authorization field. If the chunk holds only complaint content, a penalty cannot be carried out on that basis.
- Treating a successful upload as an updated index. The input is the upload record after a material change. Compare the actual training and index state. If training has not completed, retrieval results may still point to the old chunk.
How to verify the setup
- Input the hypothetical sample source text and the draft sentence, compare the status field values, and if the draft shows "completed" while the source shows "pending verification," judge the draft as unsupported.
- Input the anti-counterfeit code range A1001 to A1050 into full-text retrieval, compare whether the returned chunk contains the complete code range, and if the chunk is truncated so the range is missing, adjust the chunking method.
- Input an empty-result scenario and check whether the downstream design handles it as an empty reference. Do not claim that an invented rule necessarily returns an empty array. For libraries on different models, first check the identifier and exclude non-matching ones, record the actual observation, and do not run a cross-model retrieval to compare missed recalls.