What the documents look like
Rebate calculation happens after sales. The core work is policy interpretation and financial treatment, which differs from the timing logic of inventory replenishment. The main data sources are ERP sales records, dealer remittances, contract ledgers and the finance system. The documents handled day to day are rebate policies, sales achievement tables, reconciliation statements and red-letter invoices. The most time-consuming steps are checking the achievement basis, deducting returned goods amounts and offsetting invoicing. Data updates monthly or quarterly, and after any material change the actual training and index status must be checked.
For this page the task is inventory procurement and supplier materials, and the feature is document parsing and chunking. Rebate policies carry tiered conditions and validity periods. Sales achievement tables carry entity and quantity bases. Reconciliation statements carry return deductions. Red-letter invoices carry offset targets. Chunking must keep each condition together with the content it constrains, and must avoid splitting a policy validity period away from its tier threshold. Original data and retrieval indexes are distinct, and one group of original data can map to multiple indexes. Semantic recall and relevance serve content matching only; they do not perform permission or entity isolation. The same vector model requirement applies within a retrieval library and cannot be extended to claim that an external business registration system must use the same model.
Constraints at the document chunking step
Chunking sits between parsing and retrieval. A rebate policy clause typically states a subject, a period, a threshold and a ratio. If the period lands in one chunk and the threshold in another, a retrieval hit on the threshold arrives without the period that limits it. The same risk applies to reconciliation statements where a return deduction is separated from the sales line it reduces, and to red-letter invoices where the offset target is separated from the amount.
The documented storage structure matters here. In FastGPT the knowledge base consists of library, collection and data. The smallest search unit is the library, so a search runs across the whole library, while collections only classify data and do not affect search results. In MongoDB the dataset.datas table stores original data information together with an indexes field that records the corresponding vector IDs as an array, so one group of data can correspond to multiple vectors. In PostgresSQL a vector field stores the vectors, and during retrieval the vectors are recalled first, then the original content is located in MongoDB by vector ID; when several vectors correspond to the same group of original data they are merged and the highest score is taken. This is why annotating data blocks can raise block precision: adding multiple vectors to a longer text means that a hit on any one vector recalls that data.
Search mode also shapes chunking. Semantic retrieval depends on vector distance and suits natural language questions and semantically close content. Full-text retrieval depends on keyword hits and suits numbers, models, proper nouns and error codes. Hybrid retrieval uses both and merges them through RRF. ReRank performs a second ordering and yields a 0-1 score. The minimum relevance value is a filter threshold that takes effect only under semantic retrieval or when result reranking is used; it is not a final ordering rule. The citation limit caps how many tokens are cited per search, which is why chunk length interacts directly with how much of a clause survives trimming.
A hypothetical paper exercise sample, used only for checking, is as follows. The source text of a brand's 2025 Dealer Rebate Policy reads: "Dealer A reaches a purchase amount of 2,000,000 yuan in the second quarter of 2025, rebate ratio 3%; returned goods amounts are deducted from the achievement amount." A draft sentence under review reads: "Dealer A's second quarter rebate has been assessed at 3% and is pending payment." Judging this draft sentence requires comparing subject, period, field and status. The source text states only the policy condition. It does not state Dealer A's actual achievement amount, the result of the return deduction, any assessment action, or any payment status.
Configuration and implementation checks
| Configuration item | Hypothetical trial starting point | Reason and comparison method |
|---|---|---|
| Search mode | Hybrid retrieval | Policy numbers and natural language clauses coexist; compare the content recalled by the semantic path and the full-text path |
| Minimum relevance | 0.5 | Hypothetical starting point; compare whether policy clauses are filtered out at different thresholds |
| Citation limit | 3000 tokens | Hypothetical starting point; compare clause completeness after chunk length and citation trimming |
| Question optimization | Enabled | Completes references during follow-up questions on rebate basis; compare recalled content before and after enabling |
| Result reranking | ReRank enabled | Second ordering when candidates are numerous; compare rerank scores against vector scores |
| Chunk annotation | Annotated by policy clause | One group of original data maps to multiple indexes; compare hits on each index |
All values above are hypothetical trial starting points for this exercise and are not official defaults. RRF is the fusion process for hybrid retrieval results and is not an independent switch.
Three common errors
- Splitting a rebate policy tier threshold and its validity period into different chunks. Retrieval then hits the threshold while the period is missing, so the basis judgment is incomplete. Check: input the policy source text, compare whether each chunk after splitting retains subject, period and threshold, and observe the separation.
- Treating the draft sentence "the rebate has been assessed and is pending payment" as a fact of the source text. Check: input that draft sentence together with the policy source text, compare whether the assessment action and the payment status appear in the source text, and observe which fields lack support.
- Using minimum relevance to exclude dealer records that are not authorized for viewing. Check: input a cross-dealer query, compare whether returned content is determined by similarity alone, and observe whether permission filtering is absent.
How to verify the setup
- Input the hypothetical sample policy source text and the draft sentence, compare subject, period, field and status. When the assessment and payment status in the draft sentence are not stated in the source text, judge the sentence as unsupported.
- Input the policy number and a natural language clause as separate searches, compare whether the two recall paths cover both threshold and period, and observe whether chunking preserves the complete condition.
- Input an empty-result carry-over case, using the paper input
[]to check the downstream design, and compare whether the system handles empty citations as designed, without claiming that a fictional rule search must return empty. After a material change, check the actual training and index status; a successful upload does not mean retrieval has been updated.