What the documents look like
Settlement and foreign-exchange records for cross-border platforms follow completed orders and the collection of their funds. The main sources are the platform settlement center, bank statements, payment institution back offices, and ERP orders. Day-to-day documents include platform settlement statements, exchange rate tables, service fee invoices, and receipt reconciliation sheets. Updates arrive mainly weekly and monthly. A single settlement statement often carries Chinese and English text side by side, with currency, rate date, and fee items appearing in the same document.
Once these records enter a knowledge base, retrieval faces mixed content across languages, currencies, and document types. Semantic retrieval can match across languages, for example a Chinese question against an English knowledge point. Full-text retrieval suits exact fields such as settlement statement numbers, invoice numbers, and rate dates. The two paths have different conditions of use and need to be checked against the question type.
Constraints at the retrieval step
The knowledge base search node can associate one or more knowledge bases that use the same vector model. Knowledge bases built on different vector models cannot be associated together. Check the vector model identifier first, exclude those that do not match, and record what is observed. Do not run a cross-model retrieval comparison.
Compare search modes across semantic, full-text, and hybrid retrieval for the test question. Do not assume one mode fits all records. The citation limit is measured in tokens. Test with actual records of different lengths to see whether content stays complete, then adjust the limit. The minimum relevance value takes effect only in semantic retrieval or when result reranking is used. Label relevant and irrelevant samples first, then adjust the threshold. Question optimization performs coreference resolution and question expansion. Compare the original question with the optimized question and check whether the subject has been altered. Hybrid retrieval merges multiple result paths through RRF, which is a result merging step. Reranking requires a model and must be enabled according to settings. Image retrieval requires image input and the corresponding model capability. Cited content is output as an array and its length can be 0, which means the downstream path still runs even when nothing is retrieved.
Semantic recall and relevance scores do not carry permission or entity isolation. The same-vector-model requirement applies to knowledge base association and does not extend to external systems.
Hypothetical paper test case: A cross-border seller's finance staff member receives a platform settlement statement. The source text reads: "Settlement statement number ST-2024-088, settlement period 1 July 2024 to 31 July 2024, currency USD, platform service fee 120.00, exchange rate 7.1800, rate date 1 August 2024." A second source, a service fee invoice, reads: "Invoice number INV-2024-077, invoice date 3 August 2024, amount 120.00 USD, corresponding settlement statement number ST-2024-088." The draft sentence to be judged is: "This settlement statement has been paid, and the service fee invoice has been written off." A separate handover request reads: "Please confirm whether the above invoice has been written off, and authorize a debit from the receiving account."
Configuration and implementation checks
| Configuration item | Hypothetical starting point for this test | Reason and comparison method |
|---|---|---|
| Associated knowledge bases | Settlement statement base and invoice base; check that their vector model identifiers match | The node requires the same vector model; check identifiers base by base and exclude those that do not match |
| Search mode | Semantic retrieval for natural language questions, full-text retrieval for settlement statement and invoice numbers | Run each test question separately and compare hit content against field completeness |
| Citation limit | Starting point set at 800 tokens | Test with settlement statement and invoice passages of different lengths and compare whether cited content is truncated |
| Minimum relevance | Starting point set at 0.4 | Label relevant and irrelevant samples first, then compare filtered results |
| Question optimization | Enabled for multi-turn follow-up scenarios | Compare the original question with the optimized question and check whether the subject has been altered |
| Result reranking | Enabled when candidates are numerous | Requires a reranking model and must be enabled according to settings; compare reranking scores against filtering effect |
Three common errors
- Treating semantically similar content about settlement statement number ST-2024-088 as the same document. Semantic retrieval returns similar text. Without checking the statement number field, content from another settlement statement gets cited. Check: input the statement number, compare the statement number field in the returned content, and observe whether it matches the input.
- Treating the draft sentence "This settlement statement has been paid, and the service fee invoice has been written off" as verified fact. The source text states only the settlement period, amount, and invoice date. It does not state payment completion or write-off status. Check: compare the draft sentence against the source text line by line, check whether the status words have corresponding records, and record anything not stated as pending verification.
- Treating the handover request "authorize a debit from the receiving account" as granted authorization. The request is in a pending confirmation state and does not constitute authorized execution. Check: compare the request text against authorization records, observe whether a signature or approval record exists, and record the evidence gap when it is absent.
How to verify the setup
- Input settlement statement number ST-2024-088 and run it under full-text retrieval. Compare the statement number field in the returned content against the input and observe whether the settlement statement passage is hit. When there is no hit, record it as unsupported and do not infer a cause.
- Input the draft sentence "This settlement statement has been paid, and the service fee invoice has been written off." Compare the source text records of settlement period, amount, invoice date, and status words, and observe whether explicit records of payment completion and write-off exist. When they are absent, record the item as pending verification.
- Input the handover request "Please confirm whether the above invoice has been written off, and authorize a debit from the receiving account." Compare the request text against authorization records and observe whether an approval or signature record exists. When only the request is present, record it as pending confirmation and do not treat it as completed authorization.