What the documents look like
This category centres on disposable plastic products reporting and environmental supervision. The working materials are plastic bag procurement ledgers, sales vouchers, half-year reports and self-discipline commitments. Procurement ledgers and sales vouchers are typically produced daily; half-year reports are compiled once per half year; self-discipline commitments take written form at particular points. Together these materials support product and selection questions: which plastic bag specifications were purchased, what sales records exist, how report figures are scoped, and what the commitment covers.
Retrieval and recall operate on the text form of those materials. A procurement ledger records supplier, specification, quantity and date. A sales voucher records the transaction and its link to a product. A half-year report aggregates usage and recycling information. A self-discipline commitment records the subject and the committed items. Whether these fields are complete, whether they cover the same period, and whether they concern the same subject directly determine whether a question can be answered in a checkable way.
Constraints at the retrieval step
The knowledge base search node can associate one or more knowledge bases that use the same vector model. Knowledge bases with different vector models cannot be associated together; the vector model identifier is checked first and non-matching bases are excluded. A cross-model retrieval is not run in order to compare missed recalls.
Search mode is compared across semantic retrieval, full-text retrieval and hybrid retrieval for the test question; no single mode is assumed to suit all materials. The reference limit is measured in tokens, and materials of different lengths are used to check whether content stays complete before the limit is adjusted. The minimum relevance value takes effect only under semantic retrieval or when result reranking is used; relevant and irrelevant samples are labelled before the threshold is tuned. Question optimisation performs coreference resolution and question expansion; the original question and the optimised question are compared to check whether the subject has been altered incorrectly.
Reference content is output as an array whose length can be 0; when nothing is retrieved, the output path still completes. Retrieval produces candidate content and does not perform permission or entity isolation. Forms collect values; business verification, approval, database writes and external synchronisation each require their own downstream implementation. RRF is a fusion process for mixed results and is not written as a switch. Image retrieval requires a corresponding image input and model capability, and reranking requires a model and is enabled according to its settings.
Hypothetical paper exercise sample: the source text to be checked consists of three sentences. Sentence one: "In the first half of 2024, a supermarket store purchased 120,000 disposable plastic bags in total, from supplier Jia Packaging Factory, in a degradable material specification." Sentence two: "For the same period, the store's plastic bag sales vouchers show that 100,000 charged plastic bags were sold." Sentence three: "The store has signed a self-discipline commitment, whose content is to reduce the use of non-degradable plastic bags." One draft sentence to be assessed reads: "The store's disposable plastic bag purchase volume and sales volume for the first half of 2024 are consistent, and the recycling reporting has been completed."
Configuration and implementation checks
| Configuration or check item | Hypothetical starting point for this exercise | Reason for the choice and comparison object |
|---|---|---|
| Associated knowledge bases | Associate only bases using the same vector model | Check the vector model identifier first and exclude bases with a different model |
| Search mode | Test semantic retrieval and full-text retrieval separately | Compare with hybrid retrieval results and observe candidate differences for the same question |
| Reference limit | Start at a token value covering the longest ledger passage | Compare with the shorter sales voucher passage and check whether content is truncated |
| Minimum relevance | Start at a middle value | Compare with labelled relevant and irrelevant samples and observe filtering changes |
| Question optimisation | Enable for testing on questions containing coreference | Compare with the unoptimised question and check whether the subject is altered incorrectly |
| Result reranking | Enable according to settings when a rerank model is available | Compare with unreranked candidates and observe ordering changes |
Three common errors
- Associating knowledge bases that use different vector models. Input: two bases whose vector model identifiers differ. Compare their vector model identifiers and observe whether the non-matching one is excluded. If identifiers differ and the bases are still associated, the configuration is wrong.
- Treating minimum relevance as a permission or entity filter. Input: labelled relevant and irrelevant samples. Compare candidates before and after filtering and observe whether irrelevant samples are excluded by the threshold. Treating the threshold as a means of business isolation is a misunderstanding.
- Treating retrieval or a form as the execution of authorisation. Input: a handover request awaiting review. Compare the request status with the authorisation status and observe whether it is treated as approved. A pending review does not demonstrate a lack of operating permission, and it does not equal approval.
How to verify the setup
- Using sentence one and sentence two of the hypothetical source text, enter the question "What were the store's plastic bag purchase volume and sales volume for the first half of 2024 respectively?" Compare whether the retrieved content contains both the purchase quantity and the sales quantity, and observe whether the reference content is complete. If only one of the two sentences is retrieved, check the reference limit and the chunking.
- Using the draft sentence "The purchase volume and sales volume are consistent, and the recycling reporting has been completed" as a check request, compare whether the purchase volume, sales volume and recycling reporting in the source text share the same subject, the same period and the same field. The source text does not state recycling reporting; that part is unsupported and the evidence gap is recorded.
- For empty-result handling, enter an explicit paper input of an empty array and compare whether the downstream design completes the output path when the reference content length is 0, observing whether any interruption occurs. No claim is made that a fictional rule retrieval necessarily returns an empty array.