What the documents look like
Returns and claims handling covers reverse logistics and liability determination after export. Source materials come from customer emails, quality inspection reports, customs records, and insurance claim materials. Return agreements, claim letters, quality reports, and return customs declarations are updated per case, and formats are not uniform. The same shipment may simultaneously have a procurement contract, an inspection conclusion, and a return declaration text. The vector model and indexes must split and index these materials so that retrieval can locate specific material passages for manual verification.
Constraints at the vector indexes step
Knowledge base search uses the library as the smallest search unit; collections are only for classification management and do not affect search results. Vector retrieval is handled by the PG Vector plugin for PostgresSQL, and the index is HNSW. MongoDB's dataset.datas table stores vector source data, where the indexes field records the corresponding vector IDs. It is an array, so one group of data can correspond to multiple vectors. During retrieval, vectors are recalled first, then the vector IDs are used to return to MongoDB to find the source data. If the same group of source data is hit by multiple routes, it is merged, and the vector score takes the highest score.
Knowledge bases used in associated retrieval must use the same vector model. When selecting multiple knowledge bases, first check the actual model and input type. Libraries across different vector models cannot be associated together. Semantic retrieval depends on vector distance and is suitable for natural language questions; full-text retrieval depends on keyword hits and is suitable for numbers, model numbers, and proper nouns. Hybrid retrieval uses both at the same time and fuses them through RRF. Re-ranking requires a ReRank model and must be enabled according to settings; after re-ranking, a score of 0-1 is obtained. Minimum relevance only takes effect during semantic retrieval or when result re-ranking is used; it is a filtering threshold. The citation limit is measured in tokens. Retrieval or forms do not constitute authorized execution, and semantic recall and relevance are not responsible for permissions or entity isolation. Node output is an array, and its length can be 0.
Hypothetical exercise sample: the following content is constructed for this paper-based check and does not come from any real customer or system record.
Material text one (return agreement excerpt): "Party A signed this agreement with Party B on March 12, 2024 regarding the return of batch W2403. The reason for the return was that appearance dimensions were found to be out of tolerance during arrival inspection, and the return quantity was three hundred pieces. This agreement takes effect on the date of signing by both parties."
Material text two (claim letter excerpt): "The claimant submits a claim regarding batch W2403. The claim amount is determined based on a third-party quality report, with attached report number QC-2403-17. The submission date of this letter is April 2, 2024, and written confirmation from your side is still pending."
Draft sentence to be judged: "The return of batch W2403 has been completed, and the claim has been confirmed by the other party."
Material text one shows that the agreement has been signed and taken effect, but does not state that the return has been completed; material text two shows that the claim letter has been submitted and is pending confirmation. Signing, submission, completion, and confirmation are different facts. Neither "has been completed" nor "has been confirmed" in the draft sentence is supported.
Configuration and implementation checks
| Check item | Input | Comparison and observation |
|---|---|---|
| Vector model consistency | Identifiers of multiple knowledge bases to be associated | They can be associated together only when model identifiers are consistent; exclude inconsistent ones and record actual observations |
| Index and source data correspondence | indexes field in dataset.datas | One group of source data can correspond to multiple vectors; check the index and the original text content separately |
| Index content length | Index text built for W2403 materials | When the index is shorter and more focused, retrieval precision improves and retrieval scope narrows accordingly |
| Image index | Whether image descriptions or image vectors are generated at ingestion | Uploading an image on the query side is not enough for a hit; check whether the ingestion index was built |
| Citation limit | Maximum number of tokens cited per search | In this hypothetical test, the starting point is set to 2000 tokens, compared with a 1000-token starting point for citation completeness |
| Minimum relevance | Value from 0-1 | Takes effect only during semantic retrieval or result re-ranking; in this hypothetical test, the starting point is set to 0.3, compared with 0.5 for the number of recalled items |
Three common errors
- Treating pending confirmation as confirmed. The original claim letter says "written confirmation from your side is still pending," while the draft sentence says "the claim has been confirmed." The input is the original claim letter; compare "pending confirmation" with "confirmed." The observation is inconsistent status, which is unsupported.
- Treating signing and effectiveness as return completion. The original return agreement says "takes effect on the date of signing by both parties" and does not state that the return has been completed, while the draft sentence says "the return has been completed." The input is the original agreement; compare agreement effectiveness with return completion. The observation is that the original text does not state a completion status, which is an evidence gap.
- Forcing knowledge bases across models to be associated together. The input is the model identifiers of two knowledge bases; compare whether the two identifiers are consistent. The observation is that when the identifiers are inconsistent, exclude the latter and do not execute cross-model retrieval to compare missed recalls.
How to verify the setup
- Use material text one as input and check whether retrieval returns that return agreement passage. Compare whether the returned content is consistent with the original text's "March 12, 2024," "batch W2403," and "three hundred pieces." The observation meets the condition if the returned passage contains the above elements; if the return is an empty array or elements are missing, record it as a miss and check whether the index has been built.
- Use material text two as input and check whether retrieval returns the claim letter passage. Compare whether the returned content contains "QC-2403-17" and "written confirmation from your side is still pending." The observation meets the condition if both items appear; if only the report number appears and the pending-confirmation wording does not, record it as a partial hit and do not use it to determine that the claim has been confirmed.
- Use the draft sentence to be judged as input and check the two facts in the draft sentence. Compare "has been completed" and "has been confirmed" in the draft sentence with the stated status in the two original texts. The observation meets the condition if the draft sentence retains only facts supported by the original texts; if the draft sentence contains facts not stated, record it as unsupported and retain the evidence gap.
For empty-result handoff, an explicit paper input [] can be used to check the downstream design. After model or index adjustments, use the same set of questions to compare cited original texts, and do not promise results based on vector scores alone.