What the documents look like
Amazon FBM fulfillment places fulfillment and return-address setup with the seller, and it does not rely on platform warehouse inspection. The materials relevant to operating reviews and exception retrospectives come from Amazon Seller Central, carrier systems, ERP orders, and customer service email. Day-to-day documents include order pick sheets, shipment confirmation records, return-address settings, and logistics tracking tables. Data updates per order and per day.
For an operating review, the retrieval task is to locate the shipment, tracking number, return address, and undelivered-complaint records of a specific order, then use the retrieved content to support a retrospective conclusion. The materials differ in what each one records. A pick sheet records SKU and quantity. A shipment confirmation records a tracking number and shipment date. A return-address setting records an address and an effective date. A tracking table records movement events. Retrieval does not decide authorization, and a similarity score does not exclude unauthorized records.
Constraints at the retrieval step
The knowledge base search node can associate one or more knowledge bases that use the same vector model. Semantic retrieval relies on vector distance; full-text retrieval relies on keyword matches; hybrid retrieval merges the two result paths through RRF. The reference cap is measured in tokens. The minimum relevance value takes effect only under semantic retrieval or when result reranking is used. Question optimization performs coreference resolution and question expansion. Retrieval output is an array, and its length can be 0, which means the downstream path still runs even when nothing is retrieved.
The following hypothetical paper exercise sample is used for the checks below.
Sample material, three sentences. First sentence: "The pick sheet for order A-001 records SKU X1, quantity 2, pick date March 10, 2025." Second sentence: "The shipment confirmation records order A-001 tracking number TR-8801, shipment date March 11, 2025, carrier local carrier L." Third sentence: "The return-address setting page records return address R, effective date March 1, 2025."
Draft sentence to be judged: "Order A-001 completed shipment on March 11, 2025, and tracking number TR-8801 has been handed to the carrier."
Retrieval request: "Find the shipment confirmation and return-address setting for order A-001 in March 2025." The request must be checked for subject order A-001, period March 2025, and fields shipment confirmation and return-address setting. The phrase "has been handed to the carrier" in the draft sentence is not stated in the material, so it is unsupported.
Configuration and implementation checks
| Configuration item | Hypothetical trial starting point | Reason and comparison method |
|---|---|---|
| Associated knowledge base | Select one knowledge base containing order pick sheets and shipment confirmations | Check that its vector model identifier matches the other knowledge bases to be associated; exclude those that do not match |
| Search mode | Try hybrid retrieval first | Compare semantic, full-text, and hybrid recall differences using order number A-001 and tracking number TR-8801 from the sample |
| Reference cap | Set to a token amount that covers the three sample sentences | Use the full readout of the sample material as the comparison baseline; observe whether truncation drops the tracking number or a date |
| Minimum relevance | Start at 0.5 for this trial | Compare scores of the sample draft sentence against an unrelated sentence; observe whether the shipment confirmation is retained after filtering |
| Question optimization | Off for this trial | The sample question already contains the order number and period; compare whether enabling it alters the subject |
| Result reranking | Not enabled for this trial | The sample has few candidates; compare whether enabling it changes the position of the shipment confirmation |
Three common errors
- Treating retrieval output as an authorization credential. Input is the sample retrieval request. The comparison object is the shipment confirmation text against the draft sentence. The observable result is that retrieving the shipment confirmation does not make "has been handed to the carrier" supported. That field is absent from the material and should be recorded as unsupported.
- Treating minimum relevance as an entity filter. Input is sample order number A-001. The comparison object is the similarity score against the order subject. The observable result is that a high or low score does not exclude records of other orders, so the subject field needs manual checking.
- Treating RRF in hybrid retrieval as a switch that can be turned on. Input is the sample search-mode choice. The comparison object is the semantic path and the full-text path. The observable result is that RRF is a fusion process and does not appear as a separate switch, so the actual returned content should be checked for whether it contains hits from both paths.
How to verify the setup
- Input the three sample material sentences and the draft sentence. Compare "has been handed to the carrier" in the draft sentence against the fields recorded in the shipment confirmation. The observable result is that the shipment confirmation records only the tracking number and shipment date, and does not record handover to the carrier, so the sentence is judged unsupported.
- Input the sample retrieval request "Find the shipment confirmation and return-address setting for order A-001 in March 2025." Compare the subject, period, and fields of the returned content. The observable result is that the returned content contains order A-001, March 2025, the shipment confirmation, and the return-address setting, and the return-address effective date March 1, 2025 falls within the period, so the retrieval is judged a hit.
- Input an empty-result continuation case, using the paper input [] to check the downstream design. Compare whether the downstream path runs when the reference content array length is 0. The observable result is that the output path runs and produces no reference content, so empty-result continuation is judged normal. This case does not claim that a fictional rule retrieval will necessarily return [].