What the documents look like
Brand outlet discount operations exist to clear past-season inventory. The working documents for this page are discount approvals, slow-moving inventory lists, price tags, and gross margin calculation sheets. These materials share a common property: the same goods are referenced repeatedly across procurement, discounting, labeling, and margin calculation. SKU code, supplier name, contract batch, discount range, and validity period are the fields that link the documents together. Updates arrive weekly and monthly. Procurement and supplier materials are produced when a batch arrives or a contract is signed.
During ingestion and parsing, a single supplier document may contain quotation terms, supply scope, and settlement notes in one file. Fixed-length splitting can place a quotation validity period in a different chunk from the supply period, so retrieval returns only one fragment. One set of source data can map to multiple vector indexes, and adding several indexes to a longer text improves recall completeness.
Constraints at the document chunking step
Chunking follows the structure of each material. The approval levels and effective conditions of a discount approval, the SKU and inventory-age range of a slow-moving list, the product-to-discount mapping of a price tag, and the cost and sale-price fields of a margin sheet each need to stay complete inside one chunk. When a condition, its explanation, and the content it constrains are split apart, retrieval returns an incomplete fragment.
Source data and retrieval indexes are separate. In MongoDB, the dataset.datas table stores source data and an indexes field records the corresponding vector IDs. In PostgresSQL, a vector field stores vectors. Retrieval recalls vectors first, then returns to MongoDB to merge source data, and takes the highest score when several vectors map to one source record. A successful upload does not mean retrieval has been updated; after a material changes, the actual training and index state needs to be checked.
Semantic retrieval, full-text retrieval, hybrid retrieval, RRF, and Rerank are distinct recall paths. Exact queries such as a code, contract number, or SKU suit full-text or hybrid retrieval. Natural-language descriptions suit semantic retrieval. The retrieval step does not perform permission or entity isolation, and a similarity score is a relevance measure that cannot exclude unauthorized records or separate supplier entities.
Configuration and implementation checks
The settings below concern parsing and chunking for procurement and supplier materials. Each is a documented setting or an implementation check. The numeric values are hypothetical starting points for this page's comparison, not official defaults.
| Configuration item | Hypothetical starting point | Reason and comparison method |
|---|---|---|
| Search mode | Hybrid retrieval | Supplier materials contain contract numbers and natural-language terms; compare semantic recall hits against full-text recall hits |
| Reference limit | Hypothetical start of 3000 tokens | Use one complete supplier-material chunk as reference; compare whether raising it brings in unrelated fragments |
| Minimum relevance | Hypothetical start of 0.4 | Compare against Rerank scores and observe whether valid terms are lost after filtering |
| Question optimization | Enabled for multi-turn follow-up | Compare recalled content before and after completion for short questions and multi-turn follow-ups |
| Result reranking | Enabled when a model is available | Compare candidate text order before and after reranking and observe the 0-1 score distribution |
| Chunk completeness check | Manual review | Check each discount approval, slow-moving list, price tag, and margin sheet for whether conditions and constraints stay in one chunk |
RRF is the fusion of multiple result paths after hybrid retrieval and is not a switch. Image retrieval requires image input and a supporting model, and reranking requires a model enabled by configuration; these paths do not run on every request.
Three common errors
- Treating a successful upload as an updated retrieval index. After a material changes, the training and index state is not checked, and retrieval still returns old content. Check: enter a keyword from the changed clause, compare the returned content against the latest source text, and observe whether an older version field still appears.
- Using a similarity score to filter supplier entities. Similarity is a relevance measure and does not perform permission or entity isolation. Check: enter one supplier's contract number, compare whether returned results include records from other suppliers, and observe whether score level matches entity ownership.
- Treating a quotation validity period as a supply period. The two fields have different meanings; only mutually exclusive values in the same field form a contradiction. Check: enter the quotation validity period field, compare whether the returned text describes it as a supply period, and observe whether the field name and value correspond.
How to verify the setup
Hypothetical paper exercise. Source text: "Supplier A submitted a batch quotation on March 2025, with a quotation validity period ending 30 June 2025; the supply scope is limited to past-season footwear for the current season, and settlement is based on actual shipped quantity. The quotation is still pending review by the procurement lead." Draft sentence under review: "Supplier A's supply period ends 30 June 2025."
- Verification one: enter the "supply period" field from the draft sentence and the "quotation validity period" field from the source text, and compare whether they are the same field. The source text states only a quotation validity period and does not state a supply period, so the draft is unsupported and the evidence gap is recorded.
- Verification two: enter "Supplier A review status," compare the source text phrase "still pending review by the procurement lead" against whether the draft sentence mentions completed review. Pending review does not constitute approval, so the item remains to be confirmed.
- Verification three: use the paper input of an empty reference
[]to check downstream design, and observe whether downstream handles it as an empty result. Do not claim that a fictional rule retrieval necessarily returns an empty array; record only the actually observed return state.
All three verifications require the same entity, the same period, and the same field. Libraries using different vector models cannot be jointly associated; first check the identifier and exclude non-matching ones, record the actual observation, and do not infer an operating result after joint association.