Complaints and official replies: document chunking for procurement

Review document chunking for procurement in Complaints and official replies. Check sources, input boundaries, error handling and repeatable test evidence.

What the documents look like

Complaints and official replies are addressed to a platform or a regulatory authority, so the file set carries formal evidence and remediation text. Within this category, procurement and supplier records form one material group: purchase orders, supplier qualification files, inbound receipts, and inventory movement logs. These records serve three jobs — reconstructing the transaction sequence, assembling an evidence package, and submitting a remediation reply. The people who work with them are customer service supervisors, legal and compliance staff, and store owners. An error in this material can lead to an administrative investigation, a platform penalty, or brand exposure. The records are updated irregularly, case by case, so parsing and chunking need to keep a traceable document identity and version.

In FastGPT, a knowledge base consists of libraries, collections, and data. The smallest search unit is the library; collections only classify data and do not affect search results. One data item can map to multiple vectors, and the indexes field records the corresponding vector IDs. During retrieval, vectors are recalled first, then the original data is fetched from MongoDB dataset.datas by vector ID. When the same original data is hit through several paths, the results merge and the highest score is taken. Chunking therefore has to keep a condition, its explanation, and the content it constrains from being split apart. The vector field in PostgresSQL stores vectors only, with an HNSW index. Full-text retrieval on other vector stores uses MongoDB $text with jieba tokenization. Hybrid retrieval merges through RRF. Reranking requires a model and must be enabled by configuration. Image retrieval requires image input and matching model capability. Minimum relevance takes effect only under semantic retrieval or result reranking; it is a filter threshold and does not perform permission or entity isolation.

Constraints at the document chunking step

For procurement and supplier material, the constraints follow from what the documents contain. A quotation carries a validity period. An order carries a quantity and a date. A supplier qualification file carries a scope of validity. An inventory log carries movement records. Each of these is a distinct field, and a chunk that separates a period from the thing it qualifies can produce a retrieved passage that reads as a complete statement while missing its limiting condition.

A hypothetical paper exercise sample, labelled as such and not drawn from any real record: the source text reads, "Supplier A submitted a quotation on 1 March 2025, valid until 31 March 2025; the buyer issued an order on 10 March 2025 stating a quantity of 200 units." The sentence under review reads, "Supplier A has confirmed the supply period and completed delivery." The comparison object is the same supplier, the same period, and the same field. The source states only a quotation validity period. It does not state a supply period or a delivery fact. The draft sentence is unsupported.

Chunking decisions interact with retrieval mode. Number and model queries rely on full-text recall; natural language questions rely on semantic recall. Hybrid retrieval uses both and merges through RRF. Reranking produces a score between 0 and 1 that represents relevance and can be used for filtering. Minimum relevance applies only under semantic retrieval or reranking. None of these mechanisms establishes authorization, approval, or execution status.

Configuration and implementation checks

Configuration itemValue or check on this pageBasis and explanation
Search modeHybrid retrievalNumber and model queries depend on full-text recall; natural language questions depend on semantic recall
Minimum relevanceHypothetical trial starting value 0.3 for this exerciseUsed to observe candidate count after filtering; compared against 0.2 and 0.4; not an official default
Reference capCounted in tokens, not top kDocumentation states that chunk length varies widely in mixed libraries, making top k unstable
Question optimizationEnabled by decision for multi-turn follow-up scenariosAdds one model call and improves retrieval in continuous dialogue
Result rerankingRequires a model and must be enabled by configurationRerank score is 0-1 and can be used for filtering
Post-update reviewCheck actual training and index statusA successful upload does not mean retrieval has updated

Three common errors

  1. Treating a quotation validity period as a supply period. The source states only that the quotation is valid until 31 March 2025. The draft sentence claims a confirmed supply period and completed delivery. The fields differ and the facts are not stated.
  2. Treating a retrieval hit as an authorization basis. A retrieved order does not mean the order was approved or executed. Authorization status requires a separate check.
  3. Associating libraries across different vector models. Documentation states that retrieval operates at the library level. Libraries using different vector models should have their identifiers checked first and non-matching ones excluded. Cross-model retrieval comparison is not performed to assess missed recall.

How to verify the setup

  1. Enter the source text and the draft sentence. Compare whether "quotation validity period" and "supply period" are the same field. The support condition is met only when the source explicitly states a supply period. Otherwise record the result as unsupported.
  2. Enter a term, a number, and a natural language question as separate retrievals. Observe whether the chunking preserves conditions and constrained content. A failure branch is a key condition being split apart or not recalled.
  3. After a material change, check the actual training and index status. Observe whether retrieval results reflect the new version. A successful upload does not constitute evidence that the update is complete.