What the documents look like
Fresh-food weighing and sorting sits inside the warehouse fulfilment stage, and its central facts are weight and specification. The documents handled day to day are sorting task sheets, weighing labels, short-weight compensation forms and quality inspection records. Data arrives from the WMS, electronic scales, the order system and inspection photographs, and it is updated per order or per day. Errors at this stage surface as compensation for short shipments, gross-margin loss and poor customer ratings.
Three features of this material shape document parsing and chunking. First, the same product can appear across different forms with only the specification or weight range differing, so a chunk must retain the full set of conditions needed for a judgement. Second, weighing labels and short-weight compensation forms frequently carry identifiers and precise values, which suits full-text retrieval. Third, quality inspection records include photographs, and whether a photograph can participate in retrieval depends on whether a corresponding index was created at ingestion time.
Constraints at the document chunking step
A FastGPT knowledge base is made up of library, collection and data. The smallest search unit is the library; collections exist for classification management and do not affect search results. The MongoDB dataset.datas table stores the original vector data, and its indexes field records the corresponding vector IDs, so one set of data can map to several vectors. PostgresSQL with the PG Vector extension and an HNSW index carries vector retrieval. During retrieval, vectors are recalled first, then the vector ID is used to locate the original content in MongoDB; when several routes hit the same set of original data, the results are merged and the vector score takes the highest value.
Chunk completeness directly affects retrieval precision. When a passage is structurally and semantically complete and covers a single topic, precision improves. If a specification condition, a weight range and the product it applies to are split across separate chunks in fresh-food weighing and sorting, the retrieved fragment may lack the subject or scope needed for a judgement. Multi-vector mapping allows several vectors to be attached to one longer text, so that if any one of those vectors is retrieved, the data is recalled.
Hypothetical paper exercise sample: the source text to be checked reads, "Sorting task sheet category A: leafy vegetables are weighed within the 300 g to 350 g range; items outside the range require label review. Weighing label identifiers begin with SC; short-weight compensation form identifiers begin with BC. Quality inspection records include photographs, and the photograph index is generated at ingestion time." A draft sentence to be assessed reads, "Short-weight compensation form identifiers begin with BC, therefore all forms whose identifiers begin with BC are short-weight compensation forms." The draft sentence extends a description of an identifier prefix into a classification judgement. The source states the prefix only; it does not state a unique correspondence between the prefix and the document type, so the sentence is unsupported.
Configuration and implementation checks
The table below lists officially documented settings or implementation checks relevant to this feature and the primary task, tied to the sample above. The values are starting points for this hypothetical exercise, used for comparison, and are not official defaults.
| Configuration item | Hypothetical starting point for this exercise | Reason for the choice and comparison method |
|---|---|---|
| Search mode | Hybrid retrieval | The sample contains identifiers and natural-language phrasing; hybrid retrieval uses semantic recall and full-text recall together and merges them through RRF, so hit behaviour can be compared with a single mode |
| Reference limit | Hypothetical starting point of 2000 tokens | Reference volume is controlled in tokens, which avoids instability caused by differing chunk lengths; compare with a smaller value to see whether judgement conditions are truncated |
| Minimum relevance | Hypothetical starting point of 0.4 | This takes effect only under semantic retrieval or result reranking; compare with a higher value to see whether fragments containing identifiers are filtered out |
| Question optimisation | Enabled | The sample may involve follow-up questions; question optimisation performs coreference resolution and question expansion, so compare with the disabled state on short questions |
| Result reranking | ReRank enabled | After hybrid retrieval the candidate range is large, and the rerank score is more precise and can be used for filtering; compare ordering with the unranked state |
| Image index check | Check whether an image vector index was generated at ingestion | The sample inspection record contains photographs; image retrieval depends on the ingestion index and model capability, so verify the identifier and record the observation |
Three common errors
- Treating an identifier prefix as a basis for determining document type. The sample source states only that short-weight compensation form identifiers begin with BC; it does not state that every form whose identifier begins with BC is a short-weight compensation form. The incorrect action is to decide the document type from the prefix alone. The correct action is to check the document type field and record the item as pending verification when that field is absent.
- Treating a successful upload as an updated retrieval index. After source material changes, the actual training and index state must be checked. Uploading and index updating are two separate stages, and a successful upload does not by itself mean retrieval results already include the new content.
- Treating retrieval results as an authorization boundary. Semantic recall and relevance scores do not perform entity isolation or authorization decisions. Retrieving a piece of content does not mean the current requester is entitled to use it, and the authorization state must be checked separately.
How to verify the setup
- Input the sample source text and the draft sentence, then compare whether the draft sentence goes beyond what the source states about the identifier prefix. If retrieval returns only the identifier-prefix description and does not return a unique correspondence between prefix and document type, the draft sentence is unsupported and must not be used to determine document type.
- Input a query containing an identifier and a query in natural language, then compare the fragments returned under hybrid retrieval. If the identifier query hits a weighing label or compensation form fragment and the natural-language query hits a specification-range fragment, the chunking has retained the conditions needed for a judgement. If either query returns an empty result, record the observation without inferring a cause.
- Input an explicit paper-based empty result
[]to check the downstream design, and compare whether the downstream handles it as an empty result. An empty result does not mean that retrieval of a fictional rule will necessarily return[]; it is used only to check how the downstream accepts empty references. Libraries built on different vector models cannot be associated together; verify the identifier first, exclude those that do not match, and record the observation rather than running a cross-model retrieval to compare missed recalls.