What the documents look like
Brand-owned stores are staffed and accounted for directly by the brand, so daily sales reports, shift schedules, display photos and stock transfer orders stay under brand management. In cross-border operations these materials arrive in Chinese, English or both, and one body of material must support retrieval across languages. Store inspections and POS data update daily, and document versions move with the store operating rhythm. Document parsing and chunking handles the intake structure of these materials: cutting a report or a transfer order into retrievable segments while keeping the document identity and version that point back to the source.
The four material types differ in structure. A daily sales report carries a header, store name, date and amount fields. A shift schedule carries shifts and staff. A display photo depends on a vision model to produce a description or an image vector. A stock transfer order carries an order number, an outbound store, an inbound store, a quantity and an approval state. Chunking must preserve each structure together with the conditions it constrains, and must keep the approval state and the quantity of one transfer order inside the same segment.
In FastGPT a knowledge base consists of library, collection and data. Search operates on the library as the smallest search unit, and a collection only classifies data. One data item can map to multiple vectors, and the indexes field records the vector ID array. Retrieval recalls vectors first, then uses the vector IDs to find the original data in the MongoDB dataset.datas table. When several routes hit the same original data item, the results merge and the vector score takes the highest value. Semantic retrieval supports cross-language understanding, where a Chinese question can match an English knowledge point, but it depends on model training quality, and precision is affected by keywords and sentence completeness. Exact queries such as order numbers and model numbers suit full-text retrieval or hybrid retrieval.
Hypothetical paper exercise sample: the source material offered for comparison is three sentences — "Transfer order DB-2024-017, outbound store: Shanghai Nanjing West Road Store, inbound store: Suzhou Guanqian Street Store, item: summer short-sleeve size M, quantity 30 pieces. Applicant: store manager. Approval state: pending regional manager review." One draft sentence to be judged reads: "This transfer order has completed approval and the goods have been transferred out." This sample is used for paper comparison only and points to no real store or document.
Constraints at the document chunking step
The structural differences above set the chunking constraints. A transfer order's approval state, outbound store, inbound store and quantity form one constrained unit; splitting them across segments lets a retrieved quantity segment appear while the approval state stays elsewhere. A daily sales report's header, store name, date and amount fields need to stay with the rows they describe. A shift schedule's shift and staff pairing needs to stay intact. A display photo needs an image index built at intake when a vision model or an image-capable vector model is available; text-only indexing leaves image search unavailable.
The official documents state that a relevance score is a filter threshold, not a final ranking rule. It takes effect only under semantic retrieval or when result reranking is used. The citation limit is measured in tokens, and top k is not used. RRF is the fusion step for hybrid retrieval results and is not a standalone switch. Image retrieval requires an image input and model capability. Reranking requires a model and must be enabled in settings. Retrieval itself does not perform authorization; an approval action recorded nowhere in the material stays pending verification.
Configuration and implementation checks
The settings below support parsing and retrieval comparison for the sample above. The numeric values are hypothetical starting points for this paper exercise.
| Configuration item | Hypothetical starting point | Reason and comparison method |
|---|---|---|
| Chunking method | Split at document field boundaries | Compare with no splitting of the whole order; check whether approval state and quantity stay in one segment |
| Search mode | Hybrid retrieval | Compare with semantic retrieval alone; check whether order number DB-2024-017 is hit |
| Minimum relevance | Hypothetical starting point 0.3 | Compare with 0.5; observe whether unrelated segments enter the citation |
| Citation limit | Hypothetical starting point 2000 tokens | Compare with 1000; observe whether the full transfer order fields survive trimming |
| Question optimization | Enabled for multi-turn follow-up | Compare with disabled; check short-question retrieval and added latency |
| Image index | Generated at intake for display photos | Compare with text index only; check whether image-to-image search is available |
Minimum relevance takes effect only under semantic retrieval or with result reranking. The citation limit is counted in tokens. RRF is the fusion process for hybrid retrieval results. Image retrieval needs an image input and model capability, and reranking needs a model enabled in settings. A pending review state does not by itself show that an approval permission is absent.
Three common errors
- Splitting a transfer order's approval state from its quantity. A retrieved quantity segment then suggests the whole order state is known. In the sample the approval state reads "pending regional manager review," and a pending state does not show that approval was completed.
- Using semantic retrieval for an exact query such as order number DB-2024-017. Vector similarity can recall a nearby order number. Full-text retrieval or hybrid retrieval fits this query, and the returned segment must be checked for the complete order number.
- Treating a successful upload as an updated index. After material changes, the actual training and index state must be checked. Upload and index availability are two separate steps.
How to verify the setup
- Input the three sample source sentences and the draft sentence "This transfer order has completed approval and the goods have been transferred out." Compare the draft sentence with the approval state field in the source. The source reads "pending regional manager review" and the draft reads "completed approval." For one order and one field these values are mutually exclusive, so the draft sentence is unsupported.
- Input order number DB-2024-017 in hybrid retrieval and compare whether the returned segment contains the outbound store, inbound store, quantity and approval state. A segment missing the approval state shows that chunking did not preserve the field constraint, and the split boundary needs adjustment and a repeat check.
- Input "summer short-sleeve size M transfer" in semantic retrieval and compare the returned results with the sample order. If an unrelated store segment enters the citation, record the actual observation and check the minimum relevance setting. Retrieval does not perform authorization, and an approval action not recorded in the material stays pending verification.