Peak-sale fulfillment: document chunking for document comparison

Review document chunking for document comparison in Peak-sale fulfillment. Check sources, input boundaries, error handling and repeatable test evidence.

What the documents look like

Peak-sale warehouse and delivery scheduling revolves around short-term peak capacity and the arrangement of people, vehicles and space. The working materials are peak-sale forecast sheets, staff rosters, courier assurance plans and daily exception reports. These materials update by event and by day, and a single peak-sale event can produce several forecast versions and several roster versions.

In this setting the contract and transaction material comparison task handles field consistency across materials: whether the same subject, the same period and the same field support each other. The materials are full of date ranges, quantity ranges, responsible parties and status descriptions. Parsing and chunking need to keep those conditions in the same data block as the content they constrain, so that a condition and its conclusion are not split into different blocks and cannot be traced back to the source text.

Constraints at the document chunking step

Chunk completeness directly affects the comparison conclusion. If a date range and a headcount in a roster sheet are split apart, a retrieved fragment may show only the number and lose the applicable period, and the comparison may place data from different events side by side.

In the FastGPT vector storage structure, the MongoDB dataset.datas table stores the original vector data, and an indexes field records the corresponding vector IDs. One set of original data can correspond to multiple vectors. During retrieval, vectors are recalled first, then the vector IDs are used to find the original data in MongoDB. When the same set of original data is hit by several paths, the results are merged and the highest score is taken. This means one material can build several index groups to improve recall precision, but the index itself does not change the original data content and does not constitute verification of the material's authenticity.

The minimum relevance value takes effect only in semantic retrieval or when result reranking is used. It is a filter threshold and does not perform subject isolation or permission filtering. Retrieval is performed across the whole library; a collection is only used to classify and manage data and is unrelated to search results.

Configuration and implementation checks

The configuration below centers on the hypothetical drill sample and lists only documented settings or implementation check items relevant to this page's primary task. The values are starting points for this hypothetical trial, used to explain the comparison method, and do not represent official defaults.

Configuration itemHypothetical starting point for this trialReason for the choice and comparison object
Search modeHybrid retrievalMaterials contain both identifiers and natural-language descriptions, so semantic recall and keyword hits both need observation; compare with semantic retrieval alone
Reference upper limitSet to 3000 tokens for this trialThe sample material contains date ranges and quantity fields and needs to hold a complete conditional sentence; compare with 1500 tokens to see whether the conditional sentence is truncated
Minimum relevanceSet to 0.35 for this trialUsed to observe whether low-relevance fragments are filtered; compare with 0.2 for the number and content of returned fragments
Question optimizationEnabled for this trialThe draft sentence contains a reference, so the retrieval terms need to be observed for subject and period after completion; compare with retrieval terms when disabled
Chunk boundary checkManual checkCheck whether the date range and quantity sit in the same block; compare with the retrieved fragment after splitting into two blocks
Training status checkManual checkAfter a successful upload, check the actual training and index status; compare with only viewing the upload success notice

Three common errors

  1. Treating a successful upload as an updated retrieval index. After a material is re-uploaded, the training and index status need a separate check. If the comparison proceeds on the upload completion notice alone, an older version fragment may still be retrieved. Check method: input a field value newly added in the material and compare whether the returned fragment contains that value; if it returns the old value or returns nothing, record it as pending verification.
  2. Treating minimum relevance as a permission or subject filter. Minimum relevance is a similarity threshold and does not exclude unrelated subjects or unauthorized records. Check method: search with the name of a different subject from the sample and observe whether the returned fragment contains content from another subject; if it does, record the observation and do not infer that permissions have failed.
  3. Writing the RRF fusion of hybrid retrieval as a switch or a separate step. RRF is the fusion process for results from several recall channels, not a configuration item that can be turned on. Check method: inspect the configuration interface for a switch named RRF; if none exists, record that the documents do not describe such a switch and do not add one.

How to verify the setup

Hypothetical drill sample: the source material reads "Party A: a warehouse service unit; Party B: a courier business group; assurance period: 1 November 2025 to 15 November 2025; daily pickup capacity: not less than 8,000 pieces." The draft sentence reads "the courier business group has a daily pickup capacity of 8,000 pieces during the assurance period." The comparison task is to judge whether the draft sentence is supported by the source material.

  • Verification one: input the draft sentence "the courier business group has a daily pickup capacity of 8,000 pieces during the assurance period" and compare the value of the same subject, same period and same field in the source material. The source writes "not less than 8,000 pieces" while the draft writes "is 8,000 pieces." These are different value statements for the same field, and the draft sentence is not supported by the source. Observe whether the returned fragment contains the subject, the period and the quantity field at the same time; if the fragment is missing the period, return to the original data to check the chunk boundary.
  • Verification two: input the identifier or term from the source material, "1 November 2025 to 15 November 2025," and compare whether the returned fragment contains that date range together with the quantity field it constrains. If the returned fragment has only the date and no quantity, record that the chunk may have separated the condition from the content, and check the original data manually.
  • Verification three: input a query term that does not appear in the material and compare the returned result. An empty result or a low-relevance fragment is possible, and an empty result is not guaranteed. If the returned fragment contains unrelated content, record the actual observation and do not infer that filtering has failed. For the empty-result case, a clearly defined paper input can be used to check the downstream design, without claiming that a fictional rule retrieval necessarily returns an empty array.

The verifications above describe expectations and comparison methods only. They were not actually executed and do not confirm any system behavior.