What the documents look like
Creator special-session sales collaboration centers on external creator fulfillment and commission settlement, with materials drawn from selected alliance records, creator marketplace data, creator historical data and store transaction details. The documents handled day to day are creator cooperation agreements, commission setting sheets, sample shipment forms and livestream performance reports, refreshed per session or per week. For document parsing and chunking, this means one pipeline must carry contract clauses, commission percentages, sample dispatch records and session outcomes at the same time. Chunk boundaries have to keep the subject, the object, the period and the effective scope together, so that a condition statement is not separated from the content it governs. Retrieval and forms locate and collect values; authorization status, deal completion and contract effectiveness are separate facts that require checking against the source wording item by item.
A practical consequence is that a commission percentage written in an agreement and the same percentage written on a commission setting sheet are two distinct records even when the numbers match. Chunking that merges them removes the ability to say which document supplied which value. The same applies to sample dispatch: a deadline in an agreement and an actual dispatch date on a shipment form are different fields, and only an explicit conflicting value in the same field for the same subject and period counts as a contradiction.
Constraints at the document chunking step
In FastGPT the knowledge base consists of library, collection and data, and the smallest search unit is the library. Vectors are stored in the PostgresSQL vector field with an HNSW index; MongoDB dataset.datas holds the original data plus an indexes field, and one data group can map to multiple vectors. During retrieval, vectors are recalled first, then the vector ID is used to find the original data in MongoDB; entries that map to the same original data are merged and the highest score is kept. Semantic retrieval, full-text retrieval, hybrid retrieval, RRF fusion and ReRank are distinct paths. RRF is the fusion step for mixed results, not a toggle. Minimum relevance takes effect only under semantic retrieval or result reranking; it is a filtering threshold and does not perform permission or entity isolation. Libraries built on different vector models cannot be jointly associated; identifiers are checked first and non-matching libraries are excluded.
Hypothetical drill sample: the source wording reads, "Party A commissions Party B to run one special-session livestream on June 1, 2025, with a commission of fifteen percent of transaction value; samples are dispatched before May 20, 2025." The draft sentence under review reads, "Party B completed sample dispatch on May 18, 2025, and commission is settled at fifteen percent." The source states a dispatch deadline and a commission percentage; it does not state the actual dispatch date and does not state that settlement is complete. The draft sentence is unsupported.
Configuration and implementation checks
| Configuration item | Setting for this page | Verification method |
|---|---|---|
| Data storage | Library, collection and data in three tiers; smallest search unit is the library | Confirm the materials under review fall within the same library |
| Vector and index | PostgresSQL vector field, HNSW index | Check the index identifier against the library identifier |
| Original data mapping | indexes array in MongoDB dataset.datas | Confirm one original data group can map to multiple vectors |
| Retrieval mode | Semantic retrieval, full-text retrieval or hybrid retrieval | Retrieve separately by term, identifier and natural language |
| Fusion and reranking | RRF fusion, ReRank secondary ordering | Confirm the rerank model is enabled per configuration |
| Reference limit | Each search references at most n tokens | Record this hypothetical trial starting value and compare |
The reference limit n is a hypothetical trial starting value for this page. It is chosen because it makes the effect of chunk length on recall observable, and the comparison object is the referenced content of the same material batch under different n values. This value is not an official default.
Three common errors
- Treating a successful upload as an updated retrieval index, without checking actual training and index status. Check: input the same creator cooperation agreement, compare the upload time against the index status, and observe whether retrieval returns the content of that version.
- Treating minimum relevance as a permission or entity filter, assuming a higher value excludes unrelated creators. Check: input the same query, compare the returned sets under different minimum relevance values, and observe whether unrelated subjects still appear.
- Treating a pending review request as approved and executed. Check: input one commission setting sheet request in pending status, compare the request status against execution records, and observe whether any settlement action appears.
How to verify the setup
- Verification one: input the hypothetical sample source wording and the draft sentence, compare "dispatched before May 20, 2025" against "completed dispatch on May 18, 2025". If the retrieval result returns only the source wording and does not return the completion fact asserted by the draft sentence, the expectation is met.
- Verification two: input three query types — a term, an identifier and a natural language question — and compare what each path returns. If full-text retrieval matches the identifier and semantic retrieval matches the natural language description, the expectation is met; if a path returns nothing, record it as empty and keep the gap.
- Verification three: input an explicit paper-level empty result [], and check whether the downstream design handles it as an empty reference. If the downstream produces no fabricated referenced content, the expectation is met. All of the above are paper-level checks; no retrieval or settlement was actually executed.