What this category's data looks like
The business materials of a peer review organization include three types: review invitations, expert review comments, and author revision notes. A review invitation records the manuscript title, the date sent for review, and the expert's signature; expert review comments record the review conclusion, the comment date, and the reviewer's signature; author revision notes record the revision items, the reply date, and the author's receipt. The three types of materials are linked by manuscript number and version number, and the update cadence advances round by round as the manuscript moves through the process. When the materials enter reranking and hybrid retrieval, the review invitation provides the basis for sending for review and signatures, the expert review comments provide the basis for the review conclusion, and the author revision notes provide the basis for revisions and receipts; together, the three support back-referencing of evidence items.
Configuration and implementation checks
This step is performed by the reranking model in the retrieval service, which outputs the sorted candidates and rerank scores. Evidence items that can be back-referenced to the source text are then completed by citation trimming and answer generation. Hybrid retrieval runs both the vector and full-text paths and merges the results; after merging, the scope is large and similarity filtering cannot be applied directly, so the reranking model is usually then used to sort and filter by rerank score.
| Object | Check | Verification action |
|---|---|---|
| Review invitation | The date sent for review and the expert's signature must match the review date and reviewer's signature in the expert review comments | Check the dates and signatures of the two materials under the same manuscript number, and record the number of candidates and the rerank score |
| Expert review comments | The review conclusion and comment date must match the revision items and reply date in the author revision notes | Check the names, quantities, and dates of the two materials, check whether the version numbers are in sync, and record the number of candidates and the rerank score |
| Author revision notes | The revision items and author receipt must be consistent with the current version; a missing signature or receipt is regarded as incomplete evidence | Check the receipt and signature, confirm the version is current, and record the number of candidates and the rerank score |
| Hybrid retrieval and reranking | Both the vector and full-text paths are enabled and merged, and the reranking model is bound and used for sorting and filtering | Check searchMode, usingReRank, similarity, and limit in the search parameters, and compare the ordering and scores before and after reranking |
Three common errors
- Symptom: The name, quantity, or date of the same record does not match across the two materials. Cause: The review invitation and expert review comments are not linked by manuscript number and version number, and the entries are entered independently.
- Symptom: The version is not in sync, and the author revision notes cite an old review conclusion. Cause: The material update cadence advances as the manuscript moves through the process, the old version is not marked invalid, and retrieval still returns old-version entries.
- Symptom: A signature or receipt is missing, and the evidence item cannot be back-referenced to the source text. Cause: The signature and receipt fields of the review invitation and author revision notes are not collected in sync with the materials, so there is no verifiable source when citations are trimmed.
How to verify the setup
Acceptance for this feature: turn reranking on and off and compare the order and scores of the first three items, check the number of candidates and the rerank score, and confirm the values of searchMode, usingReRank, similarity, and limit in the search parameters.
Business test: ask a cross-material evidence question, see whether the answer gives a back-referenceable source, and check the material, version, and signature corresponding to the source.
- Check the vector and full-text path switches in the search parameters, confirm that candidates come from both paths, and record the number of candidates and the rerank score.
- Compare the ordering and scores before and after reranking, check whether the first few items are relevant, and confirm that the reranking model is bound and used for sorting and filtering.