What this category's data looks like
This page addresses the progress record and statistical verification scenario for peer review organizations. Materials come from three types of documents: review invitations, expert review comments, and author revision notes. The update cadence is rolling with review rounds, summarized weekly or monthly. Review invitations contain invitation dates, reviewer names, and response status; expert review comments contain comment dates, reviewer signatures, and comment items; author revision notes contain revision dates, point-by-point responses, and author signatures. When these three types of materials enter the knowledge base retrieval and recall function, each serves as a source of candidate segments: review invitations provide invitation and response fields, expert review comments provide comment and signature fields, and author revision notes provide revision and response fields. Retrieval results are used by subsequent nodes or manual verification to check whether record counts and statistical counts match, whether the statistical criteria are written in the table header, and based on this to generate progress tables and statistical criteria descriptions, which enter weekly or monthly reports.
Configuration and implementation checks
This step is executed by the knowledge base retrieval service (retrieval node and search test), outputting candidate segments and similarity scores (after pattern, merge, rerank, filter, and limit trimming). The progress table and statistical criteria description are completed by subsequent nodes or manually.
| Object | Check | Verification action |
|---|---|---|
| Review invitation | Invitation date, reviewer name, and response status must be consistent with the corresponding records in expert review comments | In the search test, use the same reviewer name to search the two types of materials separately, compare the date and response fields in candidate segments, and record the number of candidates and scores |
| Expert review comments | Comment date, reviewer signature, and number of comment items must correspond to the response items in author revision notes | Switch searchMode to hybrid, search for comment item keywords, check whether returned candidates cover both comment and response sides, and check whether signatures or responses are missing |
| Author revision notes | Revision date, point-by-point responses, and author signature must align with the header fields of the statistical criteria description | Use one week's records to search revision notes, check whether criteria and exceptions are clearly stated when verifying totals, separately record the number of items in the citation array, and verify the tokens budget of the citation text |
| Progress table and statistical criteria description | Record counts and statistical counts match, criteria are written in the header, and exceptions are noted | Search the three types of materials on a weekly basis, compare names, quantities, and dates in candidate segments, and confirm version synchronization and complete signatures and responses |
Three common errors
- Phenomenon: The same reviewer's name spelling or dates do not match between the invitation and comment materials. Cause: Materials are entered separately and versions are not synchronized; after retrieval hits, associated fields are not verified.
- Phenomenon: The number of expert review comment items is inconsistent with the number of responses in author revision notes. Cause: Revision notes do not respond point by point or lack signatures/responses, and statistical criteria do not state exceptions in the header.
- Phenomenon: There are many candidate segments but the top-ranked ones are irrelevant to progress verification. Cause: Retrieval mode does not match material form, rerank is not enabled, or similarity filtering and citation limit trimming are not performed.
How to verify the setup
Acceptance of this feature: Test once each with synonym-rewritten questions, compare candidates and scores, then toggle rerank on and off to observe the ordering of the top few results; at the same time adjust similarity, limit, and usingReRank, and record changes in the number of candidates and scores.
Business test: Use one week's records to verify totals, check whether criteria and exceptions are clearly stated; record the number of citations separately by array length, and for the citation limit only verify the tokens of the citation text, without equality comparison.
- In the knowledge base search test, switch
searchMode, search review invitations, expert review comments, and author revision notes once each, record the number of candidates and scores, and verify whether names, quantities, and dates are consistent. - Observe the order of the top few results once before and after enabling rerank, check whether signature and response fields are returned with candidate segments; summarize one week's records into a progress table, write statistical criteria in the header, and annotate exceptions.