What this category's data looks like
Materials from peer review organizations usually fall into three categories: review invitations, expert review comments, and author revision notes. Each category of material carries fields, dates, and signature information, for example the manuscript name and submission date in the invitation letter, the review conclusion and expert signature in the review comments, and the item-by-item responses and author receipt in the revision notes. When these materials enter the knowledge base retrieval and recall function, each plays a different role: review invitations provide the record starting point and time anchor, expert review comments provide the basis for judgment and professional conclusions, and author revision notes provide follow-up responses and version clues. The material update cadence is synchronized with review rounds, and each round may produce a new version. Common field characteristics include the need to cross-compare the name, quantity, and date of the same record across different materials, the need to confirm whether a version is current, and the need for signatures or receipts to be complete. During retrieval, these materials are recalled as fragments for subsequent nodes or manual verification.
Configuration and implementation checks
This step is executed by the knowledge base retrieval service (retrieval node and search test), outputting candidate fragments and similarity scores (after mode, merge, rerank, filter, and limit trimming). The basis entries that can be traced back to the original text are then completed by subsequent nodes or manually.
| Object | Check | Verification action |
|---|---|---|
| Review invitation | Whether the fields, dates, and signature are complete, and consistent with the name, quantity, and date in the expert review comments | In the search test, switch searchMode to semantic, full-text, or hybrid, check whether the candidate fragments cover the key fields of the invitation letter, and record the number of candidates and the scores |
| Expert review comments | Whether the review conclusion, expert signature, and date correspond to the response entries in the author revision notes, and whether the version is current | Turn usingReRank on or off to compare the ordering of the top few results, check whether the reranked candidates still retain the key fragments of the original text, and record the number of citations separately |
| Author revision notes | Whether the item-by-item responses, author receipt, and date correspond one-to-one with the comment entries in the review comments, and whether the original text can be traced back when a signature or receipt is missing | Adjust the similarity threshold to observe changes in candidates, confirm that fragments that can be traced back to the original text are still retained after filtering, and check the tokens budget of the cited text |
| Knowledge base retrieval service | Whether the candidate fragments and similarity scores come from this category's materials and can be traced back to the original text | Test once each with synonym-rewritten questions, compare the number of candidates and the scores, and check the array length after limit trimming and the cited text tokens, recording them separately |
Three common errors
- Symptom: The name, quantity, or date of the same record does not match between the review invitation and the expert review comments. Cause: Field alignment was not done when the materials were entered, and after a retrieval hit the original text was not traced back for verification.
- Symptom: The author revision notes have been updated, but retrieval still recalls fragments of the old version. Cause: The version was not synchronized, the knowledge base did not replace the current materials in time, and candidates were not filtered by the current version.
- Symptom: The review comments lack an expert signature or author receipt, but the answer still gives basis entries. Cause: Retrieval only outputs candidates and scores; the completeness of signatures and receipts must be verified by subsequent nodes or manually.
How to verify the setup
Acceptance for this feature: test once each with synonym-rewritten questions, compare candidates and scores, then toggle rerank to see the ordering of the top few results; switch searchMode and adjust similarity, recording the number of candidates and the cited text tokens separately.
Business test: ask a cross-material basis question and see whether the answer gives a traceable source; check whether the basis comes from this category's materials, whether the version is current, and whether the entries match the original text.
- In the search test, retrieve the same question in semantic, full-text, and hybrid modes respectively, record the number of candidates and the scores, and turn rerank on or off to compare the ordering of the top few results.
- Trace each recalled fragment back to the original text, and check whether the fields, dates, and signatures in the review invitation, expert review comments, and author revision notes are consistent, and whether the version is current.
This step is executed by the knowledge base retrieval service (retrieval node and search test), outputting candidate fragments and similarity scores (after mode, merge, rerank, filter, and limit trimming). The basis entries that can be traced back to the original text are then completed by subsequent nodes or manually.
- Symptom: The name, quantity, or date of the same record does not match between the review invitation and the expert review comments. Cause: Field alignment was not done when the materials were entered, and after a retrieval hit the original text was not traced back for verification.
- Symptom: The author revision notes have been updated, but retrieval still recalls fragments of the old version. Cause: The version was not synchronized, the knowledge base did not replace the current materials in time, and candidates were not filtered by the current version.
- Symptom: The review comments lack an expert signature or author receipt, but the answer still gives basis entries. Cause: Retrieval only outputs candidates and scores; the completeness of signatures and receipts must be verified by subsequent nodes or manually.
Acceptance for this feature: test once each with synonym-rewritten questions, compare candidates and scores, then toggle rerank to see the ordering of the top few results; switch searchMode and adjust similarity, recording the number of candidates and the cited text tokens separately.