Reranking and Hybrid Retrieval for Research Project Task Agreement Signing: Find Supporting Evidence

Research Agreements: review reranking for evidence lookup. Check source versions, input mappings and the values received by downstream nodes.

What this category's data looks like

This category's data looks like this: signing a research project task book involves three types of materials: the project task book, the topic task breakdown table, and the budget book. The project task book gives the research objectives, assessment indicators, and start and end dates; the topic task breakdown table assigns tasks to topics and responsible persons, including task names, quantities, and completion times; the budget book lists subject amounts, calculation bases, and approvals. The three types of materials come from different sources and have different update rhythms: the task book is updated with project initiation and changes, the breakdown table is updated with topic adjustments, and the budget book is updated with funding approval and reallocation. When entering this function, the task book and breakdown table provide basis entries, and the budget book provides the funding basis; together, the three support cross-material basis queries. The materials themselves remain as they are and only participate in retrieval and reranking as candidate sources.

Configuration and implementation checks

Configuration and implementation checks: this step is executed by the reranking model in the retrieval service, which outputs the sorted candidates and reranking scores. "Basis entries that can be traced back to the original text" are then completed by citation trimming and answer generation. Hybrid retrieval runs both vector and full-text paths and merges the results; after merging, the scope is large and similarity filtering cannot be applied directly, so the reranking model is usually used again to sort and filter by reranking score.

ObjectCheckVerification action
Project task bookFields include project name, start and end dates, signatures, and seals; the version must be currentCheck the name, dates, and signature fields, and confirm consistency with the breakdown table and budget book
Topic task breakdown tableFields include topic name, task quantity, completion time, and responsible personCompare the name, quantity, and dates of the same task in the two materials, and record differences
Budget bookFields include subject amounts, calculation bases, and approvalsCheck the correspondence between budget subjects and task book indicators, and confirm that signatures or receipts are complete
Reranking modelBound and used for sorting and filtering, outputting the number of candidate entries and scoresCompare the order and scores of the top three entries before and after reranking, and check usingReRank, searchMode, and limit

Three common errors

  • Phenomenon: the same topic name is written differently in the task book and the breakdown table. Cause: the two materials are maintained separately, and names and quantities were not aligned.
  • Phenomenon: the budget book is still the old version while the task book has already changed. Cause: versions were not synchronized, update rhythms are inconsistent, and the current version was not marked.
  • Phenomenon: basis entries lack signatures or receipts, so they cannot be traced back in the answer. Cause: the signing process was not closed-loop, and material archiving is incomplete.

How to verify the setup

Acceptance of this function: compare the order and scores of the top three entries before and after enabling reranking, check the number of candidate entries and score changes, and confirm the status of usingReRank, searchMode, and limit.

Business test: ask a cross-material basis question, see whether the answer provides traceable sources, and check the tokens of the cited text and the number of array entries and record them separately.

  • Check the names, quantities, dates, and signatures of the task book, breakdown table, and budget book one by one, and mark inconsistencies.
  • Confirm that the material versions are current, update entries whose updates were not synchronized, and complete missing signatures or receipts.