Document Parsing and Chunking for Self-Study Exam Records and Graduation Review: Find Supporting Evidence

Self-Study Exams: review parsing for evidence lookup. Compare the source records, required inputs and downstream results before handing the task over.

What this category's data looks like

Self-taught examination registration records and graduation review materials include registration archives, course pass certificates, and graduation application forms. The sources are archives of provincial examination offices and main examining institutions, updated by exam session and batch. Registration archives record the candidate's basic information and enrollment status; course pass certificates record the course name, score, and date obtained; graduation application forms record the major, main examining institution, signature, and receipt. After these three types of materials enter the parsing and training task (server-side task), each becomes a searchable data row, aligned item by item by the fields, dates, and signatures of registration archives, course pass certificates, and graduation application forms, for use in cross-material basis queries. Material versions change with exam sessions; old versions are retained but no longer serve as the current basis.

Configuration and implementation checks

This step is executed by the parsing and training task (server-side task), outputting data rows and index status (parsed, indexing, indexed). Basis entries that can be traced back to the original text are then completed by retrieval and subsequent processing. After parsing is completed, parsing chunks are immediately persisted as visible data rows and bound to a unique data identifier; the index transitions through the three states parsed, indexing, and indexed.

ObjectCheckVerification action
Registration archivesThe name, admission ticket number, enrollment status, and archive date must be consistent item by item with the course pass certificate and graduation application form; pages missing a signature or receipt must be markedOpen the data row to check the status, compare the name and admission ticket number in the registration archive and course pass certificate, and check the signature and receipt fields of the graduation application form
Course pass certificateThe course name, score, and date obtained must be consistent with the course records in the registration archive; the name and date of the same course in the two materials must not mismatchCheck the file records and data rows separately, compare item by item by course name and date obtained, and record inconsistent entries
Graduation application formThe major, main examining institution, signature, and receipt date must be consistent with the current version of the registration archive; pages whose version has not been synchronized must be markedLook at the data rows covered by the rebuild task, confirm that the current version of the graduation application form has entered the indexed state, and check the signature and receipt date
Parsing chunk configurationThe chunking parameters and training mode must match the material type; the values of chunkSize, chunkSplitter, and trainingType must correspond to the original text paragraphsCheck that the data row status is consistent with the interface display, and check the chunking results of chunkSize, chunkSplitter, and trainingType corresponding to the material

Three common errors

  • Symptom: the same course name is written differently in the registration archive and the course pass certificate, and the dates also differ by one day. Cause: the two materials were archived in different exam sessions, and the names and dates were not aligned and checked.
  • Symptom: the signature field of the graduation application form is blank or the receipt is missing, yet it is still cited as the current basis. Cause: the material version was not synchronized, and pages missing a signature or receipt were not marked during parsing.
  • Symptom: a newly uploaded material cannot be found, or empty data rows are left after training fails. Cause: the data row is in the parsed or indexing state and has not entered indexed, so the retrieval layer naturally isolates it and it is not recalled.

How to verify the setup

This feature's acceptance: upload a material and observe the data row status transitions and the coverage of the rebuild task. Check whether the status transitions through the three states parsed, indexing, and indexed and is consistent with the interface display, and whether after changing the model the rebuild task covers only data rows whose indexing has been completed.

Business test: ask a cross-material basis question and see whether the answer gives a traceable source. Check that the number of citations and the tokens of the citation text are recorded separately; quoteMaxToken is checked separately as the citation content budget and is not compared for equality with the number of citations.

  • Open the data row to check the status, confirm that it is in one of the three states parsed, indexing, or indexed and is consistent with the interface display, and check the file records and data rows separately
  • Look at the data rows covered by the rebuild task, confirm that it covers only data whose indexing has been completed, and check that chunkSize, chunkSplitter, and trainingType match the material type