Document Parsing and Chunking for Vocational College Curriculum Planning: Compare Documents and Locate Differences

Vocational Curricula: review parsing for document checks. Check source versions, input mappings and the values received by downstream nodes.

What this category's data looks like

Formulating talent cultivation programs in vocational colleges involves three types of materials: talent cultivation programs, course standards, and job competency analysis tables. The sources are academic affairs offices, secondary schools/departments, and industry-enterprise research. The update cadence changes with program adjustments, curriculum reform, and changes in job competencies. The talent cultivation program is the master outline, the course standard is the refinement at the course level, and the job competency analysis table is the source of competencies. When the materials enter this feature, the talent cultivation program and course standards serve as the objects to be parsed and chunked, and the job competency analysis table serves as the basis for comparison of competency fields. Field characteristics include program name, course name, class hours and credits, semester offered, signature, and date. The name, quantity, and date of the same record in different materials must remain consistent.

Configuration and implementation checks

This step is executed by the parsing and training task (server-side task), outputting data rows and index status (parsed, indexing, indexed). The "discrepancy list" is then completed by retrieval and subsequent processing. After parsing is complete, parsed chunks are immediately persisted as visible data rows and bound to a unique data identifier. The index transitions through three states, and data pending indexing does not participate in recall.

ObjectCheckVerification action
Data row statusIs in one of the three states parsed, indexing, indexed and is consistent with the interface displayOpen the data row to check the status, confirm that data pending indexing does not participate in recall, and check the chunking results corresponding to chunkSize and chunkSplitter
Identifiers of files and data rowsThe identities of the file record and the data row are distinguishable from each otherCheck the file record and the data row separately, confirm that the unique data identifier is bound correctly, and check trainingType and the index status
Talent cultivation programProgram name, course name, class hours and credits, date, and signature are consistent with the course standardsCheck the two materials field by field, record discrepancies in name, quantity, and date, and confirm that versions are synchronized and signature receipts are complete
Course standardsCourse name, class hours and credits, and semester offered correspond to the job competency analysis tableCheck the course-to-competency mapping, confirm that versions are synchronized and signature receipts are complete, and confirm that the data row is in the indexed state

Three common errors

  • Symptom: Materials that were just uploaded cannot be found. Cause: The data row is in parsed or indexing, and data pending indexing is naturally isolated at the retrieval layer and is not recalled.
  • Symptom: Empty data rows remain after training fails. Cause: Training interruption leaves orphan data behind, and the failed data must be directly edited or deleted.
  • Symptom: Rebuilding takes abnormally long after switching models. Cause: Rebuilding only targets data that has already been indexed, and data pending indexing does not participate in the rebuild chain.

How to verify the setup

Acceptance for this feature: Upload one material, observe the data row status transitions and the coverage scope of the rebuild task, and check chunkSize, chunkSplitter, trainingType, and the three states parsed, indexing, and indexed.

Business test: Use one real record to find fields that are inconsistent with the ledger, and confirm discrepancies in name, quantity, date, and signature receipts.

  • Check the program name, course name, class hours and credits, and date fields of the talent cultivation program and the course standards
  • Check the course mapping and signature receipts of the job competency analysis table, and confirm that versions are synchronized