Document Parsing and Chunking for New Employee Onboarding and Training: Find Supporting Evidence

Employee Onboarding: review parsing for evidence lookup. Compare the source records, required inputs and downstream results before handing the task over.

What this category's data looks like

Enterprise new employee onboarding training materials usually include three types: onboarding training plans, course sign-in sheets, and job descriptions. The onboarding training plan comes from the human resources department and is updated by batch. Its fields include course name, date, instructor, and participating positions. The course sign-in sheet is summarized by the training executor after class. Its fields include name, employee ID, sign-in time, and signature column. The job description is maintained by the hiring department. Its fields include position name, responsibility items, qualification requirements, and effective date. When the three types of materials enter this feature, their roles differ: the onboarding training plan serves as the primary basis, used to answer course arrangements and their correspondence to positions; the course sign-in sheet serves as attendance evidence, used to verify participation records; the job description serves as the basis for responsibilities, used to answer job duties and qualification conditions. After materials are uploaded as files, parsing and chunking are immediately stored as visible data rows upon completion of parsing and bound to a unique data identifier. File records and data rows each have separate identities, making it convenient to verify them separately.

Configuration and implementation checks

This step is executed by parsing and training tasks (server-side tasks), outputting data rows and index status (parsed, indexing, indexed). Basis items that can be traced back to the original text are then completed by retrieval and subsequent processing. The index transitions through three states: parsed, indexing, and indexed. Data pending indexing does not participate in recall; changing the model only rebuilds data that has completed indexing, while the initial training covers data that is pending indexing and indexing.

ObjectCheckVerification action
Onboarding training planFields include course name, date, instructor, and participating positions, and the version is currentOpen the data row to check whether the status is indexed, and verify whether the course name and date match those in the sign-in sheet
Course sign-in sheetFields include name, employee ID, sign-in time, and signature column, and signatures and receipts are completeVerify the file record and the data row separately, and confirm that the sign-in records match the course names, quantities, and dates in the training plan
Job descriptionFields include position name, responsibility items, qualification requirements, and effective date, and the effective date is the current versionVerify whether the effective date is synchronized with the training plan batch, and check whether the responsibility items can be traced back to the original text
Data rowIs in one of the three states parsed, indexing, or indexed and is consistent with the interface displayVerify the chunking results corresponding to trainingType, chunkSize, and chunkSplitter, and confirm that the rebuild task only covers data that has completed indexing

Three common errors

  • Symptom: The name, quantity, and date of the same course do not match between the training plan and the sign-in sheet. Cause: The two materials are maintained by different stages, and the fields were not verified against the same batch.
  • Symptom: The effective date of the job description is not synchronized, and answers cite the old version of responsibilities. Cause: The version was not synchronized, the rebuild only targets data that has completed indexing, and the old data is still recalled.
  • Symptom: The sign-in sheet lacks signatures or receipts, and the attendance basis is incomplete. Cause: The signature column was not completed during post-class summarization, and the material does not meet the requirement of being traceable back to the original text.

How to verify the setup

Acceptance for this feature: Upload one material and observe the data row status transitions and the coverage scope of the rebuild task. Data rows should appear in sequence as parsed, indexing, and indexed; after changing the model, the rebuild only covers data that has completed indexing, and data pending indexing does not participate in the rebuild.

Business test: Ask a basis question that spans materials and see whether the answer provides a traceable source. For example, ask whether a certain position's courses in the training plan correspond to attendance in the sign-in sheet; the answer should trace back to the specific material and data row.

  • Open the data row to verify whether the status is consistent with the interface display, and confirm that the file record and the data row each have separate identities.
  • Check the data rows covered by the rebuild task, confirm that it only covers data that has completed indexing, and verify trainingType and the chunking parameters.