Document Parsing and Chunking for Laboratory Hazardous Chemical Purchasing and Records: Compare Documents and Locate Differences

Lab Chemicals: review parsing for document checks. Compare the source records, required inputs and downstream results before handing the task over.

What this category's data looks like

The data for this category consists of three types of materials: laboratory hazardous chemical procurement applications, usage registration, and inventory ledgers. Sources are procurement system exports, scanned paper usage records, and inventory spreadsheets. The update cadence changes with each procurement, usage, and monthly inventory count. Fields include product name, specification, quantity, date, handler signature, and receipt number. After procurement applications enter the parsing and training task, they become data rows to be indexed. Usage registration and inventory ledgers are parsed into independent data rows, each bound to a unique data identifier, for subsequent retrieval and comparison of whether the name, quantity, and date of the same object across materials are consistent.

Configuration and implementation checks

This step is executed by the parsing and training task (server-side task), outputting data rows and index status (parsed, indexing, indexed). The discrepancy list is then completed by retrieval and subsequent processing. After parsing is completed, parsed chunks are immediately persisted as visible data rows and bound to a unique data identifier. The index transitions through three states: parsed, indexing, indexed.

ObjectCheckVerification action
Procurement application materialProduct name, quantity, date, and handler signature must be consistent with the corresponding record in the usage registrationVerify the data row status transitions, and check whether fields are complete after chunking by chunkSize and chunkSplitter
Usage registration materialUsage quantity, date, and receipt number must be consistent with the corresponding record in the inventory ledgerVerify the file record and data row identifier separately, and confirm trainingType and index status
Inventory ledger materialBalance quantity, inventory date, and signature must be cross-consistent with the procurement application and usage registrationInspect the data rows covered by the rebuild task; only data in indexed status should be covered
Data row statusMust be one of parsed, indexing, indexed and consistent with the interface displayOpen the data row to verify the status, and confirm that data to be indexed does not participate in recall

Three common errors

  • Symptom: The same hazardous chemical has different name spellings in the procurement application and inventory ledger, so retrieval comparison fails to match. Cause: Parsing and chunking did not unify the product name field, and the two materials each became separate rows.
  • Symptom: There is a discrepancy between the usage registration quantity and the inventory ledger balance quantity, and the discrepancy list cannot localize it. Cause: Versions were not synchronized, and old-version data rows are still in indexed status and participate in recall.
  • Symptom: The procurement application lacks a handler signature or the usage receipt number, so review cannot write back. Cause: Fields are missing from the scanned material, and the parsed data row is not marked as missing items.

How to verify the setup

Upload one hazardous chemical procurement application material, observe the data row status transition from parsed to indexing to indexed, and confirm that the rebuild task only covers data rows in indexed status.

Use one real usage record to find fields inconsistent with the inventory ledger, and verify the three items: name, quantity, and date.

  • Verify the file record and data row separately, and confirm that the file and data row identifiers are distinguishable
  • Inspect the data rows covered by the rebuild task, and confirm that it targets only indexed data