This document is written based on FastGPT v4.17.0
What this category of data looks like
Data for this category comes from enterprise metrology rooms, metrology management systems in CNAS system laboratories, external calibration institution certificates, standard device data, LIMS systems, and environmental monitoring records. Daily documents include gauge ledgers, calibration plans, calibration certificates, interim check records, original test records, reference material ledgers, and deviation handling documents.
Data updates are triggered by calibration cycles, single test results, or abnormal situations. Documents contain fields such as traceability number, calibration value, measurement uncertainty, instrument status, and include professional units like mm, mg, kPa.
Constraints on document parsing and chunking
Metrology documents include precise numerical values with professional units and complete traceability chain information. Chunking must preserve field association, and avoid splitting content across different traceability numbers.
Documents such as calibration certificates and original test records have fixed structures. Fixed separators must be used to complete chunking.
Data update frequency fluctuates with calibration cycles, test results, or abnormal situations. Chunking must support incremental parsing to adapt to frequently updated ledgers and certificates.
Traceability-related information must be fully retained in a single chunk. This prevents losing associated calibration results and instrument status during retrieval, avoids judgment errors caused by broken traceability, and eliminates the inability to trace the product range affected by inaccurate instruments.
How to set configurations
| Configuration Item | Recommended Value | Rationale |
|---|---|---|
chunkSize | 800–1200 characters | Adapts to the length of single segments in metrology documents that include traceability numbers and calibration values, and avoids splitting complete traceability chains |
chunkSplitter | Fixed separators (such as "Calibration Date:" "Uncertainty:") plus line breaks | Matches the fixed field separation format of metrology documents, and preserves field association |
customPdfParse | Enabled | Adapts to fixed-format PDF documents such as calibration certificates and test records, and retains table and field layout |
PARSE_FILE_TIMEOUT_SECONDS | 120–180 seconds | Reserves sufficient parsing time for documents like calibration certificates that may contain multi-page detailed data |
chunkSettingMode | Manual configuration | Adjusts chunking rules for different types of metrology documents such as ledgers and certificates, and adapts to diverse document structures |
Three common errors
- Phenomenon: Metrology tables in uploaded calibration certificate PDFs are not displayed in knowledge base search results. Cause: The
customPdfParseconfiguration is not enabled. The default parsing mode cannot retain PDF table structures. - Phenomenon: Traceability numbers and their corresponding calibration values are split into different chunks in search results. Cause: The
chunkSplitterconfiguration does not match the fixed field separators of metrology documents, leading to split associated information. - Phenomenon: After using a local vector model to split documents and uploading them to the server, unit loss or numerical splitting errors occur during retrieval. Cause: Unified chunking rules are not used. Different splitting logic disrupts the field and unit association of metrology documents.
How to confirm the configuration is correct
- Upload a standard calibration certificate PDF, and check if the parsed document retains tables and all metrology fields.
- Manually adjust the
chunkSizeparameter range, upload a gauge ledger document containing a long traceability chain, and confirm that chunks do not split complete traceability entries. - Verify the
PARSE_FILE_TIMEOUT_SECONDSconfiguration, upload multi-page test records, and confirm that the parsing process has no timeout errors. - Test different types of metrology documents such as reference material ledgers and deviation handling documents, and confirm that the chunking rules adapt to various document structures.
The values provided are common starting points and should be measured against the reader's own samples.