What data for this category looks like
The main entities in this category are flow, pressure, temperature, and analytical instrument manufacturers, as well as metrology laboratories. Daily documents include assembly and commissioning sheets, calibration procedures, original calibration records, error curves, metrology certificates, and software version logs. Data sources cover design documents, standard instrument and environmental monitoring data, calibration software, laboratory information systems, and customer range requirements. The update frequency is once per calibration for each instrument. Documents contain fixed business fields such as instrument number, calibration date, error value, unit of measurement, and traceability chain information. Some documents include charts and formulas.
What constraints these characteristics impose on document parsing and chunking
Documents in this category have diverse types, and include structured fields, charts, and formulas. Precise splitting is required to retain business context. Calibration records for a single instrument are independent business units. Chunking must not cross the boundaries of a single instrument’s documents. Otherwise, traceability chain information will be confused, leading to failure in metrology audits. Document update frequency is high, so fast parsing for small batches of documents must be supported. Some documents are exported from different systems, with varying formats. Custom parsing adaptation is needed. Error values and units of measurement are core audit fields. Splitting must ensure these fields are not split apart.
How to set configurations
| Configuration Item | Recommended Setting | Rationale |
|---|---|---|
chunkSize | 800–1200 characters | Documents in this category include single sets of calibration data and core information from single-page certificates. 800–1200 characters can cover complete business units and avoid splitting key fields |
chunkSplitter | Split by structured blocks + header and footer | Daily documents have structured modules such as calibration procedures and error curves. Splitting by block retains business context and avoids splitting traceability chain information |
customPdfParse | Enabled | Documents in this category include formulas and error curve charts. Custom parsing adapts to non-standard PDF formats and fully extracts formula and chart text |
PARSE_FILE_TIMEOUT_SECONDS | 300–600 seconds | Complete calibration records for a single instrument may include multiple pages of error curves. This duration prevents timeout failures |
UPLOAD_FILE_MAX_SIZE | 500 MB | Batch-uploaded collections of calibration records may have large file sizes. This setting supports batch supplier audit scenarios |
chunkSettingMode | Manual configuration | Documents in this category have fixed structures but strict field requirements. Manual configuration allows precise control of chunk boundaries and ensures core audit information is not split |
Three common errors
- The symptom is that after uploading a metrology certificate PDF, the chunk preview returns "unable to read the file content". The cause is that
customPdfParseis not enabled, and the format of industrial instrument documents with charts and formulas is not adapted. - The symptom is that when deploying FastGPT v4.8.12, complex error formulas cannot be parsed, while the online version (FastGPT v4.17.0) returns results normally. The cause is a version difference between the deployed instance and the online version, and a custom parsing environment adapted for formula parsing is not configured.
- The symptom is that after batch uploading calibration records for multiple instruments, chunk content crosses the document boundaries of a single instrument. The cause is that
chunkSplitteris not configured to split by header identifiers, leading to confusion of traceability chain information and reduced audit efficiency.
How to confirm the configuration is correct
- Upload a standard metrology certificate PDF, and check if the chunk preview retains core fields such as instrument number, calibration date, error value, and unit of measurement.
- Upload a calibration procedure document containing complex error formulas, and check if the parsing result fully extracts formula text and related descriptions.
- Upload complete calibration records for a single instrument, and check if chunks do not cross the document boundaries of the single instrument, ensuring traceability chain information is complete.
- Test uploading a batch collection of calibration records under 500 MB, and confirm that parsing does not trigger timeout or file size limit errors.
The values provided are common starting points and should be assessed against the relevant samples in use.