What data for this category looks like
Daily documents for this category include assembly and commissioning sheets, calibration procedures, original calibration records, error curves, metrology certificates, and software version logs. Data sources cover design documents, calibration standards and environmental monitoring data, calibration software outputs, laboratory information system records, and customer range requirements. Data is updated each time a single instrument is calibrated, generating new records. Individual documents contain exclusive fields such as instrument serial number, calibration date, error value, and traceability chain number. Some documents like error curves include unstructured chart text and professional numerical units.
What constraints these characteristics impose on the vector model and index link
This category has high data update frequency and strong exclusivity of individual records, requiring indexes to support incremental updates and precise recall for a single device. Documents contain unstructured error curve text and traceability chain association information, requiring chunking strategies to preserve cross-chunk associated fields. Exclusive fields such as instrument serial number and calibration date need to be used as index filter conditions to avoid mixing data from other devices in search results. Some documents have long lengths, requiring vector models and indexes to adapt to long text chunk processing while preserving professional numerical units and traceability information.
How to set configurations
| Configuration Item | Recommended Range | Rationale |
|---|---|---|
| Chunk Length | 800–1200 characters | Adapts to the standard text length of calibration procedures and error curves, avoiding truncation of professional formulas and traceability chain content |
| Overlap Length | 150–200 characters | Preserves associated fields such as calibration date and instrument serial number across chunks, ensuring complete associated documents can be recalled during retrieval |
| Similarity Threshold | 0.75–0.85 | Filters low-match general instrument knowledge, prioritizing recall of calibration records containing exclusive fields for this instrument |
maxToken | 8192 | Accommodates complete metrology certificates and long error curve text chunks, preventing truncation of critical information |
| Single Search Limit | Top 6 results | Matches the number of associated documents required for single instrument calibration, avoiding redundant results interfering with retrieval |
| Reranking | Enabled | Prioritizes recall of highly relevant documents containing core business fields such as traceability chain and calibration results |
Three common mistakes
- Phenomenon: Calibration records for multiple instruments appear in search results, and data for the target device cannot be precisely recalled. Cause: Instrument serial number was not configured as an index filter condition, so the index did not distinguish exclusive data for individual devices.
- Phenomenon: Long text error curves or metrology certificates are truncated, making complete error values and traceability information unavailable. Cause: Chunk length was set too small and the
maxTokenparameter was not adjusted, leading to splitting or truncation of long text chunks. - Phenomenon: Duplicate calibration record entries are generated during incremental index updates. Cause: A unique identifier field was not generated for each calibration, leading to duplicate writing of the same calibration data to the index.
How to confirm the configuration is correct
- Upload a calibration procedure and supporting error curve for a single instrument, initiate a search targeting that instrument’s serial number, and confirm the recalled results only contain exclusive documents for that device.
- Adjust the chunk length parameter, upload a long-text metrology certificate, and check that the chunked text fully preserves traceability chain and numerical unit information.
- Submit an incremental update of simulated calibration data, and verify that no duplicate entries are generated in the FastGPT v4.17.0 index logs.
- Adjust the similarity threshold parameter, test changes in the number of search results, and confirm that the number of recalled documents meets business requirements after threshold adjustment.
The values provided are common starting points and should be evaluated against the reader’s own samples.