What the data for this category looks like
Data for this category primarily comes from cell capacity grading data, MES systems, welding equipment, torque tools, BMS software version records, and EOL test benches. Commonly generated documents include grouping rules, BOM lists, welding and torque records, BMS flashing records, insulation and voltage withstand reports, EOL test reports, and system configuration lists. Data is updated once per pack or per cabinet, and synchronized whenever the software version changes. Document fields include cell capacity (Ah), welding torque (N·m), insulation resistance (MΩ), BMS version number, and others. Some documents are test records in scanned image format.
What constraints do these characteristics impose on the document parsing and chunking link?
Multiple heterogeneous data sources create mixed document formats. These include structured BOM lists, unstructured test records, and some documents in scanned image format. Different parsing logic must be supported to accommodate these variations. The high-frequency update requirement of once per pack or per cabinet means the parsing and chunking link must maintain low latency to avoid disrupting production workflows. Structured fields with clear units require precise extraction to prevent loss of the association between units and their associated fields. Documents in scanned image format cannot have text extracted directly, so additional OCR parsing capability must be configured. High-frequency, small-batch documents require parameter settings optimized for fast upload and parsing to avoid blocking document review processes in production workflows.
How to set the configurations
This guide is adapted for FastGPT v4.17.0. The following configuration items are set based on the document characteristics of this category:
| Configuration Item | Recommended Setting | Rationale |
|---|---|---|
chunkSize | 800–1200 characters | This category of documents includes structured fields with units and long-text test records. This range balances context integrity and retrieval accuracy |
chunkSplitter | Split by paragraph and punctuation | Documents include structured sections such as welding records and BMS configurations. Splitting by paragraph preserves the integrity of fields and associated information |
customPdfParse | Enable OCR mode | This category includes scanned image format insulation and voltage withstand reports and EOL test records. OCR extracts text content from scanned documents |
PARSE_FILE_TIMEOUT_SECONDS | 600 seconds | Large BOM lists or multi-page scanned documents take longer to parse. This duration avoids timeout errors |
UPLOAD_FILE_MAX_SIZE | 1000 MB | The configuration list for a single cabinet system in this category may include multiple associated documents, so large file upload support is required |
Three common mistakes
- Symptom: After uploading a scanned insulation and voltage withstand report, search results are empty. Cause: The OCR mode for
customPdfParsewas not enabled, so text from scanned images cannot be extracted. - Symptom: A platform error triggers when parsing a large BOM list, with a timeout prompt. Cause: The
PARSE_FILE_TIMEOUT_SECONDSparameter was not adjusted. The default duration is insufficient to complete long document parsing. - Symptom: The unit for welding torque is missing from chunked results, and searches cannot match query conditions that include units. Cause: The
chunkSizesetting is too small. Fields with units are split across different chunks, breaking the association between the field and its unit.
How to confirm the configuration is correct
- Upload a scanned EOL test report. Check if the parsed text includes test data and units to confirm the OCR configuration is active.
- Upload a BOM list with more than 50 pages. Wait for parsing to complete and check for timeout errors to confirm the timeout parameter setting is appropriate.
- Extract a chunked entry that includes torque parameters. Check if the torque value and N·m unit are retained to confirm the chunking logic did not break field associations.
- Upload a system configuration list larger than 100 MB. Confirm the upload proceeds normally and parsing completes to confirm the file size limit setting meets requirements.
The values provided are common starting points and should be measured against the reader's own samples.