What the Data for This Category Looks Like
Typical entities include manufacturers of power battery packs, energy storage cabinets, and BMS systems. Daily documents cover grouping rules, BOMs, welding and torque records, BMS flashing records, insulation and voltage withstand reports, EOL test reports, and system configuration lists. Data sources include cell capacity grading data, MES systems, welding equipment, torque tools, BMS software versions, and EOL test benches. Update frequency is once per pack or cabinet, or triggered by BMS software version changes. Documents include structured tables, standardized test reports, and configuration lists, with fields covering cell parameters, assembly parameters, test results, and more. Some data is exported directly from connected devices, with minor format differences.
What Constraints Do These Characteristics Impose on the Document Parsing and Chunking Link
The document characteristics of this category impose multiple constraints on the parsing and chunking process. Documents that mix structured tables and unstructured process text require parsing tools to retain both table row-column associations and paragraph context. The high-frequency, small-granularity update feature requires support for fast incremental parsing and batch import adaptation. Raw data from multiple devices has inconsistent formats, requiring the parsing link to support multiple export formats. Process specifications have strong parameter relevance, and chunking must avoid breaking cross-paragraph parameter descriptions and rule associations.
How to Configure the Settings
| Configuration Item | Recommended Setting | Rationale |
|---|---|---|
chunkSize | 800–1200 characters | This category’s documents contain long paragraphs of grouping rules and test procedures. This range balances context integrity and retrieval granularity |
chunkSplitter | By paragraph + table boundary | Documents mix structured tables and unstructured process text. Full table structure must be retained to avoid splitting rows or columns |
customPdfParse | Enabled | The insulation and voltage withstand reports and EOL test reports for this category use standardized PDF formats. Enabling this allows accurate extraction of tables and fixed fields |
PARSE_FILE_TIMEOUT_SECONDS | 300–600 seconds | A single energy storage system integration document may contain multiple test reports and logs, resulting in long parsing times |
UPLOAD_FILE_MAX_SIZE | 500 MB | Bulk-imported cell capacity grading data and MES log files have large file sizes, requiring adaptation to batch upload requirements |
chunkSettingMode | Custom mode | Chunking rules must be adjusted separately for structured tables and process text to adapt to the category-specific document structure |
Three Common Misconfigurations
- Symptom: After importing Excel-format BOMs or welding records, table column correspondences are lost, and specified parameters cannot be matched during retrieval. Cause: Structured table parsing mode is not enabled, and the default chunking logic does not retain table row-column association structures.
- Symptom: Full table content can be previewed in the knowledge base, but table fields are empty or not displayed in retrieval results. Cause: Tables were not treated as independent units during chunking, or the
chunkSplitterconfiguration does not include table boundary rules. - Symptom: After uploading local documents chunked by other vector models, retrieval matching accuracy drops significantly. Cause: Different chunking logic will break document context associations. FastGPT’s unified chunking configuration must be used to ensure chunking consistency.
How to Verify Correct Configuration
- Upload a single typical document, such as a grouping rule PDF, and view the parsed chunk list to confirm that tables and long paragraphs are not incorrectly split.
- Import an Excel-format BOM file, search for exclusive keywords such as "cell capacity threshold", and confirm that the returned results include complete table row-column data.
- Upload batch log files, check the parsing progress and system prompts to confirm that the
PARSE_FILE_TIMEOUT_SECONDSconfiguration is adapted to the file size. - In the FastGPT v4.17.0 environment, switch between different vector models to test retrieval effects, and confirm that chunking rules do not cause context breaks due to model changes.
The values provided are common starting points and should be measured against the reader's own samples.