What the data for this category looks like
Data for this category comes from the electrode sheet, assembly, formation, and quality departments of power, energy storage, and consumer battery companies. Daily documents include batching records, coating and rolling parameters, winding/stacking records, liquid injection records, formation and grading data, OCV and safety test reports. Data sources include raw material batch ledgers, MES systems, slurry and coating testing equipment, dew point monitoring devices, formation and grading cabinets, and laboratory data. Updates occur per batch, per individual cell, or in real time. Document fields include batch numbers, process parameter values, environmental monitoring values, and test results. Most documents use PDF or spreadsheet formats.
What constraints do these characteristics impose on document parsing and chunking
Multi-source heterogeneous document structures, real-time update characteristics, and strong batch correlation create multiple constraints for document parsing and chunking. Process and test documents in different formats require tailored parsing rules to fully extract spreadsheet parameters and long-text logs. Strong batch correlation requires retaining batch IDs as context anchors during chunking to avoid mixing data across batches. The length of real-time generated formation and grading data logs fluctuates widely. Precise chunk boundary control prevents single chunks from spanning multiple process steps. Fields involving sensitive parameters such as moisture content and cleanliness must be fully preserved to avoid parsing omissions that disrupt subsequent quality analysis.
How to configure the settings
| Configuration Item | Recommended Setting | Rationale |
|---|---|---|
chunkSize | 800–1200 characters | Documents for single batches in this category mostly contain process parameters and test data. Chunks that are too long will span multiple batches, while chunks that are too short will break parameter correlations. |
chunkSplitter | By paragraph + by table row | Documents in this category include process parameter spreadsheets and long-text logs. Mixed chunking preserves both parameter integrity and log continuity. |
customPdfParse | Enabled | PDF documents in this category often include encrypted test reports and structured spreadsheets. Custom parsing enables field extraction adapted to specific formats. |
PARSE_FILE_TIMEOUT_SECONDS | 300–600 seconds | Single-batch formation and grading data logs may be lengthy. Sufficient parsing time must be reserved to avoid timeout interruptions. |
UPLOAD_FILE_MAX_SIZE | 1000 MB | Bulk uploads of multiple batch documents require support for larger file capacities to match daily enterprise bulk archiving needs. |
chunkSettingMode | Manual mode | Document structures vary widely across this category. Manual configuration allows precise adjustment of chunking rules to adapt to different types of process and test documents. |
Three common configuration mistakes
- Symptom: After upgrading to FastGPT v4.17.0, the file parsing function fails to operate correctly. Switching to an older version or a third-party parsing tool restores normal operation. Cause: The
customPdfParseparameter was not enabled correctly, or the configured parsing rules do not match the structured spreadsheet format of this category. - Symptom: When bulk uploading multiple batch documents, parsing tasks time out and fail directly, with no valid parsing results. Cause: The
PARSE_FILE_TIMEOUT_SECONDSparameter was not adjusted. The default timeout duration is insufficient to process long-text formation and grading data logs. - Symptom: After parsing uploaded structured spreadsheet documents, some core process parameter fields are empty. Cause: The
customPdfParseparameter was not enabled. The built-in default parsing rules cannot recognize the specialized spreadsheet field format of this category.
How to confirm correct configuration
- Upload a single batching record PDF, and verify that core fields such as batch numbers and slurry parameters are fully extracted in the parsing results with no content loss.
- Upload a single batch of formation and grading data logs, and verify that chunking results are split by process nodes, with no cross-batch content concatenation.
- Adjust
PARSE_FILE_TIMEOUT_SECONDSto 600 seconds, upload a 100 MB archived document, and confirm that the parsing task completes normally with no timeout errors. - Check that
chunkSettingModeis set to manual mode in the configuration panel, and confirm that chunking rules match the document structure of this category.
The values provided are common starting points and should be measured against applicable samples.