What data for this category looks like
Data sources for this category include raw material batch records, production MES systems, slurry and coating testing equipment, dew point monitoring devices, formation and grading cabinets, and laboratory test data. Data updates in real time for each battery batch and individual cell. Daily documents include batching records, coating and calendering parameters, winding or lamination records, liquid injection records, formation and grading data, OCV and safety test reports. Document types mix structured tables, time-series curves, and pure process text. Fields include material batch numbers, process parameter values, test results, timestamps, and unit identifiers.
What constraints do these characteristics impose on the document parsing and chunking link
The high-frequency real-time updated data and multi-dimensional batch-associated documents of this category require document parsing and chunking to retain batch association. It is necessary to avoid separating process parameters and test results from the same batch. Mixed document types with structured tables, time-series curves, and unstructured text need to adapt to both table field extraction and unstructured text chunking logic. Single document volume varies widely, from a few lines of batching records to dozens of pages of formation and grading reports. Chunking must adapt to inputs of different lengths. Data precision requirements are high. Chunking must retain complete parameter units and timestamp information to avoid losing key process details.
All following configurations are based on FastGPT v4.17.0.
How to set the configurations
| Configuration Item | Recommended Setting | Rationale |
|---|---|---|
chunkSize | 800–1200 characters | This category of documents contains mixed table data and process parameter text. This range balances the completeness of information per chunk and retrieval efficiency |
chunkSplitter | Split by paragraph + table | Daily documents contain structured tables and process description paragraphs. Splitting this way preserves the integrity of fields within tables |
customPdfParse | Enabled | PDF documents for this category often contain embedded tables and time-series curves. Enabling custom parsing improves the accuracy of table field extraction |
PARSE_FILE_TIMEOUT_SECONDS | 300–600 seconds | Single formation and grading reports may contain large amounts of time-series data. Sufficient parsing time must be reserved |
UPLOAD_FILE_MAX_SIZE | 1000 MB | Batch-uploaded collections of batch documents may reach large sizes. This upper limit covers the document needs of most production batches |
chunkSettingMode | Manual configuration | Document structures vary widely across this category. Manual configuration allows flexible adjustment of chunking rules to adapt to different document types |
Three common mistakes
- Phenomenon: After uploading a formation and grading report, the interface displays a parsing timeout error, or the request fails after exceeding the preset threshold. Cause: The
PARSE_FILE_TIMEOUT_SECONDSparameter is not adjusted for the long time-series documents of this category. Using a generic short timeout configuration cannot complete parsing of large-volume documents. - Phenomenon: After uploading a table document containing coating and calendering parameters, only text content is retained in the knowledge base, and table field data is lost. Cause: The
customPdfParseconfiguration is not enabled. The default parsing engine cannot accurately extract structured table fields dedicated to lithium battery manufacturing, resulting in data being filtered out. - Phenomenon: When searching for process data from the same batch, the corresponding liquid injection parameters cannot be associated. The chunking results split the complete batch process group into different chunks. Cause: The
chunkSizesetting is too small, breaking the association of data from the same batch.
How to confirm the configuration is correct
- Upload a typical batching record PDF, and check if the parsed text retains all table fields and parameter units.
- Upload a long time-series formation and grading report, and verify that the parsing time falls within the preset range of
PARSE_FILE_TIMEOUT_SECONDS, with no timeout errors occurring. - Search for process data from the same batch, and confirm that associated parameter chunks are not split, allowing complete batch information to be obtained in a single search.
- Upload a batch collection of batch documents, and confirm that the uploaded file size does not trigger the
UPLOAD_FILE_MAX_SIZElimit.
The values provided are common starting points and should be measured against the reader's own samples.