What the data for this category looks like
Documents for this category come from raw material batch ledgers, production MES systems, slurry and coating testing equipment, dew point monitoring terminals, formation and grading cabinets, and laboratory testing systems. Daily documents include batching records, coating and rolling parameter sheets, winding or lamination process records, liquid injection ledgers, formation and grading data files, and OCV and safety test reports. Data updates occur per battery batch, per individual cell, or in real time. Documents contain numerous parameter fields with specific units, and some documents use batch record formats with multiple tables.
What constraints these characteristics impose on document parsing and chunking
Real-time or high-frequency updated documents require parsing processes with low latency, to avoid data lag that reduces cross-review efficiency. Multiple tables and parameter fields with units require chunking logic that does not break table structures or the parameter-unit binding relationship. If this binding is broken, it will be impossible to match corresponding parameters and test results during cross-review. Large batch record files require support for large-file parsing. Chunking must also avoid truncating complete test data across pages, to prevent loss of critical parameters. Electrochemical systems have strict data integrity requirements. Missing a single formation curve or safety test data will directly reduce review accuracy.
How to configure settings
This configuration is based on FastGPT v4.17.0.
| Configuration Item | Recommended Setting | Rationale |
|---|---|---|
chunkSize | 800–1200 characters | Matches the length of parameter lines and table paragraphs in lithium-ion cell manufacturing documents, to avoid truncating parameter data bound to specific units |
chunkSplitter | Split by paragraph + table boundaries | Preserves complete table structures from batching records and formation and grading reports, preventing separation of parameters and their corresponding units |
customPdfParse | Enable OCR mode | Adapts to old scanned batch ledgers and quality inspection reports with handwritten annotations, extracting printed or handwritten parameter content |
UPLOAD_FILE_MAX_SIZE | 1000 MB | Supports batch upload and parsing of single-batch summary test reports and continuous formation and grading data |
PARSE_FILE_TIMEOUT_SECONDS | 600 seconds | Adapts to batch parsing processes for large-volume historical data, preventing task interruptions due to excessive time consumption |
chunkSettingMode | Manual configuration | Flexibly adjust paragraph depth and chunking rules to match business needs for different document types such as parameter ledgers and test reports |
Three common configuration errors
- Unable to select table datasets when building a knowledge base. The cause is that the table recognition switch for
customPdfParseis not enabled. After parsing, table content is merged into plain text paragraphs and cannot be recognized as selectable table datasets. - Parsed file content is incomplete, with missing table rows for coating rolling parameters in returned results. The cause is that
chunkSizeis set too small, truncating complete table blocks across pages and preventing full extraction of some parameter rows. - Parsing tasks return a timeout status code. The cause is that
PARSE_FILE_TIMEOUT_SECONDSis set lower than actual parsing time, making full parsing impossible when processing multi-batch formation and grading data.
How to confirm the configuration is properly set
- Upload a document containing a coating rolling parameter table, and check whether the parsed result retains the complete table structure and the parameter and unit content of each cell.
- Check the size of the uploaded file, and confirm that it does not exceed the
UPLOAD_FILE_MAX_SIZElimit to avoid being blocked due to excessive file size. - Run a parsing test for a single large-volume formation and grading data file, and confirm that the task duration does not exceed the
PARSE_FILE_TIMEOUT_SECONDSsetting, with no timeout errors. - Adjust
chunkSizeto the 800–1200 character range, and verify that parsed chunks do not truncate the binding relationship between a single set of parameters and their corresponding units.
The values provided are common starting points and should be measured against the reader's own samples.