What this category of data looks like
The data for this category comes from raw material batch ledgers, production MES systems, slurry and coating testing equipment, dew point monitoring devices, formation and grading cabinets, and laboratory test reports. Daily documents include batching records, coating and calendering parameter sheets, winding or lamination process records, liquid injection ledgers, formation and grading data reports, OCV and safety test reports. Data update frequency is adjusted in real time based on production batches and individual cell progress. Each individual document contains fields such as batch number, process parameters, test values, and equipment number. Some reports include multiple sets of continuous test time-series data.
What constraints these characteristics impose on the "document parsing and chunking" link
This category of data features high real-time performance and includes structured fields and time-series test data, which creates multiple constraints for document parsing and chunking. First, real-time updated batch data must retain contextual connections between processes, to avoid breaking parameter logic due to splitting. Second, structured documents with multiple associated fields must retain the binding relationship between fields and their corresponding values, and should not split complete information across fields. Third, electrochemical production is sensitive to constraint fields such as environmental parameters and equipment numbers. Parsing must fully retain this associated information, and no key details can be omitted. Fourth, document structures vary significantly across multiple processes. Parsing rules for different document types must be accurately distinguished, to avoid mixing parameter logic from different processes.
How to set the configurations
The following configurations are based on FastGPT v4.17.0.
| Configuration Item | Recommended Setting | Rationale |
|---|---|---|
chunkSize | 800–1200 characters | This category of documents contains multi-field structured data and short time-series segments. This range can retain complete associations of single-process parameters and avoid splitting critical information |
chunkSplitter | By paragraph + semantic boundary | This category of documents includes process records and test data. Splitting by semantic boundaries can retain the binding relationship between parameters and their corresponding test results |
customPdfParse | Enable structured parsing | Most PDF documents in this category are standardized reports. Enabling this feature can extract the corresponding relationship between fields and values, improving chunking accuracy |
PARSE_FILE_TIMEOUT_SECONDS | 600 seconds | Single formation and grading reports in this category have large data volumes. Sufficient parsing time must be reserved to avoid timeouts |
chunkSettingMode | Custom mode | Document structures for this category vary significantly across processes. Custom mode can adapt to chunking rules for different process documents |
UPLOAD_FILE_MAX_SIZE | 1000 MB | Batch production documents in this category may include multi-batch data. Large file upload and parsing support is required |
Three common mistakes
- Issue: Chunking results split parameters and test values from the same process segment, making it impossible to associate complete information during retrieval. Cause:
chunkSplitterwas not correctly set to semantic boundary mode. The default fixed-character splitting causes field association breaks. - Issue: The knowledge base text stream returned by the API is in plain text format, with no paragraph or semantic separators, leading to monotonous front-end display. Cause: The format retention function of
customPdfParsewas not enabled, so the parsed text does not retain the original document's paragraph structure. - Issue: After uploading multiple files in parallel when creating a knowledge base, some document parsing results are empty or cannot achieve long-document contextual association. Cause:
UPLOAD_FILE_MAX_SIZEwas not adjusted to adapt to large files, or the parent-child chunking mode underchunkSettingModewas not enabled, leading to failed batch uploads or long-document chunking.
How to confirm the configurations are correct
- Upload a single coating and calendering parameter PDF, and check whether the chunking results retain the complete association of calendering thickness, areal density, and corresponding equipment number.
- Upload multiple documents from different processes, verify the parallel execution status of parsing tasks, and confirm no batch timeouts or failures occur.
- View the knowledge base parsing logs, confirm that structured fields have been correctly extracted, with no missing or misaligned fields.
- Test retrieval of formation and grading documents containing time-series data, confirm that the chunking results retain continuous test curve parameters.
The values provided here are common starting points, and should be evaluated against the reader's own samples.