What this category of data looks like
Data for this category comes from rubber product enterprises including tire, seal, and hose manufacturers. Sources include formula systems, raw material batch ledgers, internal mixers, Mooney viscometers, rheometers, vulcanizers, and laboratory testing equipment. Daily documents include formula sheets, weighing records, mixing curves, rubber quick test reports, vulcanization process cards, mold usage records, and finished product performance test reports. Documents are generated per truckload of rubber, per batch of raw materials, and per mold run of products. Each single document contains fields such as batch number, feeding amount, process parameters, and test results. Units follow industry general standards matching the document type.
Constraints imposed on the document parsing and chunking link
High-frequency, small-sized multi-document sets require chunking configurations optimized for efficient batch processing of small files, to avoid excessive time spent chunking single files. Significant structural differences exist across document types. Some are plain-text formula sheets, while others include time-series mixing curve tables. Chunking rules must preserve contextual integrity for all document types. Closely related business data fields (such as mixing temperature and corresponding time, vulcanization parameters and production batch) require that chunking does not break binding relationships between fields. Otherwise, the inspection and audit process cannot match critical information. Business errors have severe consequences, so chunking must retain sufficient context to support subsequent compliance audit workflows.
Configuration settings
These configurations are based on FastGPT v4.17.0.
| Configuration Item | Recommended Setting | Rationale |
|---|---|---|
chunkSize | 800–1200 characters | Balances contextual integrity and chunking efficiency for documents containing associated data such as mixing curves and process parameters, and avoids splitting critical fields across chunks |
chunkSplitter | Paragraph + semantic chunking | Adapts to diverse document structures, preserves paragraph logic for formula sheets and process cards, and splits time-series data blocks from mixing curves |
PARSE_FILE_TIMEOUT_SECONDS | 300–600 seconds | Prevents parsing timeout errors for documents with large volumes of time-series data such as mixing curves, which require longer parsing time |
customPdfParse | Enabled | Adapts to structured PDF documents exported from internal mixers and vulcanizers, and accurately extracts critical fields including batch numbers and feeding amounts from tables |
UPLOAD_FILE_MAX_SIZE | 500 MB | Supports batch upload of multiple documents associated with a single batch of raw materials, meeting the requirements of frequently updated data submissions |
chunkSettingMode | Batch chunking mode | Adapts to batch processing scenarios for multiple documents, and avoids index exceptions caused by exceeding the allowable number of chunks for a single batch of documents |
Three common configuration errors
- Chunk count exceeds 3000, and an index exception prompt displays in the interface. The cause is failure to set
chunkSettingModeto batch chunking mode, leading to exceeding the allowable number of chunks for a single batch of documents. - CSV-format weighing records fail to parse after a version upgrade, with logs showing format incompatibility. The cause is failure to adapt to the compatibility rules of the new version of
customPdfParse, or parameter values outside the allowable range of the new version. - Parsed mixing curve documents lack temperature-time corresponding fields, making them unavailable for process audit. The cause is failure to enable
customPdfParsecustom parsing; the default parsing does not extract time-series associated data from tables.
How to verify correct configuration
- Upload a single vulcanization process card PDF, check the chunk preview interface, and confirm that critical parameters such as vulcanization temperature and time are not split across chunks.
- Check system parsing logs, confirm that
PARSE_FILE_TIMEOUT_SECONDSmatches document parsing time, and no timeout errors occur. - Upload multiple associated documents such as a formula sheet and mixing curve, and verify that chunk results do not lose batch association information between documents.
- Adjust the
chunkSizevalue, and verify that the chunk count meets business requirements and no limit-exceeding prompts appear.
The values provided are common starting points, and should be evaluated against the user's own samples.