What data for this category looks like
The primary entities in this category are rubber product enterprises including tire, seal, and hose manufacturers. Daily documents include formula sheets, weighing records, mixing curves, vulcanization process cards, and finished product performance reports. Data sources cover formula systems, internal mixers, Mooney viscometers, vulcanizers, and laboratory testing equipment. Update frequency follows per batch of rubber, per batch of raw materials, or per mold production cycle. Document structures include structured tables such as weighing records and vulcanization parameter tables, numerical fields with units such as rubber compound phr, mixing temperature in ℃, and vulcanization time in minutes, plus curve data. A single document can contain tens of thousands of table rows or 100,000 Chinese characters.
What constraints do these characteristics impose on document parsing and chunking
Multi-source, heterogeneous document types require parsing to adapt to text, table, and curve formats. This avoids breaking parameter associations after chunking. High update frequency creates demand for batch parsing. Single task duration must be controlled to match production cycles. Long tables and large-volume documents cause default chunking logic to fail. This leads to split parameters within chunks or chunks exceeding vector model supported lengths. Differences in field units across documents such as phr, MPa, and seconds require retaining full context during chunking. This avoids ambiguity caused by separating units from parameters.
How to set the configuration
This configuration is based on FastGPT v4.17.0. It is customized for document characteristics of rubber mixing and product vulcanization scenarios.
| Configuration Item | Recommended Setting | Rationale |
|---|---|---|
chunkSize | 800–1200 characters | Matches the combined parameter length of process cards and mixing curves for this category. Prevents splitting critical associated information such as weighing sequences and temperature curves |
chunkSplitter | Split by paragraph and table row boundaries | Adapts to the large number of structured weighing records and vulcanization parameter tables in documents for this category. Prevents default text splitting from damaging table structure |
customPdfParse | Enabled | Extracts complete embedded mixing curve parameters and process card tables from PDF documents for this category. Avoids loss of structured data from default parsing |
PARSE_FILE_TIMEOUT_SECONDS | 300–600 seconds | Matches parsing duration for single 100,000-character Word documents or 15,000-row Excel documents. Prevents parsing timeouts for large-volume documents |
UPLOAD_FILE_MAX_SIZE | 1000 MB | Accommodates batch upload needs for single-batch mixing records and multiple process cards for this category |
chunkSettingMode | Manual mode | Allows custom chunking rules for different document types such as table-based and text-based documents. Adapts to the diverse document structures for this category |
Three common errors
- Chunked data blocks are too large. This causes vector generation errors or insufficient recall accuracy. The cause is failing to adjust
chunkSizefor structured tables in this category. General default values are used instead. Chunks are not split by process parameter combinations. - Parsing timeouts occur when uploading 15,000-row Excel or 100,000-character Word documents. The cause is failing to adjust
PARSE_FILE_TIMEOUT_SECONDS. Default short timeout thresholds are used instead. The parsing duration for large-volume documents is not matched. - Parsed process card fields are empty or units are missing. The cause is failing to enable
customPdfParse. Default parsing cannot extract embedded mixing curve parameters and weighing record unit information for this category.
How to confirm the configuration is correct
- Upload a single 15,000-row weighing record Excel file. Check that parsed chunk count matches expectations. No overly long chunks are present.
- Enable
customPdfParse, then upload a mixing process PDF. Check that complete vulcanization temperature, time parameters and corresponding units are extracted. - Adjust
PARSE_FILE_TIMEOUT_SECONDSto 600 seconds. Upload a 100,000-character mixing curve document. Confirm no parsing timeout errors. - Test chunked data blocks. Check that associated parameters such as rubber compound ratios and weighing sequences are fully included. No information is split.
The values provided here are common starting points. Adjust them against your own samples to validate performance.