What data looks like for this category
Data comes from dyeing and finishing factories and fabric companies, covering dyeing, setting, laboratory, and quality departments. Daily documents include dyeing formulas, chemical dosing records, dye vat curves, sample confirmation reports, color difference reports, setting process sheets, and wastewater ledgers. Data sources include customer color swatches, formula libraries, dye chemical batches, dye vat control equipment, colorimeters, setting machines, and environmental monitoring devices. Update frequency is per vat, per batch, or per shift. Most documents are structured tables, time-series curve text, and fielded records. They include professional parameters such as ΔE color difference, dye vat temperature, and chemical dosage, with units including ℃, g/L, mg/L, and others.
Constraints for Document Parsing and Chunking
High-frequency updated per-vat or per-batch data requires chunking to retain context links for core parameters, and avoid splitting critical parameter combinations. Structured tables and time-series curve documents require parsing tools to adapt to non-pure text content, and fully extract table fields and time-series data. Binding between professional parameters, units, and batches requires that chunking does not break these associations, as this will affect subsequent audit verification. Batch multi-document processing needs require that parsing configurations balance single-document integrity and batch processing efficiency.
How to Set Configurations
This configuration is based on FastGPT v4.17.0, and is adapted to document parsing and chunking requirements for dyeing and finishing scenarios:
| Configuration Item | Recommended Value | Rationale |
|---|---|---|
chunkSize | 800–1200 characters | Ensures context integrity for core parameter combinations in single documents such as dyeing formulas and color difference reports, and avoids splitting associated fields like dye dosage and process temperature |
chunkSplitter | Split by paragraph + table block | Adapts to time-series dye vat curve data and structured tables on setting process sheets, and retains complete parameter links for each table row |
customPdfParse | Enable structured parsing mode | Extracts original fields and professional parameters from embedded tables and image-embedded text in dye vat curve PDFs and setting process sheets, and retains links between units and batches |
PARSE_FILE_TIMEOUT_SECONDS | 600 seconds | Adapts to parsing duration for dye vat curve PDFs with large amounts of time-series data, and prevents interruptions during large file parsing |
UPLOAD_FILE_MAX_SIZE | 500 MB | Adapts to batch uploads of dye vat curve PDF packages and multiple ledger documents, and prevents failures due to exceeding upload limits |
chunkSettingMode | Manual configuration | Flexibly adjusts parameters to fit different scenarios such as formula sheets and wastewater ledgers, addressing differentiated chunking needs for different document types |
Three Common Configuration Mistakes
- Phenomenon: After uploading a Word-format setting process sheet, parsed chunks lose embedded process parameter tables and image-embedded color difference values. Cause: Structured parsing mode for
customPdfParseis not enabled. The default setting only extracts pure text paragraphs, and cannot recognize embedded tables and image text in Word files. - Phenomenon: Parsing timeout triggers when batch uploading multiple dye vat curve PDFs. Cause:
PARSE_FILE_TIMEOUT_SECONDSis not adjusted to a reasonable duration, and using the default short duration causes large file parsing to interrupt before completion. - Phenomenon: Chunked content cannot link dyeing formulas and corresponding chemical dosing records, and the split content loses associated parameters. Cause:
chunkSizeis set too small, splitting associated content between formulas and chemical batch numbers, and making parameter matching impossible during audits.
How to Verify Correct Configuration
- Upload a typical dyeing formula sheet PDF, check if parsed chunks retain complete combinations of dye names, dosages, and process temperatures.
- Upload a dye vat curve PDF containing time-series data, check if the parsed result includes complete temperature and time sequence data, with no missing fields.
- Batch upload 3 to 5 documents of the same type, verify that parsing duration does not exceed the preset
PARSE_FILE_TIMEOUT_SECONDSvalue, with no timeout errors. - Review chunked content, confirm that dye chemical batch numbers and corresponding vat numbers appear in the same chunk of text, with no broken links.
The values provided are common starting points and should be measured against the reader's own samples.