What this category's data looks like
Typical data for this category comes from dyeing and finishing factories, and the dyeing, setting, laboratory, and quality departments of fabric enterprises. Daily documents include dyeing formulas, chemical dosing records, dye vat curves, sample confirmation reports, color difference reports, setting process sheets, and sewage ledgers. Data sources cover customer color samples, formula libraries, dye chemical batches, dye vat control equipment, colorimeters, setting machines, and environmental monitoring systems. Update frequency is per vat, per batch, per shift. Documents contain structured fields such as dye vat number, batch number, temperature values, and color difference ΔE values, with units including ℃, g/L, %, and others.
What constraints these characteristics impose on the "document parsing and chunking" link
The high-frequency, small-batch update feature requires parsing and chunking to support fast batch processing of single-vat or single-batch documents, and avoid repeated association of cross-batch data. Document types include structured tables, time-series curves, and scanned reports. Different chunking logic must be adapted to avoid splitting critical parameter groups. Professional units and associated identifiers in structured fields require the parsing process to retain field associations, preventing loss of parameter correspondence after chunking. The high-frequency updated sewage ledgers and dye vat curves have large data volumes, so reasonable file size and parsing time limits must be adapted.
How to set the configuration
This configuration is based on FastGPT v4.17.0.
| Configuration Item | Recommended Value | Rationale |
|---|---|---|
chunkSize | 800–1200 characters | Adapts to complete semantic blocks of structured fields and time-series curves in dyeing documents, avoiding splitting critical parameter groups |
chunkSplitter | Customized by document type | Uses time-series chunking for dye vat curves, row-column chunking for formula tables, adapting to mixed document scenarios |
customPdfParse | Enable OCR mode and retain original images | Adapts to scanned sample confirmation reports and color difference reports, avoiding loss of original images and corresponding text annotations |
UPLOAD_FILE_MAX_SIZE | 500 MB | Meets batch upload requirements for complete single-vat dye vat curves and multi-batch ledgers |
PARSE_FILE_TIMEOUT_SECONDS | 300–600 seconds | Satisfies complete parsing time for large dye vat curve files, avoiding timeout interruptions |
chunkSettingMode | Manual configuration | Adjusts chunking rules for different document types, adapting to mixed upload scenarios of structured and unstructured documents |
Three common mistakes
- After uploading Excel-format dyeing formulas or chemical dosing records, index results have empty fields or association errors. This occurs because structured table parsing rules are not configured, and critical associated fields such as dye vat number and batch number are not retained.
- After uploading scanned PDF-format sample confirmation reports or color difference reports, OCR recognition results lack professional terms or cannot match the original images. This occurs because the professional print OCR mode of
customPdfParseis not enabled. - After uploading large dye vat curve files, parsing times out or files are truncated. This occurs because
PARSE_FILE_TIMEOUT_SECONDSis set too short, orUPLOAD_FILE_MAX_SIZEdoes not match the actual file size.
How to confirm the configuration is correct
- Upload a single dyeing formula Excel file, check if the parsed fields include key business information such as components, proportions, and batch numbers.
- Upload a scanned color difference report PDF file, check if the OCR results retain core content such as ΔE values and detection dates.
- Upload a 100 MB dye vat curve file, confirm that the parsing progress does not show timeout errors, and the chunking results completely retain the single-vat process parameters.
- Adjust the
chunkSizeparameter, view the chunking preview results, and confirm that no single-vat complete parameters are split.
The values provided are common starting points and should be measured against the reader's own samples.