What Data Looks Like for This Category
Daily documents for cotton spinning, chemical fiber spinning, woven, and knitting enterprises include cotton blending lists, process setup sheets, equipment operation logs, yarn inspection reports, grey fabric inspection reports, defect records, and more. Data sources include raw fiber inspection systems, ERP systems, spinning and loom data collection devices, evenness testers, fabric inspectors, and order specification documents. Update frequency occurs per batch of raw materials, per production shift, or per piece of fabric produced. Documents contain structured numerical fields such as yarn count, grey fabric width, twist, and equipment runtime. Some documents include actual defect photos. Supported formats cover PDF and Excel. Some fields have dedicated units.
Constraints on Document Parsing and Chunking from These Characteristics
High-frequency updates of category documents require chunking logic to adapt to small-batch, multi-batch upload scenarios. This prevents repeated parsing or resource waste caused by rigid chunking rules. Mixed uploads of multiple document types require parsing logic to support both structured tables and image-containing PDF files, while preserving the integrity of professional fields. Specialized textile terminology and units require the parsing module to identify professional terms, avoiding damage to parameter binding relationships during field splitting. An example is the association between twist and the corresponding yarn model. Significant structural differences exist across documents: cotton blending lists are multi-column numerical tables, process setup sheets are paragraph-style parameter lists, and defect records are mixed text-image documents. Chunking rules must adapt to different document structures, preventing incorrect splitting of cross-type content.
Configuration Settings
This configuration is adapted for FastGPT v4.17.0.
| Configuration Item | Recommended Value | Rationale |
|---|---|---|
chunkSize | 800–1200 characters | Adapts to the parameter segment length of process setup sheets and cotton blending lists, avoiding splitting associated parameters such as twist and fabric width |
chunkSplitter | Split by paragraph + table boundary | Matches the mixed structure of category documents, preserving the integrity of tables from Excel cotton blending lists and process sheets |
customPdfParse | Enable OCR mode | Adapts to PDF documents with defect photos, enabling simultaneous recall of text and image text |
PARSE_FILE_TIMEOUT_SECONDS | 120 seconds | Prevents parsing interruptions from overly long single task durations when processing batch equipment operation log PDFs |
UPLOAD_FILE_MAX_SIZE | 1000 MB | Adapts to batch document upload requirements for entire raw material batches, supporting combined uploads of multiple small files |
chunkSettingMode | Manual configuration mode | Adjusts chunking parameters for different document types, adapting to differentiated requirements of mixed text-image and pure table documents |
Three Common Configuration Mistakes
- Phenomenon: After uploading a PDF with defect photos, only text content is recalled, and the defect description text in the image is not extracted. Cause: The OCR mode of
customPdfParseis not enabled, and the image text parsing switch is not configured. - Phenomenon: After uploading an Excel-format cotton blending list, table columns are split into different chunks, making it impossible to match complete fiber usage parameters during retrieval. Cause:
chunkSplitteris configured to split only by character count, and table boundary recognition rules are not enabled. - Phenomenon: Parsing tasks time out and fail when batch uploading operation log PDFs from multiple looms. Cause:
PARSE_FILE_TIMEOUT_SECONDSis set too low, failing to adapt to the processing duration of batch documents in a single batch.
How to Verify Correct Configuration
- Upload a single PDF document with defect photos, and check whether the retrieval results include the defect description text in the image.
- Upload an Excel-format cotton blending list, retrieve complete fiber types and corresponding usage parameters, and confirm that the parameters are not split into different chunks.
- Upload batch equipment operation log PDFs, and check whether the parsing task completes within the set
PARSE_FILE_TIMEOUT_SECONDS. - Adjust the
chunkSizeparameter, compare chunking results of process sheets of different lengths, and confirm that associated process parameters are not split.
The values provided here are common starting points, and should be evaluated against the applicable sample sets.