What this category of data includes
It covers daily production documents from cotton spinning, chemical fiber spinning, woven and knitted fabric enterprises. These documents include cotton blending or ingredient lists, process setup sheets, equipment operation records, yarn inspection reports, grey fabric inspection reports, and defect logs. Data sources include raw fiber inspection, ERP systems, spinning and loom equipment data, evenness testers, fabric inspection machines, and order specifications. The update frequency adjusts based on each raw material batch, each shift’s production, and each piece of fabric produced. Documents include fields such as fiber performance indicators, twist, weaving structure parameters, defect types and quantities. Units used include professional production units like grams, thread counts, meters, and pieces.
What constraints these characteristics impose on document parsing and chunking
Documents in this category have diverse formats. They include both structured tabular ingredient lists and setup sheets, plus unstructured text-based defect logs. Parsing and chunking must adapt to content in different formats. Frequently updated small-batch documents require the parsing process to handle single or small numbers of documents quickly. This avoids delays in production workflows. Data from multiple sources must retain corresponding metadata, such as equipment numbers, production batches, and inspection times. This ensures complete business information can be linked during retrieval. Professional fields and units require parsing rules to accurately identify and retain production-related proprietary information. This prevents field loss or unit confusion.
Configuration Settings
This setup is compatible with FastGPT v4.17.0. For document parsing and chunking in spinning and weaving scenarios, refer to the following configurations:
| Configuration Item | Recommended Setting | Rationale |
|---|---|---|
chunkSize | 800–1200 characters | Documents in this category include tables with multi-line fields and long-text records. This range balances field completeness and retrieval accuracy |
chunkSplitter | Split by paragraph + table | Documents in this category include both structured tables and unstructured text. This setting supports splitting logic for both formats |
UPLOAD_FILE_MAX_SIZE | 500 MB | A single batch of raw material process documents may include multiple setup sheets and inspection records. This setting supports large file uploads |
PARSE_FILE_TIMEOUT_SECONDS | 300–600 seconds | Large process documents require processing multi-page tables and batch data. This timeout range covers the complete parsing workflow |
customPdfParse | Enabled | PDF documents in this category often include fixed-format process parameter columns. Custom parsing accurately extracts core fields |
chunkSettingMode | Manual configuration | Chunking parameters must be adjusted for different document types, such as ingredient lists and inspection records. This adapts to exclusive document structures |
Three Common Misconfigurations
- Issue: An unsupported file format error appears when uploading an Excel-format cotton blending list. Cause: Corresponding format parsing rules are not configured, or custom parsing adapted to table fields is not enabled.
- Issue: A parsing timeout error triggers when uploading a 3 MB process PDF. Cause: The value of
PARSE_FILE_TIMEOUT_SECONDSis too small, and does not cover the complete parsing workflow for large documents. - Issue: Retrieved chunks only contain text content, and do not include metadata such as equipment numbers or batch numbers. Cause: Metadata extraction configuration is not enabled, or original business metadata associated with the document is not retained during chunking.
How to Verify Proper Configuration
- Upload a typical cotton blending list document, and check if core production fields such as fiber content and cotton blending ratio are retained after parsing.
- Upload a single collection of process documents under 300 MB, and confirm the parsing task completes within the set timeout period.
- Initiate a retrieval request, and check if returned results include both chunked text and metadata such as batch numbers and equipment numbers for the corresponding document.
- Upload multiple documents in different formats, including PDF setup sheets and Excel inspection records, and confirm all formats can be parsed and chunked successfully.
The values provided are common starting points and should be measured against the user's own samples.