What the data for this category looks like
This category's data comes from daily outputs and production equipment data from dyeing and finishing factories, fabric enterprises' dyeing, setting, laboratory, and quality departments. Daily documents include dyeing formulas, chemical weighing records, dye vat curves, sample confirmation sheets, color difference reports, setting process sheets, sewage ledgers, and more. Data sources include customer color samples, formula libraries, dye chemical batch records, dye vat control systems, colorimeters, setting machines, and environmental monitoring equipment. Data is updated per vat, per batch, per shift. Individual documents contain fields such as specific process parameters, batch information, and test results. Some equipment-exported data includes standardized units.
What constraints these characteristics impose on the vector model and indexing workflow
The high-frequency, multi-dimensional production data characteristics of this category impose multiple constraints on the vector model and indexing workflow. First, real-time updated data per vat and per batch requires indexes to support incremental synchronization and low-latency refresh. This avoids delayed data impacting cross-review efficiency. Second, documents contain mixed-format content including structured process parameters, non-standardized test descriptions, and time-series curves exported by equipment. This requires vector models to adapt to vectorization conversion of multiple data types, while indexes must support precise matching of structured fields and semantic retrieval. In addition, individual documents are associated with specific production batches. Indexes must establish batch-dimensional association mappings to quickly locate relevant documents by production unit.
How to set configurations
The following configurations are adapted for FastGPT v4.17.0.
| Configuration Item | Recommended Setting | Rationale for This Setting |
|---|---|---|
| Chunk Length | 800–1200 characters | Documents in this category mostly contain continuous process parameters and complete test records. This range preserves the integrity of a single set of process logic |
maxToken | 16384 token | Meets the vectorization requirements for long documents such as dyeing formulas and dye vat curves, and avoids truncation of critical parameters |
defaultToken | 512 token | Used for efficient vectorization of short texts such as color difference reports and setting process sheet summaries |
| Similarity Threshold | 0.72–0.85 | Balances recall accuracy and recall volume, and avoids incorrectly recalling irrelevant documents from different production batches |
| Single Search Limit | Top 10 entries | Covers relevant documents from the same batch and same process, and meets the document coverage requirements for cross-review |
| Reranking | Enabled | Optimizes semantic association ranking for time-series data exported by equipment and text descriptions, and improves review accuracy |
Three common mistakes
- Issue: Index recall results mix dye vat curve data from different batches, and the number of returned entries exceeds the set limit. Cause: No index association established by production batch, and the single search limit configuration does not match the business scenario.
- Issue: A token limit error is prompted when calling the vector model, and long document vectorization cannot be completed. Cause: The
maxTokensetting does not match the actual length of the document, and truncation of long documents leads to parameter exceptions. - Issue: Multimodal vector model calls fail, and color sample image data exported by equipment cannot be processed. Cause: Call parameters for the multimodal model are not configured correctly, or the image format exported by equipment is not compatible.
How to confirm that configurations are set correctly
- Upload a complete dyeing formula document, check whether the token count after vectorization falls within the configured range, and there are no truncation error prompts.
- Submit a cross-review request for the same batch, verify that all recalled documents belong to the same production batch, and the quantity matches the set single search limit.
- Adjust the similarity threshold, confirm that changes in recall result relevance align with business expectations, with no obvious incorrect or missed recalls.
- Test the multimodal vector model's vectorization call for color sample images, confirm there are no format errors or call failure prompts.
All provided configuration values are common starting points, and should be evaluated against the reader's own samples.