What data for this category includes
Data for this category typically comes from order design systems, sheet and wood batch records, CNC cutting and drilling equipment, ERP/MES systems, and moisture content testing equipment. Daily documents include order breakdown drawings, cutting lists, edge banding and drilling programs, coating process documents, hardware lists, trial assembly and packaging records.
Data updates trigger with orders, daily cycles, or per material batch. Most documents include fields like panel dimensions, process parameters, and hardware model numbers. No unified fixed format exists for these documents.
What constraints do these characteristics impose on vector models and indexing
Data for this category comes from scattered sources and updates frequently. Indexes must support incremental updates to avoid excessive time spent on full reconstruction.
Documents include both structured lists and unstructured process files. When splitting text, retain business-related information to avoid breaking critical content such as panel dimensions and hole positions.
Custom orders drive large numbers of panels per single batch. Limit the number of results returned per single recall to fit the context window.
Document formats vary widely across different sources. The system must support vectorization processing for multiple file types.
Configuration Settings
| Configuration Item | Recommended Setting | Rationale |
|---|---|---|
| Chunk Length | 800–1200 characters | Fits the single-segment business logic length of cutting lists and process documents for this category. Avoids splitting that breaks panel dimensions and hardware-related information |
| Chunk Overlap Length | 100–150 characters | Retains context between adjacent chunks. Prevents hole position and process step information in order breakdown drawings from being split apart |
| Similarity Threshold | 0.75–0.85 | Equipment inspection and maintenance Q&A requires precise matching of equipment models and process parameters. A threshold that is too low may introduce irrelevant results. A threshold that is too high may lead to insufficient recall results |
| Single Search Limit | Top 10 results | Large numbers of panels per single order. Limits initial recall volume to control total context length |
maxToken | Set to the selected embedding model/service maximum supported input tokens | maxToken is the input-token maximum supported by the selected embedding model/service; verify it in that provider configuration. See the v4.17.2 model-field reference. It is not retrieval/LLM context or a FastGPT-wide fixed cap. |
defaultToken | Set based on actual measurement | Lengths of sheet and process documents vary across different batches. Adjust the base token allocation based on actually uploaded documents |
Three Common Mistakes
- Phenomenon: When uploading a document, the interface shows 8 chunks. Later viewing shows 13 chunks with duplicate segments. Cause: No fixed chunk length is set. Line breaks and spaces in the document trigger automatic re-splitting. Old version segments are not cleaned up in the incremental index.
- Phenomenon: After upgrading from an older version to FastGPT v4.17.0, previously queryable equipment maintenance content can no longer be retrieved. Cause: The index format changed after the upgrade. The vector index was not regenerated.
- Phenomenon: After uploading an Excel-format cutting list, the vectorized index results have incorrect matching. Cause: Merged cells and hidden columns in the Excel file were not cleaned up. Invalid blank fields were extracted during chunking.
How to Verify Correct Configuration
- Upload a single cutting list document. View the chunk preview. Confirm each segment contains complete panel dimensions and process parameter information. No broken splits are present.
- Run two identical equipment inspection queries. Compare the number of returned segments. Confirm there are no duplicate index segments.
- After upgrading the FastGPT version, perform a reindex operation on existing equipment maintenance documents. Verify that content can be retrieved normally.
- Upload an Excel-format hardware list. Check that the chunk preview extracts valid fields. No blank or incorrect content is included.
The values provided are common starting points and should be measured against the reader's own samples.