Note: The configurations on this page apply to FastGPT v4.17.0.
What this category of data looks like
Typical data for this category comes from machine shops, precision parts factories, and machining workshops of equipment manufacturing enterprises. Daily documents include process routes, operation cards, tool lists, CNC programs, first-piece inspection records, patrol inspection forms, and measuring tool calibration records. Data sources cover product drawings and 3D models, CAPP/CAM systems, machine tool collected data, tool management systems, MES systems, and measuring and inspection tool data. Update frequency is triggered by work orders, individual workpieces, sampling batches, or tool replacement actions. Most documents contain structured fields with specific units, such as process dimensions, tool life, and inspection parameters.
Constraints on Document Parsing and Chunking
The document characteristics of this category impose multiple constraints on parsing and chunking. First, diverse document types are present, including text, tables, CNC code and other formats, requiring adaptation to different splitting logic. Second, update frequency is high, and single documents may be adjusted iteratively alongside work orders, requiring support for fast parsing. Third, many structured fields with specific units are included, so chunking must retain field associations to avoid information fragmentation. Fourth, some documents have large file sizes, requiring support for large-file parsing without damaging content integrity.
Configuration Settings
| Configuration Item | Recommended Setting | Rationale |
|---|---|---|
chunkSize | 800–1200 characters | Metal cutting machining documents mostly contain continuous process parameters and dimensional tolerances. This length preserves the association of process logic and avoids overly fragmented or overly long chunks |
chunkSplitter | Split by paragraph + table | Adapts to the table formats of process routes and patrol inspection forms, as well as the code block format of CNC programs, and retains the integrity of structured content |
customPdfParse | Enabled | For PDF format documents such as operation cards and inspection records, retains table structures and field associations to avoid chaotic content after parsing |
UPLOAD_FILE_MAX_SIZE | 1000 MB | Supports upload of large CNC programs and complete process manuals, adapting to the upload needs of workshop batch work order documents |
PARSE_FILE_TIMEOUT_SECONDS | 600 seconds | Large machining documents take a long time to parse. This duration avoids timeout interruptions and ensures complete parsing |
chunkSettingMode | Manual configuration | Adjust chunking parameters for different document types (operation cards, CNC programs) to adapt to category-specific characteristics |
Three Common Mistakes
- Files larger than 10 MB generate over 1000 chunks after upload, with abnormal vectorization of some chunks. The phenomenon is that after uploading large-volume process manuals or CNC programs, the knowledge base chunk list shows a large number of short chunks, and some chunks have empty content or cannot be vectorized normally. The cause is that the
chunkSizeparameter has not been adjusted for category documents, and overly fragmented chunking leads to context breaks during vectorization, making it impossible to associate continuous process parameters. - The exception disappears after repeatedly clicking retry. The phenomenon is that after an upload exception occurs, clicking the retry button multiple times makes the abnormal state disappear but does not fix the root cause. The cause is that the parsing process is affected by temporary node load or network fluctuations. Retrying temporarily can bypass temporary failures, but does not fix the adaptation issues of the chunking logic.
- Fields are empty after importing Feishu online spreadsheets. The phenomenon is that after importing Feishu patrol inspection forms or measuring tool calibration spreadsheets, only the header is retained in the knowledge base, with no specific inspection data. The cause is that structured parsing configuration has not been enabled, so structured fields with units in the table cannot be extracted correctly.
How to Verify Proper Configuration
- Upload a typical operation card PDF, and check whether continuous fields such as operation number, dimensional tolerance, and inspection standards are retained after chunking.
- Upload a single CNC program file larger than 100 MB, and confirm that no timeout error appears in the parsing progress.
- View the knowledge base chunk list, and confirm that no table breaks or code block truncation appear in the content of each chunk.
- Trigger a batch upload test, and check whether the chunk quantity and content integrity of multiple work order documents meet expectations.
The values provided are common starting points and should be evaluated against relevant samples.