This document is written based on FastGPT v4.17.0.
What the data for this category looks like
Daily documents for this category include process routes, operation cards, tool lists, CNC programs, first-piece inspection records, patrol inspection forms, and measuring tool calibration records. Data comes from product drawings and 3D models, CAPP/CAM systems, machine tool collection terminals, tool management systems, MES, and measuring and inspection tool data. Update frequency is triggered by work orders, individual products, sampling batches, or tool replacement actions. Document fields include professional parameters such as tool life (unit: cutting edges per hour), process tolerance (unit: mm), and machine tool speed (unit: rpm). The document structure covers text paragraphs, nested tables, and code snippets.
What constraints do these characteristics impose on the document parsing and chunking link
Document types for this category cover text paragraphs, nested tables, and CNC program code. The parsing link must retain structured units and unit information for professional parameters. Data updates are triggered by work orders, individual products, or tool replacements, with high frequency. Chunking must adapt to efficiency requirements for small-batch batch processing. Documents aggregated from multiple sources need to be associated with their original sources. The parsing link must retain document metadata to support subsequent traceability. The volume of different documents varies significantly, from single-page operation cards to dozens of pages of CNC program collections. Configuration must support parsing and chunking logic for different file sizes.
How to set the configuration
For the document characteristics of metal cutting machining, use the following configuration items:
| Configuration Item | Recommended Value | Rationale |
|---|---|---|
chunkSize | 800–1200 characters | The process documents for metal cutting machining include professional parameters with units. Chunks that are too long will lose the contextual association between parameters and processes. Chunks that are too short will damage the integrity of process logic. |
chunkSplitter | Split by paragraph + table unit | Documents in this category include multi-row tables such as tool lists and patrol inspection forms. Splitting by fixed characters will break the relational connection between table rows. Structured table units must be retained. |
customPdfParse | Enable Marker parsing mode | Most process documents for this category include embedded CAD thumbnails and nested tables. General PDF parsing easily loses table column alignment information and embedded content. |
PARSE_FILE_TIMEOUT_SECONDS | 600 seconds | CNC program files have large volumes. Parsing requires processing a large amount of time-series data collected by machine tools. A longer timeout period is needed to avoid parsing interruptions. |
UPLOAD_FILE_MAX_SIZE | 1000 MB | A single work order may include multiple operation cards, tool lists, and CNC programs. Support for large-file batch upload and parsing is required. |
Three common mistakes
- Phenomenon: After deploying Marker and configuring environment variables, processing PDF files returns a JSON response with error information. Cause: The parsing dependencies required for
customPdfParseare not correctly associated, or the container memory quota is insufficient to support Marker parsing of process files with nested tables and CAD thumbnails. - Phenomenon: After chunking and vectorizing the knowledge base, the returned answer does not associate the source information of the corresponding document. Cause: The metadata retention logic during chunking is not enabled, and the document ID is not bound to the text chunk.
- Phenomenon: XLSX-format tool lists or patrol inspection forms are split into scattered text chunks. Cause:
chunkSplitteris not set to split by table unit, or thechunkSizevalue is too small, causing table rows to be forcibly truncated.
How to confirm the configuration is correct
- Upload a process PDF file that includes CNC program code and nested tables. Check whether the parsed text retains table column alignment and code block integrity.
- Initiate a query that includes process parameters. Verify whether the returned result includes the metadata information of the corresponding document.
- Upload a single 500 MB XLSX file of a tool list. Confirm that the parsing task does not trigger a timeout or file size limit error.
- Adjust
chunkSizeto 1000 characters, upload an operation card, and check whether the chunks retain complete process steps and corresponding tolerance parameters.
The values provided are common starting points and should be measured against the reader's own samples.