What this category of data looks like
This data comes from engineering, production, quality control, and laboratory teams at printed circuit board factories. It covers engineering production instructions, Gerber parsing records, drilling and lamination parameters, chemical solution analysis reports, AOI inspection reports, electrical test reports, and cross-section analysis reports. Data sources include Gerber, drill tape, and BOM files provided by customers, CAM output files, equipment operating parameters, bath solution test results, and data collected by AOI and electrical test equipment. Data is updated per production batch, per shift, or via continuous collection. Most documents contain specialized process parameters and electrical test results.
What constraints do these characteristics impose on document parsing and chunking?
The data characteristics of this category impose multiple constraints on the document parsing and chunking workflow. Multi-source and heterogeneous data formats, including binary Gerber files, tabular process parameter documents, and scanned inspection reports, require the parsing process to support multiple file types. The high-frequency, batch-based update rhythm requires chunking logic to match the contextual association of production batches, to avoid mixing data across batches. There are many specialized process parameter and electrical test fields, so chunking must preserve the integrity of fields and units, to prevent splitting that breaks parameter logic. Structured data directly output by equipment and unstructured documents filled out manually coexist, so chunking rules must distinguish between these two types of content.
How to set the configurations
The following configurations are based on FastGPT v4.17.0.
| Configuration Item | Recommended Setting | Rationale |
|---|---|---|
chunkSize | 800–1200 characters | PCB documents contain long parameter strings and test results. This range preserves the integrity of process logic and avoids split breaks |
chunkSplitter | By paragraph + specialized terminology boundary | PCB documents mostly include coherent process steps and parameter groups. Splitting by terminology boundaries preserves contextual association |
customPdfParse | Enable scanned PDF OCR + original image retention | This category includes scanned AOI and cross-section reports. It is necessary to extract text and retain original images for subsequent audits |
PARSE_FILE_TIMEOUT_SECONDS | 300–600 seconds | Parsing large Gerber files and batch inspection reports takes a long time. This range covers parsing needs for most scenarios |
UPLOAD_FILE_MAX_SIZE | 2000 MB | The total size of Gerber files and associated documents for a single production batch can reach this scale, so large file uploads must be supported |
chunkSettingMode | Manual configuration | PCB documents have large format differences. Manual configuration allows flexible adjustment of chunking rules to adapt to different document types |
Three common mistakes
- Symptom: After uploading an Excel-format engineering BOM table, vector indexing has field misalignment or loss of numerical units, and corresponding process parameters cannot be matched during retrieval. Cause: The parsing rules for structured tables are not adapted, and the default chunking logic breaks the row and column associations of the table.
- Symptom: After uploading a scanned cross-section analysis report, the extracted text has garbled characters or fails to recognize specialized parameters such as hole copper thickness. Cause: The scanned PDF parsing mode of
customPdfParseis not enabled, resulting in insufficient OCR adaptation. - Symptom: When uploading a Gerber file and associated documents whose total size exceeds the limit for a single batch, upload fails or a parsing task timeout error is triggered. Cause: The configuration values of
UPLOAD_FILE_MAX_SIZEorPARSE_FILE_TIMEOUT_SECONDSdo not match the actual size and parsing time of PCB documents.
How to confirm configurations are correctly set
- Upload a single typical PCB document, such as a Gerber parsing record or AOI inspection report, and check the parsed chunking results to confirm that specialized parameters and units are not split broken.
- Upload a scanned inspection report, check whether the parsed result includes both text content and original image retention, and confirm that the
customPdfParseconfiguration is effective. - Upload a file package with a total size that meets production requirements for a single batch, and verify that the upload and parsing processes do not trigger timeout or size limit errors.
- For Excel-format BOM tables, check whether the chunking results retain the row and column structure of the table, and confirm that the parsing rules are adapted to structured tables.
The values provided here are common starting points, and should be evaluated against the user's own samples.