This document is compatible with FastGPT v4.17.0.
What the data for this category looks like
The data for this category comes from BOMs, Gerber files, coordinate files provided by customers, as well as internal factory MES systems, solder paste and component batch ledgers, placement machines, SPI/AOI testing equipment, reflow oven temperature records, and more. Daily documents include multi-column structured tables, mounting files with coordinate parameters, line change work orders, stencil and furnace temperature logs, first article confirmation reports, defect detection records, and similar materials. Fields include part numbers, reference designators, component specifications, test values, timestamps, and more. Update frequency changes in real time with work orders and line change operations.
What constraints do these characteristics impose on the document parsing and chunking process
Multi-column structured BOMs, coordinate files, and other documents in this category contain a large number of coding and numerical fields. During parsing, accurately match columns with their business meanings to avoid breaking field associations during chunking. High-frequency documents updated in real time for work orders and line changes require parsing and chunking to adapt to rapid upload and retrieval efficiency. Significant structural differences exist across different document types (tables, PDF test reports, plain text logs), so adjust chunking rules as needed. Some documents contain strongly correlated business data such as component batches and furnace temperature parameters. Retain contextual associations during chunking to prevent key information from being split apart.
How to set the configurations
| Configuration Item | Recommended Setting | Rationale |
|---|---|---|
chunkSize | 800–1200 characters | Adapts to the content length of multi-column tables and parameter-containing test reports in SMT documents, avoiding including too much cross-page or cross-field information in a single chunk |
chunkSplitter | Chunk by paragraph + semantics | Adapts to different document types, such as splitting tables by row groups and PDF reports by chapter, to match the logical associations of business data |
customPdfParse | Enable structured parsing mode | Accurately extract column fields for structured PDFs such as BOMs and coordinate files, avoiding misidentification of layout elements |
PARSE_FILE_TIMEOUT_SECONDS | 120–180 seconds | Adapts to large-volume batch work order documents, avoiding upload failures caused by parsing timeouts |
chunkSettingMode | Manual configuration | Adjust chunking rules for different document types (such as BOM tables, SPI reports) to meet the association requirements of business fields |
Three common mistakes
- Symptom: When uploading multi-column BOMs or coordinate files, only the first two columns of data are identified, and the remaining columns are not retrieved. Cause: The default chunking configuration is not adapted to multi-column structured documents, and the custom mode of
chunkSettingModeis not enabled, resulting in incorrect column field matching. - Symptom: When using the PDF enhanced parsing function, key business fields such as component batches and furnace temperatures are missing from the parsing results. Cause: The structured parsing rules for
customPdfParseare not configured correctly, and only general PDF parsing is used, which cannot recognize the dedicated field formats of SMT documents. - Symptom: Boolean type test results (such as solder joint qualification markers) parsed from HTTP components become null values after entering conditional judgment, and the corresponding verification logic cannot be triggered. Cause: The type information of the original data is not retained during chunking, and the original identifier of the boolean value is lost during field mapping.
How to confirm the configuration is correct
- Upload a standard SMT BOM table, check whether all column fields are included in the parsed chunks, with no missing or misaligned content.
- Upload a SPI/AOI test report PDF, verify that the parsing results accurately extract business fields such as defect locations and test values.
- Upload a real-time updated line change work order document, confirm that the parsing time meets expectations and no timeout error is triggered.
- Simulate uploading multi-column structured documents, check whether the chunked content retains the logical association between fields to facilitate subsequent retrieval and matching.
The values provided here are common starting points and should be evaluated against in-house samples.