Data for This Category
The data for this category comes from Gerber files, drill files, BOMs provided by customers, as well as internal CAM output materials, equipment operating parameters, bath test results, AOI inspection reports, electrical test reports, laboratory slice records and other documents from the factory. Daily documents include production process instructions, drilling lamination parameter logs, chemical analysis reports and more. The data update frequency changes with production batches and team shifts. Some equipment data is collected continuously. A full production batch will generate multiple types of related documents, and each document contains dedicated process parameters, test results and batch identification fields.
Constraints Imposed on Vector Models and Indexes
The multi-source, heterogeneous data characteristics of this category impose multiple constraints on the vector indexing and model workflow. Significant differences exist across various document formats, so chunking rules adapted to different structures are required to avoid breaking the correlation of process parameters. Continuously collected equipment data is updated frequently, so the index must support incremental synchronization to reduce resource consumption from full reconstruction. A single production batch is associated with multiple documents, so the index must support fast and accurate recall based on batch identifiers. Gerber and CAM files contain precise process parameters, so chunking logic must retain parameter associations to prevent process information from being disrupted during splitting. Some documents contain numerical test results and qualitative descriptions, so a unified embedding format is needed to ensure retrieval consistency.
Configuration Settings
Configuration references for FastGPT v4.17.0 are as follows:
| Configuration Item | Recommended Setting | Rationale |
|---|---|---|
| Chunk Length | 800–1200 characters | Adapts to the structure of long process parameters and test records in PCB documents, retains parameter correlations |
| Overlap Length | 100–150 characters | Fills context gaps after chunking long documents |
defaultToken | 8192 | Matches the token length requirements of most PCB documents, balances embedding accuracy and calculation efficiency |
| Similarity Threshold | 0.72–0.80 | Meets the accuracy requirements for PCB process parameter matching, filters low-relevance search results |
| Single Search Limit | Top 8 entries | Covers core process and test documents for a single production batch |
maxToken | 4000–6000 | Ensures complete context of multiple related documents can be recalled for audit scenarios |
Three Common Configuration Mistakes
- The knowledge base upload status stays in indexing for a long time. The cause is that incremental indexing rules are not configured, and the number of multi-source documents uploaded in a single batch is too large, resulting in excessive time consumption for full index calculation.
- An embedding rate limit exceeded error occurs during the vectorization process. The cause is that parallel thread parameters are not adjusted, and the default number of threads is too high, causing the call frequency to exceed the limit. The rate can be reduced by adjusting the number of threads.
- Block loss occurs when the chunk length is set to 3000 characters. The cause is that a single segment of text exceeds the upper limit of
defaultToken, causing the system to automatically truncate part of the content that is not correctly embedded.
How to Validate Configuration
- Upload a single typical PCB process document, check the chunking preview results, confirm that the chunk length falls within the preset range and no content loss occurs.
- Initiate a retrieval request, verify that the similarity scores of returned results fall within the preset threshold interval, and filter out irrelevant results.
- Simulate continuous uploads of multiple documents from the same batch, check the index update status, confirm that the incremental synchronization function works properly.
- View the vectorization logs, confirm that the embedding call frequency meets the preset limits, and no rate limit exceeded errors occur.
The values provided are common starting points and should be measured against the reader's own samples.