What the Data for This Category Looks Like
Typical data sources for this category include paint supplier TDS/SDS documents, real-time data collected from spray booths and ovens, test reports from color difference meters and film thickness gauges, environmental monitoring logs, and MES work orders. Daily documents cover coating process cards, paint batch sheets, paint mixing records, film thickness and color difference reports, oven curves, defect repair sheets, and VOCs logs, among others. Data updates occur per paint batch, per shift, or via continuous collection. Documents include dedicated fields such as batch number, test values (e.g., film thickness in μm, color difference ΔE, oven temperature in ℃), and timestamps. Supported formats include PDF, structured tables, and plain text.
Constraints for Document Parsing and Chunking
Multi-source heterogeneous data sources and mixed-format documents require parsing workflows to support splitting both structured tables and unstructured text. The high-frequency update feature means chunking must adapt to incremental upload scenarios, avoiding repeated parsing of full datasets. Dedicated fields and units require parsing processes to retain the binding relationship between fields and units, and not randomly break parameter combinations. Additionally, coating processes follow pre-treatment, spraying, and curing windows. Chunking must retain contextual association between adjacent process steps to prevent damage to the integrity of process logic.
Configuration Settings
This configuration applies to FastGPT v4.17.0.
| Configuration Item | Recommended Value | Rationale |
|---|---|---|
chunkSize | 800–1200 characters | Coating process cards contain continuous process steps and parameters. Chunks that are too long will lose context, while chunks that are too short will break parameter associations. Calibrated through actual testing to adapt to multiple document types |
chunkSplitter | By paragraph + fixed character boundary | Coating documents contain both structured tables (such as oven curves) and process text. Mixed splitting can balance table parsing and text coherence |
customPdfParse | Enable table structured parsing | Coating reports and process cards mostly contain table data such as film thickness and color difference. Enabling this feature retains the corresponding relationship between fields and units |
PARSE_FILE_TIMEOUT_SECONDS | 180–240 seconds | Parsing large oven curve PDFs or multi-batch logs takes a long time, to avoid timeout interruptions |
UPLOAD_FILE_MAX_SIZE | 100 MB | When uploading multiple documents packaged for a single paint batch, limit the single file size to adapt to batch parsing scenarios |
chunkSettingMode | Manual configuration | Adjustment of splitting rules is required for different document types (such as TDS and on-site reports). Automatic mode cannot meet specific field requirements |
Three Common Mistakes
- Phenomenon: After uploading a coating process card PDF, the parsed chunks separate film thickness values from their units. Cause: The table structured parsing function of
customPdfParseis not enabled, causing table fields to be broken apart. - Phenomenon: Chunk IDs displayed on the front-end page cannot be selected and copied. Cause: The front-end chunk ID copy control configuration is not enabled, which matches common issues in actual implementation.
- Phenomenon: After uploading a custom-split paint batch sheet, the knowledge base automatically deletes duplicate chunks, causing index order confusion. Cause: The knowledge base’s automatic deduplication function is not disabled, or the batch number is not retained as a unique identifier during chunking.
How to Verify Proper Configuration
- Upload a standard coating process card PDF, and confirm that parsed chunks retain the corresponding relationship between fields and units such as film thickness and oven temperature.
- Upload a VOCs log larger than 50 MB, and confirm that the parsing task does not trigger a timeout error.
- Custom-split a batch of paint batch sheets, and verify that the knowledge base does not automatically delete valid duplicate chunks after upload.
- View the ID items in the chunk list, and confirm that they can be selected and copied.
The values provided here are common starting points and should be measured against the reader's own samples.