What the data for this category looks like
Typical documents for this category include recipes and ingredient lists, production records, baking/steaming curves, metal detection logs, cleaning and disinfection logs, and factory inspection reports for bread, pastry, biscuit, instant noodle, and frozen pastry manufacturing facilities. Data sources include raw material qualification certificates, ERP/MES systems, weighing systems, oven and frying line equipment, laboratory test data, and food traceability ledgers. Updates occur per production batch, shift, or daily. Documents contain structured tables, numerical curves, and text descriptions, with fields that include clear units such as ingredient dosage (grams/kilograms), baking temperature (℃), duration (minutes), and allergen labels.
Constraints these characteristics impose on document parsing and chunking
Documents for this category have multi-type structures, high update frequencies, and strongly associated fields, which create multiple constraints for parsing and chunking. First, documents include structured tables and numerical curves. Chunking must retain table integrity and the association between curve parameters to avoid splitting critical data. Second, the per-batch and per-shift update requirement means chunking must use production batches as boundaries, to prevent mixing information across batches. Third, recipe and allergen fields have strong binding relationships. Chunking must retain the association between these fields, to avoid losing complete information required for cross-review after splitting. Fourth, differences in data formats across multiple sources require parsing tools to adapt extraction rules for different document types.
How to configure settings
These configurations are based on FastGPT v4.17.0.
| Configuration Item | Recommended Setting | Rationale |
|---|---|---|
chunkSize | 800–1200 characters | Adapts to the length of core single-segment information for recipes and production records in this category, avoiding splitting critical associated fields |
chunkSplitter | Split by paragraph + table | Adapts to the mixed format of text paragraphs and structured tables in category documents, retaining table integrity |
PARSE_FILE_TIMEOUT_SECONDS | 300–600 seconds | Single-batch production records contain multiple curves and test data, leading to long parsing times; this avoids mid-task timeouts |
customPdfParse | Enabled | Supports custom extraction of core review fields such as recipes and allergens for this category |
UPLOAD_FILE_MAX_SIZE | 1000 MB | Adapts to the file size requirements for batch importing multi-batch production records |
chunkSettingMode | Manual configuration | Allows adjustment of chunking rules for different document types, adapting to the diverse document formats of the category |
Three common configuration errors
- Issue: Data processing returns empty after uploading an Excel recipe file. Cause: Custom parsing rules for
customPdfParseare not enabled, and the default parsing fails to recognize structured fields in ingredient lists. - Issue: Parsing tasks time out when batch importing multi-batch production records. Cause: The value set for
PARSE_FILE_TIMEOUT_SECONDSis lower than actual parsing time, failing to adapt to the complex structure of documents in this category. - Issue: Vectorization speed is too slow. Cause:
chunkSizeis set too small, generating an excessive number of chunks which leads to a sharp increase in vectorization task volume.
How to confirm correct configuration
- Upload a single standard production record document, view the parsed chunk preview, and confirm that ingredient list tables and baking curve parameters are not split.
- Upload a batch production record file, verify that no timeout errors appear in parsing tasks, and confirm that the
PARSE_FILE_TIMEOUT_SECONDSsetting is reasonable. - Test uploading production records for new batches, confirm that only new content is parsed, and no historical data is parsed repeatedly.
- Check allergen fields in chunks, confirm that they are in the same chunk as corresponding recipe content, meeting the association requirements for cross-review.
The values provided are common starting points and should be evaluated against the reader’s own samples.