What the data for this category looks like
Daily documents for bakeries, pastry producers, cookie, instant noodle, and frozen dough factories include formulas and feeding lists, production records, baking or steaming curves, metal detection logs, cleaning and disinfection logs, and factory inspection reports. Data sources include raw and auxiliary material qualification certificates, ERP/MES systems, weighing systems, ovens or frying lines, laboratories, and food traceability ledgers. Update frequency is per batch, per shift, or daily. Most document structures are structured tables, timeline-based curve files, and standardized forms. Fields include material codes, batch numbers, process parameter ranges, inspection results, and similar items. Units include grams, degrees Celsius, minutes, shelf life days, and other standard food industry units.
Constraints imposed on document parsing and chunking
Documents in this category contain large numbers of structured tables and unstructured process descriptions. Standard plain text chunking breaks field associations within tables. Parsing logic must be adapted for table content. Multi-batch production records are updated by time or batch. Chunking must retain the link between batches and timestamps. Baking or steaming curves are unstructured files. Support must be added for extracting and linking curve data. Allergen labeling fields are critical for compliance. Chunking must preserve these fields accurately to avoid data loss. Frequent line changes create multiple document versions. Support must be added for linking corresponding chunked content to specific batches or timestamps.
Configuration Settings
This configuration applies to FastGPT v4.17.0.
| Configuration Item | Recommended Setting | Rationale |
|---|---|---|
chunkSize | 800–1200 characters | This category’s documents include long-form process descriptions and structured tables. This range preserves complete information for a single batch’s feeding list or a single segment’s baking parameters |
chunkSplitter | Split by paragraph + table block | Daily documents mix unstructured text and structured tables. This splitting method balances text logic and table integrity |
customPdfParse | Enable table recognition mode | Daily documents include PDF files with tables such as baking curves and feeding lists. Enabling this mode preserves the row and column field associations of tables |
PARSE_FILE_TIMEOUT_SECONDS | 120–180 seconds | Large production record PDFs or multi-batch Excel files take longer to parse. This range covers the parsing needs of most documents |
UPLOAD_FILE_MAX_SIZE | 500 MB | This category requires uploading multi-batch archived production records and inspection reports. This upper limit meets the upload needs of batch files |
chunkSettingMode | Manual configuration | This category’s documents have large structural differences. Chunking rules must be adjusted for different document types |
Three Common Errors
- Scenario: After uploading an Excel feeding list, search returns fewer foreign origin material entries than the source file. Cause: Default chunking does not retain full column associations for tables. Origin fields are split and lost, so matching cannot fully retrieve corresponding entries.
- Scenario: After uploading a DOCX format process procedure, RAG search results do not include section titles. Cause: Chunking configuration does not enable section title binding. Titles and main text are split into separate chunks, so search cannot associate and match them.
- Scenario: After uploading a baking curve PDF larger than 500 MB, the parsing task times out and fails. Cause: The
PARSE_FILE_TIMEOUT_SECONDSparameter is not adjusted. The default timeout duration is insufficient to parse large unstructured PDF files.
How to Verify Proper Configuration
- Upload a single batch’s feeding list Excel file. Review the parsed chunked content. Confirm that fields such as material names, usage quantities, and allergen labels are fully associated, with no splits or losses.
- Upload a baking curve PDF that includes tables. Enable
customPdfParse. Confirm that the parsed result retains the table’s row and column structure and numerical information. - Upload a multi-batch production record archive file under 500 MB. Confirm that the parsing task completes within 180 seconds, with no timeout errors.
- Search for allergen-related queries. Confirm that search results include the corresponding section titles and main text content, with no field omissions.
The values provided are common starting points and should be measured against the reader's own samples.