What the data for this category looks like
Daily documents of baking and convenience food factories include formulas and feeding lists, production records, baking/steaming curves, metal detection records, cleaning and disinfection records, factory inspection reports, and more. Data sources include raw and auxiliary material qualification certificates, ERP/MES systems, weighing systems, oven or frying line equipment, laboratories, and food traceability ledgers. Update frequency follows per batch, per shift, or daily. Most documents contain fields such as allergen labels, batch numbers, temperature parameters, shelf life, and others. Units include degrees Celsius, kilograms, dates, and more. The number of documents per single batch increases with line change frequency, and document formats exported from multiple systems vary.
What constraints do these characteristics impose on the document parsing and chunking link
The document characteristics of this category impose multiple constraints on the parsing and chunking process: Heterogeneous documents exported from multiple systems require parsing tools to support multiple file type adaptations. Frequent line changes and high-frequency updates require chunking rules to be flexibly adjusted to adapt to the document structures of different batches. Baking/steaming curves containing time-series data and formula documents with multiple fields require chunking to not destroy field associations and time-series continuity. The accuracy requirements for core business fields such as allergens and foreign matter control mean that the parsing process must not lose key information, otherwise compliance or safety risks may arise.
How to set the configurations
| Configuration Item | Recommended Setting | Rationale |
|---|---|---|
chunkSize | 800–1200 characters | This category of documents contains multi-field time-series data and structured tables. Too small a chunk size will split field associations, while too large a chunk size will introduce redundant context. The calibrated range is adapted based on actual testing |
chunkSplitter | Semantic + fixed length hybrid | Documents contain mixed structures such as formulas, curves, and records. Pure semantic splitting may easily break business fields, while pure fixed-length splitting may easily destroy time-series continuity. The hybrid mode can balance accuracy and completeness |
customPdfParse | Enabled | Most documents in this category contain tables with merged cells and multi-unit fields. The default parsing will lose key business information. Enabling custom parsing can retain field alignment and content integrity |
UPLOAD_FILE_MAX_SIZE | 1000 MB | Single-batch production records may include multiple days of equipment operation data. Large file upload support is required to avoid parsing interruptions caused by oversized files |
PARSE_FILE_TIMEOUT_SECONDS | 600 seconds | Parsing large baking curve PDFs or multi-sheet Excel documents takes a long time. 600 seconds covers the parsing needs of most business scenarios |
chunkSettingMode | Manual configuration | The document formats of this category vary significantly. Manual configuration allows precise adjustment of chunking rules to adapt to different types of business documents |
Three common mistakes
- Symptom: Core fields such as feeding volume and allergen labels are missing from the parsing result after uploading an Excel spreadsheet. Cause: The
customPdfParseconfiguration is not enabled, and the default parsing does not adapt to merged cells and multi-unit fields in food industry tables. - Symptom: Request failure is displayed when parsing large baking curve PDF files, with a timeout prompt in the interface. Cause: The
PARSE_FILE_TIMEOUT_SECONDSsetting value is lower than the actual parsing time, and no sufficient time is reserved for processing long time-series documents. - Symptom: Training data is incomplete after uploading multiple batches of documents to the knowledge base. Cause: The
chunkSizesetting is too small, splitting a single complete formula into multiple chunks, causing associated fields to fail to match, or thechunkSettingModeis set to automatic mode and does not adapt to multi-format documents.
How to confirm the configuration is correct
- Upload a single formula PDF containing allergen fields, and check whether the parsing result retains allergen labels and corresponding ingredient fields.
- Upload a 100 MB multi-sheet Excel production record, and confirm that the parsing progress does not trigger a timeout prompt within 2 minutes.
- Upload a single-batch baking curve document, and check whether the chunking result retains a continuous temperature-time sequence without breaks.
- In FastGPT v4.17.0, test uploading documents in different formats (PDF, Excel, CSV) and confirm that core business fields are retained after parsing.
Note: The configuration values listed above are common starting points. They should be evaluated against the target documents and operational scenarios.