What the data for this category looks like
Typical entities in this category are bread and pastry, cookie, instant noodle, and frozen prepared pastry factories. Daily documents include formula and feeding lists, production records, baking or steaming curves, metal detection records, cleaning and disinfection records, and factory inspection reports. Data sources include raw and auxiliary material qualification certificates, ERP/MES systems, weighing systems, ovens or frying lines, laboratory testing equipment, and food traceability ledgers. Data is updated per batch, per shift, or daily. Documents contain clear batch numbers, allergen labels, test values and corresponding units, such as baking temperature, feeding weight, shelf life duration, etc. Document structures cover text paragraphs, tables, and charts.
What constraints these characteristics impose on the "document parsing and chunking" link
Frequent batch-based updates for this category require chunking to accurately associate batch information, and prevent mixing content across batches. Multiple document types (text, tables, chart images) require parsing functions to adapt to different formats. Specifically, PDF images of baking curves must support embedded text extraction. Frequent product line changes lead to frequent switches between formula and production records. Chunking must retain contextual associations between documents, to ensure full information for the target batch can be quickly located during retrieval. Strict allergen control requirements mean chunking must not split allergen labels and test results from the same cleaning or feeding record. Doing so would reduce traceability accuracy.
How to set the configurations
| Configuration Item | Recommended Setting | Rationale |
|---|---|---|
chunkSize | 800–1200 characters | Baking documents often contain batch information, formula parameters, and test values. Chunks that are too long will lose batch associations. Chunks that are too short will split the complete logic of the same test record |
chunkSplitter | Split by paragraph + semantic split | Production records and formula lists have clear paragraph separations. Semantic splitting preserves the integrity of key information such as allergens and batches, and adapts to the structured characteristics of category documents |
customPdfParse | Enable image text extraction mode | Some baking or steaming curves are stored as PDF images. Text such as temperature and time labels in the images must be extracted to ensure parsing covers all document types |
PARSE_FILE_TIMEOUT_SECONDS | 300–600 seconds | Parsing large production ledger Excel files or multi-page PDF documents takes longer. The default timeout duration cannot cover parsing requirements, preventing mid-process interruptions |
UPLOAD_FILE_MAX_SIZE | 1000 MB | Supports batch uploading of multi-batch production record files, adapting to the category's frequent small-batch upload scenarios |
chunkSettingMode | Manual configuration | Different document types (formulas, test records, cleaning ledgers) require differentiated chunking parameters. Automatic mode cannot adapt to the category's unique document structures and business needs |
Three common mistakes
- Phenomenon: After uploading a PDF image containing a baking curve, the knowledge base returns empty results or no valid text content. Cause: The image text extraction function of
customPdfParseis not enabled, so embedded chart image text cannot be parsed. - Phenomenon: When querying an Excel-format feeding list, the response mixes feeding data from different batches. Cause: Chunking does not split by batch semantics, and table rows across batches are split into the same chunk, leading to incorrect contextual associations and inability to accurately locate target batch information.
- Phenomenon: Parsing timeout errors are triggered when uploading multi-page production ledgers. Cause:
PARSE_FILE_TIMEOUT_SECONDSis not adjusted to a suitable duration. The default timeout setting cannot cover the complete parsing process of multi-page documents.
How to confirm the configuration is correct
The following verification actions are based on the knowledge base function of FastGPT v4.17.0:
- Upload a single PDF image containing a baking curve, check if the parsing result includes labeled text such as temperature and time from the image, to confirm that the image text extraction function is active.
- Upload batch Excel feeding lists, retrieve feeding data for a specific batch, confirm that the returned results only contain content for the corresponding batch, with no cross-batch mixing.
- Adjust
chunkSizeto different ranges, compare the completeness and relevance of retrieval results, and select a value range that meets business needs. - Upload a production ledger file at the maximum allowed size, confirm that the parsing process does not trigger timeout errors, verifying the reasonableness of the timeout setting.
The values provided here are common starting points, and should be measured against the reader's own samples.