What the data for this category looks like
The main entities in this category are carbonated drink, juice, tea, functional beverage, and packaged drinking water factories. Daily documents include formula sheets, water treatment records, blending records, sterilization or aseptic filling records, packaging material inspection reports, and finished product inspection reports. Data sources cover water source and raw material testing, formula systems, online sugar and acidity monitoring equipment, filling equipment, visual inspection systems, and LIMS systems. Documents are updated per batch, per shift, or via continuous collection. Most document fields contain precise numerical content, such as concentrate dosage, sugar content values, sterilization temperature, and turbidity indicators, with clear unit labels attached.
What constraints these characteristics impose on the document parsing and chunking link
Documents in this category are mostly multi-page mixed formats. They contain structured tables, embedded numerical values and units. Update frequency is high, and the number of documents per batch is large. Continuously collected real-time data must be split by batch to avoid mixing content across batches. Precise numerical values in formula sheets and inspection reports must retain their original format. The binding relationship between values and units must not be broken during chunking. In high-speed filling scenarios, filling record sheets have compact content per page. Chunking must avoid truncating key inspection items and anomaly annotations. Most documents are exported from different systems, with minor format differences. Parsing logic for multiple types of structured and semi-structured documents must be adapted.
How to set configurations
The following configurations are adapted for FastGPT v4.17.0.
| Configuration Item | Recommended Setting | Rationale |
|---|---|---|
chunkSize | 800–1200 characters | Adapts to the compact inspection data and formula fields in documents of this category. It avoids truncation of key content such as single-item dosage and detection values |
chunkSplitter | Split by paragraph + table | Matches the mixed layout of structured tables and explanatory paragraphs in formula sheets and inspection reports. It retains logical content associations |
customPdfParse | Enable mixed image-text parsing mode | Adapts to scenarios where this category’s documents include packaging material photos and filling process diagrams. It fully extracts image-text associated information |
PARSE_FILE_TIMEOUT_SECONDS | 300–600 seconds | Covers the parsing time of single-batch multi-page documents. It avoids timeout failures for batch documents collected continuously |
UPLOAD_FILE_MAX_SIZE | 1000 MB | Supports batch upload of multiple inspection reports and continuously collected filling records |
chunkSettingMode | Manual configuration | Adjust chunking rules for different document types, such as formula sheets and water treatment records. This adapts to differentiated content structures |
Three common errors
- Symptom: Images are missing or cannot be associated with corresponding text in the parsed document. Cause: The image-text binding parsing mode of
customPdfParseis not enabled, resulting in separated images and text. - Symptom: A single formula sheet is split into multiple logically unrelated chunks. Cause: The
chunkSizesetting is too small, truncating complete paragraphs that include dosage information and corresponding additive details. - Symptom: Parsing timeout errors occur for batch-uploaded water treatment records. Cause: The
PARSE_FILE_TIMEOUT_SECONDSsetting is lower than the actual parsing time, failing to adapt to batch processing requirements for multi-page documents collected continuously.
How to confirm the configuration is correct
- Upload a single packaging inspection report that includes tables and embedded images. Verify whether the parsed content retains the positional association between images and corresponding text.
- Upload a continuous filling record document with more than 10 pages. Check whether the parsing time falls within the preset range of
PARSE_FILE_TIMEOUT_SECONDSwith no timeout errors. - For a single formula sheet, check that the chunking result does not truncate a complete combination of single-item dosing parameters and corresponding units.
- Verify that the batch upload limit matches the
UPLOAD_FILE_MAX_SIZEsetting. Upload a document package close to the threshold to confirm no upload failures occur.
The values provided are common starting points and should be measured against the reader's own samples.