What this category’s data looks like
Daily documents for this category include pulp preparation sheets, beating records, paper machine operation logs, online quality curves, laboratory reports, and sheet break and product change records. Data sources include wood pulp and waste paper batches, DCS/QCS systems, paper machine online measurement equipment, laboratory testing, energy and water treatment systems. Data update frequencies follow three dimensions: real-time, per shift, and per roll. Most documents include process parameters such as pulp ratio, grammage, moisture content, and thickness. Some documents use formats with embedded charts.
What constraints do these characteristics impose on the "document parsing and chunking" link?
High-frequency real-time, per-shift, and per-roll data requires chunk granularity to match short-cycle process nodes. Avoid oversized chunks that cause context breaks. Embedded chart data in online quality curves and sheet break or product change records requires support for extracting numerical values and trend information. This meets full data requirements for process analysis. Documents such as pulp preparation sheets and laboratory reports combine structured tables and free text. Parsing rules must adapt to multiple document types. Parameters like paper machine speed and web width in paper machine operation logs are linked to production continuity. Chunking must retain contextual connections between adjacent nodes to ensure complete process logic.
How to set the configuration
This configuration is based on FastGPT v4.17.0.
| Configuration Item | Recommended Setting | Rationale |
|---|---|---|
chunkSize | 800–1200 characters | Matches the text length of paper machine operation logs and laboratory reports, retains contextual connections for process parameters |
chunkSplitter | Split by paragraph + table boundaries | Adapts to documents mixing structured tables and free text such as pulp preparation sheets, avoids splitting that damages table structure |
customPdfParse | Enable embedded chart parsing mode | Supports extraction of numerical values and trend data from online quality curves, meets process analysis requirements |
PARSE_FILE_TIMEOUT_SECONDS | 300–600 seconds | Adapts to parsing duration of large online quality curve PDFs, avoids timeout interruptions |
chunkSettingMode | Manual configuration | Adjust chunking rules for different document types (table-based, curve-based) to cover all scenario needs |
Three common mistakes
- Empty results when querying a single PDF. The cause is that embedded chart parsing for
customPdfParseis not enabled. Key data for quality curve documents is not extracted. - Garbled characters or incorrect recognition of technical terms in Chinese OCR results. The cause is that no dedicated term library is configured for pulp and paper industry-specific terms such as beating degree and wet weight. This leads to recognition errors.
- Excel tables imported normally in the knowledge base but not displayed in generated results. The cause is that chunking did not split along table boundaries. The row and column correspondence of the table is damaged.
How to confirm the configuration is correct
- Upload a PDF document containing online quality curves. Check if the parsed text includes numerical values and trend descriptions corresponding to the curves.
- Upload a pulp preparation sheet Excel table. Verify that chunked content retains the correspondence between table fields such as pulp ratio and batch number.
- Upload a single paper machine operation log document. Confirm that parsing time is within the preset
PARSE_FILE_TIMEOUT_SECONDSrange, with no timeout errors. - Initiate a query related to process parameters for a single document. Confirm that the returned result includes all key process data from the document, with no missing content.
The values provided are common starting points and should be measured against the reader's own samples.