What the data for this category looks like
Typically, production data for this category comes from concentrate and auxiliary material analysis reports, DCS systems, cell voltage and current monitoring data, laboratory test results, and energy and environmental monitoring platforms. Daily production documents include batching lists, smelting records, electrolytic cell condition records, test reports, anode and cathode quality records, and environmental protection ledgers. Data update frequency is adjusted according to production rhythm, covering real-time updates for each batch of raw materials, each furnace, or each cell. Most document fields include grade, current intensity, cell temperature, impurity content, and corresponding units such as percentage, millivolt, degrees Celsius, grams per liter, etc. Some documents must be bound to specific production unit identifiers.
What constraints do these characteristics impose on the document parsing and chunking link
This category has multi-source heterogeneous documents with significant format differences. The parsing logic for tabular batching lists and time-series electrolytic cell condition records requires separate adaptation. High-frequency real-time production data requires parsing response to match production rhythm, to avoid delays affecting subsequent audits. The binding relationship between specific fields and units requires retaining context during parsing, to prevent mismatching values and units. Documents from different production units such as different electrolytic cells, smelting furnaces need to be split by unit, to avoid audit deviations caused by cross-unit data confusion.
How to set the configuration
This adaptation applies to FastGPT v4.17.0’s document parsing and chunking feature.
| Configuration Item | Recommended Value Range | Rationale |
|---|---|---|
chunkSize | 800–1200 characters | This category’s documents contain multiple field values and text descriptions. 800–1200 characters can cover the complete business information of a single test result or single smelting furnace record |
chunkSplitter | Split by paragraph + title | This category’s documents have clear production unit titles such as 1# electrolytic cell, 3rd smelting furnace. Splitting by title ensures business independence of single chunk data |
PARSE_FILE_TIMEOUT_SECONDS | 120–180 seconds | Parsing large smelting records or batch test reports requires longer processing time. This range covers the parsing duration of most conventional production documents |
customPdfParse | Enabled | Most PDF documents in this category are standardized report templates. Custom parsing can accurately extract tables and fixed production fields |
chunkSettingMode | Manual configuration | This category’s documents have large format differences. Manual configuration can adapt to different types of production documents such as batching lists and cell condition records |
Three common errors
- Symptom: When calling the API to create a knowledge base, it is not possible to obtain parsing, ready, or failed status feedback. Cause: Failed to correctly call the status query interface, or the request parameter did not carry a valid knowledge base identifier.
- Symptom: Embedded images in uploaded PDFs cannot be bound and output with corresponding text blocks. Cause: The
customPdfParseconfiguration is not enabled, or the chunking logic does not retain the association between images and context. - Symptom: When parsing HTTP interface response messages, it is not possible to correctly extract Set-Cookie fields. Cause: Custom parsing rules are not enabled, so response header information is not included in the parsing scope.
How to confirm the configuration is properly set
- Call the knowledge base creation API, check if the returned result includes status-related fields, and confirm that status can be queried and updated normally.
- Upload a standard production document containing tables and images, and check if the parsed chunks are split by business unit, with images correctly associated with corresponding text.
- Adjust the
chunkSizeconfiguration to different value ranges, compare the parsed chunk content, and confirm that the configuration parameters take effect. - Upload batch documents within the range specified by the
UPLOAD_FILE_MAX_SIZEconfiguration, and confirm that parsing tasks are not intercepted and can be completed normally.
The values provided are common starting points and should be measured against the reader's own samples.