What this category of data looks like
Data for this category comes from professional heat treatment plants and heat treatment workshops of automobile and bearing parts factories. Daily documents include heat treatment process cards, charging charts, furnace temperature curves, hardness and metallographic reports, equipment inspection forms, and exception handling sheets. Some documents are multi-page summary reports, with single copies reaching thousands of pages. Data sources cover material grades, customer technical requirements, temperature control instruments, furnace temperature uniformity testing equipment, hardness testers and metallographic equipment, and MES systems. Update frequency is per heat and per batch. Document fields include process parameters (heating temperature, holding time, cooling rate), equipment numbers, test values (hardness values, metallographic structure grades), exception descriptions, and handling plans. Most units use degrees Celsius, hours, megapascals, HV, and similar units.
What constraints do these characteristics impose on document parsing and chunking
The high-frequency updates, multiple document types, and specialized field characteristics of this data category impose multiple constraints on document parsing and chunking. Distinguish single-heat data from batch summary content to avoid merging parameters across heats into chunks. Bind process parameters to their corresponding units to prevent separation of parsed data and units. For Excel-format inspection forms and hardness reports, accurately identify the correspondence between table headers and data rows to avoid mixing unrelated rows within a chunk. For long documents such as annual furnace temperature summary reports, locate single-heat paragraphs to prevent merging cross-batch content. Additionally, support parsing and association of non-text documents like PDF-format charging charts and furnace temperature curves, ensuring images are bound to their corresponding text parameters.
How to configure the settings
| Configuration Item | Recommended Setting | Rationale |
|---|---|---|
chunkSize | 800–1200 characters | Heat treatment documents contain continuous single-heat process parameters and test data. This range ensures a single chunk contains a complete process unit and avoids splitting parameters |
chunkSplitter | Split by paragraph + fixed character truncation | Process paragraphs and test report paragraphs in heat treatment documents have clear structures. Splitting by paragraph preserves semantic integrity. Fixed character truncation adapts to batch processing of long documents |
customPdfParse | Enable furnace number recognition + header and footer filtering | Heat treatment PDF documents often contain identifiers such as furnace numbers and batch numbers. Enabling this configuration filters irrelevant headers and footers to accurately locate single-heat content |
PARSE_FILE_TIMEOUT_SECONDS | 600 seconds | Long documents such as annual furnace temperature summary reports take a long time to parse. This duration covers complete parsing of single 1000-page documents |
UPLOAD_FILE_MAX_SIZE | 1000 MB | Adapts to batch-uploaded annual summary reports and multi-batch documents, preventing large file upload failures |
chunkSettingMode | Manual configuration | Adjust chunking parameters separately for different document types (process cards, furnace temperature curves, Excel inspection forms) to adapt to specialized field structures |
This configuration is based on FastGPT v4.17.0
Three Common Mistakes
- Phenomenon: Incomplete command prompts are returned after parsing Excel-format inspection forms and hardness reports. Cause: Header recognition and row grouping rules are not configured for Excel documents, so the large language model cannot recognize the data structure.
- Phenomenon: Parsing times out for long documents such as annual furnace temperature summary reports. Cause:
PARSE_FILE_TIMEOUT_SECONDSis not set to a sufficient duration, or parsing rules for long documents are not adjusted. - Phenomenon: Chunk granularity is too large, containing process content from multiple heats. Cause:
chunkSizeis set beyond the length range of single-heat process parameters, and chunking is not combined with furnace number identifiers.
How to Verify Correct Configuration
- Upload a single heat treatment process card PDF. Confirm parsed chunks contain complete heating temperature, holding time, and cooling rate parameters, with no residual headers or footers.
- Upload an Excel-format inspection form. Confirm chunks are grouped by table headers, with each row of data bound to its corresponding header, and no missing fields.
- Upload a 1000-page annual furnace temperature summary report. Confirm the parsing process does not trigger a timeout, and chunks are divided by furnace number or batch.
- Test mixed uploads of multiple document types. Confirm chunking rules automatically adapt to different document types, with no cross-type content merging.
The values listed here are common starting points and should be validated against individual use cases.