What this category of data looks like
Data for this category mainly comes from daily documents of ready-mixed concrete batching plants and project laboratories. These documents include mix proportion notifications, batch opening verifications, production and transport slips, raw material inspection reports, test block records, strength test reports, and more. Data sources cover engineering orders, sand, gravel, cement, and admixture test results, moisture content measurement data, batching plant control system logs, and laboratory test records. Updates follow production and testing milestones. Updates occur per vehicle production, per batch material inspection, and after daily production scheduling. Most documents are structured forms. Some are PDF files with tables or fixed-format documents. Fields include mix proportion parameters, moisture content, slump, strength grade, pouring location, transport time, and more. Units comply with construction engineering measurement standards.
Constraints on Document Parsing and Chunking
The structured documents and high-frequency update characteristics of this category impose specific requirements on the parsing and chunking process. The document parsing link must accurately extract key fields such as mix proportion, moisture content, and strength from tables. It must avoid splitting associated data into separate chunks. The high-frequency update rhythm requires chunk length to match business logic units. This prevents splitting the same production or test data into multiple chunks. Data from multiple sources leads to large variations in document formats. The parsing tool must support fixed-format forms and system-exported documents. Additionally, abnormal parameters such as slump and strength may trigger material return or shutdown risks. The parsing link must fully retain document metadata and core business associations. When chunking, context logic must be preserved to ensure corresponding raw material inspection and production records can be linked during retrieval.
Configuration Settings
These configuration items are based on the standard configuration options for FastGPT v4.17.0:
| Configuration Item | Recommended Setting | Rationale |
|---|---|---|
chunkSize | 800–1200 characters | Most documents in this category are structured forms and parameter-containing paragraphs. The chunk length covers the complete logic unit of a single production or test data, avoiding splitting associated data of mix proportion and moisture content correction |
chunkSplitter | Merge chunks by paragraph and table | This category of documents contains a large number of tabular mix proportion and test data. Merging chunks by paragraph and table can retain the integrity of structured data and avoid incorrect splitting of table content |
customPdfParse | Enabled | Most documents in this category are fixed-format engineering forms. Enabling custom parsing can accurately extract table fields and metadata, avoiding missing key parameters |
PARSE_FILE_TIMEOUT_SECONDS | 120–180 seconds | This category of documents contains a large number of tables and system-exported PDFs. The timeout setting reserves sufficient parsing time to avoid task failure due to parsing timeout |
UPLOAD_FILE_MAX_SIZE | 1000 MB | This category may contain batch production records or historical documents. Setting a reasonable upper limit avoids resource occupation during upload and parsing |
chunkSettingMode | Manual configuration | The document formats of this category are fixed. Manual configuration can accurately match the chunking and parsing requirements of business data, avoiding inapplicable parameters from automatic configuration |
Three Common Misconfigurations
- The symptom is missing images in the parsing result after uploading a PDF. This occurs when the
customPdfParseconfiguration is not enabled. The parsing tool only extracts text content and does not capture charts or images in the document. - The symptom is inability to switch document parsing tools. This happens when parameters other than
customPdfParseare not configured, or the custom parsing tool option is not enabled in the interface settings. This prevents replacement of the parsing tool. - The symptom is no associated metadata returned after retrieving a text block. This is caused by missing metadata extraction and binding logic in the chunking link, or an excessively large
chunkSizesetting. A large chunk size splits metadata and text blocks, making it impossible to link corresponding business data.
How to Verify Correct Configuration
- Upload a ready-mixed concrete document that includes images and tables. Check if the parsing result contains images and complete table fields.
- Access the configuration interface. Confirm that
customPdfParseis enabled and the custom parsing tool can be switched. - Run a retrieval test. Enter keywords such as mix proportion or strength grade. Check if the retrieval result links to corresponding business metadata.
- Upload an Excel format document. Confirm that the upload and parsing process completes normally.
The values listed here are common starting points and should be evaluated against the reader's own samples.