What the data for this category looks like
Data for iron ore intelligent due diligence mainly comes from mine-issued quality inspection reports, port customs clearance documents, bulk commodity trade contracts, and monthly industry association briefings. It is updated per logistics node of each shipment, with monthly updates for industry briefings. Documents are primarily in PDF format, supplemented by Word trade contracts and Excel ledgers. Structures include structured tables with fixed headers and plain text trade clauses. Fields cover dry basis total iron content, particle size distribution range, wet basis moisture, port of shipment information, and more. Particle size is measured in millimeters, and content fields use dry basis or wet basis as the measurement standard.
What constraints do these characteristics impose on the "document parsing and chunking" link
Structured tables account for a large proportion of documents. Directly splitting into plain text will lose the column correspondence of indicators such as content and particle size, so table structure must be retained during chunking. Exclusive fields for each batch of documents (such as batch number, origin) need to be bound to their context to avoid confusion across batches. Long documents are mostly monthly industry briefings, which should be chunked by chapter logic instead of fixed length. Excel ledgers with multiple sheets must be parsed per sheet to avoid mixing content across sheets. Annotations in PDF documents must be included in the parsing scope to ensure the completeness of due diligence information.
How to set the configurations
| Configuration Item | Recommended Value | Rationale |
|---|---|---|
PARSE_TABLE_ENABLE | Enabled | Iron ore due diligence documents contain a large number of structured tables. Retaining table structure ensures that column correspondence of indicators such as content and particle size is not lost |
PARSE_CHUNK_SIZE | 800–1200 characters | Monthly industry briefing documents are relatively long. This range balances the integrity of information per chunk and retrieval recall accuracy, and adapts to the chapter content density of due diligence reports |
PARSE_CHUNK_OVERLAP | 100–150 characters | Prevents key cross-chapter information from being split, and ensures contextual coherence of exclusive fields such as batch number and origin |
PARSE_EXCEL_SHEET_MODE | Chunk per sheet | Excel ledgers usually store independent data for different batches. Chunking per sheet avoids mixing data across batches |
PARSE_PDF_ANNOTATION_ENABLE | Enabled | Trade annotations in PDF documents are core due diligence information and must be included in parsed content |
PARSE_FILE_TIMEOUT_SECONDS | 600 seconds | Parsing large monthly briefing documents takes a long time. This duration covers the parsing needs of most documents |
The parameter values provided on this page are conventional recommendations used as a starting point for configuration. Actual values are affected by material form, data volume, and business rules. Specific issues require case-by-case analysis. It is recommended to test on your own samples before finalizing settings.
Three common mistakes
- Symptom: After uploading an iron ore quality inspection PDF, no results are returned when searching for total iron content in the knowledge base. The interface shows parsing succeeded but retrieval is empty. Cause: The
PARSE_TABLE_ENABLEconfiguration is not enabled. Table content is directly converted to unstructured plain text, causing keyword matching to fail. - Symptom: After updating the platform version, the original miner-u parsing entry disappears, and iron ore documents cannot be parsed normally. Cause: The platform has integrated the miner-u parsing function into the global file parsing configuration. Adjust parameters in the "Document Parsing" module of knowledge base settings, do not enable the corresponding plugin separately.
- Symptom: After uploading an Excel-format iron ore ledger, multi-sheet data is merged into a single chunk. Cause:
PARSE_EXCEL_SHEET_MODEis not configured to chunk per sheet, resulting in mixed data across sheets.
How to confirm the configuration is set correctly
- Upload a single iron ore quality inspection PDF, enter the "Document Parsing Logs" of the knowledge base to check, confirm that the table structure is fully retained and no field misalignment occurs.
- Enter the "Document Parsing" module of knowledge base settings, verify that the values of configuration items such as
PARSE_TABLE_ENABLEandPARSE_EXCEL_SHEET_MODEmatch the preset plan. - Initiate a knowledge base search test, input exclusive keywords such as total iron content and particle size distribution, confirm that chunked content of the corresponding documents can be retrieved.
- Upload a multi-sheet Excel ledger, check the chunk list, confirm that each sheet corresponds to an independent chunk unit.
Question material comes from public community discussions. Configuration values are common starting points and should be measured against your own samples. Verified on 2026-09-14.