What this category of data looks like
Oil and gas exploration and development documents and materials mainly come from drilling logs, logging analysis reports, reserve assessment plans, core scanning documents, and so on. The update rhythm is adjusted according to project progress: logs are generated in real time during the drilling phase, and annual assessment reports are updated regularly. Most documents are long-form PDFs, containing nested professional tables, quantitative data with units, and scanned core images. Fields include well depth, porosity, permeability, oil saturation, and other related items. Common units are meters, %, and millidarcies.
What constraints these characteristics impose on the document parsing and chunking link
Long-form documents have large individual file sizes, which place clear requirements on upload and parsing timeout settings. Nested tables and professional fields account for a high proportion, so ordinary parsing processes are prone to losing format and field unit information. Scanned core documents account for a non-negligible proportion, requiring enhanced OCR parsing capabilities to fully extract content. Professional terms are dense and highly correlated, so chunking operations must retain semantic integrity, avoiding splitting professional expressions and causing semantic breaks.
How to set configurations
| Configuration Item | Recommended Setting | Rationale |
|---|---|---|
chunkSize | 800–1200 characters | Oil and gas exploration documents mostly contain long professional sentences. This range can retain semantic integrity of single segments and avoid splitting professional terms |
chunkSplitter | Split by paragraph + heading | Exploration documents have clear hierarchical headings. Splitting this way can retain the logical structure of the document |
customPdfParse | Enable enhanced parsing | Adapts to nested tables and scanned PDF core reports, retaining formats and professional fields |
PARSE_FILE_TIMEOUT_SECONDS | 600 seconds | Large drilling log PDF files take a long time to parse, avoiding mid-parsing timeout interruptions |
UPLOAD_FILE_MAX_SIZE | 1000 MB | Supports uploading complete single exploration project reports, avoiding file splitting |
chunkSettingMode | Manual configuration | Adjust chunking parameters for professional documents, adapting to specific formats in the oil and gas field |
Three common errors
- Phenomenon: Table fields are still lost after enabling enhanced PDF parsing. Cause: The locally deployed MinerU service port is not correctly bound in the FastGPT knowledge base settings, or the
customPdfParseparameter is not checked. - Phenomenon: A 504 timeout error is returned when parsing large exploration reports. Cause: The
PARSE_FILE_TIMEOUT_SECONDSsetting value is lower than the actual parsing time, and does not match the processing cycle for large documents. - Phenomenon: Multi-vector support cannot be enabled for knowledge base table data. Cause: Multi-vector support is not separately configured for table-type documents, or chunking parameters do not adapt to table content density.
How to confirm the configuration is correct
- Upload a typical logging PDF report, check the parsed text in FastGPT v4.17.0, and confirm that professional fields such as well number, porosity, permeability and their corresponding units are retained.
- Enter the settings page of the target knowledge base, confirm that the
customPdfParseoption is checked, and that the deployment address and port of the local MinerU service are correctly configured. - Upload a drilling log PDF larger than 500 MB, verify that the parsing task does not show a timeout failure prompt, and the task status shows completed.
- Select the parsed table-type document, confirm that the multi-vector support configuration item can be enabled separately in the knowledge base settings.
The values provided are common starting points and should be measured against the reader's own samples.
Question material comes from public community discussions. Configuration values are common starting points and should be measured against your own samples. Verified on 2026-09-28.