What the data for this category looks like
Data primarily comes from exploration project approval documents, drilling operation files, reserve calculation reports, environmental assessment approval files, and similar sources. Update rhythm matches project progress milestones, with staged updates within a single project cycle. Documents are mostly in mixed formats, including long professional technical texts, structured tables such as drilling parameter tables and reservoir evaluation tables, and associated text embedded with well location diagrams. Fields include well number, drilling depth, reservoir temperature, and fluid density, with corresponding units of meters, degrees Celsius, and grams per cubic centimeter respectively. The length of individual documents varies widely, with some comprehensive files being quite lengthy.
What Constraints These Characteristics Impose on the Document Parsing and Chunking Process
Mixed-format document structures require the parsing process to adapt to both plain text and table content. General-purpose parsers often lose the column association relationships of tables. Single files of large-volume comprehensive files exceed conventional parsing thresholds, requiring splitting while preserving context coherence of professional terms. Fields in structured tables have exclusive units and professional definitions. The binding relationship between fields and their corresponding values must not be broken during chunking. Associated text embedded in diagrams must be parsed synchronously with the main document to avoid separation of text and accompanying diagram descriptions. Staged updated document content requires the chunking logic to support incremental identification of new content, avoiding repeated parsing of existing data.
How to Configure Settings
Reference only; verify against the target FastGPT release and deployment documentation.
| Configuration Item | Recommended Setting | Rationale |
|---|---|---|
chunkSize | 800–1200 characters | Adapts to the coherence of professional terms in oil and gas exploration documents, avoids losing context after splitting, and controls the embedding length of single chunk data |
chunkSplitter | Split by paragraph + table boundaries | Matches the mixed format characteristics of documents, retains complete row groups of tables and semantic integrity of paragraphs |
customPdfParse | Enabled | For embedded charts and structured tables in oil and gas exploration documents, custom parsing rules improve the accuracy of table field extraction |
UPLOAD_FILE_MAX_SIZE | 1000 MB | Covers the maximum possible size of a single comprehensive file, preventing large file uploads from being blocked |
PARSE_FILE_TIMEOUT_SECONDS | 600 seconds | Adapts to the parsing time of large-volume documents, preventing task failure due to parsing timeout |
chunkSettingMode | Manual configuration | Allows adjustment of chunking parameters for different types of files, such as drilling logs and reserve reports, adapting to diverse document structures |
Three Common Misconfigurations
- Phenomenon: Fields are empty or table rows are split across multiple chunks after parsing uploaded drilling table documents. Cause: The
customPdfParseconfiguration is not enabled. General-purpose parsers cannot recognize the structure of oil and gas exploration-specific tables, leading to failed field extraction or splitting logic that breaks table associations. - Phenomenon: Parsing tasks return timeout errors, or task status remains in the parsing state for an extended period. Cause: The
PARSE_FILE_TIMEOUT_SECONDSparameter is not adjusted. The default timeout duration is insufficient to complete the parsing process for large-volume comprehensive files. - Phenomenon: Chunked content has fragmented professional terms, such as "porosity" and its corresponding value being split into different chunks. Cause: The
chunkSizevalue is too small, forcing semantically complete professional paragraphs to be split.
How to Confirm the Configuration Is Correct
A standard drilling parameter table document is uploaded. The parsed text is checked to confirm it retains complete column titles and corresponding value rows. A lengthy comprehensive file document is uploaded. The parsing task is checked to confirm it does not trigger a timeout error. The chunkSize parameter is adjusted. Chunking results under different values are compared to confirm professional terms are not fragmented. The chunk list in the knowledge base is viewed to confirm the chunk content of table documents does not have separation of fields and their corresponding values.
The values provided here are common starting points, and should be validated against local sample data.
Question material comes from public community discussions. Configuration values are common starting points and should be measured against your own samples. Verified on 2026-09-28.