What the data for this category looks like
Coal transportation and sales documents and materials mainly include annual transportation and sales plans, monthly settlement statements, weigh tickets, transportation contracts, ownership certificates, and other types. Data sources are electronic documents exported from internal transportation and sales management systems of central state-owned enterprises, or PDF and image files generated by scanning offline paper documents. Updates follow business nodes. One document is created per transportation run. Monthly summary documents are updated monthly. Most documents have fixed business fields: shipment volume, coal calorific value, unit price, settlement amount, transportation route, ownership party, and others. Units include tons, large calories, yuan/ton, yuan, and more.
What constraints do these characteristics impose on the document parsing and chunking link?
The fixed fields and multi-batch data overlay characteristics of coal transportation and sales documents require that the parsing process does not damage the integrity of business fields. It also requires avoiding splitting continuous settlement details or contract clauses. The high proportion of scanned documents requires parsing tools to support unstructured text extraction and table recognition. Document update frequency fluctuates with business nodes. Large summary files take longer to parse, so reasonable timeout thresholds must be adapted. Scenarios with batch uploads of multiple files require parsing results to bind the metadata of the corresponding files. This avoids mixing content from different documents.
How to set the configuration
Reference only; verify against the target FastGPT release and deployment documentation.
| Configuration Item | Recommended Value | Rationale |
|---|---|---|
chunkSize | 800–1200 characters | Coal transportation and sales documents contain continuous business fields and paragraphs. A chunk length in this range preserves complete business context, and avoids splitting core settlement or contract clauses |
chunkSplitter | By paragraph + fixed field boundary | Contract clauses and settlement details in transportation and sales documents are divided by paragraph. Splitting by paragraph preserves the integrity of business logic |
customPdfParse | Enabled | Most coal transportation and sales documents include scanned documents or PDFs with embedded tables. Enabling custom parsing accurately extracts fields from weigh tickets and settlement statements in tables |
PARSE_FILE_TIMEOUT_SECONDS | 120–180 seconds | A single monthly summary PDF may contain multiple batches of transportation and sales data. Parsing takes longer, and this range covers the parsing needs of most large files |
UPLOAD_FILE_MAX_SIZE | 1000 MB | Central state-owned enterprise transportation and sales documents may include scanned copies of batch historical documents. This upper limit supports most batch upload scenarios |
chunkSettingMode | Manual configuration | The field structure of transportation and sales documents is fixed. Manually configuring chunking parameters allows precise adaptation to business data characteristics |
Three common mistakes
- Phenomenon: PDF parsing suddenly reports an error with no clear error message or returns empty content. Cause: The
customPdfParseparameter is not configured correctly, orPARSE_FILE_TIMEOUT_SECONDSis not adjusted to adapt to large file parsing requirements. - Phenomenon: After uploading multiple coal transportation and sales documents, the parsing results cannot distinguish between corresponding files. Cause: File metadata binding configuration is not enabled, or file identification fields are not retained during chunking.
- Phenomenon: Knowledge base search tests fail to hit chunked content. Cause:
chunkSizeis set too small, splitting continuous business fields, which prevents matching complete business context during search.
How to confirm the configuration is correct
- Upload a single standard coal transportation and sales PDF, and check if the parsing result extracts core business fields such as weigh tickets and settlement amounts.
- Upload multiple different types of transportation and sales documents, and check if each file's parsing result has an independent file identifier.
- Adjust the
chunkSizeparameter, perform a knowledge base search test, and confirm that the matching degree of hit chunked content meets business expectations. - Upload batch document files within 1000 MB, and check if parsing completes within the set timeout period.
The values provided are common starting points and should be measured against the reader's own samples.
Question material comes from public community discussions. Configuration values are common starting points and should be measured against your own samples. Verified on 2026-09-28.