What this category of data looks like
Coal transportation and sales service consultation data mainly comes from transportation ledgers, weigh slips, purchase and sales contracts, monthly settlement reports, and similar sources. Data updates trigger with business processes, and are synced during single shipments and monthly settlements. Most documents use structured table formats. Excel files contain fields such as shipment date, mine site, rail car number, measured tonnage, settlement unit price, and receivable amount, with units mostly tons, yuan per ton, and ten thousand yuan. Word and PDF documents typically use fixed clauses paired with attached tables. Some scanned paper document attachments include minor handwritten annotations.
What constraints these characteristics impose on document parsing and chunking
Structured tables have many fields and mixed units. Automatic chunking may merge cross-field rows, leading to semantic fragmentation during retrieval. Large monthly settlement reports often reach hundreds of megabytes, creating high timeout risk for conventional parsing. Format differences across document types are significant: merged cells in Excel files and scanned PDF tables disrupt automatic splitting logic. The service consultation scenario requires precise matching of single business entries. Chunks must strictly correspond to single shipment or settlement records, and prevent cross-entry content from appearing in the same chunk.
How to set the configuration
Reference only; verify against the target FastGPT release and deployment documentation.
| Configuration Item | Recommended Setting | Rationale |
|---|---|---|
chunkSize | 800–1200 characters | The text length of a single coal transportation and sales business record mostly falls between 500 and 1000 characters. Reserve redundancy to avoid splitting breaks |
chunkSplitter | Split by line | Adapt to the structured row format of Excel files and ledgers, and ensure each chunk corresponds to a single shipment or settlement record |
UPLOAD_FILE_MAX_SIZE | 1000 MB | Cover the upload requirements for large files such as coal transportation and sales monthly settlement reports |
PARSE_FILE_TIMEOUT_SECONDS | 600 seconds | Reserve sufficient time for complex table parsing to avoid mid-process timeout interruptions |
chunkSettingMode | Custom mode | Depart from general chunking logic, and manually configure rules to adapt to coal transportation and sales business field requirements |
customPdfParse | Enable table recognition mode | For scanned weigh slip PDF files, enable dedicated table recognition to restore structured data |
Three common configuration errors
- Phenomenon: Multiple rows merge into one chunk after Excel import, making precise matching of single business entries impossible during retrieval. Cause: The line-separated chunking rule is not configured, and the default semantic segmentation logic is used, ignoring the row boundaries of structured tables.
- Phenomenon: Timeout errors trigger when parsing Word files larger than 10 MB. Cause: The
PARSE_FILE_TIMEOUT_SECONDSparameter is not adjusted, and the default short timeout setting is used, which cannot cover the parsing time of large files. - Phenomenon: Table fields are missing from parsing results after uploading scanned weigh slip PDFs. Cause: The table recognition mode of
customPdfParseis not enabled, and ordinary text parsing is used directly, which cannot restore the table structure.
How to confirm the configuration is properly set
- Upload a single coal shipment Excel record, and check if each record corresponds to an independent chunk in the chunking result.
- Upload a monthly settlement report larger than 10 MB, and confirm no timeout prompt triggers during the parsing process.
- Adjust the
chunkSizeparameter, and verify that the chunk length matches the preset character range. - Upload a scanned weigh slip PDF, and check if the parsed table fields fully restore the original data.
The values provided are common starting points and should be measured against the reader's own samples.
Question material comes from public community discussions. Configuration values are common starting points and should be measured against your own samples. Verified on 2026-09-28.