What the data for this category looks like
Coal transportation and sales ledger data primarily comes from documents such as weigh tickets, coal quality test reports, purchase and sales contracts, and settlement statements. Update cycles follow daily vehicle trips, weekly summaries, and monthly settlements. Document formats include digitally signed PDF files, scanned PDF files, and CSV tables exported from systems. Most content uses structured tables with fixed headers, with a small amount of explanatory text. Core fields include vehicle number, gross weight, tare weight, net weight (unit: ton), coal type, calorific value (unit: large calories per kilogram), transportation unit, and purchase and sales party information. Some documents also include electronic signatures and cross-page seals.
What constraints do these characteristics impose on the document parsing and chunking process
Multiple document formats require parsing tools to support both native PDF and scanned PDF parsing. This avoids missing ledger data from non-electronic documents. High-frequency updates of large volumes of daily vehicle data require small chunk sizes. Larger chunks increase subsequent summary and retrieval time. Structured tables with fixed headers require chunking tools to preserve table structure. They must not forcibly split fields within cells and their associated units. Electronic signatures and cross-page seals require the parsing process to preserve document integrity identifiers. This avoids impacting subsequent review steps.
Reference only; verify against the target FastGPT release and deployment documentation.
How to set the configurations
| Configuration Item | Recommended Setting | Rationale |
|---|---|---|
customPdfParse | Enabled | Coal transportation and sales ledger PDFs mostly contain structured tables with fixed formats and seals. Custom parsing can fully preserve table structure and field associations |
chunkSize | 800–1200 characters | A single complete ledger record is approximately 300–500 characters. This value range can hold 1 to 2 complete records, and avoids splitting core fields of a single ledger |
chunkSplitter | Split by paragraph + table | Coal transportation and sales documents mostly mix structured tables and explanatory paragraphs. This splitting method preserves data relevance and readability |
UPLOAD_FILE_MAX_SIZE | 1000 MB | Supports uploading multiple ledger files for monthly summaries in a single batch, adapting to bulk import business requirements |
PARSE_FILE_TIMEOUT_SECONDS | 600 seconds | Large monthly ledger PDF files take longer to parse. Extending the timeout period avoids mid-interruption of bulk parsing tasks |
chunkSettingMode | Manual configuration | Adjusting chunking logic to match the fixed field rules for coal transportation and sales is required. Manual configuration fits business scenarios better than automatic mode |
Three common mistakes
- Symptom: No parsing progress is shown after uploading a PDF via the dialog box, and no error logs appear in the background. Cause: The
customPdfParseconfiguration is not enabled. The default general parsing cannot recognize the fixed table structure of coal transportation and sales ledgers, leading to parsing failure without valid prompts. - Symptom: Service memory usage increases sharply after enabling the parsing function. Cause: The
UPLOAD_FILE_MAX_SIZElimit is not set. Oversized bulk ledger files that exceed server carrying capacity are uploaded, triggering memory resource exhaustion. - Symptom: Core fields such as coal type and net weight are split into different chunks in the chunking result. Cause: The
chunkSizesetting is too small, and the table-based chunking rule is not enabled. This leads to forced splitting of a single complete ledger record.
How to verify the configuration is correct
- Upload a single typical coal weigh ticket PDF, and check if the parsed text retains complete fields and table structure such as weigh time, vehicle number, gross weight and tare weight.
- Upload monthly ledger files in bulk, and check if parsing tasks complete within the time set by the
PARSE_FILE_TIMEOUT_SECONDSconfiguration, with no timeout interruption prompts. - Call the API to upload a file, and verify that the knowledge base chunk list contains complete ledger entries with no abnormal field splitting.
- Adjust the
chunkSizeparameter, and compare chunking results across different values to confirm that chunks do not split single complete ledger records.
The values provided here are common starting points and should be measured against the reader's own samples.
Question material comes from public community discussions. Configuration values are common starting points and should be measured against your own samples. Verified on 2026-09-28.