What this category of data looks like
Data sources for internal office and employee services in coal transportation and sales mainly include purchase-sales contracts, weigh slips, monthly settlement reports exported from internal sales and transportation management systems, employee on-duty training manuals, attendance records, and other materials. Document formats cover Word and PDF, with a large number of scanned weigh slips and contracts. Data update rhythms vary significantly: weigh data is updated daily, monthly settlement reports are updated monthly, and training manuals are revised as needed. Document fields include mine site name, rail car number, carried tonnage, settlement unit price, employee ID number, job title, and others. Units include tons, yuan, trips, person-times, and others.
What constraints these characteristics impose on the "document parsing and chunking" link
Coal transportation and sales documents have a high proportion of nested tables and scanned copies. Parsing processes must retain table structure and support OCR recognition. Failure to do so splits cross-row and cross-column business data, or loses key information from scanned content. Document sizes vary widely, from single-page weigh slips to dozens of pages of annual summary documents. Adjustments to parsing and chunking parameters are required to accommodate these variations. Employee training documents contain job-specific terminology. Chunking processes must retain contextual associations to avoid splitting complete job operation procedure logic.
How to set the configurations
Reference only; verify against the target FastGPT release and deployment documentation.
| Configuration Item | Recommended Value Range | Rationale |
|---|---|---|
chunkSize | 800–1200 characters | Coal transportation and sales documents contain nested tables and professional terminology. The chunk length covers a single settlement detail or job operation instruction to avoid splitting key business information |
chunkSplitter | Split by paragraphs + filter headers and footers | Transportation and sales documents have repeated mine site headers and footers. Split by paragraphs and filter repeated headers and footers to retain core business content |
customPdfParse | Enable OCR mode | A large number of scanned weigh slips and contracts exist. OCR mode can recognize text and table content in scanned copies |
UPLOAD_FILE_MAX_SIZE | 1000 MB | Support uploading large annual summary PDF documents to avoid failure to parse due to exceeding file size limits |
PARSE_FILE_TIMEOUT_SECONDS | 600 seconds | Large settlement documents take a long time to parse. Extend the timeout period to avoid mid-interruption of parsing tasks |
chunkSettingMode | Manual configuration | Adjust chunking parameters for different document types (ledgers/contracts/employee documents) to adapt to diverse document structures |
Three common mistakes
- Phenomenon: After uploading a sales contract or employee training document with embedded images, the knowledge base preview displays image links. Question-and-answer tests cannot access content related to the images. Reason: The embedded image text extraction configuration of
customPdfParseis not enabled. Only the main document text is parsed, and explanatory text embedded in images is not captured. - Phenomenon: When uploading a monthly settlement summary PDF, the parsing task times out and fails, returning status code 504. Reason: The
PARSE_FILE_TIMEOUT_SECONDSconfiguration is not adjusted. The default timeout period is insufficient to complete the parsing of large documents. - Phenomenon: After parsing scanned weigh slips, core fields such as rail car number and carried tonnage are missing in the chunking results. Reason: OCR parsing mode is not enabled. Only the blank layer text of the PDF is read, and business data in the scanned content is not recognized.
How to confirm the configurations are set correctly
- Upload a scanned weigh slip. Check if parsed text includes core fields such as weigh weight and rail car number to confirm OCR mode is active.
- Upload an employee training document with embedded images. Check if the knowledge base preview includes explanatory text corresponding to images to confirm image text extraction configuration is active.
- Adjust
chunkSizeto different ranges. Upload a long-text transportation and sales ledger. Check if chunking results split according to the set length without truncating key information. - Upload a document exceeding the platform's default size. Confirm successful upload and completed parsing to verify the
UPLOAD_FILE_MAX_SIZEconfiguration is active.
The values provided are common starting points and should be measured against the reader's own samples.
Question material comes from public community discussions. Configuration values are common starting points and should be measured against your own samples. Verified on 2026-09-28.