What This Category of Data Looks Like
Non-ferrous metal smelting ledger data primarily comes from shift inspection records of production teams, workshop batching ledgers, quality inspection test reports, and monthly summary reports. Data update frequencies cover real-time, daily, and monthly cycles. Document formats include collaborative documents with multi-level directories, structured Excel tables, and PDF reports converted from scanned files. Document fields include material name, batch number, processing volume, energy consumption, test results, and timestamp. Units include tons, cubic meters, and kilowatt-hours. Documents are organized into multi-level directories by workshop and team, and a single document may contain dozens of pages of tables and text content.
Constraints Imposed on Document Parsing and Chunking
Multi-level directory document structures require the parsing process to traverse all documents under subdirectories to avoid missing team-level ledger data. Structured Excel tables and PDF quality inspection reports require complete extraction of cell content without format damage or splitting. Real-time updated inspection records require the parsing process to adapt to frequently updated documents, while long documents may carry parsing timeout risks. Multi-field ledger data requires retaining field association during chunking to avoid information breakage across fields caused by splitting.
Configuration Settings
Reference only; verify against the target FastGPT release and deployment documentation.
| Configuration Item | Recommended Value | Rationale |
|---|---|---|
chunkSize | 800–1200 characters | Non-ferrous metal ledgers contain multi-field single records and complete tables. This range preserves full information for a single ledger entry and avoids truncating core content. |
chunkSplitter | By paragraph + table | Ledgers include both unstructured text from inspection descriptions and structured tables for batching and quality inspection. Mixed chunking retains the connection between both formats and their data. |
customPdfParse | Enable OCR mode | Some ledgers are PDFs generated from scanned paper records. OCR mode accurately extracts text and table content from scanned documents. |
PARSE_FILE_TIMEOUT_SECONDS | 600 seconds | Monthly summary ledgers may contain dozens of pages of tables and text. This duration supports complete parsing of long documents. |
chunkSettingMode | Custom mode | Allows adjustment of chunking rules based on the structured characteristics of ledgers, adapting to multi-field document structures. |
UPLOAD_FILE_MAX_SIZE | 1000 MB | Some ledgers include large-capacity Excel attachments and multi-page PDF documents. This setting allows uploading complete monthly summary documents. |
Three Common Configuration Errors
- Issue: Subdocuments under multi-level directories are not parsed or retrieved. Cause: Directory traversal parsing configuration is not enabled, only documents under the root directory are parsed.
- Issue: API call to upload documents returns success but no parsing results. Cause:
PARSE_FILE_TIMEOUT_SECONDSis set too small, long document parsing is not completed. - Issue: Logs prompt split errors. Cause:
chunkSizeis set too small, truncating complete rows of structured tables during splitting, leading to chunking failure.
How to Verify Proper Configuration
- Upload a test ledger document with multi-level directories, and confirm the parsed text includes content from all documents under subdirectories.
- Call the API to upload the test document, and verify the returned parsing results include complete text and table content.
- View the chunked results, and confirm each chunk contains a complete single ledger record or table content.
- Adjust
chunkSize, upload the same test document, and compare the completeness and relevance of the chunked results.
The values provided are common starting points and should be measured against the reader's own samples.
Question material comes from public community discussions. Configuration values are common starting points and should be measured against your own samples. Verified on 2026-09-28.