This document applies to FastGPT v4.17.0.
What data for this category looks like
Data for this category comes from asset management systems, procurement systems, financial fixed asset modules, and terminal management platforms. Update frequency is monthly, quarterly, or annually. Commonly processed documents include asset ledgers, inventory sheets, loan and return forms, and scrap applications. Fields in these documents include asset ID, name, specification model, user, storage location, purchase date, recorded value, and more. Units are mostly units, pieces, yuan, and date formats. Some documents are structured formats exported from systems, while others are unstructured documents filled offline.
Constraints imposed by these characteristics on document parsing and chunking
Document formats vary widely across sources, with mixed structured and unstructured content. Parsing and chunking must balance field extraction accuracy and context integrity. Core fields for asset documents, such as asset ID and user, must be fully retained within a single chunk. This avoids information fragmentation caused by splitting. Document sizes differ significantly based on update cycles. Monthly ledgers are relatively short, while quarterly summary files may contain large numbers of entries. Parsing and chunking rules must adapt to different file sizes. Additionally, numerical and identification fields in documents must be accurately identified. This prevents disruption of asset information association during chunking.
How to set configurations
| Configuration Item | Recommended Value | Rationale |
|---|---|---|
chunkSize | 800–1200 characters | Asset ledgers contain asset entries with combined multiple fields. A single chunk must cover a complete entry group to avoid splitting associated information for a single asset |
chunkSplitter | By paragraph + fixed character count | Loan and return forms and inventory sheets are divided by paragraphs of loan or inventory records. Character count is used as a fallback to prevent parsing failures for overly long paragraphs |
customPdfParse | Enabled | Some scrap applications are scanned PDF files. Enabling custom parsing optimizes extraction accuracy for tables and text, adapting to recognition of structured asset data |
UPLOAD_FILE_MAX_SIZE | 1000 MB | Quarterly inventory summary files may contain multiple batches of asset data. Support for large file uploads is required to meet batch processing needs |
PARSE_FILE_TIMEOUT_SECONDS | 600 seconds | Parsing large summary files takes a long time. Setting a longer timeout prevents mid-task parsing interruptions |
chunkSettingMode | Manual configuration | Asset document fields are fixed but formats vary. Manual configuration adapts chunking rules for different ledger templates |
Three common mistakes
- Phenomenon: Parsed pandas data is not presented in table form. Cause: Table extraction configuration for
customPdfParseis not enabled, or context association for table structure is not retained during chunking. This causes table content to be split into scattered text. - Phenomenon: After calling the API to create a knowledge base, parsing status (parsing, ready, failed) cannot be obtained. Cause: Status return parameters are not configured in the API request, or the specified status query interface is not called to obtain real-time parsing progress.
- Phenomenon: The HTTP request module cannot extract Set-Cookie fields, and the code execution module cannot call API interfaces. Cause: Header field extraction configuration is not enabled in custom parsing rules, or context passing for external API calls is not allowed during chunking. This prevents relevant fields from being identified.
How to confirm configurations are correct
- Upload a quarterly asset inventory sheet PDF. Check if the parsed text retains table structures for fields such as asset ID and user. Confirm that chunking does not break associated information.
- Call the API to create a knowledge base. Check if the returned result contains a parsing status field. Manually trigger parsing, then poll to confirm that the status updates with progress.
- Configure custom parsing rules. Upload an interface response message containing Set-Cookie fields. Verify that the corresponding fields are extracted in the parsed result.
- Upload a single summary file under 1000 MB. Confirm that the parsing task completes normally without timing out, and the number of chunks matches expectations.
The values provided are common starting points and should be measured against the reader's own samples.
Question material comes from public community discussions. Configuration values are common starting points and should be measured against your own samples. Verified on 2026-10-19.