What the data for this category looks like
Reference only; verify against the target FastGPT release and deployment documentation.
What constraints these characteristics impose on the "document parsing and chunking" step
Mixed document formats from multiple sources, including structured tables, scanned PDFs, and nested attachments, place demands on the format adaptation capabilities of the parsing module. Daily updated batch documents require efficient parsing processes to avoid delays that disrupt compliance review progress. The most time-consuming verification steps in the relevant business require that core compliance content such as scoring methods and procurement method clauses retain contextual associations. This content must not be split arbitrarily. Some documents contain sensitive content. Parsing must prevent irrelevant content from being mistakenly included in chunks, while ensuring the integrity of original attachments.
How to Configure the Settings
| Configuration Item | Recommended Setting | Rationale |
|---|---|---|
chunkSize | 800–1200 characters | Adapts to the length of core paragraphs such as procurement plans and scoring rules in tendering and procurement documents, and avoids splitting key compliance clauses |
chunkSplitter | Split by paragraph + heading | Matches the heading hierarchy structure of bidding documents and bid evaluation records, and retains contextual associations |
customPdfParse | Enable image-text separation mode | Addresses scanned bid evaluation reports and supplier qualification PDFs, retains original images and extracts text, and avoids mistakenly converting image content into OCR text |
UPLOAD_FILE_MAX_SIZE | 100 MB | Adapts to batch uploads of procurement plans and contract files, and avoids upload failures caused by oversized files |
PARSE_FILE_TIMEOUT_SECONDS | 600 seconds | Reserves sufficient parsing time when processing large Excel procurement lists and multi-page scanned PDFs |
chunkSettingMode | Manual configuration | Adjusts chunking parameters for different document types (such as bidding documents and contracts) to adapt to multi-format documents in the tendering and procurement scenario |
Three Common Configuration Errors
- Phenomenon: Content in uploaded Excel procurement lists is missing or disordered in the vector index. Cause: Chunking is not adapted to table structures. Default character-based chunking causes cell data to be truncated across cells.
- Phenomenon: Scanned bidding document PDFs have no original images and have inaccurate OCR text after parsing. Cause: The image-text separation mode of
customPdfParseis not enabled. Default OCR processing overwrites original image content. - Phenomenon: Parsing times out when batch uploading multiple bid evaluation records. Cause:
PARSE_FILE_TIMEOUT_SECONDSis not adjusted. The default timeout duration is insufficient for processing large multi-page documents.
How to Confirm Correct Configuration
- Upload a single Excel procurement list, and check whether the chunking results retain complete single procurement item content without cross-cell truncation.
- Upload a scanned PDF bidding document, and check whether the parsing results include both original images and extracted text content.
- Upload batch contract files under 100 MB, and confirm that parsing tasks complete within 600 seconds without errors.
- Adjust the
chunkSizeparameter, and test whether scoring rule paragraphs are fully retained in a single chunk.
The values listed above are common starting points. They should be evaluated against the samples relevant to the specific deployment scenario.
Question material comes from public community discussions. Configuration values are common starting points and should be measured against your own samples. Verified on 2026-09-28.