What the data for this category looks like
The data for this category primarily comes from audio files, annotation results exported from annotation platforms, custom term lists, and project spot-check samples. Commonly processed documents include transcription specification documents, pending audio processing lists, quality inspection reports, and speaker annotation tables. Data updates follow a daily or single-batch project schedule. Fields vary across different projects. Common fields include audio duration, annotator identifiers, quality inspection scores, and speaker classification tags. There are no unified mandatory format requirements.
What constraints do these characteristics impose on the document parsing and chunking link
Documents associated with audio must bind both text and audio metadata simultaneously. Chunking operations must not break the association between text and corresponding audio segments. They must retain the correspondence between metadata and text. Structured documents such as transcription specifications and quality inspection reports have clear entry hierarchies. Chunking must retain these hierarchies to avoid splitting core rule entries. Daily batch-updated data increases parsing concurrency pressure. Reasonable timeout and sharding strategies must be adopted. Custom term list content must be fully retained in chunks. This prevents errors in subsequent annotation verification caused by splitting proprietary terms. The strategy must also adapt to the update schedule of custom term lists for different projects.
How to set configurations
These configurations are written for FastGPT v4.17.0.
| Configuration Item | Recommended Setting | Rationale |
|---|---|---|
chunkSize | 800–1200 characters | Adapts to the typical length of single transcription segments and rule entries in speech annotation documents, avoiding splitting of custom terms |
chunkSplitter | Split by paragraph + semantic segment | Retains semantic integrity of transcription content and specification documents, and meets structured entry splitting requirements |
customPdfParse | Enabled | Preserves original layout and hierarchical structure for PDF documents such as transcription specifications and quality inspection reports |
UPLOAD_FILE_MAX_SIZE | 1000 MB | Meets upload requirements for batch audio files and associated documents, preventing upload failures due to oversized files |
PARSE_FILE_TIMEOUT_SECONDS | 600 seconds | Addresses time spent on metadata association and audio verification during batch parsing, preventing parsing interruptions due to timeout |
chunkSettingMode | Manual configuration | Flexibly adjusts chunking parameters to adapt to custom term lists and document structures for different batch speech annotation projects |
Three common errors
- Phenomenon: After uploading a PDF document associated with audio to the knowledge base, vectorization processing takes too long. Cause: The
chunkSizeparameter is not adjusted to fit long audio transcription text. Oversized chunks lead to excessive per-batch processing load. - Phenomenon: When directly asking text questions in the latest main branch code, the document parsing process is triggered. Cause:
chunkSettingModeis not configured correctly. General questions are included in the document parsing link, directly calling the question-and-answer interface. - Phenomenon: After local deployment, uploading documents results in empty data processing results and no parsing completion prompt. Cause: A reasonable
PARSE_FILE_TIMEOUT_SECONDSvalue is not set. Batch parsing returns empty results before completion due to timeout, and the parsing status push interface is not configured.
How to confirm configurations are properly set
- Upload a single transcription specification PDF document, check the parsed chunk content, and confirm that the original entry hierarchy is retained.
- Upload a single long audio transcription text, verify that the chunk length falls within the range set by
chunkSize. - Test batch upload of 10 associated documents, confirm that the parsing process does not experience timeout interruptions, and that generated chunk data is complete.
- Trigger a direct text question, confirm that the document parsing process is not triggered, and verify that the
chunkSettingModeconfiguration is effective.
The values provided are common starting points and should be measured against the reader's own samples.
Question material comes from public community discussions. Configuration values are common starting points and should be measured against your own samples. Verified on 2026-10-19.