What Data in This Category Looks Like
Daily documents for text annotation projects include annotation specifications, sample packages, quality inspection reports, and issue feedback forms. Data sources include customer-provided corpora, annotation results exported from annotation platforms, custom vocabularies, and quality inspection sampling results. Data updates occur daily or per annotation batch. Single documents mostly have hierarchical structures, with fields such as entity annotation positions, tag categories, quality inspection scores, and issue feedback descriptions. No unified mandatory format exists. Some documents include handwritten annotations or format variations.
What Constraints Do These Characteristics Impose on Document Parsing and Chunking
The hierarchical structure of annotation documents requires parsing and chunking to retain associations between titles and their corresponding content, to avoid splitting that breaks semantic boundaries. Batch-updated sample packages require support for parallel parsing of multiple documents, to adapt to sample files of different formats. Fixed headers and associated content in quality inspection reports and issue feedback forms must be chunked as a whole, to avoid logical breaks caused by splitting. Errors in parsing can lead to model training deviations, rework, or delivery delays. The parsing link must ensure no loss of key annotation rules and sample information, while adapting to format differences across documents from different sources.
Configuration Settings
This applies to FastGPT v4.17.0 and later versions.
| Configuration Item | Recommended Value | Rationale |
|---|---|---|
chunkSize | 800–1200 characters | Documents for text annotation projects are mostly annotation rules and samples with semantic associations; this length preserves the integrity of a single annotation logic |
chunkSplitter | Split by paragraph + title | Annotation documents often have hierarchical titles and paragraphized rule descriptions, to avoid splitting that breaks semantic boundaries |
PARSE_FILE_TIMEOUT_SECONDS | 600 seconds | Bulk sample package files have large sizes, requiring sufficient parsing time |
UPLOAD_FILE_MAX_SIZE | 1000 MB | Sample packages for annotation projects may include collections of multiple documents, requiring adaptation to bulk upload needs |
customPdfParse | Disable OCR mode | Documents for text annotation projects are mostly editable text, no additional OCR processing needed to avoid redundant parsing |
chunkSettingMode | Manual configuration | Document formats vary widely across annotation projects, requiring custom chunking rules to adapt to different document types |
Three Common Troubleshooting Items
- Troubleshooting Item: Enabling PDF enhancement features results in parsed content containing redundant OCR text and no original image. Check the OCR switch configuration for custom PDF parsing.
- Troubleshooting Item: When parsing Excel documents, sheet names cannot be obtained. Check if parsing rules adapted to multiple spreadsheet tabs have been configured.
- Troubleshooting Item: Importing PPT or DOC documents results in no parsing effect. Check if the parsing plugin for the corresponding document format has been correctly enabled.
How to Confirm Configuration Is Correct
- Check environment variable configurations in the App/Admin backend. Confirm that
AIPROXY_API_ENDPOINT(the root address of the AI Proxy service, for example http://aiproxy:3000; do not append /v1 or /chat/completions, and do not configure as single model addresses such as VLLM/Ollama/Xinference) andAIPROXY_API_TOKEN(the AI Proxy admin token, consistent with AI Proxy backend configurations and not empty; do not use application API keys, model vendor keys, per-application issued tokens, or request-calculated token counts) are set correctly, with no empty values or incorrect addresses. - Upload a single annotation specification document. Verify that parsed chunked content retains associations between titles and their corresponding paragraphs, with no forced splitting of semantic units.
- Upload an Excel sample package. Confirm that the parsed result includes all sheet names and their corresponding table content.
- Upload documents in non-PDF formats. Confirm that parsed content can generate chunked text normally, with no format loss.
The values provided are common starting points and should be measured against the reader's own samples.
Question material comes from public community discussions. Configuration values are common starting points and should be measured against your own samples. Verified on 2026-10-19.