What this category of data looks like
This category of data comes from China's Disease Prevention and Control Information System, medical institution HIS systems, infectious disease reporting systems, and laboratory testing systems. The update frequency is daily. Daily documents include infectious disease reporting cards, epidemic analysis reports, case correction records, clustered epidemic reports, and epidemiological investigation reports. These include both table documents with fixed headers and long-text analysis reports with hundreds of thousands of Chinese characters. Document fields include legally required reporting-related content such as case number, diagnosis time, reporting unit, and case classification. Supported formats cover the two mainstream office document types: Excel and Word.
Constraints imposed on the document parsing and chunking link
The data characteristics of this category create multiple constraints for the document parsing and chunking process. First, documents imported from multiple source systems have mixed formats. Some are Excel tables with over 15,000 rows, others are long-text reports with hundreds of thousands of Chinese characters. This requires chunking rules to balance the integrity of structured tables and long texts. Second, the daily update frequency demands an efficient parsing process, to avoid overly long wait times. Third, the rigor of legally required reporting means chunking must not break the associated information of cases and correction records. Otherwise, this will impact subsequent verification and early warning work. In addition, the need to process large-volume documents requires setting sufficient upload and parsing resource thresholds to ensure all data is parsed correctly.
How to configure the settings
| Configuration Item | Recommended Setting | Rationale |
|---|---|---|
chunkSize | 800–1200 characters | Adapts to the per-chunk business information volume of this category, avoids excessive context resource consumption caused by overly large chunks, and preserves complete associations between cases and correction records |
chunkSplitter | Split by paragraph + table boundaries | Adapts to document structures that include both long-text epidemic analysis reports and tabular reporting cards, prevents splitting from destroying structured fields of tables |
PARSE_FILE_TIMEOUT_SECONDS | 600 seconds | Reserves sufficient parsing time when processing 100,000-character epidemiological investigation reports or 15,000-row case data, prevents parsing failures caused by timeouts |
UPLOAD_FILE_MAX_SIZE | 1000 MB | Adapts to upload requirements for single large-volume epidemic analysis reports or multi-batch reported files |
chunkSettingMode | Manual mode | For documents with fixed business structures in this category, allows custom chunking rules to adapt to different types of reporting documents |
customPdfParse | Enable structured table parsing | For table documents such as infectious disease reporting cards with fixed headers, extracts structured fields and avoids chaotic parsed text |
Three Common Mistakes
- Phenomenon: Incomplete command prompts are returned when parsing Excel documents. Cause: Structured table parsing is not enabled, and
chunkSizeis set too large, causing single-chunk data to exceed the model's processing range and trigger default prompts. - Phenomenon: Document parsing tasks time out without completion. Cause: The value set for
PARSE_FILE_TIMEOUT_SECONDSis lower than 300 seconds, making it impossible to complete the complete parsing process when processing 100,000-character epidemiological investigation reports or 15,000-row case data. - Phenomenon: Associated information of case correction records is split into different chunks. Cause:
chunkSplitteronly splits by character boundaries, and does not preserve paragraph and table boundaries, leading to fragmentation of the context of the same correction record.
How to Confirm the Configuration Is Correct
Reference only; verify against the target FastGPT release and deployment documentation.
- Upload an Excel infectious disease reporting card with 15,000 rows, and check whether the complete header and structured fields of each row are retained after parsing, with no missing or chaotic content.
- View the chunking results, confirm that each case correction record is not split into different chunks, and the character count per chunk falls within the range set by
chunkSize. - Initiate a retrieval test that includes the case number, confirm that the system can retrieve complete associated case information from the corresponding chunks, with no context loss.
The values given are common starting points and should be measured against the reader's own samples.
Question material comes from public community discussions. Configuration values are common starting points and should be measured against your own samples. Verified on 2026-09-28.