What this category of data looks like
The main entities of this category of data are disease control institutions, medical institutions, and health management departments. Data sources include the Chinese Disease Prevention and Control Information System, medical institution HIS systems, and laboratory testing systems. Daily documents include infectious disease report cards, epidemic analysis reports, case correction records, clustered epidemic reports, and epidemiological investigation reports, updated daily. Most document structures include fixed statutory fields such as case unique identifier, diagnosis time, reporting time limit, and case classification. Field units are mostly dates, number of people, case classification codes, etc. Document formats exported from different systems vary, and some PDF files have non-standard headers and footers.
What constraints do these characteristics impose on the document parsing and chunking link
The daily updated business rhythm requires the parsing process to have stable batch processing capabilities to avoid delays in epidemic monitoring and reporting. The fixed statutory field document structure requires that chunking must retain the integrity of key information, and must not truncate core reporting fields such as case number and diagnosis time. Non-standard format documents from multiple sources require parsing tools to adapt to different export styles to avoid field omission or misidentification. Statutory reporting rules require parsing results to accurately match reporting requirements. Chunking errors may lead to underreporting or delayed reporting risks in subsequent review links. At the same time, the content length of different document types varies greatly, and chunking parameters need to be flexibly adjusted to adapt to different scenarios.
Reference only; verify against the target FastGPT release and deployment documentation.
How to set configurations
| Configuration Item | Recommended Value | Rationale |
|---|---|---|
chunkSize | 800–1200 characters | Adapts to the conventional content length of infectious disease report cards and epidemic analysis reports, avoiding truncation of core statutory fields |
chunkSplitter | By paragraph + statutory field boundary | Matches the fixed field structure of documents, ensuring that key information such as case number and diagnosis time is retained after chunking |
PARSE_FILE_TIMEOUT_SECONDS | 600 seconds | Covers the complete parsing time of large epidemic analysis reports, avoiding timeout errors during batch processing |
customPdfParse | Enabled | Adapts to non-standard PDF formats exported by disease control systems, correcting parsing abnormalities caused by headers and footers |
UPLOAD_FILE_MAX_SIZE | 1000 MB | Supports the upload requirement of epidemic summary files for batch reporting |
chunkSettingMode | Manual mode | Adjust chunking parameters separately for different document types (report cards, epidemiological investigation reports) |
Three common mistakes
- An error "Cannot read properties of undefined (reading 'xxx')" occurs when using
customPdfParseto parse PDFs exported by disease control systems. The cause is failure to configure custom parsing rules to adapt to the document's non-standard headers and footers. - Duplicate or missing key fields appear after chunking. The cause is failure to set
chunkSplitteraccording to statutory field boundaries, only using the default character-based chunking logic. - Parsing timeout is triggered when batch uploading multiple large epidemic analysis reports. The cause is that the value of
PARSE_FILE_TIMEOUT_SECONDSis less than the actual parsing time.
How to confirm the configuration is correct
- Upload a single standard-format infectious disease report card PDF, check that the parsed chunked content includes complete statutory fields such as case unique identifier and diagnosis date, with no truncation or omission.
- Batch upload 3 disease control documents of different types (report card, epidemiological investigation report, epidemic analysis report), check that no timeout errors related to
PARSE_FILE_TIMEOUT_SECONDSare triggered. - Adjust the
chunkSizeparameter, check that the chunked content covers complete single-paragraph report information, with no cross-paragraph splicing caused by overly large chunks or field splitting caused by overly small chunks. - After enabling
customPdfParse, check that non-standard PDF parsing results do not have headers and footers mistakenly identified as body content.
The values given are common starting points and should be measured against the reader's own samples.
Question material comes from public community discussions. Configuration values are common starting points and should be measured against your own samples. Verified on 2026-09-28.