Data Characteristics for This Category
Daily documents for this category include cell bank or seed lot records, culture and purification records, aseptic filling records, environmental monitoring data, media fill simulation results, sterility test reports, and batch release documents. Data sources cover cell bank and seed lot systems, process analysis platforms, clean room monitoring systems, MES, LIMS, and equipment integrity testing tools. Updates occur per batch, in real time, or per fixed monitoring plans. Most documents include fields such as batch number, process parameters, and test values. Units include degrees Celsius, milliliters, colony-forming units per plate, hours, and others.
Constraints Imposed on Document Parsing and Chunking
Mixed-source, heterogeneous document formats and varying update frequencies require parsing functions to support mixed scenarios of structured tables, semi-structured records, and plain-text reports. Process parameters with units and batch-bound information require chunking to retain contextual associations. This avoids splitting the correspondence between critical process steps and test results. High-frequency, real-time clean room monitoring data requires support for incremental parsing and resumable uploads. This prevents repeated parsing of full historical datasets. Strict quality control mandates that parsing results must not lose critical exception descriptions and associated evidence. Loss of this data compromises the accuracy of subsequent exception report drafting.
Configuration Settings
The following configurations are based on FastGPT v4.17.0.
| Configuration Item | Recommended Setting | Rationale |
|---|---|---|
chunkSize | 800–1200 characters | Documents contain large volumes of process parameters with units and batch-associated information. Chunking must retain the binding relationship between parameters and context, to avoid splitting critical process steps |
chunkSplitter | Split by paragraph + semantic boundary | Process documents and exception reports have clear operational step and problem description paragraphs. Semantic segmentation preserves information integrity |
customPdfParse | Enable structured parsing mode | Some documents originate from table-containing PDF exports from MES and LIMS. Structured parsing extracts table fields and associated data |
UPLOAD_FILE_MAX_SIZE | 1000 MB | Supports batch upload of large-volume documents such as batch release materials, and adapts to scenarios where multiple batch files are uploaded together |
PARSE_FILE_TIMEOUT_SECONDS | 600 seconds | Large aseptic filling records or environmental monitoring summary documents require longer parsing times. This setting avoids forced termination of parsing tasks mid-process |
chunkSettingMode | Manual configuration | Chunking parameters must be adjusted for different document types, to adapt to differences between structured records and semi-structured exception reports |
Three Common Configuration Errors
- Phenomenon: Large batch document uploads interrupt mid-process. Parsing cannot resume after restarting Docker. Cause: No reasonable
PARSE_FILE_TIMEOUT_SECONDSconfigured. Large summary documents time out during parsing and are forced to terminate. Resumable parsing mechanisms are not enabled. - Phenomenon: After document parsing, the model does not answer questions based on the content. Cause:
chunkSplitteris configured to split by fixed character count. This does not retain contextual associations between process parameters and batch numbers. Critical associated information is lost during chunking. - Phenomenon: No parsing feedback appears when uploading a file via the dialog box. No errors appear in the background. Cause:
customPdfParseis not enabled. Table fields in PDFs are not correctly extracted. This results in empty parsing results.
How to Verify Correct Configuration
- A single typical aseptic filling record PDF is uploaded. Parsed chunks are checked to confirm retention of associated content for filling temperature, time, and batch number.
- Batch release materials exceeding 500 MB are uploaded. The parsing process is confirmed to have no timeouts or interruptions. Task status shows completed.
PARSE_FILE_TIMEOUT_SECONDSis configured to 600 seconds. A single large environmental monitoring summary document is uploaded. Parsing duration is verified to meet expectations.chunkSettingModeis switched to manual configuration.chunkSizeis adjusted to 1000 characters. Chunk length is verified to match the set value.
The values provided are common starting points and should be evaluated against relevant sample datasets.