What the data for this category looks like
Data related to pharmaceutical preparations production comes from approved prescription processes, MES electronic batch records, LIMS quality management systems, and equipment and environmental monitoring data. Daily documents include master production documents, batch production/packaging records, cleaning records, inspection records, deviation CAPA, and batch release review forms. Data is updated per production batch, key process steps, and monitoring plans. Document fields include material batch numbers, process parameter values, test results, and deviation descriptions. Units align with the document type: milligrams and degrees Celsius for quality parameters, and date-time stamps for operation records.
Constraints imposed on document parsing and chunking
Documents for this category are generated per production batch. Single-batch records include fixed compliance fields, and data integrity requirements are strict. Document parsing and chunking must not break the contextual association of a single batch. Different document types have notable structural differences: batch records use a mixed format of chaptered tables, while inspection reports are standardized data forms. The parsing engine must adapt to multiple formats. Data updates occur frequently, and archived documents uploaded in batches have large volumes. Parsing and chunking must balance efficiency and integrity, and avoid missing key compliance fields.
Configuration Settings
| Configuration Item | Recommended Value | Rationale |
|---|---|---|
chunkSize | 800–1200 characters | Aligns with the length of single-segment process descriptions in pharmaceutical batch records, and avoids splitting continuous process steps or cross-page tables |
chunkSplitter | Split by title + paragraph | Matches the fixed chapter structure of batch records, retains contextual integrity for each chapter, and supports subsequent traceability of process deviations |
customPdfParse | Enabled | Adapts to pharmaceutical document formats with compliant headers, footers, and embedded tables, and accurately extracts core fields such as batch numbers and test data |
PARSE_FILE_TIMEOUT_SECONDS | 300–600 seconds | Meets parsing time requirements for large-volume batch records and batch uploads of multiple documents, and prevents parsing interruptions caused by timeouts |
UPLOAD_FILE_MAX_SIZE | 1000 MB | Supports enterprise archived full-batch production record files, and enables batch upload of compliance documents |
chunkSettingMode | Manual mode | Allows adjustment of chunking parameters for different document types such as batch records and inspection reports, and adapts to diverse document structures |
Three Common Mistakes
- Symptom: No activity is observed on the parsing node after file upload. Cause:
customPdfParseis not enabled to adapt to compliant PDF formats, so the parsing engine cannot recognize the document structure. - Symptom: Only partial data is captured in database tables. Cause:
chunkSizeis set too small, which truncates cross-page table rows during splitting, leading to incomplete key fields. - Symptom: Model output Markdown tables are truncated and cannot render properly. Cause:
PARSE_FILE_TIMEOUT_SECONDSis not adjusted, and parsing timeout causes the second half of long tables to not be fully extracted.
How to Confirm Configurations Are Set Correctly
- A single standard batch record PDF is uploaded, and the parsed text is viewed in FastGPT v4.17.0 to confirm that core fields such as batch numbers and process parameters are fully extracted.
- 3 to 5 documents of the same type are uploaded in batch, and all parsing tasks are confirmed to start and complete normally with no abnormal errors.
- The
chunkSizeparameter is adjusted to different ranges, and chunking results are compared to ensure that continuous process steps or cross-page tables are not split. - An archived document larger than 500 MB is uploaded, and the upload and parsing processes are confirmed to complete normally with no file size limit prompts.
The values provided are common starting points and should be measured against the reader's own samples.