What the Data for This Category Looks Like
Typical data for this category originates from production, quality assurance, quality control, and warehousing departments of pharmaceutical preparation enterprises. Daily documents include master production files, batch production records, batch packaging records, cleaning records, inspection records, deviation/CAPA/change documents, and batch release review forms. Data sources include approved formulation processes, material batch information, MES/electronic batch records, equipment and environmental monitoring data, LIMS systems, and quality management systems. Updates occur after each batch production, each key process step, and per established monitoring plans. Documents contain structured tables (such as material feeding lists and deviation record sheets) and unstructured process descriptions. Fields include specific units (such as mg, ml, and batch numbers) and operation node information required by GMP.
Constraints Imposed on Document Parsing and Chunking
Document types are diverse and have inconsistent structures. They include continuous text paragraphs and nested structured tables. Chunking must avoid splitting key process steps or table content. Otherwise, context accuracy required for Q&A will be compromised. Data update frequency is high. New documents are generated for every batch production. The parsing link must support batch uploads and fast processing requirements. Fields include specific units and compliance identifiers. Parsing must retain all this information completely. No omissions or alterations are permitted. GMP requires complete data traceability. Chunked content must retain full operational logic and associated data. Simple character-based splitting must not damage business context.
Configuration Settings
The following configurations are based on FastGPT v4.17.0.
| Configuration Item | Recommended Setting | Rationale |
|---|---|---|
chunkSize | 800–1200 characters | Process documents for chemical pharmaceutical preparations manufacturing include continuous process steps and associated tables. This range avoids splitting key operational logic, while ensuring context integrity of chunked content and optimizing vectorization processing efficiency |
chunkSplitter | Paragraph + table recognition | Adapts to mixed text paragraphs and structured tables such as material lists and deviation records in documents. Prevents content breaks caused by table splitting |
customPdfParse | Enabled | Filters redundant content such as fixed headers and footers in GMP documents. Retains core process and quality record information |
UPLOAD_FILE_MAX_SIZE | 1000 MB | Adapts to large batch production record files containing multi-page scans and attachments. Prevents upload interception |
PARSE_FILE_TIMEOUT_SECONDS | 600 seconds | Large batch record files take a long time to parse. This threshold covers standard parsing durations and avoids mid-parsing timeout failures |
chunkSettingMode | Manual mode | Allows separate adjustment of chunking parameters for different document types (such as batch production records, deviation reports). Adapts to diverse document structures |
Three Common Configuration Errors
- Uploading an Excel-format material list triggers an unsupported format prompt. The cause is failure to enable parsing adaptation for structured tables. The default parsing cannot recognize special table columns containing material batch numbers and weighing values in pharmaceutical documents.
- Using a third-party deployed model and uploading a file results in a parsing error. The cause is setting
PARSE_FILE_TIMEOUT_SECONDSto a value lower than 300 seconds. Parsing duration for large batch record files exceeds the threshold, leading to interruption. - Vectorized Q&A results include irrelevant header and footer content. The cause is failure to enable
customPdfParse. Non-core, repeatedly occurring information in documents is not filtered out.
How to Verify Correct Configuration
- Upload a typical batch production record document. Check if parsed text completely retains all table content and continuous process step paragraphs.
- View the file upload record. Confirm no
UPLOAD_FILE_MAX_SIZElimit prompt is triggered. - Randomly extract chunked text fragments. Confirm no non-core content such as headers or footers is included.
- Adjust the
chunkSizeparameter. Verify that chunks do not split key paragraphs containing feeding sequences.
The values provided are common starting points and should be measured against the reader's own samples.