What This Category of Data Looks Like
Data sources include approved formulation processes, material batches, MES/electronic batch records, equipment and environmental monitoring data, LIMS systems, and quality management systems. Updates are synchronized per a fixed monitoring schedule after each production batch and each key process step. Daily documents include master production files, batch production records, batch packaging records, cleaning records, inspection records, deviation/CAPA/change documents, and batch release review forms. Document fields include material batch numbers, process parameter values, test results, deviation descriptions, release approval opinions, and other relevant fields. Units include professional measurement identifiers such as mass, volume, temperature, and time.
Constraints for Document Parsing and Chunking
Data is generated independently per production batch and must comply with GMP requirements. Chunking operations must not split content across batches, to prevent mixing of process and test data from different batches. Documents contain numerous fields with precise numerical values and unique identifiers. Parsing must retain full field integrity, and key information must not be split arbitrarily. Single-batch record document lengths vary significantly, with some including multi-page scans or electronic forms. Parsing timeout settings must be adjusted to handle long documents. Documents must also be synchronously linked to operation times and approval nodes. Chunking must retain contextual connections to ensure full process traceability during audits.
Configuration Settings
These configurations are adapted for FastGPT v4.17.0:
| Configuration Item | Recommended Setting | Rationale |
|---|---|---|
chunkSize | 800–1200 characters | Adapts to the length of single-segment key information in pharmaceutical dosage form batch records, avoiding splitting of process steps or unique batch number fields |
chunkSplitter | Split by paragraph + heading | Matches the structure of GMP chapters and batch record sections, retaining hierarchical associations for compliance documents |
customPdfParse | Enabled | Retains field values and stamp metadata in PDF forms, meeting archiving requirements for GMP records |
PARSE_FILE_TIMEOUT_SECONDS | 600 seconds | Adapts to long batch record documents containing multi-page scans, avoiding task failure due to parsing timeout |
UPLOAD_FILE_MAX_SIZE | 1000 MB | Adapts to batch record documents containing multi-page scans and high-definition images |
chunkSettingMode | Manual configuration | Allows adjustment of chunking rules for different types of quality audit documents, adapting to the different structures of deviation reports and release review forms |
Three Common Mistakes
- After calling the knowledge base creation API, parsing status cannot be obtained, and progress can only be viewed via the interface. The cause is that API status push configuration is not enabled, and the corresponding status query interface is not called.
- After uploading a batch record PDF containing inspection report images, the AI output does not include text content associated with the images. The cause is that image recognition function is not enabled, or image metadata is discarded in chunking configuration.
- Parsing the Set-Cookie field of HTTP interface requests fails, causing external documents that rely on the interface to fail to load correctly. The cause is that HTTP header processing configuration for custom PDF parsing is not enabled, so session identifier fields cannot be recognized.
How to Verify Correct Configuration
- Initiate a knowledge base creation API request, confirm that the returned result includes parsing-related status identification fields.
- Upload a batch record document containing inspection report images, verify that the generated knowledge base content associates image information with corresponding text.
- View parsing task logs, confirm that the Set-Cookie field of HTTP requests is parsed normally, with no error records.
- Check parameter settings on the configuration page to ensure they match the preset chunking rules, confirming that parameters such as
chunkSizeandchunkSplitteralign with the document structure of the current category.
The values provided are common starting points and should be measured against the reader's own samples.