What Data for This Category Looks Like
This category's data comes from approved prescription processes, material batch ledgers, MES electronic batch records, LIMS laboratory management systems, and quality management systems. Daily documents include master production files, batch production records, batch packaging records, cleaning records, inspection records, deviation/CAPA/change documents, and batch release review forms. Data is updated per production batch, each key operation step, and fixed monitoring plans. Document fields include material batch numbers, process execution parameters, test values, deviation rectification records, and more. Units involved include mass, volume, temperature, and others.
What Constraints These Characteristics Impose on Document Parsing and Chunking
Exclusive documents for each batch mix with general production files. Chunking must avoid splicing content across batches. Split units based on the production batch and operation link. Documents include GMP compliance records. Chunking must retain logically complete units such as deviation rectification and batch release review. Do not split key approval nodes. Fields include precise values and unique batch numbers. Parsing must retain field relevance. Do not compromise data integrity. Frequently updated documents create parsing pressure. Limit single chunk length and parsing timeout to prevent timeout interruptions. Records with inspection images require parsing of associated text descriptions. Do not retain only empty content or truncate key information.
Configuration Settings
This configuration is based on FastGPT v4.17.0. Specific settings are listed below:
| Configuration Item | Recommended Setting | Rationale |
|---|---|---|
chunkSize | 800–1200 characters | Matches the length of single-batch document units, avoids splitting key process or approval records |
chunkSplitter | Split by paragraph + heading | Retains the logical structure of GMP documents, avoids breaking complete units of batch records or deviation records |
PARSE_FILE_TIMEOUT_SECONDS | 600 seconds | Accommodates large documents with multiple batch records, avoids timeout errors from excessive parsing time |
UPLOAD_FILE_MAX_SIZE | 1000 MB | Stores complete production and inspection document collections for a single batch |
customPdfParse | Enable text-first mode | Prioritizes extracting structured text from documents, retains the relevance of batch numbers and value fields |
chunkSettingMode | Manual configuration | Customizes chunking rules for this category's compliance document structure, does not use generic templates |
Three Common Misconfigurations
- Setting
chunkSizetoo small truncates Markdown table content. The...[hide 38432 char]prompt appears in the interface. This occurs because long tables in this category's documents, such as batch record material lists, have unmatched length. Chunking splits the table's complete structure. - Uploading a docx file with inspection images returns the
Invalid image fileerror. This occurs because the image text association mode ofcustomPdfParseis not enabled. Only image links are extracted, without processing embedded text. This leads to parsing failure. - Parsing large batch record collections triggers the
slow operation xxxxmslog error. This occurs because thePARSE_FILE_TIMEOUT_SECONDSparameter is not adjusted. The default timeout duration is insufficient to complete parsing of multi-batch documents.
How to Verify Successful Configuration
- Upload complete batch record documents for a single batch. Check that parsed chunks include complete material lists and process parameter paragraphs. No cross-batch content splicing is present.
- Upload a docx document with embedded inspection images. Confirm that the parsed result retains text descriptions corresponding to the images. No
Invalid image fileerrors appear. - Upload a multi-batch document collection under 1000 MB. Check that the parsing process has no timeout logs. The number of chunks matches expectations.
- Verify the Markdown table parsing result. Confirm that no content is truncated. Complete table content loads normally.
The values provided here are common starting points, and should be evaluated against the reader's own samples.