What the Data for This Category Looks Like
Data sources for this category include approved formulation processes, MES/electronic batch records, LIMS, quality management systems, and others. Daily documents include master production documents, batch production records, batch packaging records, cleaning records, inspection records, deviation/CAPA/change documents, and batch release review forms. Data is updated per production batch, each critical process step, and as per monitoring plans. Most documents contain fixed fields such as material batch numbers, weighing values, process parameters, and equipment numbers, and use standard industry units including kilograms, milliliters, degrees Celsius.
What Constraints These Characteristics Impose on Document Parsing and Chunking
Documents in this category have fixed structures synchronized with production operations, with strict requirements for field associations. Chunking must retain the integrity of a single business logic unit, and avoid splitting content across batches or process steps. Data is updated frequently and tied to specific production batches, so chunking must enable quick retrieval by batch or process step. If chunks are too large, retrieval accuracy decreases; if too small, field associations within a single complete record may be broken. Additionally, under compliance requirements, document parsing must not omit key fields, as this will affect the accuracy of subsequent cross-review.
Configuration Settings
The following configuration settings are based on the logic of FastGPT v4.17.0.
| Configuration Item | Recommended Setting | Rationale |
|---|---|---|
chunkSize | 800–1200 characters | Matches the character volume of a single process step or single inspection record, avoiding splitting complete business units |
chunkSplitter | By paragraph + fixed character boundary | Adapts to the fixed format of documents in this category, retaining field associations and semantic integrity within paragraphs |
customPdfParse | Enabled | Extracts embedded table data from PDFs, preventing structured tables in batch records and inspection reports from being incorrectly split |
PARSE_FILE_TIMEOUT_SECONDS | 600 seconds | Adapts to the parsing duration of large multi-batch archived documents, avoiding parsing failures due to timeout |
UPLOAD_FILE_MAX_SIZE | 1000 MB | Supports uploading archived document collections across multiple batches, meeting enterprise bulk upload needs |
chunkSettingMode | Manual configuration | Adjust chunking parameters to adapt to different document types such as batch records and deviation reports, to match business scenarios |
Three Common Misconfigurations
- Phenomenon: When batch records or inspection records in Excel format are uploaded, only the first two columns of data are identified. Cause: Custom table parsing rules are not configured, and FastGPT defaults to recognizing only two-column question-and-answer format datasets.
- Phenomenon: A timeout error occurs when a single electronic batch record PDF with more than 500 pages is parsed. Cause: The
PARSE_FILE_TIMEOUT_SECONDSparameter is not adjusted, and the default timeout duration is insufficient to complete parsing of large-volume documents. - Phenomenon: When searching for weighing data for a specific batch, only some associated fields are returned. Cause: The
chunkSizesetting is too small, splitting a single weighing record into multiple text blocks and losing the association information between the batch number and the weighing value.
How to Verify Correct Configuration
A single batch record PDF is uploaded, then the parsed text block list is reviewed to confirm that structured table fields are fully extracted and no cross-process step splitting has occurred. The background logs of the parsing task are checked to confirm that no timeout errors appear, and the effectiveness of the PARSE_FILE_TIMEOUT_SECONDS parameter is verified. A specific material batch number is entered for retrieval, confirming that complete document blocks containing that batch are matched, with no missing fields or broken associations. An Excel inspection record with multiple columns of data is uploaded, confirming that all column contents are correctly identified and chunked.
The values provided are common starting points and should be measured against the reader's own samples.