What Data for This Category Looks Like
Typical data for this category comes from formulation and process libraries, raw material batches, reactor DCS systems, weighing systems, LIMS, and warehouse management systems. Update frequency is per batch or each key production step. Daily generated documents include batch production instructions, feeding verification sheets, reaction process records, cleaning records, inspection reports, deviation and change orders. Documents contain professional fields such as feeding volume (kg, L), reaction temperature (℃), batch number, cleaning time, and others. Due to production switching between multiple product varieties by batch, a single batch document is linked to multiple different types of business records.
What Constraints Do These Characteristics Impose on Document Parsing and Chunking
This category has characteristics including multiple batch production switches, dense professional fields, and step-by-step updates. These impose multiple constraints on document parsing and chunking. Parsing must be limited to the single batch document currently uploaded, to avoid mixing in historical data. Professional fields and contextual associations of business steps must be retained, to avoid breaking the logical integrity of processes such as feeding and reactions after chunking. The system must adapt to hierarchical structures of multiple document types, and support OCR recognition of scanned documents to cover paper record parsing needs. Due to high data update frequency, parsing duration must be controlled to match production pace.
How to Configure Settings
The following configurations are adapted for FastGPT v4.17.0.
| Configuration Item | Recommended Setting | Rationale |
|---|---|---|
chunkSize | 800–1200 characters | Documents for batch synthesis in fine chemicals contain complete business units such as single-step feeding and reactions. This length preserves complete context for a single business unit |
chunkSplitter | Split by paragraph + title, set chunk overlap to 100–150 characters | Documents such as batch production instructions and reaction records have clear title and paragraph hierarchies. Splitting this way preserves the relevance of business logic, and the overlap setting prevents key fields from being split apart |
customPdfParse | Enabled | Structured professional fields such as batch numbers and feeding volumes must be extracted from documents, and custom parsing adapts to the document formats of this category |
PARSE_FILE_TIMEOUT_SECONDS | 600 seconds | Single batch documents may contain multiple pages of reaction process records, reserving sufficient parsing time to avoid timeout interruptions |
UPLOAD_FILE_MAX_SIZE | 500 MB | Supports batch upload of multiple associated documents for a single batch, such as feeding sheets, inspection reports, and cleaning records |
chunkSettingMode | Manual configuration | Adjustment of chunking rules is required for different document types such as cleaning records and deviation orders, and manual mode provides greater flexibility for business needs |
Three Common Configuration Mistakes
- Parsing results include historical documents not from the current upload. The symptom is that business records from other batches appear during retrieval. The cause is failure to configure a single-document filtering rule via
customPdfParse, and the system loads all associated historical data by default. - Garbled Chinese characters appear in OCR results after uploading scanned documents. The symptom is garbled professional fields in parsed scanned feeding verification sheets. The cause is failure to enable OCR configuration adapted for professional terminology, or failure to specify the correct Chinese language pack.
- Asking a question about a single PDF document returns irrelevant content. The symptom is that after asking a question about a batch production instruction for a specific batch, records from other batches are returned. The cause is that
chunkSizedoes not match the step length of documents for this category, leading to chunking that breaks the contextual association of the single document.
How to Verify Correct Configuration
- Upload a single batch document, and check that the parsing result only contains content from that document, with no historical data from other batches.
- Upload a PDF document with scanned images, and verify that the OCR-recognized professional fields have no garbled characters.
- After configuring
chunkSize, split a single batch production instruction, and confirm that each chunk contains a complete single-step feeding or reaction record. - Upload multiple associated documents for a single batch, and verify that the
UPLOAD_FILE_MAX_SIZElimit does not trigger an upload error.
The values provided are common starting points and must be measured against the reader's own samples.