Data Characteristics for This Category
Cosmetics manufacturing enterprises generate daily documents. These include formula process sheets, weighing and feeding records, emulsification and filling records, cleaning and clearance records, semi-finished and finished product inspection reports, and label review documents for skin care, hair care, and makeup product lines. Data comes from registered and filed formulas, raw material safety materials, production equipment, ERP/MES systems, laboratory test results, and packaging material version documents. Documents receive updates with each production batch or line change. Single documents contain fixed fields: raw material name and dosage, process temperature parameters, test results, batch number, and packaging material specification. Some documents include complex nested tables and multiple linked explanatory sections.
Constraints on Document Parsing and Chunking
Documents update dynamically with each production batch. Parsing and chunking must adapt to frequently changing field content. This avoids breaking field associations across batches. Documents contain large amounts of hierarchical professional content. Examples include corresponding explanations for formula raw materials and restricted substances. Proper chunking retains the binding relationship between titles and content. Without this, field misalignment occurs during retrieval. The complex structure of raw material lists and inspection tables easily splits under default parsing logic. Targeted configuration is required to preserve formatting. Differences in document formats across multiple sources exist. Examples include PDF process sheets and Excel feeding records. Parsing logic must support multiple file types to prevent formatting loss.
Configuration Settings
| Configuration Item | Recommended Setting | Rationale |
|---|---|---|
chunkSize | 800–1200 characters | Adapts to the paragraph length of raw material lists and process descriptions in cosmetics manufacturing documents, avoiding splitting that breaks the association between raw materials and their dosages |
chunkSplitter | Split by paragraph + title | Matches the title-content structure of process sheets and inspection records, retaining field hierarchy and associated logic |
customPdfParse | Enabled | Adapts to PDF-format formula process sheets and label review documents, preserving table and layout structures to avoid formatting chaos caused by default parsing |
PARSE_FILE_TIMEOUT_SECONDS | 300–600 seconds | Addresses large formula documents with complex nested tables, preventing timeout errors during the parsing process |
UPLOAD_FILE_MAX_SIZE | 100 MB | Adapts to record documents from batch production runs, preventing upload failures caused by excessive file size |
chunkSettingMode | Manual configuration | Adjusts chunking parameters for different document types (formula sheets, feeding records, inspection reports) to match their respective field granularity |
Three Common Configuration Errors
- Symptom: After uploading a PDF formula process sheet, only plain text appears in the knowledge base. Original table content scatters into disjoint paragraphs. Cause: The
customPdfParseconfiguration is not enabled. Default parsing logic does not preserve PDF table structures. - Symptom: HTTP requests calling the document parsing service return a 504 timeout status code. Cause: The
PARSE_FILE_TIMEOUT_SECONDSsetting is lower than the actual parsing time of complex formula documents. This interrupts the parsing process. - Symptom: The same raw material name in chunking results splits into two separate blocks. Full association with the corresponding dosage becomes impossible. Cause: The
chunkSizesetting is too small. This truncates the complete paragraph of raw materials and their dosages during splitting.
How to Verify Proper Configuration
- Upload a single standard PDF formula process sheet. Check if the parsed document retains the complete raw material list and process parameter table structure. Ensure no disjoint or misformatted text appears.
- Adjust
chunkSizeto different range values. Upload the same feeding record document. Compare whether raw material names in the chunking results fully retain within a single block. - Upload a batch record compressed file larger than 50 MB. Confirm the upload and parsing processes do not trigger file size limit or timeout errors.
- Check the parsing logs in FastGPT v4.17.0. Confirm the
customPdfParseconfiguration is active. Ensure no formatting loss prompts from default parsing appear.
The values provided are common starting points and should be evaluated against the reader's own samples.