Data characteristics for this category
Data for this category comes from daily business documents of skincare, haircare, makeup, and special cosmetics manufacturers. These documents include formulation process sheets, weighing and feeding records, emulsification and filling records, cleaning and site clearance records, semi-finished and finished product inspection reports, and label review documents.
Data sources include registered and archived formulations, raw material safety materials, production equipment, ERP/MES systems, laboratory data, and packaging material version documents. Updates trigger per production batch or each line change.
Documents include fields such as formulation raw material details, process parameters, inspection indicators, and packaging material claim information. Some documents have quantified data with units like temperature, weight, and concentration.
Constraints on document parsing and chunking
This category’s multi-source heterogeneous document structures, high-frequency update cycles, and strict compliance requirements create multiple constraints for document parsing and chunking.
Different document types — such as structured feeding records and unstructured exception reports — have large format differences. Parsing tools must adapt to diverse layouts.
High-frequency updates per batch or line change require parsing processes to have low latency. This avoids disrupting production rhythms.
Documents subject to filing requirements, such as formulations and labels, need accurate extraction of fields matching archived information. This ensures consistency of parsing results.
Quantified process parameters with units must be accurately identified, and their original units retained. This prevents data deviations in subsequent processing.
How to configure settings
This configuration uses the knowledge base parsing module of FastGPT v4.17.0, adjusted for the document characteristics of the cosmetics manufacturing scenario:
| Configuration Item | Recommended Setting | Rationale |
|---|---|---|
chunkSize | 800–1200 characters | Adapts to the mixed structure of cosmetics manufacturing documents, which include both long paragraph process descriptions and short field parameters. Prevents single chunks from being too long to cause retrieval fragmentation, or too short to retain complete semantic meaning |
chunkSplitter | Split by semantic meaning + paragraphs | Matches the content logic of process documents divided by process steps and operating procedures. Ensures each chunk after splitting has independent business significance |
customPdfParse | Enable MinerU enhanced parsing | Adapts to PDF documents such as formulation sheets and inspection reports. Retains text and layout information from tables and embedded images, and solves table content loss issues with standard parsing tools |
PARSE_FILE_TIMEOUT_SECONDS | 600 seconds | Addresses the large size and complex layout of single batch production record documents. Prevents task failure from parsing timeouts |
chunkSettingMode | Custom mode | Allows adjustment of chunking parameters for different document types, such as feeding records vs label review documents. Adapts to the diverse document formats within this category |
Three common configuration mistakes
- Symptom: After uploading a PDF document containing external image URLs, the image-associated text in the parsing result is empty or cannot be extracted. Cause: The
customPdfParseconfiguration is not enabled. The default parsing tool cannot pull associated content from external image URLs. - Symptom: After deploying FastGPT locally and enabling the enhanced PDF parsing function, parsing tasks continue to fail and return a timeout status code. Cause: The API port mapping for MinerU is not correctly configured. FastGPT cannot connect to the locally deployed MinerU service.
- Symptom: After parsing table documents such as weighing and feeding records, multiple columns of data are not correctly split into independent semantic chunks. Cause: The
chunkSplitterconfiguration is not adjusted for table data. The default splitting logic does not adapt to table row and column structures.
How to confirm correct configuration
- Upload a typical cosmetics formulation process sheet PDF. Confirm that the raw material list and process parameters in the parsing result retain the original layout and field information.
- Navigate to the knowledge base settings page. Confirm that the enhanced PDF parsing option is enabled, and that the connectivity test for the MinerU service passes.
- Upload a weighing and feeding record table containing multiple columns of data. Confirm that the chunking result splits independent semantic entries according to business logic.
- Adjust the
chunkSizeparameter, then upload a long paragraph exception report document. Confirm that the chunk length matches the expected configuration.
The values provided are common starting points, and should be evaluated against the user's own samples.