This document applies to FastGPT v4.17.0.
What the data for this category looks like
Daily documents for cosmetics production include formulation process sheets, weighing and feeding records, emulsification and filling records, cleaning and site clearance records, semi-finished and finished product inspection reports, label review documents, and more. Data sources include registered and filed formulations, raw material safety materials, MES/ERP systems, laboratory equipment, and packaging material version documents. Update frequency is per production batch or each line change. Document fields include raw material name, restricted substance content, batch number, operating temperature, inspection results, and more. Units include grams, degrees Celsius, hours, batch numbers, and more. Most structures are compliance records that mix tables and paragraphs.
What constraints these characteristics impose on the document parsing and chunking link
High-frequency updated documents per batch require chunking rules to adapt to rapidly iterating production data, to avoid breaking field associations due to overly coarse or fine chunking. Mixed multi-type document structures require parsing logic to handle both tables and paragraphs, to prevent cutting the correspondence between raw material names and feeding amounts during splitting. Extraction of compliance fields needs to retain the context of original formatting, to ensure accurate matching of filed data and production records during subsequent ledger summary. It also needs to support batch upload of multiple documents of the same type, to meet batch summary requirements.
How to set the configurations
| Configuration Item | Recommended Setting | Rationale |
|---|---|---|
PARSE_FILE_TIMEOUT_SECONDS | 600–900 seconds | Address scenarios where single bulk ledger documents have large file sizes and long parsing times, to avoid timeout errors |
chunkSize | 800–1200 characters | Match the field density of cosmetics ledgers, to avoid cutting key associated information such as raw material names and batch numbers during chunking |
chunkSplitter | Split by paragraph + header | Adapt to the mixed table and paragraph structure of ledgers, retain the integrity of each group of operation records |
customPdfParse | Enabled | Adapt to formatted PDF documents such as formulation sheets and inspection reports, retain field positions from original formatting |
UPLOAD_FILE_MAX_SIZE | 1000 MB | Support batch upload of summary files for single-batch production ledgers, meet the needs of merging and uploading multiple documents |
chunkSettingMode | Manual configuration | Adjust chunking rules for different types of ledger documents such as feeding records and inspection reports to adapt to field differences |
Three common mistakes
- Parsing timeout errors occur when uploading large ledger PDF files. The cause is failure to adjust the
PARSE_FILE_TIMEOUT_SECONDSparameter; the default timeout duration is insufficient to complete parsing of complex formatted documents. - The number of returned Excel ledger data entries is less than expected. The cause is failure to retain the context of associated fields such as "usage location" and "batch attribution" during chunking, leading to missed content with cross-field associations during matching.
- Core fields are empty after uploading ledger documents. The cause is failure to enable the
customPdfParseconfiguration, leading to incorrect extraction of field content from custom-formatted ledger documents.
How to confirm the configuration is correct
- Upload a standard cosmetics feeding record PDF, check if the parsed text retains complete paragraphs of key fields such as raw material names, feeding amounts, and time.
- Adjust
PARSE_FILE_TIMEOUT_SECONDSto 700 seconds, then upload a single ledger file larger than 500 MB, confirm no timeout errors occur. - Search for ledger data containing "foreign usage location", confirm all eligible entries are returned in the results.
- Compare parsing results before and after enabling
customPdfParse, confirm that formatting and field extraction are more complete for formulation sheet documents with tables.
The values provided are common starting points and should be measured against the reader's own samples.