What this category's data looks like
Manufacturing enterprise PMC, planning, shop scheduling, and material control departments provide most of the data for this category. Daily documents include sales orders, master production schedules, material requirements plans, weekly and daily production schedules, work orders, material shortage lists, and delivery change notices. Data sources cover ERP orders and inventory, BOMs, process routes, equipment capacity, staff shifts, work in progress, and supplier arrivals. Updates happen daily, weekly, or when triggered by orders or exceptional events. Most documents contain structured tables and specific business fields: order numbers, material codes, capacity values, shift times, and more. Common units include pieces, hours, and days.
What constraints these characteristics impose on the document parsing and chunking step
Document sizes vary widely. Some single work orders have only tens of pages, while weekly production summaries can reach hundreds of pages. Parsing tools must handle files of different sizes. Updates happen frequently and are triggered by exceptions. The parsing process needs fast response times to avoid delaying subsequent schedule adjustments. A large share of documents contain structured tables with closely linked fields. Chunking operations must not break the integrity of business units. Some temporary documents have inconsistent formats, such as emergency work orders created after order changes. Parsing and chunking logic needs appropriate flexibility while preserving accuracy of core business fields.
How to set configurations
| Configuration Item | Recommended Value | Rationale |
|---|---|---|
chunkSize | 800–1200 characters | These documents contain numerous tables and paragraphs linked to processes and materials. An overly long chunk length will cause splicing across business units, while an overly short length will split core business associations. |
chunkSplitter | Split by paragraph + table boundaries | Tables in production schedules and work orders are core business units. Splitting must retain the full structure of tables, to avoid splitting content from the same process across different chunks. |
PARSE_FILE_TIMEOUT_SECONDS | 300–600 seconds | Parsing multi-hundred-page summary PDFs takes longer time. Extending the timeout threshold prevents parsing failures for large files. |
UPLOAD_FILE_MAX_SIZE | 1000 MB | Adapt to multi-document packaged files exported from ERP, to prevent upload failure due to exceeding file size limits. |
customPdfParse | Enabled | For standardized business table formats, custom parsing improves extraction accuracy for specific fields such as order numbers and material codes. |
Three common mistakes
- Calling the parsing tool to process a hundreds-page PDF file triggers an error, while tens-page files parse normally. Cause: The
PARSE_FILE_TIMEOUT_SECONDSparameter is not adjusted. The large file parsing duration exceeds the default threshold, causing an interruption. - The locally deployed pdf-marker 4.9.0 page displays a Cannot read properties of undefined error. Cause: The parsing plugin dependencies associated with
customPdfParseare not configured correctly, or the chunking mode does not match the document table structure, leading to parsing exceptions. - The number of retrieved chunks after processing does not meet expectations, and chunk content has a high repetition rate. Cause: The
chunkSettingModeparameter is not adjusted based on document field characteristics. The overlapping range of adjacent chunks or chunk length settings are unreasonable, failing to accurately match business retrieval needs.
How to confirm correct configuration
- Upload a single standard 100-page weekly production schedule PDF. Check the table integrity of parsed chunks to confirm no content loss or cross-unit splicing errors.
- Upload a single test document close to the
UPLOAD_FILE_MAX_SIZElimit. Confirm the parsing task completes within the setPARSE_FILE_TIMEOUT_SECONDSvalue, with no timeout errors. - Adjust the
chunkSizeparameter. Compare chunking results across different chunk lengths to confirm that associations between core business fields such as processes and materials are not broken. - After enabling
customPdfParse, randomly select fields such as order numbers and material codes from the document. Verify that extracted results match the original document content.
This page's configurations are based on FastGPT v4.17.0
The values provided are common starting points and should be evaluated against the reader's own samples.