What the data for this category looks like
Data sources for advanced chemical material ledgers and reports include production batch reports, raw material quality inspection certificates, warehouse outbound records, and downstream delivery ledgers. Data updates are triggered with each production batch, and update intervals vary per batch. Document formats include structured Excel spreadsheets, semi-structured PDF test reports, and plain text ledger entries. Core fields include material grade, batch number, production/warehousing date, density, tensile strength, and single-batch output. Units include g/cm³, MPa, tons, and others.
What constraints these characteristics impose on the document parsing and chunking step
Structured tables in advanced chemical material ledgers have merged cells and multi-level headers. Parsing must retain cell hierarchy to avoid field misalignment. Semi-structured test reports contain formulas and process flow diagrams. Chunking must not separate charts from their corresponding explanatory text. Core fields have associated units. Chunking must bind fields to their units to avoid unit confusion during retrieval. Incrementally updated batch data has no fixed format. Parsing must accommodate dynamically added field entries to prevent missing key information in chunks.
How to set configurations
| Configuration Item | Recommended Setting | Rationale |
|---|---|---|
chunkSize | 800–1200 characters | Core information for a single entry in advanced chemical material ledgers typically ranges from 500 to 1000 characters. This range fully preserves the complete parameter set for a single batch of material |
chunkSplitter | Split by paragraph and semantic meaning | Ledgers contain combinations of multiple field types. Splitting by semantic meaning avoids breaking the parameter association for the same batch |
customPdfParse | Enable structured table parsing | Semi-structured test reports contain multi-column parameter tables. Enabling this setting preserves the table row and column structure |
UPLOAD_FILE_MAX_SIZE | 1000 MB | Single-batch production ledgers may include multiple test reports and warehouse receipts. This upper limit accommodates complete batch archive files |
PARSE_FILE_TIMEOUT_SECONDS | 600 seconds | Large multi-page test reports require extended parsing time. This duration covers standard bulk parsing scenarios |
Reference only; verify against the target FastGPT release and deployment documentation.
Three common mistakes
- Symptom: After custom chunking, documents stored in the knowledge base have duplicate chunks automatically removed, and custom index order fails. Cause: The system enables global document chunk deduplication logic by default, and this check is not disabled for custom chunking scenarios.
- Symptom: Timeouts occur during parsing of large multi-page test reports, and tasks are marked as parsing failed. Cause: The
PARSE_FILE_TIMEOUT_SECONDSparameter is not adjusted. The default duration is insufficient to complete parsing of complex chemical material test reports. - Symptom: After parsing structured Excel ledgers, content from merged cells is split across multiple chunks, and the parameter association for the same batch is lost. Cause: No structured table parsing mode is configured. The default chunking logic does not retain cell hierarchy structure.
How to confirm configurations are set correctly
- A single typical chemical material test report is uploaded, the parsed chunk preview is reviewed, and core parameters are verified to remain unsplit.
- Batch archive files are uploaded, the status return of parsing tasks is checked, and compliance with the duration requirements of the
PARSE_FILE_TIMEOUT_SECONDSconfiguration is confirmed. - Test data with custom chunking is submitted, stored in the knowledge base, and the chunk list is queried to confirm the automatic deduplication logic is not triggered.
- The API is called to create a knowledge base and bind a test document, and real-time parsing status and task progress are verified to be retrievable.
The values provided are common starting points and should be evaluated against in-house test samples.
Question material comes from public community discussions. Configuration values are common starting points and should be measured against your own samples. Verified on 2026-09-28.