This page is written based on the configuration rules of FastGPT v4.17.0.
What this category of data looks like
Data for this category comes from design BOMs, silicon steel sheet and insulating material procurement data, vacuum drying equipment logs, oil test reports, high-voltage test records from test stations, and product nameplate parameter tables. Daily documents include winding and core assembly records, vacuum drying records, insulating oil reports, partial discharge and withstand voltage test reports, and nameplate parameter tables. Each document corresponds to a single product. Most fields are technical parameters with units, such as rated voltage (kV), capacity (kVA), partial discharge value (pC), and water content (ppm). Documents are generated after each single product or batch of materials is completed.
What constraints do these characteristics impose on the document parsing and chunking workflow
Single-product exclusive documents require that chunking retain the association between documents and products to avoid mixing parameters across products. There are many scattered technical parameter fields with units, so chunk boundaries must be divided by document type, such as test reports or nameplates, to ensure complete parameter context. Frequently updated single-product documents require chunking processing speed to adapt to the number of documents uploaded in a single batch. Long table data in insulating oil reports and withstand voltage test reports must be prevented from being split across tables, which would cause logical gaps in quotation basis in tender documents. In addition, the tender scenario requires integrating documents of multiple products, so chunking must retain the context association of single products to ensure accurate retrieval of corresponding product technical parameters.
How to set the configurations
| Configuration Item | Recommended Value | Rationale |
|---|---|---|
chunkSize | 800–1200 characters | Documents in this category contain long tables and technical parameters with units. 800–1200 characters can fully cover a single set of test parameters or a nameplate entry, avoiding parameter split breaks |
chunkSplitter | By document type + paragraph separator | Documents in this category have large structural differences. Test reports need to be chunked by test items, and nameplates need to be chunked by parameter entries. Unified separators cannot adapt to multiple types of documents |
customPdfParse | Enable structured parsing | Documents in this category contain a large number of tables and fixed-format parameter fields. Enabling structured parsing can retain the correspondence between fields and units, improving subsequent retrieval accuracy |
PARSE_FILE_TIMEOUT_SECONDS | 300–600 seconds | The full set of documents for a single product, including multiple test reports, has a large volume. Sufficient time must be reserved for parsing and chunking |
chunkSettingMode | Customize by document type | The parameter density and length vary greatly across different document types, such as test reports or nameplates. Unified chunking configurations cannot adapt to all documents |
Three common mistakes
- Phenomenon: Core parameters and auxiliary data cannot be distinguished by priority during retrieval, and returned results are mixed. Cause: No chunking priority configuration is set for the unit-bearing technical parameters of this category, and core test data such as partial discharge values is not set as a priority retrieval field.
- Phenomenon: Documents chunked locally cannot be retrieved normally after being uploaded to the server. Cause: No unified adaptation rules are set between chunking configurations and vector models. The semantic encoding logic of chunking varies across different vector models, causing local chunking results to not be correctly recognized by the server.
- Phenomenon: Markdown formatting is not rendered in context references, only plain text is displayed. Cause: The document structured parsing configuration is not enabled, and original document format identifiers are not retained during chunking, causing subsequent references to fail to restore the typesetting structure of tables and parameters.
How to confirm the configuration is correct
- Upload an insulating oil report of this category, check whether the chunking result is split by test items, and whether each chunk contains complete parameters and units.
- Compare local and server-side chunked files, confirm that chunking length and separator configurations are consistent, and verify chunking compatibility across different vector models.
- View the context reference preview interface, confirm that tables and parameter entries retain the typesetting structure of the original document.
- Upload the full set of documents for a single product, check whether parsing time is within the preset
PARSE_FILE_TIMEOUT_SECONDSrange, and no timeout errors occur.
The values given are common starting points and should be measured against the reader's own samples.