What data for this category looks like
Historical query record data originates from natural language search interaction logs initiated by end users. It includes query content entered within the business terminal, query time, associated business nodes, user identifiers, dwell time, and other information. The update cadence is near-real-time, with data synced to the knowledge base data source hourly. Document structures are mostly formatted as structured data, and can be exported as CSV, Excel, or JSON files. Each individual record has clear field definitions. Fields include query_text (query text), user_id (user identifier), query_time (UTC timestamp), result_ids (associated result ID array), duration (dwell time, unit: milliseconds), and others. The structure is clear and includes nested data types.
What constraints do these characteristics impose on document parsing and chunking
The structured, multi-field nature of historical query records requires the parsing step to accurately extract specified fields, and avoid losing associated information such as user identifiers and query time. Otherwise, accurate matching for subsequent in-terminal search will be affected. The near-real-time update cadence requires chunking tasks to support incremental parsing, avoiding the computational overhead caused by full re-runs. The characteristic of short per-record length but large batch data volume requires aggregation by user session during chunking, instead of splitting individual records, to ensure context coherence. The nested array type associated result ID field requires parsing logic to retain structured fields, rather than only extracting plain text content. Otherwise, the association between queries and results will be lost.
How to set configurations
| Configuration Item | Recommended Setting | Rationale |
|---|---|---|
PARSE_INCREMENTAL_ENABLE | true | Historical query records have a high update frequency; incremental parsing avoids computational overhead of full re-runs, adapting to near-real-time sync requirements |
CHUNK_SIZE | 800–1200 characters | Content aggregated by user session fits in-terminal search windows, avoiding context overflow from overly large chunks |
CHUNK_OVERLAP | 100–150 characters | Preserves continuity of session context, avoiding critical information gaps between adjacent chunks |
STRUCTURED_FIELD_EXTRACT | ["query_text", "query_time", "user_id"] | Only retain core search-related fields, filter redundant business fields, reducing redundancy in parsing and search |
PARSE_SHEET_NAME_ENABLE | true | Historical query records are often stored in separate worksheets by business scenario; retaining sheet names helps distinguish log content from different data sources |
MAX_PARSE_TIMEOUT | 300 seconds | Reasonable time threshold for parsing bulk historical logs, preventing task interruptions from excessive data volume |
The parameter values provided on this page are common starting points for configuration. Actual values are affected by material format, data volume, and business rules. Specific issues require targeted analysis, and it is recommended to test against your own samples before finalizing settings.
Three common mistakes
- When parsing exported Excel historical query records, only the content of the first worksheet is returned, and data from other worksheets is not included. The cause is that the
PARSE_SHEET_NAME_ENABLEconfiguration is not enabled; the default parsing logic only reads the first worksheet. - When using PDF enhanced mode to parse non-PDF documents of historical query records, no structured parsing results are obtained. The cause is that the PDF enhancement function only adapts to PDF format documents, and has no additional structured parsing capability for formats such as Word, PPT, etc.
- When bulk importing CSV files of historical query records, only the first two columns of data are read. The cause is that
STRUCTURED_FIELD_EXTRACTis not configured to specify target fields; the default CSV parsing only reads the first two columns as text content.
How to confirm configuration is complete
- Upload a single Excel test file for historical query records containing multiple worksheets. Confirm the parsing result includes content from all worksheets, and that the
sheet_namefield has been extracted. - Import a CSV file of historical query records containing multiple fields. Confirm the parsing result includes all fields specified in the configuration, rather than only the first two columns of content.
- Initiate an incremental parsing task. Confirm only newly added historical query records are processed, and that no full re-run of all historical data is performed.
- Review the parsing task’s running logs. Confirm no timeout errors occur, and that the per-chunk parsing duration matches the expected configuration.
Question material comes from public community discussions. Configuration values are common starting points and should be measured against your own samples. Verified on 2026-09-14.