What this category of data looks like
Data for this category primarily originates from engine Profiler export files, crash platform logs, real-device test frame rate and resource usage metrics, and performance test reports generated during version iterations. Commonly used documents include model adaptation lists, optimization record ledgers, crash analysis reports, and more. Most document structures include clear technical fields such as model number, frame rate value, memory utilization rate, and crash stack fragments. Some documents include unit labels, such as frame rate in fps and memory usage in MB. Data update frequency is mostly daily test data output, or batch updates accompanying version iterations.
What constraints do these characteristics impose on the document parsing and chunking link
The multi-source, heterogeneous data characteristics of this category require the document parsing link to support multiple formats including HTML tables, PDF text blocks, and structured logs. Technical fields with clear units are included in the documents. During chunking, the contextual association between fields, their corresponding units, and test conditions must be retained to avoid data ambiguity after splitting. Frequently updated test data and version documents require the chunking logic to support dividing content chunks by test batch, model group, or version dimension, to prevent unrelated performance data across batches from being grouped into the same chunk. Additionally, stack fragments in crash analysis reports are continuous technical content; chunking must not interrupt complete stack lines, as this will affect the semantic integrity of subsequent interface Q&A.
How to set the configurations
The following configurations are formulated based on the official logic of FastGPT v4.17.0:
| Configuration Item | Recommended Value | Rationale |
|---|---|---|
chunkSize | 800–1200 characters | Adapts to the length of single-batch test data in performance test reports, avoiding loss of association between test conditions and results after splitting |
chunkSplitter | By paragraph + title hierarchy | Matches the title hierarchy of model adaptation lists and optimization records, ensuring semantic completeness of chunked content |
customPdfParse | Enable OCR mode | Adapts to some exported PDF performance reports, supporting extraction of test data text from images embedded in PDFs |
UPLOAD_FILE_MAX_SIZE | 1000 MB | Accommodates the size of performance test log packages exported in bulk for a single version, preventing large file upload failures |
PARSE_FILE_TIMEOUT_SECONDS | 300–600 seconds | Adapts to parsing time of multi-page PDF performance reports or large HTML interface documents, preventing timeout interruptions |
chunkSettingMode | Manual configuration | Adjusts chunking rules for the exclusive document structure of game client performance optimization, adapting to industry-specific data characteristics |
Three common mistakes
- No parsing results after uploading javadoc-generated HTML interface documents. The cause is that the parsing switch for HTML tables is not enabled. The default parsing logic only extracts plain text content and does not process embedded interface parameter table structures.
- Unable to extract test data from images when parsing PDF documents. The cause is that the OCR mode of
customPdfParseis not enabled. Only the plain text layer embedded in the PDF is extracted, and frame rate and model adaptation information in images are not recognized. - Separation of model number and adaptation status occurs after chunking. The cause is that the chunking logic only splits by character length and does not group by title or paragraph, causing associated data across paragraphs to be split into different chunks.
How to confirm configurations are correct
- Upload a single javadoc HTML interface document, check if the parsed text blocks contain the complete content of the interface parameter table, confirming that the parsing switch is correctly configured.
- Upload a PDF performance report containing embedded test charts, check if the parsing result includes the text descriptions and test data next to the charts, confirming that OCR mode is enabled.
- Import a model adaptation list document, check if the chunked content retains the complete adaptation information for a single model, confirming that the chunking rule matches the document structure.
- Upload a performance log package exceeding 500 MB, confirm that the upload and parsing process does not trigger timeout errors, verifying that the file size limit configuration is reasonable.
The values provided are common starting points and should be measured against the reader's own samples.
Question material comes from public community discussions. Configuration values are common starting points and should be measured against your own samples. Verified on 2026-10-19.