What Data for This Category Looks Like
Data for this category primarily originates from internal company new hire onboarding documents, including employee handbooks, attendance management policies, salary and benefits guides, department contact lists, onboarding training materials, and more. Supported file formats include PDF, Word, CSV, and others. Update frequency aligns with company policy adjustments or new hire batch schedules, typically occurring quarterly or annually. Corresponding documents are updated immediately following temporary policy changes. Document structures include fixed chapters, process steps, and structured tables. Fields cover department names, contact person information, attendance days, reimbursement workflows, and more. Units include days, hours, yuan, and others.
What Constraints Do These Characteristics Impose on the Document Parsing and Chunking Step
Multi-format document sources require the parsing step to support different file types. For example, CSV files need correct extraction of multiple column fields, while encrypted or customized PDFs require enabling custom parsing services. The diversity of document structures requires the chunking step to retain the logical integrity of chapters and processes, avoiding splitting a single complete process description. Uncertain update frequencies require parsing and chunking parameters to be flexibly adjusted to adapt to new or revised document content. Single document volumes vary widely, ranging from a few-page process guides to dozens of pages of training collections. This requires balancing parsing efficiency and completeness.
Configuration Settings
Reference only; verify against the target FastGPT release and deployment documentation.
| Configuration Item | Recommended Value | Rationale |
|---|---|---|
chunkSize | 800–1200 characters | New hire onboarding documents are mostly paragraph descriptions and process steps. This length can fully cover a single process or chapter content, avoiding logical breaks from splitting |
chunkSplitter | Split by paragraph + title | Onboarding documents have clear chapter titles and paragraph separations. Splitting this way retains the document's logical structure, facilitating subsequent context matching for question answering |
customPdfParse | Enabled, configure custom parsing interface | Some onboarding documents include encrypted PDFs or customized formats. Enabling this adapts to parsing needs for non-standard PDFs, solving the issue where native parsing cannot extract table content |
PARSE_FILE_TIMEOUT_SECONDS | 600 seconds | Single onboarding documents may have large volume, and custom parsing requires calling external interfaces. This duration covers the parsing process for most large-volume documents |
UPLOAD_FILE_MAX_SIZE | 1000 MB | Some onboarding training materials include high-definition videos or multi-page PDF collections. This upper limit meets conventional bulk document upload requirements |
chunkSettingMode | Manual configuration | Onboarding documents from different departments have large structural differences. Manual configuration allows flexible adjustment of chunking parameters to adapt to various scenario needs |
Three Common Configuration Mistakes
- Phenomenon: PDF uploads result in prolonged unresponsiveness or timeout errors. Cause: The
PARSE_FILE_TIMEOUT_SECONDSparameter was not adjusted. The default duration is insufficient to cover the custom parsing interface call time. - Phenomenon: Only the first two columns of data are displayed after CSV file import. Cause: Corresponding parsing rules were not configured. The default parsing logic only extracts the first two column fields.
- Phenomenon: Process steps are split after chunking, causing question answering to fail to associate complete processes. Cause: The
chunkSizesetting is too small, splitting a single complete process description.
How to Verify Correct Configuration
- Upload a test onboarding PDF document, check if the parsed text content includes all tables and chapter content, confirming that the
customPdfParseconfiguration takes effect. - Upload a test onboarding CSV file, verify that the imported data columns cover all required fields, confirming that the parsing rule configuration is correct.
- Adjust the
chunkSizeparameter, check the chunking preview interface to confirm that single chunk content does not split complete processes or chapters. - Upload a large-volume onboarding document, wait for parsing to complete, confirm that no timeout error is triggered, verifying that the
PARSE_FILE_TIMEOUT_SECONDSconfiguration is valid.
The values provided are common starting points and should be evaluated against the reader's own samples.
Question material comes from public community discussions. Configuration values are common starting points and should be measured against your own samples. Verified on 2026-09-28.