What this type of data consists of
Port and shipping business documents mainly originate from port operation systems, vessel scheduling platforms, customs declaration systems, and route planning tools. Document update frequency varies by business scenario: vessel berthing operation documents are updated in real time based on berthing times, customs declaration documents are updated periodically per declaration batch, and annual route planning documents are updated once per quarter.
Document structure includes fixed header fields such as vessel name, voyage number, berth number, and operation type, long-text operation instructions, compliance clauses, and embedded visual content such as cargo inventory tables and berth layout diagrams. Fields and units follow clear standards: voyage numbers use a combination of letters and numbers, operation duration is measured in hours, and cargo weight is measured in tons.
What constraints these characteristics impose on document parsing and chunking
The real-time update nature of port and shipping documents requires the parsing process to have low latency, to avoid business data synchronization delays caused by overly long parsing times. The presence of long-text operation instructions and compliance clauses requires chunking rules to balance contextual integrity and embedding model input limits. Chunking that is too fragmented will break business logical associations, while chunking that is too long will exceed the model's processing upper limit.
Embedded tables and visual content require parsing tools to support extraction and structuring of non-pure text elements, otherwise key operation information will be lost. Fixed-format fields and units require that chunking preserves contextual associations, to avoid splitting fields from their corresponding descriptions, which would affect subsequent retrieval accuracy.
How to configure the settings
Reference only; verify against the target FastGPT release and deployment documentation.
| Configuration Item | Recommended Value | Rationale |
|---|---|---|
chunkSize | 800–1200 characters | Port and shipping documents include long operation instructions and compliance clauses. This range preserves context and fits the input limits of most embedding models |
chunkSplitter | Split by paragraph + heading | Port documents have fixed heading hierarchies and paragraph structures. Splitting this way preserves business logic integrity |
customPdfParse | Enabled | Port documents often contain embedded cargo inventory tables and berth layout diagrams. Enabling this allows complete parsing of non-pure text content |
UPLOAD_FILE_MAX_SIZE | 1000 MB | Single port customs declaration documents or vessel scheduling logs may include multiple batches of operation data, requiring support for large file uploads |
PARSE_FILE_TIMEOUT_SECONDS | 600 seconds | Parsing large port documents takes longer time. This avoids parsing failures caused by timeout |
chunkSettingMode | Manual configuration | Port documents have large structural differences. Manual configuration allows flexible adjustment of chunking rules to adapt to different types of documents |
Three common misconfigurations
- Symptom: After using
chunkmode to call the pushdata API to upload large port documents, the index status remains displayed for a long time. Cause: ThePARSE_FILE_TIMEOUT_SECONDSconfiguration is not set to a reasonable duration for port documents. The default timeout threshold is insufficient to complete large file parsing, causing the task to hang. - Symptom: After uploading a PDF port document containing a berth layout diagram, the parsing result only includes plain text with no image-related content. Cause: The
customPdfParseconfiguration is not enabled. The default parsing mode only extracts plain text content and ignores embedded visual elements. - Symptom: After uploading a javadoc-generated HTML port operation API document, no valid parsed content is obtained. Cause: The specific tag structure of HTML documents is not adapted. The default parsing rules cannot identify interface fields and description text, resulting in empty parsing results.
How to verify correct configuration
- Upload a standard port cargo inventory PDF, check the match between the number of parsed text blocks and the number of paragraphs in the original document, and adjust
chunkSizeuntil the chunking logic aligns with business process coherence. - Upload a port operation document containing an embedded table, check whether the parsing result completely extracts the cargo name, quantity, and weight information in the table, and confirm that the
chunkSplitterconfiguration adapts to the table structure. - Call the pushdata API to upload a single port scheduling log with a volume not exceeding 1000 MB, check whether the task status is completed within the preset duration, and confirm that the
PARSE_FILE_TIMEOUT_SECONDSvalue adapts to the document volume. - Upload a javadoc-format port operation API document, check whether the parsing result includes interface fields and descriptions, and confirm that the parsing rules have adapted to the tag structure of HTML documents.
The values provided are common starting points and should be measured against the reader's own samples.
Question material comes from public community discussions. Configuration values are common starting points and should be measured against your own samples. Verified on 2026-09-28.