What this category of data looks like
Port and shipping declaration materials and file review data mainly comes from port operation management systems, vessel entry and exit declaration platforms, and customs supervision data interfaces. Data update rhythm follows single-operation triggers or daily batch synchronization. Single documents contain fixed structured fields and unstructured attachments. Structured fields include vessel name, voyage number, bill of lading number, total cargo tonnage, and container quantity. Units include TEU, ton, and hour. Unstructured attachments are mostly PDF customs declarations and manifest scans, with some including paper scans with handwritten annotations.
What constraints these characteristics impose on the document parsing and chunking step
The mixed structured and unstructured features of port and shipping declaration materials require the parsing link to support both structured field extraction and non-text content transcription. Single documents have wide size variation, ranging from a few KB electronic declaration forms to dozens of pages of multimodal transport files. Parsing and chunking logic must adapt to different lengths. Fixed field formats require retaining field association relationships during chunking to avoid losing business context after splitting. Some paper scans have tilt and noise, requiring image preprocessing before parsing; otherwise, chunked content will be disorganized. Some documents contain repeated fields such as multiple bills of lading for the same voyage, requiring retention of original association relationships, and no random splitting or deduplication.
How to configure
Reference only; verify against the target FastGPT release and deployment documentation.
| Configuration Item | Recommended Setting | Rationale |
|---|---|---|
chunkSize | 800–1200 characters | Adapts to the mixed length of structured fields and unstructured content in port and shipping documents, avoiding loss of business context after splitting |
chunkSplitter | By paragraph + semantic boundary | Adapts to the paragraph structure of customs declarations and manifests, retaining complete logic of declaration items |
customPdfParse | Enabled | Targets the fixed template format of port and shipping documents, enabling custom extraction of header fields and attachment content |
UPLOAD_FILE_MAX_SIZE | 1000 MB | Adapts to the upload requirements of large-volume multimodal transport files |
PARSE_FILE_TIMEOUT_SECONDS | 600 seconds | Avoids parsing timeout for large-volume files, ensuring complete content extraction |
chunkSettingMode | Manual mode | Supports separate adjustment of chunking parameters for different types of declaration documents such as bills of lading and manifests |
Three common mistakes
- Phenomenon: After custom chunking, the knowledge base automatically deletes duplicate document chunks, resulting in a mismatch between the custom chunk index order and the actual stored content. Cause: The system's default duplicate chunk deduplication logic is not turned off, or chunk parameters do not adapt to common fields such as repeated bills of lading and container quantities in port and shipping documents.
- Phenomenon: Parsed documents cannot return structured field content in table format. Cause: The
customPdfParsecustom parsing mode is not enabled, or field extraction rules for port and shipping fixed templates are not configured. - Phenomenon: When calling the API to create a knowledge base, status information for parsing in progress, ready, and parsing failed cannot be obtained. Cause: The parsing status return identifier is not specified in the API request parameters, or the status query interface is not called correctly.
How to confirm the configuration is correctly set
- A typical port and shipping declaration document is uploaded. Parsed structured fields are checked for complete extraction, and core fields such as vessel name and voyage number are verified for accuracy.
- The knowledge base document list is reviewed to confirm that large-volume multimodal transport files are not truncated and parsing progress completes normally.
- The API is called to query document parsing status, confirming that corresponding status identifiers for parsing in progress, ready, and failed can be obtained.
- Chunked content is tested to confirm that chunk boundaries do not destroy business context, such as bill of lading information not being split into two chunks.
The values listed are common starting points and should be evaluated against the reader's own samples.
Question material comes from public community discussions. Configuration values are common starting points and should be measured against your own samples. Verified on 2026-09-28.