What the data for this category looks like
Data related to document drafting and material preparation for pharmaceutical distribution mainly comes from PDF and Word documents exported from internal business systems. These include purchase approval forms, cold chain temperature control logs, drug warehouse inspection receipts, supplier qualification documents, and other types. Update rhythm follows business processes: corresponding documents are generated after a single business is completed. Some cold chain temperature control logs are updated hourly in real time. Document structures include structured tables (such as inventory ledgers), long paragraph descriptions (such as drug test reports), and qualification documents in scanned image format. Fields involve generic drug names, production batch numbers, circulation temperature ranges, supplier unified social credit codes, and more. Units include ℃, pieces, batch numbers, and others.
What constraints these characteristics impose on the document parsing and chunking link
Documents with many structured tables require the parsing stage to retain row-column structure and field correspondence. Otherwise, subsequent chunking will lose the association between key information such as drug batch numbers and circulation temperatures. Long paragraph test reports and approval forms contain fixed-format clauses within their text. Chunking must maintain semantic coherence, and must not randomly split individual business descriptions. Qualification documents in scanned image format require OCR parsing capabilities. Without these, valid text cannot be extracted. Large cold chain temperature control log files have large file sizes. This requires adaptation to longer parsing timeouts and higher upload file limits. Delimiters vary across different documents. Some documents separate paragraphs with line breaks plus numbering, while others use timestamps. This requires flexible adjustment of chunking logic.
How to set the configurations
Reference only; verify against the target FastGPT release and deployment documentation.
| Configuration Item | Suggested Value | Rationale |
|---|---|---|
chunkSize | 800–1200 characters | Pharmaceutical distribution documents mostly contain long paragraph test descriptions and structured table content. This range preserves the integrity of individual business information |
chunkSplitter | Custom delimiter + semantic segmentation | Adapts to multi-format delimiter requirements for purchase orders and temperature control logs, avoiding disruption of structured fields |
customPdfParse | Enable table structured parsing + OCR mode | Process scanned qualification documents and inventory documents with tables, retain original field correspondence |
PARSE_FILE_TIMEOUT_SECONDS | 300–600 seconds | Adapt to the parsing duration of large cold chain temperature control log files, avoid timeout interruptions |
UPLOAD_FILE_MAX_SIZE | 1000 MB | Adapt to the upload requirements of single large cold chain temperature control log files, avoid upload failures |
chunkSettingMode | Manual configuration mode | Adjust chunking parameters for different document types to meet chunking needs for multi-format documents |
Three common errors
- Phenomenon: After configuring custom delimiters and
chunkSize, chunking results either merge multiple independent paragraphs or split a single long paragraph into multiple unexpected chunks. Cause: Failed to adjustchunkSettingModebased on the structured characteristics of pharmaceutical distribution documents, only relying on fixed delimiters or a single length parameter, without considering semantic coherence. - Phenomenon: An error occurs when deploying Marker for PDF parsing, and the log returns
{"detail":"Error message"}. Cause: Failed to correctly configure the environment variables corresponding tocustomPdfParse, or the uploaded file exceeded theUPLOAD_FILE_MAX_SIZElimit, causing the parsing process to abort abnormally. - Phenomenon: The vector results after chunking cannot be associated with specific chapters and file sources of the original document. Cause: Failed to configure document metadata retention parameters, or failed to carry the original file's identification information during chunking.
How to confirm the configuration is correct
- Upload a single pharmaceutical distribution inventory ledger PDF, check if the parsed text retains the row-column structure and field correspondence of the table, confirming that the
customPdfParseconfiguration takes effect. - Upload a 100MB cold chain temperature control log file, wait for parsing to complete, confirm that no timeout-related errors are triggered.
- Manually set
chunkSizeto 1000 characters, upload a single long paragraph drug test report, check if the chunking results do not split individual test clauses into unexpected chunks. - Configure the custom delimiter as "Approval No.: ", upload a purchase approval form, check if chunking is reasonably split by approval number as the unit.
The values provided are common starting points and should be measured against the reader's own samples.
Question material comes from public community discussions. Configuration values are common starting points and should be measured against your own samples. Verified on 2026-09-28.