What the data for this category looks like
Duty-free investment research data primarily comes from commodity price lists published by off-island duty-free shops, offshore passenger shopping policy documents released by the General Administration of Customs, duty-free supply agreements from brand parties, and off-island passenger flow statistics. Policy documents are updated irregularly. Commodity price lists are updated monthly alongside brand price adjustments. Passenger flow data is synchronized daily.
Documents are divided into three categories:
- Policy documents include document number, effective date, applicable scope, and enforcement clauses.
- Commodity documents include SKU code, category, dutiable unit price, duty-free unit price, and single-purchase limit.
- Passenger flow documents include time period, off-island type, and consumption amount.
Field units are, respectively: document number, code, yuan, yuan, pieces/passenger trips, time period identifier, passenger trips, yuan.
What constraints these characteristics impose on database and operations workflows
Irregular updates and version differences in policy documents require the database to support incremental synchronization and historical version retention. This prevents overwriting old policy data that could lead to investment research errors.
The large number of commodity SKUs, small per-document size but fast overall growth, require reasonable database sharding configuration to support high-concurrency queries.
Fields include multi-dimensional amount and limit parameters. Indexes must be created for fields such as dutiable unit price and duty-free unit price to improve retrieval efficiency.
High-frequency synchronization of time-series passenger flow data requires the database connection pool configuration to adapt to concurrent write requirements. Data audit logs must also be retained to meet compliance requirements.
How to set configurations
| Configuration Item | Recommended Value | Rationale |
|---|---|---|
recall count | Top 10-15 entries | The duty-free category has a large number of SKUs, so more relevant products and policies must be covered to avoid missing valid information |
PARSE_FILE_TIMEOUT_SECONDS | 300 seconds | Policy documents often contain long tables and nested clauses, with longer parsing time than general documents |
UPLOAD_FILE_MAX_SIZE | 500 MB | Brand supply agreements are mostly multi-page PDF files with large single-file size |
similarity threshold | 0.75-0.85 | Duty-free policy statements are precise, so false recalls must be reduced while covering policy clauses with similar wording |
MONGODB_CONNECTION_POOL_SIZE | 20-30 | Balances the concurrent requirements of high-frequency writes and retrieval queries for passenger flow data |
database sharding threshold | 1,000,000 records | Adapts to the growth rate of total duty-free SKU and policy document volume |
The parameter values provided on this page are common recommendations used as a starting point for configuration. Actual values are affected by material format, data volume, and business rules. Specific issues require individual analysis. Testing should be conducted using local sample datasets before finalizing settings.
Three common configuration mistakes
- Symptom: The query conversation history interface returns an empty array. Cause:
MONGODB_URIis not configured to point to the correct database instance, or the connection string does not include the target database name. - Symptom: The knowledge base retrieval returns an insufficient number of results. Cause: The
recall countconfiguration value is set too low, not adapting to the multi-SKU characteristics of the duty-free category. - Symptom: The file parsing task reports an
ETIMEDOUTerror. Cause: ThePARSE_FILE_TIMEOUT_SECONDSconfiguration value is less than the actual parsing time of long policy documents.
How to verify correct configuration
- Upload a duty-free policy PDF document, verify that the parsed fields include preset fields such as
policy document numberandeffective date. - Initiate a retrieval test, enter a query related to duty-free, verify that the number of returned results falls within the configured
recall countrange. - Check the database connection logs, confirm there are no errors such as
connection refusedorauthentication failed. - Test the tool call workflow by entering a question with no matching knowledge base content, and verify that a preset specified response is returned.
Question material comes from public community discussions. Configuration values are common starting points and should be measured against your own samples. Verified on 2026-09-14.