What does the data for this category look like?
Data sources for biologics yield and market data include public pharmaceutical circulation quotation APIs, official biologic product batch issuance databases, and sales data disclosed regularly by listed companies. Full category market data for the previous calendar day is updated every early morning. Historical batch issuance-related data is synchronized quarterly. Documents use structured tables or JSON format. Each data entry includes product generic name, dosage form, specification, manufacturing enterprise, quotation date, current day’s transaction price, previous day’s transaction price, daily yield, circulation channel level. Unit specifications: Transaction price is measured in yuan per unit, daily yield uses percentage format, and dates follow the YYYY-MM-DD standard format.
What constraints do these characteristics impose on database and operations workflows?
Multiple data sources require the database to support multi-source data merging and deduplication, to avoid duplicate entry of market data for the same product on the same date. The fixed daily update schedule requires scheduled tasks to be configured, with data import and index rebuilding performed during business low-peak hours to avoid impacting real-time queries. Structured data with multiple specifications and fields requires the database to set a composite primary key (product name + specification + quotation date) to prevent duplicate import entries. The percentage-format yield field requires the database to configure numeric type validation to avoid dirty data entering the index system. Additionally, the daily report broadcast business scenario requires the database query response speed to meet standards, so partitioned indexes must be created for commonly used query fields to optimize performance.
How to set configurations
| Configuration Item | Recommended Value | Rationale |
|---|---|---|
DB_INSERT_BATCH_SIZE | 500–1000 entries per batch | Biologics daily report data has a large number of entries. A single batch that is too large will easily trigger table locking in PostgreSQL databases, while a batch that is too small will reduce import efficiency. |
PARSE_FILE_TIMEOUT_SECONDS | 300 seconds | Biologics market documents contain data for multiple specifications and enterprises, so parsing takes longer than general documents. |
INDEX_REBUILD_INTERVAL | 86400 seconds | Data is updated once daily. Rebuilding indexes daily ensures query efficiency and data consistency. |
UPLOAD_FILE_MAX_SIZE | 200 MB | Full biologics market reports imported in bulk typically do not exceed this size, preventing upload timeouts. |
QUERY_TIMEOUT | 60 seconds | The yield broadcast business needs to quickly return market data for multiple products. Timeouts will affect broadcast timeliness. |
SIMILARITY_THRESHOLD | 0.75 | A reasonable similarity threshold is required to distinguish yield data for the same product with different specifications, avoiding data confusion. |
The parameter values provided on this page are common starting points for configuration. Actual values are affected by material form, data volume, and business rules. Specific issues require specific analysis. It is recommended to test on your own samples before finalizing values.
Three common errors
- Phenomenon: An
Invalid array lengtherror pops up when executing knowledge base question answering splitting. Cause: The imported biologics market document contains an emptydaily_yieldfield array, and empty array entries were not filtered during splitting. - Phenomenon: After deploying a PostgreSQL database, the knowledge base index can never be created. The host configuration is 8 cores 16 GB, no GPU. Cause: The
DB_INSERT_BATCH_SIZEparameter was not adjusted. An excessively large single import batch caused memory usage to exceed the virtual machine limit. - Phenomenon: After modifying database and service passwords, the service cannot connect to the database normally. Cause: The
DB_PASSWORDandSERVICE_PASSWORDparameters in the FastGPT configuration file were not updated synchronously, causing a mismatch between configuration and actual passwords.
How to confirm correct configuration
- Run the database import script, check that the import log has no
Invalid array lengtherrors, and the number of imported entries matches the count from the source file. - Access the FastGPT index management page, confirm that the index status of the biologics knowledge base is "Completed", with no failure prompts or abnormal logs.
- Execute the yield query API, verify that the returned results include required fields such as
product_nameanddaily_yield, and that the response time meets the business threshold. - After modifying the database password, restart the service and test the database connection, confirm that there are no error logs related to connection failure.
Question material comes from public community discussions. Configuration values are common starting points and should be measured against your own samples. Verified on 2026-09-14.