Data Characteristics
Patient Assistance Program (PAP) registration documents include project proposals, pharmaceutical research data, clinical trial reports, safety data, ethics approval documents, and various approvals and compliance files from project execution. This information originates from internal pharmaceutical R&D, clinical, and regulatory departments, as well as external CROs (Contract Research Organizations) and charitable organizations. Data updates frequently, especially when project proposals are revised, clinical data is updated, or regulatory policies change. Document formats vary, including formal reports in PDF, project plans in Word, patient data summaries in Excel, and scanned approval documents. Specific fields include patient enrollment criteria, drug dosage, follow-up period, adverse event (AE) records, and patient informed consent numbers. Units involve milligrams, milliliters, days, weeks, and years.
Constraints from Data Characteristics on Knowledge Base Retrieval and Recall
The multi-source and diverse nature of PAP documents requires the knowledge base to support various file formats for document parsing. High update frequency demands efficient incremental update and version management mechanisms to ensure retrieval results are timely. Complex document structures, containing many specialized terms and cross-references, require the knowledge base to effectively identify contextual relationships during segmentation, preventing semantic fragmentation. Specific fields like drug dosage and follow-up period need proper structuring or entity recognition during knowledge base construction for accurate retrieval. For example, querying adverse event rates at specific dosages may lead to retrieval deviations if dosage units are not accurately parsed. Additionally, the large number of scanned documents challenges the accuracy and multilingual support of OCR (Optical Character Recognition), directly impacting text content extraction quality.
Configuration Settings
| Configuration Item | Recommended Value | Rationale |
|---|---|---|
Chunk size (Segment Length) | 800–1200 characters | Ensures each knowledge block contains sufficient context to avoid splitting critical information, while balancing recall efficiency. |
Chunk Overlap Length (Segment Overlap Length) | 100–200 characters | Maintains semantic continuity between segments, especially when processing long reports and regulatory clauses, preventing information loss due to boundary effects. |
Recall count (Recall Count) | top 5-8 entries | Balances retrieval efficiency and coverage. For the complexity of patient assistance documents, increasing recall quantity improves relevance. |
Similarity threshold (Similarity Threshold) | 0.75–0.85 | Sets a higher threshold for the strictness of regulatory documents and clinical data, ensuring the precision of recalled content. |
Rerank result count (Rerank Return Count) | top 3 entries | Further refines recall results, prioritizing content most relevant to the user's query, reducing the engineer's filtering effort. |
UPLOAD_FILE_MAX_SIZE | 500 MB | Accommodates the upload requirements for large PDF files such as clinical trial reports and pharmaceutical research reports. |
Common Pitfalls
- Symptom: The AI response states "no relevant information found in the knowledge base," but the knowledge base actually contains the corresponding content. Reason: Segment length is set too small, causing complete semantic units to be broken, or the similarity threshold is set too high, failing to recall relevant but slightly differently phrased knowledge blocks.
- Symptom: Parsing of retrieved JSON content fails with an "Unexpected token" error. Reason: Non-standard characters were introduced during format conversion or OCR recognition of original documents in the knowledge base, or the API's returned data structure does not match expectations, preventing correct processing by the frontend.
- Symptom: When adding data to a knowledge base collection via API, some data items fail to be stored or are not retrievable after storage. Reason: Field names provided during the API call do not match the Schema definition of the knowledge base collection, or data types are inconsistent, leading to data being filtered or failing to store.
How to Verify Configuration
- Select typical and representative patient assistance program documents. Conduct multiple rounds of Q&A testing to observe the accuracy and completeness of AI responses.
- In different query scenarios, examine the raw segmented content recalled by the knowledge base. Evaluate whether it contains the core information and context required for the query.
- Simulate the upload and retrieval process for various file types (PDF, Word, Excel, scanned documents). Verify the knowledge base's ability to handle multi-source heterogeneous data.
- Monitor status codes and error messages during knowledge base retrieval and recall by reviewing FastGPT's log output to confirm the absence of anomalies.
The values provided are common starting points and should be measured against the reader's own samples.
Question material comes from public community discussions. Configuration values are common starting points and should be measured against your own samples. Verified on 2026-09-21.