Test Cases and Defect Analysis for User Growth and Retention Operations: Citation Sources and Traceability

Daily documents for this category include growth experiment plans, user segmentation tables, outreach copy, and retention analysis reports. Data

What Data for This Category Looks Like

Daily documents for this category include growth experiment plans, user segmentation tables, outreach copy, and retention analysis reports. Data sources include user behavior logs, push systems, experiment platforms, and CRM. Data update frequencies are split into daily and weekly: user behavior logs update daily, while user segmentation tables and retention analysis reports update weekly. Document fields include user ID, outreach time, experiment group, retention days, outreach channel, and others. Units include person-times, pieces, days, and others. Field naming varies slightly across different data sources.

Constraints on Citation Sources and Traceability

Differences in fields across multiple data sources require clear field mapping rules in configuration to avoid confusing user identifiers from different systems. Differences in update frequencies require matching the time range of corresponding data during traceability, ensuring cited experiment data aligns with the current analysis cycle. Diverse document types require configuring different citation scopes for each document type, preventing overly long citation content from interfering with analysis results. Errors can lead to user dissatisfaction or wasted budget, requiring traceability information to be accurate and linked back to the original data source.

How to Configure Settings

Configuration ItemRecommended SettingRationale
quoteMaxToken800–1200 tokenMatches the average paragraph length of growth operation documents, avoids truncating critical content
responseDataReturn source document path, update time, document typeMeets the need to confirm data sources and update cycles during traceability
detailEnabledDisplays the full content of original outreach copy or experiment plans, facilitating defect troubleshooting
Single recall limitTop 3 entriesControls the volume of cited content, avoids exceeding the context window
Field mapping rulesBind user ID, experiment group fieldsMatches naming differences across data sources, ensures accurate data association

Three Common Misconfigurations

  • Symptom: Generated growth plans cite incorrect user segmentation data, with empty fields or mismatched associations. Cause: No field mapping rules configured, confusing user ID fields from experiment platforms and CRM.
  • Symptom: Cited content in generated outreach copy is truncated, unable to fully display the original copy. Cause: quoteMaxToken is set too small, not matching the actual length of the target document.
  • Symptom: Source document update times cannot be viewed during traceability, making it impossible to troubleshoot time deviations in retention analysis reports. Cause: The detail parameter is not enabled, and required return items for responseData are not configured.

How to Verify Correct Configuration

  • In FastGPT v4.17.0, import a user segmentation table, enter a test user ID, and confirm the returned citation source matches the corresponding data in CRM.
  • Generate a retention analysis report, check the traceability link at the end of the report, and confirm it points to the correct original document path and update time.
  • Trigger a generation test for outreach copy, confirm the cited copy content exactly matches the original outreach copy.
  • Check the single recall limit in the configuration, confirm the number of returned cited content meets the preset top 3 entry requirement.

The values provided are common starting points and should be evaluated against the reader's own samples.

Question material comes from public community discussions. Configuration values are common starting points and should be measured against your own samples. Verified on 2026-10-19.