Segment Discovery Application Overview

Segment Discovery is a web-based analytics tool designed to analyze A/B testing results and identify feature segments that have the greatest impact on the outcome.

Workflow

1. Upload: User uploads a CSV file with A/B test results (features as column names).
2. Inspection: The app examines the file for format and consistency.
3. Selection: User specifies the variant and outcome columns.
4. Outcome Type: App determines if the outcome is binary or continuous.
5. Feature Handling:
   • Categorical features with too many levels → user can bin them.
   • Continuous features → user can create ranges.
6. Compliance Check: Once everything is prepared, the analysis can start.

Analysis Steps

1. Main Effects: Analyze each feature individually.
2. Two-Way Interactions: Evaluate combined effects between pairs of features.

Modeling Approach

• Binary outcomes → Logistic regression
• Continuous outcomes → Elastic model

For each feature/interaction, we calculate:

• p-values
• Odds or likelihood
• Confidence intervals (low and high)

Two-way interaction results are further compared using p-tests to highlight the added value of our approach.

AI-Generated Report

After analysis, the app generates a report including:

• Top 20 most influential segments
• Plots and visualizations
• Clear explanations of main and interaction effects