# Analysis Techniques — When to Use Each

## Hypothesis Testing

**Use when:** Comparing two groups to determine if a difference is real or random chance.

**Technique selection:**
| Data type | Groups | Test |
|-----------|--------|------|
| Continuous | 2 | t-test (if normal) or Mann-Whitney |
| Continuous | 3+ | ANOVA or Kruskal-Wallis |
| Proportions | 2 | Chi-square or Fisher's exact |
| Paired data | 2 | Paired t-test or Wilcoxon signed-rank |

**Key outputs:**
- p-value (probability of seeing this difference by chance)
- Effect size (how big is the difference - Cohen's d, odds ratio)
- Confidence interval (range of plausible true values)

**Watch out for:**
- Large samples make everything "significant" - focus on effect size
- Multiple comparisons inflate false positives
- Normality assumptions (use non-parametric if violated)

---

## Cohort Analysis

**Use when:** Understanding how user behavior changes over time, segmented by when they started.

**Types:**
- **Retention cohorts:** % of users still active N days after signup
- **Revenue cohorts:** Revenue per cohort over time
- **Behavioral cohorts:** Feature adoption by signup cohort

**Setup:**
1. Define cohort (usually signup week/month)
2. Define event (login, purchase, specific action)
3. Define time windows (day 1, 7, 30, 90)
4. Build matrix: cohort × time period

**Key outputs:**
- Retention curves (line chart by cohort)
- Cohort comparison (are newer cohorts performing better?)
- Time-to-event patterns

**Watch out for:**
- Cohort size differences (small cohorts = noisy data)
- Seasonality (December cohort behaves differently)
- Definition consistency (what counts as "active"?)

---

## Funnel Analysis

**Use when:** Understanding conversion through a multi-step process.

**Setup:**
1. Define stages (visit -> signup -> activate -> purchase)
2. Count users at each stage
3. Calculate drop-off rates between stages

**Key outputs:**
- Conversion rates per stage
- Biggest drop-off points
- Segment comparison (mobile vs desktop funnels)

**Watch out for:**
- Time window (did they convert eventually, or just not today?)
- Stage ordering (users don't always follow linear paths)
- Defining "same session" vs "ever"

---

## Regression Analysis

**Use when:** Understanding what predicts an outcome, controlling for other factors.

**Types:**
- **Linear:** Continuous outcome (revenue, time spent)
- **Logistic:** Binary outcome (churned/retained, converted/didn't)
- **Poisson:** Count outcome (purchases, logins)

**Key outputs:**
- Coefficients (effect of each variable, holding others constant)
- R² (how much variance is explained)
- p-values per variable
- Residual plots (are assumptions met?)

**Watch out for:**
- Multicollinearity (correlated predictors)
- Omitted variable bias (missing important controls)
- Extrapolation beyond data range
- Causation claims from observational data

---

## Segmentation/Clustering

**Use when:** Discovering natural groups in your data.

**Techniques:**
- **K-means:** Simple, fast, assumes spherical clusters
- **Hierarchical:** Shows cluster relationships, good for exploration
- **RFM:** Business-specific (Recency, Frequency, Monetary)

**Process:**
1. Select features (what defines a segment?)
2. Normalize features (so scale doesn't dominate)
3. Choose number of clusters (elbow method, silhouette score)
4. Profile each cluster (what makes them different?)

**Key outputs:**
- Cluster profiles (avg values per segment)
- Segment sizes
- Distinguishing characteristics

**Watch out for:**
- Garbage in, garbage out (feature selection matters)
- Cluster count is subjective
- Stability (do clusters hold with different random seeds?)

---

## Anomaly Detection

**Use when:** Finding unusual data points that warrant investigation.

**Approaches:**
- **Statistical:** Points beyond 2-3 standard deviations
- **IQR method:** Below Q1-1.5×IQR or above Q3+1.5×IQR
- **Isolation Forest:** For multivariate anomalies
- **Domain rules:** Negative revenue, future dates, impossible values

**Key outputs:**
- Flagged records with anomaly scores
- Context (why is this unusual?)
- Severity (how far from normal?)

**Watch out for:**
- Seasonality (Black Friday isn't an anomaly)
- Trends (growth makes old "normal" look like anomalies)
- False positives (investigate before acting)

---

## Time Series Analysis

**Use when:** Understanding patterns in data over time.

**Components:**
- **Trend:** Long-term direction
- **Seasonality:** Repeating patterns (daily, weekly, yearly)
- **Noise:** Random variation

**Techniques:**
- **Moving averages:** Smooth out noise
- **Decomposition:** Separate trend, seasonal, residual
- **Year-over-year:** Compare same period last year

**Key outputs:**
- Trend direction and strength
- Seasonal patterns identified
- Forecast with uncertainty bands

**Watch out for:**
- Comparing different lengths (months vary in days)
- Holidays/events (one-time vs recurring)
- Structural breaks (COVID, product changes)