ivangdavila_data-analysis/techniques.md

Analysis Techniques — When to Use Each

Hypothesis Testing

Use when: Comparing two groups to determine if a difference is real or random chance.

Technique selection:

Data type	Groups	Test
Continuous	2	t-test (if normal) or Mann-Whitney
Continuous	3+	ANOVA or Kruskal-Wallis
Proportions	2	Chi-square or Fisher's exact
Paired data	2	Paired t-test or Wilcoxon signed-rank

Key outputs:

• p-value (probability of seeing this difference by chance)
• Effect size (how big is the difference - Cohen's d, odds ratio)
• Confidence interval (range of plausible true values)

Watch out for:

• Large samples make everything "significant" - focus on effect size
• Multiple comparisons inflate false positives
• Normality assumptions (use non-parametric if violated)

---

Cohort Analysis

Use when: Understanding how user behavior changes over time, segmented by when they started.

Types:

• Retention cohorts: % of users still active N days after signup
• Revenue cohorts: Revenue per cohort over time
• Behavioral cohorts: Feature adoption by signup cohort

Setup:

1. Define cohort (usually signup week/month)
2. Define event (login, purchase, specific action)
3. Define time windows (day 1, 7, 30, 90)
4. Build matrix: cohort × time period

Key outputs:

• Retention curves (line chart by cohort)
• Cohort comparison (are newer cohorts performing better?)
• Time-to-event patterns

Watch out for:

• Cohort size differences (small cohorts = noisy data)
• Seasonality (December cohort behaves differently)
• Definition consistency (what counts as "active"?)

---

Funnel Analysis

Use when: Understanding conversion through a multi-step process.

Setup:

1. Define stages (visit -> signup -> activate -> purchase)
2. Count users at each stage
3. Calculate drop-off rates between stages

Key outputs:

• Conversion rates per stage
• Biggest drop-off points
• Segment comparison (mobile vs desktop funnels)

Watch out for:

• Time window (did they convert eventually, or just not today?)
• Stage ordering (users don't always follow linear paths)
• Defining "same session" vs "ever"

---

Regression Analysis

Use when: Understanding what predicts an outcome, controlling for other factors.

Types:

• Linear: Continuous outcome (revenue, time spent)
• Logistic: Binary outcome (churned/retained, converted/didn't)
• Poisson: Count outcome (purchases, logins)

Key outputs:

• Coefficients (effect of each variable, holding others constant)
• R² (how much variance is explained)
• p-values per variable
• Residual plots (are assumptions met?)

Watch out for:

• Multicollinearity (correlated predictors)
• Omitted variable bias (missing important controls)
• Extrapolation beyond data range
• Causation claims from observational data

---

Segmentation/Clustering

Use when: Discovering natural groups in your data.

Techniques:

• K-means: Simple, fast, assumes spherical clusters
• Hierarchical: Shows cluster relationships, good for exploration
• RFM: Business-specific (Recency, Frequency, Monetary)

Process:

1. Select features (what defines a segment?)
2. Normalize features (so scale doesn't dominate)
3. Choose number of clusters (elbow method, silhouette score)
4. Profile each cluster (what makes them different?)

Key outputs:

• Cluster profiles (avg values per segment)
• Segment sizes
• Distinguishing characteristics

Watch out for:

• Garbage in, garbage out (feature selection matters)
• Cluster count is subjective
• Stability (do clusters hold with different random seeds?)

---

Anomaly Detection

Use when: Finding unusual data points that warrant investigation.

Approaches:

• Statistical: Points beyond 2-3 standard deviations
• IQR method: Below Q1-1.5×IQR or above Q3+1.5×IQR
• Isolation Forest: For multivariate anomalies
• Domain rules: Negative revenue, future dates, impossible values

Key outputs:

• Flagged records with anomaly scores
• Context (why is this unusual?)
• Severity (how far from normal?)

Watch out for:

• Seasonality (Black Friday isn't an anomaly)
• Trends (growth makes old "normal" look like anomalies)
• False positives (investigate before acting)

---

Time Series Analysis

Use when: Understanding patterns in data over time.

Components:

• Trend: Long-term direction
• Seasonality: Repeating patterns (daily, weekly, yearly)
• Noise: Random variation

Techniques:

• Moving averages: Smooth out noise
• Decomposition: Separate trend, seasonal, residual
• Year-over-year: Compare same period last year

Key outputs:

• Trend direction and strength
• Seasonal patterns identified
• Forecast with uncertainty bands

Watch out for:

• Comparing different lengths (months vary in days)
• Holidays/events (one-time vs recurring)
• Structural breaks (COVID, product changes)