Notes

These notes are mini-experiments meant to record thoughts or test different ideas and assumptions. These will often be small simulations with some details written.

Adjusting classification threshold by strata

Predictive modeling

Classification

When predicting binary outcomes, if we want to threshold our estimated probabilities to obtain predicted classes, then we devise a strategy for determining the optimal…

Regression for t-test and ANOVA with unequal variances

Statistics

Hypothesis testing

Welch’s t-test or Welch’s ANOVA, used in the case of heteroskedasticity (unequal variances), is “equivalent” to a likelihood ratio test (LRT) using a generalized least…

Visualizing partial correlations with 3 variables

Visualization

Partial correlation

Consider three variables x, y, and z. The correlation between x and y is the linear association between them (a straight line in a scatter plot). But what if we think z is a…

Resampling and SMOTE to handle class imbalance

Predictive modeling

Classification

Class imbalance

Calibration

SMOTE

Findings: (1) Predicted probabilities are miscalibrated when using resampling techniques. (2) Typical inference methods are invalid. (3) Using the naive threshold of 0.5 leads to misleading performance estimates.

---
title: "Notes"
description: "These notes are mini-experiments meant to record thoughts or test different ideas and assumptions. These will often be small simulations with some details written."
listing:
  contents: notes
  sort: "date desc"
  categories: true
  sort-ui: [title, date]
  filter-ui: [title, date]
  feed: true
---