Propensity scores

• The propensity score (ps) is defined as the conditional probability of receiving a treatment given pre-treatment covariates X.

Propensity scores

• The propensity score (ps) is defined as the conditional probability of receiving a treatment given pre-treatment covariates X.

• That is,

  $$e(X) = \mathbb{Pr}[W = 1 | X] = \mathbb{E}[W | X],$$

  where $X = (X_1, \ldots, X_p)$ is the vector of $p$ covariates/predictors.

Propensity scores

• The propensity score (ps) is defined as the conditional probability of receiving a treatment given pre-treatment covariates X.

• That is,

  $$e(X) = \mathbb{Pr}[W = 1 | X] = \mathbb{E}[W | X],$$

  where $X = (X_1, \ldots, X_p)$ is the vector of $p$ covariates/predictors.

• Propensity score is a probability, analogous to a summary statistic.

Balancing property of propensity score

• Property 1. The propensity score e(X) balances the distribution of all $X$ between the treatment groups:

  $$W \perp X | e(X)$$

Balancing property of propensity score

• Property 1. The propensity score e(X) balances the distribution of all $X$ between the treatment groups:

  $$W \perp X | e(X)$$

• Equivalently,

  $$\mathbb{Pr}[W_i = 1 | X_i, e(X_i)] = \mathbb{Pr}[W_i = 1 | e(X_i)].$$

Remarks on the balancing property

• Rosenbaum and Rubin (1983) show that all balancing scores are a function of $e(X)$.

Remarks on the balancing property

• Rosenbaum and Rubin (1983) show that all balancing scores are a function of $e(X)$.

• If a subclass of units or a matched treatment-control pair are homogeneous in $e(X)$, then the treatment and control units have the same distribution of $X$.

Propensity score: unconfoundedness

• Property 2. If $W$ is unconfounded given $X$, then $W$ is unconfounded given $e(X)$, i.e.,

Propensity score: unconfoundedness

• Property 2. If $W$ is unconfounded given $X$, then $W$ is unconfounded given $e(X)$, i.e.,

• That is, if

  $${Y_i(0), Y_i(1)} \perp W_i | X_i$$

  holds, then

  $${Y_i(0), Y_i(1)} \perp W_i | e(X_i),$$

  also holds.

Propensity score: unconfoundedness

• $e(X)$ can be viewed as a summary score of the observed covariates.

Propensity score: unconfoundedness

• $e(X)$ can be viewed as a summary score of the observed covariates.

• This is great because causal inference can then be drawn through stratification, matching, regression, etc. using the scalar $e(X)$ instead of the high dimensional covariates.

Propensity score: unconfoundedness

• $e(X)$ can be viewed as a summary score of the observed covariates.

• This is great because causal inference can then be drawn through stratification, matching, regression, etc. using the scalar $e(X)$ instead of the high dimensional covariates.

• The propensity score balances the observed covariates, but does not generally balance unobserved covariates.

Causal inference using propensity scores

Propensity score analysis (in observational studies) typically involves two stages:

Causal inference using propensity scores

Propensity score analysis (in observational studies) typically involves two stages:

• Stage 1. Estimate the propensity score: by a logistic regression or machine learning methods.

Stage 1: estimating the propensity score

• The main purpose of estimating propensity score is to ensure overlap and balance of covariates between treatment groups, instead of “finding a perfect fit" of propensity score.

Stage 1: estimating the propensity score

• The main purpose of estimating propensity score is to ensure overlap and balance of covariates between treatment groups, instead of “finding a perfect fit" of propensity score.

• As long as the important covariates are balanced, model overfitting is not a concern; underfit can be a problem however.

Stage 1: estimating the propensity score

• A standard procedure for estimating propensity scores includes:
  1. initial fit;

Stage 1: estimating the propensity score

• A standard procedure for estimating propensity scores includes:

  1. initial fit;

  2. discarding outliers (with too large or too small propensity scores);

Stage 1: estimating the propensity score

• A standard procedure for estimating propensity scores includes:

  1. initial fit;

  2. discarding outliers (with too large or too small propensity scores);

  3. check covariate balance; and

Stage 1: estimating the propensity score

• Step 1. Estimate propensity score using a logistic regression:

  $$W_i | X_i \sim \textrm{Bernoulli}(\pi_i); \ \ \ \ \textrm{log}\left(\dfrac{\pi_i}{1-\pi_i}\right) = X_i\boldsymbol{\beta}.$$

Stage 1: estimating the propensity score

• Step 1. Estimate propensity score using a logistic regression:

  $$W_i | X_i \sim \textrm{Bernoulli}(\pi_i); \ \ \ \ \textrm{log}\left(\dfrac{\pi_i}{1-\pi_i}\right) = X_i\boldsymbol{\beta}.$$

  Include all covariates in this initial model or do a stepwise selection on the covariates and interactions to get an initial estimate of the propensity scores. That is,

  $$\hat{e}^0(X_i) = \dfrac{e^{X_i\hat{\boldsymbol{\beta}}}}{1 + e^{X_i\hat{\boldsymbol{\beta}}}}.$$

Stage 1: estimating the propensity score

• Step 2. Check overlap of propensity score between treatment groups. If necessary, discard the observations with non-overlapping propensity scores.

Stage 1: estimating the propensity score

• Step 2. Check overlap of propensity score between treatment groups. If necessary, discard the observations with non-overlapping propensity scores.

• Step 3. Assess balance given by initial model in Step 1.

Stage 1: estimating the propensity score

• Step 2. Check overlap of propensity score between treatment groups. If necessary, discard the observations with non-overlapping propensity scores.

• Step 3. Assess balance given by initial model in Step 1.

• Step 4. If one or more covariates are seriously unbalanced, include some of their higher order terms and/or interactions to re-fit the propensity score model and repeat Steps 1-3, until most covariates are balanced.

Stage 1: estimating the propensity score

• In practice, balance checking in the PS estimation stage can be done via sub-classification/stratification, matching or weighting.

Stage 1: estimating the propensity score

• In practice, balance checking in the PS estimation stage can be done via sub-classification/stratification, matching or weighting.
  • sub-classification/stratification: check the balance of all important covariates within $K$ blocks of $\hat{e}^0(X_i)$ based on its quantiles.

Stage 1: estimating the propensity score

• In practice, balance checking in the PS estimation stage can be done via sub-classification/stratification, matching or weighting.
  • sub-classification/stratification: check the balance of all important covariates within $K$ blocks of $\hat{e}^0(X_i)$ based on its quantiles.
  • matching: check the balance of all important covariates in the matched sample.

Stage 2: stratification

• Given the estimated propensity score, we can estimate the causal estimands through sub-classification/stratification, weighting or matching.

Stage 2: stratification

• Given the estimated propensity score, we can estimate the causal estimands through sub-classification/stratification, weighting or matching.

• Let's start with stratification.

Stage 2: stratification

• Given the estimated propensity score, we can estimate the causal estimands through sub-classification/stratification, weighting or matching.

• Let's start with stratification.

• Recall that the result of 5 strata of a single covariate removes 90% bias.

Stage 2: stratification

• Divide the subjects in to $K$ strata by the corresponding quantiles of the estimated propensity scores.

Stage 2: stratification

• Divide the subjects in to $K$ strata by the corresponding quantiles of the estimated propensity scores.

• ATE: estimate ATE within each stratum and then average by the block size. That is,

  $$\hat{\tau}^{ATE} = \sum_{k=1}^K \left(\bar{Y}_{k,1} - \bar{Y}_{k,0} \right) \dfrac{N_{k,1}+N_{k,0}}{N},$$

  with $N_{k,1}$ and $N_{k,0}$ being the numbers of units in class $k$ under treated and control, respectively.

Stage 2: stratification

• Divide the subjects in to $K$ strata by the corresponding quantiles of the estimated propensity scores.

• ATE: estimate ATE within each stratum and then average by the block size. That is,

  $$\hat{\tau}^{ATE} = \sum_{k=1}^K \left(\bar{Y}_{k,1} - \bar{Y}_{k,0} \right) \dfrac{N_{k,1}+N_{k,0}}{N},$$

  with $N_{k,1}$ and $N_{k,0}$ being the numbers of units in class $k$ under treated and control, respectively.

• ATT: weight within-block ATE by proportion of treated units $N_{k,1}/N_1$.

Propensity score stratification: Remarks

• 5 blocks is usually not enough, consider higher number such as 10.

Propensity score stratification: Remarks

• 5 blocks is usually not enough, consider higher number such as 10.

• Stratification is a coarsened version of matching.

Propensity score stratification: Remarks

• 5 blocks is usually not enough, consider higher number such as 10.

• Stratification is a coarsened version of matching.

• Empirical results from real applications and situations: usually not as good as matching or weighting.

Propensity score stratification: Remarks

• 5 blocks is usually not enough, consider higher number such as 10.

• Stratification is a coarsened version of matching.

• Empirical results from real applications and situations: usually not as good as matching or weighting.

• Good for cases with extreme outliers (smoothing): less sensitive, but also less efficient.

Stage 2: matching

• In propensity score matching, potential matches are compared using (estimated) propensity score.

Stage 2: matching

• In propensity score matching, potential matches are compared using (estimated) propensity score.

• 1-to-n closest neighbor matching is common when the control group is large compared to treatment group.

Stage 2: matching

• In propensity score matching, potential matches are compared using (estimated) propensity score.

• 1-to-n closest neighbor matching is common when the control group is large compared to treatment group.

• In most software packages, the default is actually 1-to-1 closest neighbor matching.

Stage 2: matching

• In propensity score matching, potential matches are compared using (estimated) propensity score.

• 1-to-n closest neighbor matching is common when the control group is large compared to treatment group.

• In most software packages, the default is actually 1-to-1 closest neighbor matching.

• Pros: robust, matched pairs (so you can do within pair analysis).

Stage 2: matching

• In propensity score matching, potential matches are compared using (estimated) propensity score.

• 1-to-n closest neighbor matching is common when the control group is large compared to treatment group.

• In most software packages, the default is actually 1-to-1 closest neighbor matching.

• Pros: robust, matched pairs (so you can do within pair analysis).

• Sometimes, dimension reduction via the propensity score may be too drastic, recent methods advocate matching on the multivariate covariates directly.

Stage 2: regression

• Remember the key propensity score property:

  $${Y_i(0), Y_i(1)} \perp W_i | X_i \ \ \Rightarrow \ \ {Y_i(0), Y_i(1)} \perp W_i | e(X_i)$$

Stage 2: regression

• Remember the key propensity score property:

  $${Y_i(0), Y_i(1)} \perp W_i | X_i \ \ \Rightarrow \ \ {Y_i(0), Y_i(1)} \perp W_i | e(X_i)$$

• Idea: in a regression estimator, adjusting for $e(X)$ instead of the whole $X$; thus in regression models of $Y(w)$ use $e(X)$ as the single predictor.

Stage 2: regression

• Remember the key propensity score property:

  $${Y_i(0), Y_i(1)} \perp W_i | X_i \ \ \Rightarrow \ \ {Y_i(0), Y_i(1)} \perp W_i | e(X_i)$$

• Idea: in a regression estimator, adjusting for $e(X)$ instead of the whole $X$; thus in regression models of $Y(w)$ use $e(X)$ as the single predictor.

• Clearly, modeling $\mathbb{Pr}(Y(w)|\hat{e}(X))$ is simpler than modeling $\mathbb{Pr}(Y(w)|X)$; effectively more data to estimate essential parameters due to the dimension reduction.

Stage 2: regression

• Remember the key propensity score property:

  $${Y_i(0), Y_i(1)} \perp W_i | X_i \ \ \Rightarrow \ \ {Y_i(0), Y_i(1)} \perp W_i | e(X_i)$$

• Idea: in a regression estimator, adjusting for $e(X)$ instead of the whole $X$; thus in regression models of $Y(w)$ use $e(X)$ as the single predictor.

• Clearly, modeling $\mathbb{Pr}(Y(w)|\hat{e}(X))$ is simpler than modeling $\mathbb{Pr}(Y(w)|X)$; effectively more data to estimate essential parameters due to the dimension reduction.

• However,

  • we lose interpretation of the effects of individual covariates, e.g. age, sex; and

Stage 2: regression

• Idea: instead of using the estimated $\hat{e}(X)$ as the single predictor, use it as an additional predictor in the model. That is, $\mathbb{Pr}(Y(w)|X,\hat{e}(X))$.

Stage 2: regression

• Idea: instead of using the estimated $\hat{e}(X)$ as the single predictor, use it as an additional predictor in the model. That is, $\mathbb{Pr}(Y(w)|X,\hat{e}(X))$.

• Turns out that $\mathbb{Pr}(Y(w)|X,\hat{e}(X))$ gives both efficiency and robustness.

Stage 2: regression

• Idea: instead of using the estimated $\hat{e}(X)$ as the single predictor, use it as an additional predictor in the model. That is, $\mathbb{Pr}(Y(w)|X,\hat{e}(X))$.

• Turns out that $\mathbb{Pr}(Y(w)|X,\hat{e}(X))$ gives both efficiency and robustness.

• Also, if we are unable to achieve full balance on some of the predictors, using $\mathbb{Pr}(Y(w)|X,\hat{e}(X))$ will help further control for those unbalance predictors.

Stage 2: regression

• Idea: instead of using the estimated $\hat{e}(X)$ as the single predictor, use it as an additional predictor in the model. That is, $\mathbb{Pr}(Y(w)|X,\hat{e}(X))$.

• Turns out that $\mathbb{Pr}(Y(w)|X,\hat{e}(X))$ gives both efficiency and robustness.

• Also, if we are unable to achieve full balance on some of the predictors, using $\mathbb{Pr}(Y(w)|X,\hat{e}(X))$ will help further control for those unbalance predictors.

• Empirical evidences (e.g. simulations) support.