Article

Performance of Data Mining Methods for Propensity Score Analysis with Multilevel Data

---

Abstract

Propensity score methods have been developed primarily for single-level data structures. In educational studies, however, data typically have a hierarchical structure, where the probability of receiving treatment is a function of both individual and cluster-level factors. Using Monte Carlo simulations, this study compared two tree-based data mining approaches to two parametric models for propensity score estimation. All four estimation methods with an appropriately specified outcome model provided unbiased treatment effect estimates. However, under non-additivity and non-linearity, the data mining approaches provided a better performance on predicting the propensity score. Also, hidden bias from unobserved covariates greatly impacted the estimates of causal effect- rendering all propensity score analysis approaches invalid. The generalized boosting modeling-multilevel logistic regression combination provided more accurate treatment effect estimates among the eight estimation and outcome model combinations. 

 

 

Keywords: Causal inference, propensity score analysis, hierarchical structure, data mining approaches, GBM

How to Cite:

Fan, M., May, H. & Collier, Z., (2026) “Performance of Data Mining Methods for Propensity Score Analysis with Multilevel Data”, Practical Assessment, Research, and Evaluation 31(1): 15. doi: https://doi.org/10.7275/pare.3322