Policy Design and Causal Inference for Social Science
Course Description
I discuss empirical strategies in applied micro research aimed at evaluating the causal effects of policies or programs on desired outcomes. These strategies collectively form the contemporary toolbox for conducting causal inference in various academic domains, including public policy, development economics, labor economics, education, marketing, and corporate finance. They also find applications in industries and international organizations. This course caters to second-year Ph.D. students equipped with a quantitative background comparable to that provided in AREC623. While the course does encompass theoretical and formal econometric reasoning, its primary focus lies in fostering design-oriented identification using observational data. My exploration of strategies stems from the prerequisites needed to replicate the conditions of an ideal experiment, which serves as the foundation for addressing the underlying causal query. The lectures are structured into self-contained modules, each dedicated to specific strategies, with rotation taking place across academic years. The curriculum covers a range of methodologies, including regression and matching, instrumental variables and natural experiments, differences-in-difference designs, synthetic control methods and regression discontinuity designs. Practical coding of these empirical strategies for application in real-world scenarios will also be extensively discussed in the lectures.
Syllabus: Fall 2023
Schedule: Testudo
Webpage: Canvas (UMD students only)
Course Contents
Part I. Causality and Empirical Research: The Basic Framework
Slides for lectures (revised in August 2023).
Most of the terminology and concepts discussed in this part can be found in the following references:
Most of the terminology and concepts discussed in this part can be found in the following references:
- Angrist, Joshua D., and Jorn-Steffen Pischke (2009). Mostly Harmless Econometrics. An Empiricist’s Companion, Princeton University Press. [Chapters 1 and 2]
- Hernán, Miguel A., and Robins, James M. (2020). Causal Inference: What If, Boca Raton: Chapman & Hall/CRC. [Chapters 1, 6, 7 and 8]
- Imbens, Guido W., and Donald B. Rubin (2015). Causal Inference for Statistics, Social, and Biomedical Sciences, Cambridge University Press. [Chapters 1, 2 and 3]
- Morgan, Stephen L., and Christopher Winship (2015). Counterfactuals and Causal Inference, Cambridge University Press. [Chapter 1]
- Pearl, Judea, Madelyn Glymour, and Nicholas P. Jewell (2016). Causal Inference in Statistics: A Primer, Wiley. [Chapter 1]
- Rosembaum, Paul R. (2017). Observation and Experiment, Harvard University Press. [Chapters 1 and 2]
- Athey, Susan, and Guido W. Imbens (2017). The State of Applied Econometrics: Causality and Policy Evaluation. Journal of Economic Perspectives, 31(2), pp. 3-32.
- Heckman, James J., Smith, Jeffrey, and Nancy Clements (1997). Making The Most Out Of Programme Evaluations and Social Experiments: Accounting For Heterogeneity in Programme Impacts. The Review of Economic Studies, 64(4), pp. 487–535.
- Imbens, Guido W. (2020). Potential Outcome and Directed Acyclic Graph Approaches to Causality: Relevance for Empirical Practice in Economics. Journal of Economic Literature, 58(4), pp. 1129-79.
- Lewbel, Arthur (2019). The Identification Zoo: Meanings of Identification in Econometrics. Journal of Economic Literature, 57(4), pp. 835-903.
Part II. FOOD FOR THOUGHT: iNFERENCE In Randomized Experiments
Slides for lectures (revised in September 2023).
Most of the terminology and concepts discussed in this part can be found in the following references:
Most of the terminology and concepts discussed in this part can be found in the following references:
- Efron, Bradley, and Trevor Hastie (2016). Computer Age Statistical Inference: Algorithms, Evidence and Data Science, Cambridge University Press. [Chapters 4 and 15]
- Hernán, Miguel A., and Robins, James M. (2020). Causal Inference: What If. Boca Raton: Chapman & Hall/CRC. [Chapters 2, 3 and 10]
- Imbens, Guido W., and Donald B. Rubin (2015). Causal Inference for Statistics, Social, and Biomedical Sciences, Cambridge University Press. [Chapters 5, 6, 7, 9 and 10]
- Rosembaum, Paul R. (2017). Observation and Experiment, Harvard University Press. [Chapters 1 and 2]
- Abadie, Alberto, Susan Athey, Guido W. Imbens, and Jeffrey M. Wooldridge (2017). When Should You Adjust Standard Errors for Clustering? NBER Working Paper No. 24003.
- Abadie, Alberto, Susan Athey, Guido W. Imbens, and Jeffrey M. Wooldridge (2020). Sampling-Based versus Design-Based Uncertainty in Regression Analysis. Econometrica, 88: 265-296.
- Anderson, Michael L. (2008). Multiple Inference and Gender Differences in the Effects of Early Intervention: A Reevaluation of the Abecedarian, Perry Preschool, and Early Training Projects. Journal of the American Statistical Association, 103(484), pp. 1481-1495.
- Athey, Susan, and Imbens, G. W. (2017). The Econometrics of Randomized Experiments. In A. V. Banerjee and E. Duflo, eds, Handbook of Field Experiments, Vol. 1 of Handbook of Economic Field Experiments, North-Holland, pp. 73-140.
- Bai, Yuheao (2020). Optimality of Matched-Pair Designs in Randomized Controlled Trials. Unpublished manuscript, University of Michigan.
- de Chaisemartin, Clément, and Jaime Ramirez-Cuellar (2020). At What Level Should One Cluster Standard Errors in Paired Experiments, and in Stratified Experiments with Small Strata? NBER Working Paper No. 27609.
- Deaton, Angus (2020). Randomization in the Tropics Revisited: a Theme and Eleven Variations. NBER Working Paper No. 27600.
- Heckman, J. J., Moon, S. H., Pinto, R., Savelyev, P., and Yavitz, A. (2010). Analyzing social experiments as implemented: A reexamination of the evidence from the HighScope Perry Preschool Program. Quantitative Economics, 1, pp. 1-46.
- Westfall, Peter H., and S. Stanley Young (1993). Resampling-Based Multiple-Testing: Examples and Methods for P-value Adjustment. New York: John Wiley & Sons.
- Young, Alwyn (2019). Channeling Fisher: Randomization Tests and the Statistical Insignificance of Seemingly Significant Experimental Results. Quarterly Journal of Economics, 134(2). pp. 557–598.
- Deaton, Angus (2017). The problems with Randomised Controlled Trials. CEPR & VideoVox Economics.
- McKenzie, David (2017). Finally, a way to do easy randomization inference in Stata! World Bank Blogs.
- McKenzie, David (2017). When should you cluster standard errors? New wisdom from the econometrics oracle. World Bank Blogs.
- McKenzie, David (2021). An updated overview of multiple hypothesis testing commands in Stata. World Bank Blogs.
Part III. Estimating Causal Effects By Conditioning (not taught in fall 2023)
Most of the terminology and concepts discussed in this part can be found in the following references:
- Imbens, Guido W., and Donald B. Rubin (2015). Causal Inference for Statistics, Social, and Biomedical Sciences, Cambridge University Press. [Chapters 12-19, selected parts]
- Abdulkadiroglu, Atila, Joshua D. Angrist, and Parag Pathak (2014). The Elite Illusion: Achievement Effects at Boston and New York Exam Schools, Econometrica, 82, pp. 137-196.
- Abdulkadiroglu, Atila, Joshua D. Angrist, Yusuke Narita, and Parag Pathak (2019). Breaking Ties: Regression Discontinuity Design Meets Market Design, IZA Discussion Paper No. 12205.
- Athey, Susan, Guido Imbens, Thai Pham, and Stefan Wager (2017). Estimating Average Treatment Effects: Supplementary Analyses and Remaining Challenges. American Economic Review, 107(5), pp. 278-81.
Part IV. Estimating Causal Effects Using Instrumental Variation
Slides for lectures: Part 1 and Part 2 (revised in September 2023).
Most of the terminology and concepts discussed in this part can be found in the following references:
Most of the terminology and concepts discussed in this part can be found in the following references:
- Imbens, Guido W., and Donald B. Rubin (2015). Causal Inference for Statistics, Social, and Biomedical Sciences, Cambridge University Press. [Chapters 23 and 24]
- Abadie, Alberto (2002). Bootstrap Tests for Distributional Treatment Effects in Instrumental Variables Models, Journal of the American Statistical Association, 97, pp. 284-292.
- Abadie, Alberto (2003). Semiparametric Instrumental Variable Estimation of Treatment Response Models, Journal of Econometrics, 113, pp. 231-263.
- Brinch, Christian N., Magne Mogstad, and Matthew Wiswall (2017). Beyond LATE with a Discrete Instrument, Journal of Political Economy, 125(4), pp. 985-1039.
- Carneiro, Pedro, James J. Heckman, and Edward J. Vytlacil (2011). Estimating Marginal Returns to Education, American Economic Review, 101(6), pp. 2754-81.
- de Chaisemartin, Clément (2017). Tolerating Defiance: LATE Without Monotonicity, Quantitative Economics, 8(2), pp. 367-396.
- Heckman, James J., and Edward Vytlacil (2005). Structural Equations, Treatment Effects, and Econometric Policy Evaluation, Econometrica, 73(3), pp. 669-738.
- Kitagawa, Toru (2015). A Test for Instrument Validity, Econometrica, 83(5), pp. 2043-2063.
- Huber, Martin, and Giovanni Mellace (2015). Testing Instrument Validity for LATE Identification Based on Inequality Moment Constraints, The Review of Economics and Statistics, 97(2), pp. 398-411.
- Mogstad, Magne, and Alexander Torgovitsky (2018). Identification and Extrapolation of Causal Effects with Instrumental Variables, Annual Review of Economics, 10(1), pp. 577-613.
- Mogstad, Magne, Andres Santos, and Alexander Torgovitsky (2018). Using Instrumental Variables for Inference About Policy Relevant Treatment Parameters, Econometrica, 86(5), pp. 1589-1619.
- Mogstad, Magne, Alexander Torgovitsky, and Christopher R. Walters (2019). The Causal Interpretation of Two-Stage Least Squares with Multiple Instrumental Variables, NBER Working Paper No. 25691.
- Mogstad, Magne, Alexander Torgovitsky, and Christopher R. Walters (2020). Policy Evaluation with Multiple Instrumental Variables, Becker Friedman Institute for Economics Working Paper 2020-99.
- Mourifié, Ismael, and Yuanyuan Wan (2017). Testing Local Average Treatment Effect Assumptions, The Review of Economics and Statistics, 99(2), pp. 305-313.
- Zhou, Xiang, and Yu Xie (2019). Marginal Treatment Effects from a Propensity Score Perspective, Journal of Political Economy, 127(6), pp. 3070-3084.
- Torgovitsky, Alexander (2021). Chamberlain Seminar. Instrumental Variables with Multiple Instruments.
PART V. ESTIMATING CAUSAL EFFECTS USING Longitudinal VARIATION
Slides for lectures (revised in November 2023).
Suggested readings:
Suggested readings:
- Abadie, Alberto (2021). Using Synthetic Controls: Feasibility, Data Requirements, and Methodological Aspects, Journal of Economic Literature, 59(2), pp. 391-425.
- Abadie, Alberto, and Jérémy L'Hour (2021). A penalized synthetic control estimator for disaggregated data, Unpublished manuscript, MIT.
- Arkhangelsky, Dmitry, Susan Athey, David A. Hirshberg, Guido W. Imbens, and Stefan Wager (2020). Synthetic Difference in Differences, American Economic Review, forthcoming.
- Ben-Michael, Eli, Avi Feller, and Jesse Rothstein (2021). Synthetic Controls with Staggered Adoption. NBER Working Paper 28886.
- Borusyak, Kirill, Xavier Jaravel, and Jann Spiess (2021). Revisiting Event Study Designs: Robust and Efficient Estimation, Unpublished manuscript, University College London.
- Callaway, Brantly, and Pedro H.C. Sant’Anna (2020). Difference-in-Differences with multiple time periods, Journal of Econometrics, forthcoming.
- de Chaisemartin, Clément, and Xavier d’Haultfoeuille (2018). Fuzzy differences-in-differences, The Review of Economic Studies, 85(2), pp. 999-1028.
- de Chaisemartin, Clément, and Xavier d’Haultfoeuille (2020). Two-Way Fixed Effects Estimators with Heterogeneous Treatment Effects, American Economic Review, 110(9), pp. 2964-96.
- de Chaisemartin, Clément, and Xavier d’Haultfoeuille (2021). Difference-in-Differences Estimators of Intertemporal Treatment Effects, Unpublished manuscript, University of California at Santa Barbara.
- Doudchenko, Nikolay, and Guido W. Imbens (2016). Balancing, Regression, Difference-In-Differences and Synthetic Control Methods: A Synthesis, NBER Working Paper 22791.
- Freyaldenhoven, Simon, Christian Hansen and Jesse M. Shapiro (2019). Pre-event Trends in the Panel Event-Study Design, American Economic Review, 109(9), pp. 3307-38.
- Freyaldenhoven, Simon, Christian Hansen, Jorge Pérez, and Jesse M. Shapiro (2021). Visualization, Identification, and Estimation in the Linear Panel Event-Study Design. NBER Working Paper 29170.
- Goodman-Bacon, Andrew (2021), Difference-in-differences with variation in treatment timing, Journal of Econometrics,
forthcoming. - Kahn-Lang, Ariella, and Kevin Lang (2019). The promise and pitfalls of differences-in-differences: Reflections on 16 and pregnant and other applications, Journal of Business & Economic Statistics, 38(3), pp. 613-620.
- Rambachan, Ashesh, and Jonathan Roth (2020). An Honest Approach to Parallel Trends. Unpublished manuscript, Brown University.
- Roth, Jonathan, and Pedro H.C. Sant’Anna (2020). Efficient Estimation for Staggered Rollout Designs, Unpublished manuscript, Brown University.
- Sun, Liyang, and Sarah Abraham (2020). Estimating Dynamic Treatment Effects in Event Studies with Heterogeneous Treatment Effects, Journal of Econometrics, forthcoming.
- Abadie, Alberto (2021). Chamberlain Seminar. Tutorial on Synthetic Control Methods.
- Chamberlain Seminar (2020). Symposium on Synthetic Control Methods.
- Difference in Differences (repository for recent developments).
- McKenzie, David (2020). Revisiting the Difference-in-Differences Parallel Trends Assumption: Part I. Pre-Trend Testing. World Bank Blogs.
- McKenzie, David (2020). Revisiting the Difference-in-Differences Parallel Trends Assumption: Part II. What happens if the parallel trends assumption is (might be) violated? World Bank Blogs.
- Imbens, Guido W. (2021). RES 2021: Sargan Lecture - Causal Panel Data Models.
PART VI. Policy Analysis With Jumps and Kinks
Slides for lectures are under revision.