Past Workshops

This page provides information for our past workshops.

You can jump to a specific workshop (within this overall page) using the links below:


2013 Fourth Annual Main Causal Inference Workshop

We would like to invite you to attend the fourth annual workshop on Research Design for Causal Inference, sponsored by Northwestern University, the University of Southern California, and the Society for Empirical Legal Studies.

Monday-Friday, June 24-28, 2013, at Northwestern Law School, Chicago, IL

An Advanced Workshop on Research Design for Causal Inference will be held this year on August 12-14, 2013.

Workshop Overview

Research design for causal inference is at the heart of a "credibility revolution" in empirical research. We will cover the design of true randomized experiments and contrast them to simulations and quasi-experiments, where part of the sample is "treated" in some way, and the remainder is a control group, but the researcher controls neither the assignment of cases to treatment and control groups nor administration of the treatment. We will assess the kinds of causal inferences one can and cannot draw from a research design, threats to valid inference, and research designs that can mitigate those threats.

Most empirical methods courses begin with the methods. They survey how each method works, and what assumptions each relies on. We will begin instead with the goal of causal inference, and discuss how to design research to come closer to that goal. The methods reflect the goal and are often adapted to the needs of a particular study. Some of the methods we will discuss are covered in PhD programs, but rarely in depth, and rarely with a focus on causal inference and on which methods to prefer for messy, real-world datasets with limited sample sizes.

Statistical software
We will use Stata, and will provide attendees with a temporary Stata12 license. Versions 10 and 11 should be fine for most purposes (but you may need to “downconvert” datasets. Each day will conclude with a Stata “workshop” where we will illustrate selected methods with real data and Stata code.

Target audience
Quantitative empirical researchers (faculty and graduate students) in social science, including law, political science, economics, many business-school areas (finance, accounting, management, marketing, etc), sociology, education, psychology, etc. – indeed anywhere that causal inference is important.

Minimum prior knowledge
We will assume knowledge, at the level of an upper-level college econometrics or applied statistics course, of how to run multivariate regressions, including OLS, logit, and probit; familiarity with basic probability and statistics including conditional and compound probabilities, confidence intervals, t-statistics, and standard errors; and some understanding of instrumental variables.

Despite its modest prerequisites, this course should be suitable for most researchers with PhD level training and for empirical legal scholars with reasonable but more limited training. For recent PhD's, there will be overlap with what you already know, but much that you don't know, or don't know as well as you should.

Teaching Faculty

We are fortunate to have recruited outstanding experts in causal research design to teach the workshop sessions.

  • Guido Imbens (Stanford University)
    Professor of Economics at Stanford University, Graduate School of Business. Principal research interests: econometric theory, applied econometrics, labor economics. Coauthor with Donald Rubin of Causal Inference in Statistics and Social Sciences (draft 2012). Papers on SSRN
  • Justin McCrary (University of California, Berkeley)
    Professor of Law, University of California, Berkeley. Principal research interests: crime and urban problems, law and economics, corporations, employment discrimination, and empirical legal studies. Papers on SSRN
  • Alberto Abadie (Harvard University)
    Professor of Public Policy at the Kennedy School of Government at Harvard University. Principal research interests: econometrics; program evaluation. Papers on SSRN

Conference Organizers

  • Bernard Black (Northwestern University, Law and Kellogg School of Management)
    Nicholas J. Chabraja Professor at Northwestern University, with positions in the Law School and Kellogg School of Management. Principal research interests: law and finance, international corporate governance, health law and policy; empirical legal studies. Papers on SSRN
  • Mathew McCubbins (University of Southern California)
    Provost Professor of Business, Law and Political Economy at University of Southern California, with positions in the Marshall School of Business, the Gould School of Law, and the Department of Political Science. Principal research interests: legislative organization; communication, learning and decisionmaking; research design; network economics. Papers on SSRN

Workshop Schedule

Monday, June 24 (Guido Imbens)
Introduction to Modern Methods for Causal Inference; Analysis of Randomized Experiments
Overview of causal inference and the Rubin "potential outcomes" causal model. The "gold standard" of a randomized experiment. Treatment and control groups, and the core role of the assignment (to treatment) mechanism. Causal inference as a missing data problem, and imputation of missing potential outcomes.

Reading: Imbens and Rubin, Causal Inference in Statistics and Social Sciences, chapters 1-8 (chapter 2 is background and can be skipped).

Tuesday, June 25 (Guido Imbens)
Pure Observational Studies: Matching and Propensity Score Methods
The core, untestable "unconfoundedness" or "selection on observables" assumption. The need for overlap between treated and control units. Propensity scores, matching methods, blocking on the propensity score. Trimming to deal with lack of overlap. Average treatment effects on the treated (ATT), the controls (ATC) and the whole sample (ATE). Near-equivalence of matching and reweighting.

Reading: Imbens and Rubin, Causal Inference in Statistics and Social Sciences, chapters 12-21, 23

Wednesday, June 26 (Justin McCrary)
Instrumental variable and regression discontinuity methods
Instrumental variables (IV), including (i) the core (untestable) need to satisfy the "only through" exclusion restriction, (ii) heterogeneous treatment effects; (iii) randomized trials or quasi-experiments with noncompliance.

Reading:  Angrist and Pischke, Mostly Harmless Econometrics, chaps. 2, 4

(Regression) discontinuity (RD) research designs: sharp and fuzzy designs; bandwidth choice; need to test (not just assume) covariate balance; discontinuities as substitutes for true randomization and as sources of convincing instruments.

Readings: Justin McCrary and Heather Royer (2011), The Effect of Female Education on Fertility and Infant Health: Evidence from School Entry Laws Using Exact Date of Birth", 101 American Economic Review 158–95.

David S. Lee (2008), Randomized Experiments from Non-random Selection in U.S. House Elections, 142 Journal of Econometrics 675-697.

Thursday, June 27 (Alberto Abadie)
Difference-in-Differences (DiD), Panel Data, and Synthetic Controls
Difference-in-Differences: Simple two-period DiD; the “parallel changes” assumption. Accommodating covariates. Triple differences. Panel data methods. Synthetic controls.

Readings:  Abadie, Alberto, Alexis Diamond and Jens Hainmueller (2010), Synthetic Control Methods for Comparative Case Studies: Estimating the Effect of California's Tobacco Control Program, 105 Journal of the American Statistical Association 493-505.

Angrist, Joshua, and Jorn-Steffen Pischke (2009), Mostly Harmless Econometrics: An Empiricist’s Companion ch. 5.

Recommended (but we know it’s another book to buy):  Cameron, A. Colin, and Pravin Trivedi (2005), Microeconometrics, Methods and Applications chs. 21-23

Thursday post-lunch talk (Bernie Black)
Bloopers
Examples drawn from different areas of how to get causal inference wrong.

Friday morning, June 28 (Alberto Abadie)
Further Topics: Standard Errors, Directed Acyclic Graphs, Event Studies
Standard errors: ordinary, robust, and clustered standard errors. The bootstrap. Introduction to directed acyclic graphs. Event studies.

Readings:  Abadie, Alberto, and Javier Gardeazabal (2003), The Economic Costs of Conflict: A Case Study of the Basque Country, 93 American Economic Review 113-132.

Angrist, Joshua, and Jorn-Steffen Pischke (2009), Mostly Harmless Econometrics: An Empiricist’s Companion ch. 8.

Recommended: Cameron and Trivedi (2005), Microeconometrics, Methods and Applications ch. 11.

Morgan, Steven, and Christopher Winship (2007) Counterfactuals and Causal Inference: Methods and Principles for Social Research ch. 3.

Friday afternoon
Feedback on your own research
Attendees will have an opportunity to present their own research design questions from current work in breakout sessions. Goal: obtain feedback on research design; not present results from a complete paper. Session leaders: Bernie Black, Mat McCubbins, Alberto Abadie. Parallel sessions as needed to meet demand) (15 min to present, 15 min discussion).


[Back to top]


2013 Advanced Causal Inference Workshop

We would like to invite you to attend our first "advanced" workshop on Research Design for Causal Inference. The workshop builds on our "main" workshop. The workshop is sponsored by Northwestern Law School, the Center for the Study of Democracy and the Rule of Law at Duke Law School, and the Society for Empirical Legal Studies.

Monday-Wednesday, August 12-14, 2013, at Northwestern Law School, Chicago, IL

Our regular "Main" Workshop on Research Design for Causal Inference will be held this year on June 24-28, 2013.

Workshop Readings (login required)

Workshop Overview

There continues to be rapid change in the "research frontier" for causal inference, and in what constitutes best practice (normally somewhat inside the frontier). The goal for the advanced workshop is to provide an in-depth discussion of selected topics in causal inference that won’t fit in the main workshop. Our target audience is empirical researchers (faculty and graduate students) who are familiar with the fundamentals of causal inference (from our main workshop, a graduate-level causal inference class, or their own research), who want to extend their knowledge. Most methods are not field-specific. Thus, we expect the workshop to be suitable for researchers in a wide variety of areas, including law, political science, economics, many business-school areas (finance, accounting, management, marketing, etc), medicine, sociology, education, psychology, etc. – indeed anywhere that careful causal inference is important. 

Teaching Faculty

We are fortunate to have recruited outstanding experts in causal research design to teach the workshop sessions.

  • Donald B. Rubin (Harvard University) John L. Loeb Professor of Statistics, Harvard University. His work on what today is often called the "Rubin Causal Model" is central to modern understanding of when one can and cannot infer causation from regression. Principal research interests: statistical methods for causal inference; Bayesian statistics; analysis of incomplete data. Wikipedia
  • Ben Hansen (University of Michigan) Associate Professor of Statistics, University of Michigan, and Faculty Associate at the University of Michigan Institute for Social Research.  Principal research interests: causal inference in observational studies, design-based inference for randomized experiments, statistical computing; applications to health outcomes research, social epidemiology, political science and education evaluation. CV (pdf)
  • Justin McCrary (University of California, Berkeley) Professor of Law, University of California, Berkeley. Principal research interests: crime and urban problems, law and economics, corporations, employment discrimination, and empirical legal studies. Papers on SSRN

Conference Organizers

  • Bernard Black (Northwestern University, Law and Kellogg School of Management) Nicholas J. Chabraja Professor at Northwestern University, with positions in the Law School and Kellogg School of Management. Principal research interests: law and finance, international corporate governance, health law and policy; empirical legal studies. Papers on SSRN
  • Mathew McCubbins (Duke University) Professor of Political Science and Law.  Principal research interests: legislative organization; communication, learning and decisionmaking; research design; network economics. Papers on SSRN

Workshop Schedule

A printable detailed schedule is available.

Monday, August 12 (Don Rubin) Choosing estimands (the science). Implications of choice of estimand for choice of method. Principal stratification. Flexible matching methods. Multiple imputation of missing potential outcomes.

Suggested readings (all are posted):

Imbens, Guido, and Donald Rubin (2013), Causal Inference in Statistics and Social Sciences, chs. xx-yy(BSB note:  We posted the full draft book).

Rubin, Donald B. (2008), For Objective Causal Inference, Design Trumps Analysis, 2 Annals of Applied Statistics 808-840.

On estimands:

Rubin, Donald (2005), Causal Inference Using Potential Outcomes:  Design, Modeling, Decisions, 100 Journal of the American Statistical Association 322-331.

On principal stratification:

Comment:  Principal stratification was first used (although the term was developed later) in the context of instrumental variable estimates of “local average treatment effects” for treatment with non-compliance.  See Angrist, Joshua, Guido Imbens, and Donald Rubin (1996), 91 Journal of the American Statistical Association 444-455.  The groups relevant for this analysis (always takers, never takers, compliers, and defiers) can be understood as principal strata.  If you are not familiar with this use of principal stratification, please read Imbens and Rubin chs. 24-25.

Meulli, Fabrizia, and Donald Rubin (2003), Assumptions Allowing the Estimation of Direct Causal Effects, 112 Journal of Econometrics 79-87.  [Note:  we have not posted the long, complex paper by Peter Adams et al. that Meulli and Rubin are commenting on; you need to know only that they estimate something they call a “direct” causal effect of wealth on mortality by conditioning on health.]

Imbens, Guido, and Donald Rubin (2012), Causal Inference in Statistics and Social Sciences, ch. 28 on Principal Stratification (BSB note:  This chapter is not included in the 2013 version of this book.)

Tuesday, August 13 (Ben Hansen) Advanced matching. This session focuses on the use of propensity-score based, optimal matching methods to reliably secure covariate balance and, with additional assumptions, consistency and robustness of causal effect estimation. Methods are demonstrated with code and exercises in R. Propensity score matching is the best known of a number of techniques that play roles both in study design and in statistical analysis; the session includes a comparative discussion of alternative matching and weighting methods.

Suggested readings (all are posted):

Rosenbaum, Paul R. (2009), Design of Observational Studies, ch. 13 (Matching in R); ch. 18 (After Matching, Before Analysis) (BSB note:  We posted these chapters, but you ought to buy the whole book.  When I did so though Northwestern, it came with a pdf version, which I find very useful.)

Hosman, Ben Hansen and Paul Holland (2010), The sensitivity of linear regression coefficients' confidence limits to the omission of a confounder, 4 Annals of Applied Statistics 849-870, esp. §§ 1, 2.1, and 4.3.

Imbens and Rubin (12013), ch. 23 (Sensitivity Analysis and Bounds).

Wednesday, August 14 (Justin McCrary) Conducting simulation studies. Generalized method of moments (GMM) as a tool for estimating treatment effects and standard errors, including adjusting standard errors for two-step estimation (e.g., reweighting). Inference and testing using the bootstrap. Topics in regression discontinuity design: nonparametric estimation; Local linear regression and density estimation; choosing bandwidth and assessing sensitivity to bandwidth choice.

Suggested readings (all are posted):

Cameron, A. Colin, and Praveen Trivedi (2010), Microeconometrics Using Stata, chapter 4 and chapter 13 (BSB note:  We posted these chapters).

An additional source for simulation in Stata is Adkins, Lee, and Mary Gade (2012), Monte Carlo Experiments Using Stata:  A Primer with Examples.

Hansen, Bruce (2013), Econometrics, chs. 10-12, 14 (BSB note:  We posted the full book).

[Back to top]


2012 Third Annual Main Causal Inference Workshop

Workshop Overview
Research design for causal inference is at the heart of a "credibility revolution" in empirical research. We will cover the design of true randomized experiments and contrast them to simulations and quasi-experiments, where part of the sample is "treated" in some way, and the remainder is a control group, but the researcher controls neither the assignment of cases to treatment and control groups nor administration of the treatment. We will assess the kinds of causal inferences one can and cannot draw from a research design, threats to valid inference, and research designs that can mitigate those threats.

Most empirical methods courses begin with the methods. They survey how each method works, and what assumptions each relies on. We will begin instead with the goal of causal inference, and discuss how to design research to come closer to that goal. The methods reflect the goal and are often adapted to the needs of a particular study. Some of the methods we will discuss are covered in PhD programs, but rarely in depth, and rarely with a focus on causal inference and on which methods to prefer for messy, real-world datasets with limited sample sizes.

Each day will conclude with a Stata "workshop" where we will illustrate selected methods with real data and Stata code.

Target audience
Quantitative empirical researchers (faculty and graduate students) in social science, including law, political science, economics, many business-school areas (finance, accounting, management, marketing, etc), sociology, education, psychology, etc. - indeed anywhere that causal inference is important.

Minimum prior knowledge
We will assume knowledge, at the level of an upper-level college econometrics or applied statistics course, of how to run multivariate regressions, including OLS, logit, and probit; familiarity with basic probability and statistics including conditional and compound probabilities, confidence intervals, t-statistics, and standard errors; and some understanding of instrumental variables are and how they are used.

Despite its modest prerequisites, this course should be suitable for most researchers with PhD level training and for empirical legal scholars with reasonable even if more limited training. Especially for recent PhD's, there will be overlap with what you already know, but much that you don't know, or don't know as well as you should.

Teaching Faculty

We are fortunate to have recruited outstanding experts in causal research design to teach the workshop sessions. Click on the links below and see for yourself.

  • Donald B. Rubin (Harvard University) is John L. Loeb Professor of Statistics, Harvard University. His work on what today is often called the "Rubin Causal Model" is central to modern understanding of when one can and cannot infer causation from regression. Principal research interests: statistical methods for causal inference; Bayesian statistics; analysis of incomplete data. Wikipedia
  • Justin McCrary (University of California, Berkeley) is Professor of Law, University of California, Berkeley. Principal research interests: crime and urban problems, law and economics, corporations, employment discrimination, and empirical legal studies.
  • Alberto Abadie (Harvard University) is Professor of Public Policy at the Kennedy School of Government at Harvard University. Principal research interests: econometrics; program evaluation. Papers on SSRN
  • Jens Hainmueller (MIT, Political Science) is Associate Professor at the Massachusetts Institute of Technology. Principal research interests: applied statistics, immigration, political economy, program evaluation. Web page with link to CV and synthetic controls software.

Conference Organizers

  • Bernard Black (Northwestern University, Law and Kellogg School of Management) is Nicholas J. Chabraja Professor at Northwestern University, with positions in the Law School and Kellogg School of Management. Principal research interests: law and finance, international corporate governance, health law and policy; empirical legal studies. Papers on SSRN
  • Mathew McCubbins (University of Southern California) is Provost Professor of Business, Law and Political Economy at University of Southern California, with positions in the Marshall School of Business, the Gould School of Law, and the Department of Political Science. Principal research interests: legislative organization; communication, learning and decisionmaking; research design; network economics. Papers on SSRN

Workshop Schedule

Detailed Workshop Schedule (pdf) | Workshop Readings (login required)

Monday, August 6 (Don Rubin)
Overview of causal inference and the Rubin "potential outcomes" causal model. The "gold standard" of a randomized experiment. Treatment and control groups, and the core role of the assignment (to treatment) mechanism. Multiple imputation of missing potential outcomes.

Tuesday, August 7 (Justin McCrary)
Instrumental variables (IV), including (i) the core (untestable) need to satisfy the "only through" exclusion restriction, (ii) heterogeneous treatment effects; (iii) randomized trials or quasi-experiments with noncompliance.

(Regression) discontinuity (RD) research designs: sharp and fuzzy designs; bandwidth choice; need to test (not just assume) covariate balance; discontinuities as substitutes for true randomization and as sources of convincing instruments.

Wednesday, August 8 (Alberto Abadie)
Selection (only) on observables, matching and subclassification, comparing matching to regression, propensity score methods, handling poorly matched observations.

Thursday, August 9 (Alberto Abadie)
Matching methods continued, combining matching with DiD, synthetic controls, standard error issues (robust and clustered standard errors, bootstrapping).

Thursday post-lunch talk (Bernie Black): Bloopers
Examples drawn from different areas of how to get causal inference wrong.

Friday morning, August 10 (Jens Hainmueller)
Panel data: Fixed and random effects, and the choice between then. DiD with multiple before and after time periods. Standard errors - what do you cluster on?

Friday afternoon: Feedback on your own research
Attendees will have an opportunity to present their own research questions from current work in breakout sessions and receive group feedback. Session leaders: Bernie Black, Jens Hainmueller, Mat McCubbins,(parallel sessions as needed to meet demand) (15 min to present, 15 min discussion)

[Back to top]


2011 Second Annual Causal Inference: Frequentist Methods Workshop

Dates and Location: Monday - Friday, August 15-19, 2011, at Northwestern Law School, Chicago, IL

Overview of the Causal Inference Workshop
Research design for causal inference is at the heart of a "credibility revolution" in empirical research. We will cover the design of true randomized experiments and contrast them to simulations and quasi-experiments, where part of the sample is "treated" in some way, and the remainder is a control group, but the researcher controls neither the assignment of cases to treatment and control groups nor administration of the treatment. We will assess the kinds of causal inferences one can and cannot draw from a research design, threats to valid inference, and research designs that can mitigate those threats.

Most empirical methods courses begin with the methods. They survey how each method works, and what assumptions each relies on. We will begin instead with the goal of causal inference, and discuss how to design research to come closer to that goal. The methods reflect the goal and are often adapted to the needs of a particular study. Some of the methods we will discuss are covered in PhD programs, but rarely in depth, and rarely with a focus on causal inference and on which methods to prefer for messy, real-world datasets with limited sample sizes.

Each day will conclude with a Stata "workshop" where we will illustrate selected methods with real data and Stata code.

Target audience
Quantitative empirical researchers (faculty and graduate students) in social science, including law, political science, economics, many business-school areas (finance, accounting, management, marketing, etc), sociology, education, psychology, etc. - indeed anywhere that causal inference is important.

Minimum prior knowledge
We will assume knowledge, at the level of an upper-level college econometrics or applied statistics course, of how to run multivariate regressions, including OLS, logit, and probit; familiarity with basic probability and statistics including conditional and compound probabilities, confidence intervals, t-statistics, and standard errors; and some understanding of instrumental variables are and how they are used.

Despite its modest prerequisites, this course should be suitable for most researchers with PhD level training and for empirical legal scholars with reasonable even if more limited training. Especially for recent PhD's, there will be overlap with what you already know, but much that you don't know, or don't know as well as you should.

Teaching Faculty
We are fortunate to have recruited outstanding experts in causal research design to teach the workshop sessions. Click on the links below and see for yourself.

  • Alberto Abadie (Harvard University) is Professor of Public Policy at the Kennedy School of Government at Harvard University. Principal research interests: econometrics; program evaluation. Papers on SSRN
  • Joshua Angrist (MIT) is Ford Professor of Economics at MIT. Principal research interests: labor economics; econometrics. Author of Joshua Angrist and Jorn-Steffen Pischke, Mostly Harmless Econometrics: An Empiricist's Companion (2009). Mostly Harmless Econometrics blog. Papers on SSRN
  • Theodore Eisenberg (Cornell University) is Henry Allen Mark Professor of Law and Adjunct Professor of Statistical Sciences at Cornell University, and the founding editor of the Journal of Empirical Legal Studies. Principal research interest: empirical studies of the legal system. Papers on SSRN
  • Daniel E. Ho (Stanford University) is Professor of Law and Robert E. Paradise Faculty Fellow for Excellence in Teaching and Research at Stanford Law School. Principal research interests: empirical legal studies; administrative law; judicial behavior; antidiscrimination law. Papers on SSRN
  • Guido Imbens (Harvard University) is Professor of Economics at Harvard. Principal research interests: econometric theory, applied econometrics, labor economics. Coauthor with Donald Rubin of Causal Inference in Statistics and Social Sciences (forthcoming 2011). Papers on SSRN

Conference Organizers

  • Bernard Black (Northwestern University, Law and Kellogg School of Management) is Nicholas J. Chabraja Professor at Northwestern University, with positions in the Law School and Kellogg School of Management, Department of Finance. Principal research interests: law and finance, international corporate governance, health law and policy; empirical legal studies. Papers on SSRN
  • Mathew McCubbins (University of Southern California) is Provost Professor of Business, Law and Political Economy at University of Southern California, with positions in the Marshall School of Business, the Gould School of Law, and the Department of Political Science. Principal research interests: legislative organization; communication, learning and decisionmaking; research design; network economics. Papers on SSRN

Workshop Schedule [Detailed Schedule (pdf)]

Breakfast will be available each morning from 8:30 a.m.
A registration table will be open Monday beginning 8:00 a.m.

  • Monday, August 15 (Joshua Angrist)
    • Introduction by Bernie Black: Lessons We Hope You'll Learn This Week
    • Overview of regression analysis, and when it does and does not permit causal inference
      Angrist and Pischke, Mostly Harmless Econometrics, chaps 1-3 (mostly 3)
    • Introduction to instrumental variables with constant treatment effects
      Angrist and Pischke, Mostly Harmless Econometrics, chap 4
    Stata examples: (i) control variables and omitted variable bias; (ii) choice of method for limited dependent variables.
Monday evening
  • Reception for attendees from 5:00-6:30 p.m.

  • Tuesday, August 16 (Joshua Angrist)
    • Instrumental variables (i) with heterogeneous treatment effects; and (ii) in randomized trials or quasi-experiments with noncompliance
      Angrist and Pischke, Mostly Harmless Econometrics, chap 4
    • Difference-in-differences (DiD) analysis
      Angrist and Pischke, Mostly Harmless Econometrics, chap 5
    Stata examples: good and bad instruments; DiD analysis
  • Wednesday, August 17 (Guido Imbens)
    • Research design in observational studies
    • Methods for improving matching and covariate balance
    Stata examples: assessing and improving covariate balance
  • Thursday, August 18 (Alberto Abadie)
    • Matching methods; choice among matching, regression, or a combination
    • Synthetic controls
    • Standard error issues (robust and clustered standard errors, bootstrapping)
    Stata examples: nnmatch.ado; propensity score weighting; standard errors

Thursday post-lunch talk (Bernie Black)

    • Bloopers: Examples, drawn from different areas, of how to get causal inference wrong.
  • Friday, August 19 - morning (Daniel Ho)
    • Regression discontinuity: sharp and fuzzy designs; bandwidth choice; parametric and nonparametric options; covariate balance.
    Stata examples: sharp discontinuity design; combining RD and DiD

Friday post-lunch talk (Ted Eisenberg)

  • Strengths and Weaknesses in Empirical Studies: Observations of a Journal Editor

Friday afternoon

  • Feedback on your own research: Attendees will have an opportunity to present their own research questions from current work in breakout sessions (session leaders: Bernie Black, Ted Eisenberg, Dan Ho, Mat McCubbins)
  • Present your own research design issues; get group feedback (parallel sessions as needed to meet demand) (15 min to present, 15 min discussion)

Workshop Materials

Workshop materials are available only to participants.

Login

The username and password have been sent to all participants. If you can't locate them, please e-mail causalinference@law.northwestern.edu

[Back to top]


2010 First Annual Causal Inference: Frequentist Methods Workshop

Dates and Location: Monday - Friday, August 16-20, 2010, at Northwestern Law

Overview: Research design for causal inference is at the heart of a "credibility revolution" in empirical research in the last 15 years that spans many fields. We will cover the design of true randomized experiments and contrast them to simulations and to quasi-experiments, where part of the sample is "treated" in some way, and the remainder is a control group, but the researcher controls neither the assignment of cases to treatment and control groups nor administration of the treatment. We will carefully describe the kinds of causal inferences one can and cannot draw from a research design, various threats to valid inference, and research designs commonly used to mitigate those threats.

Most empirical methods courses begin with the methods. They survey how each method works, and what assumptions each relies on. We will begin instead with the goal of causal inference, and discuss how to design research to come closer to that goal. The methods reflect the goal and are often adapted to the needs of a particular study. Some of the methods we will discuss are covered in PhD programs, but rarely in depth, and rarely with a focus on causal inference and on which methods to prefer for messy, real-world datasets with limited sample sizes.

Target audience: Quantitative empirical researchers (faculty and graduate students) in social science, including law, political science, economics, most business-school areas (finance, accounting, marketing, etc), sociology, education, psychology – indeed anywhere that causal inference is important.

Minimum prior knowledge: We will assume knowledge, at the level of an upper-level college econometrics or applied statistics course, of how to run simple regressions, including OLS, logit, and probit; familiarity with basic probability and statistics including conditional and compound probabilities; confidence intervals, t-statistics and standard errors.

Despite its modest prerequisites, this course should be suitable for most researchers with PhD level training and for empirical legal scholars with reasonable even if more limited training. Especially for recent PhDs, there will be overlap with what you already know, but also much that you don't know, or don't know as well as you need to.

Teaching Faculty
We are fortunate to have recruited truly outstanding experts in causal research design to teach the workshop sessions.

  • Nathaniel Beck (New York University)
    Nathaniel Beck is Professor of Politics at NYU. Principal research interests: political methodology (empirical methods applied to political science research), especially research design, time-series and time-series cross section analysis; comparative political economy.
  • Thomas Cook (Northwestern University)
    Thomas Cook is Joan and Sarepta Harrison Chair in Ethics and Justice, Professor of Sociology, Psychology, Education and Social Policy, and Faculty Fellow, Institute for Policy Research, at Northwestern University. He is a leading scholar on experimental and quasi-experimental design and regression discontinuity and a coauthor of a classic text, Shadish, Cook, and Campbell, Experimental and Quasi-Experimental Designs for Generalized Causal Inference (2002).
  • Guido Imbens (Harvard University)
    Guido Imbens is Professor of Economics at Harvard. Principal research interests: econometric theory, applied econometrics, labor economics. Coauthor with Donald Rubin of Causal Inference in Statistics and Social Sciences (forthcoming 2010). Papers on SSRN
  • Jonathan N. Katz (California Institute of Technology)
    Jonathan Katz is Professor of Social Sciences and Statistics and Chair of the Division of the Humanities and Social Sciences at Caltech. Principal research interests: American politics, political methodology; formal political theory.
  • Kevin Quinn (University of California, Berkeley)
    Kevin Quinn is Professor of Law at University of California, Berkeley, School of Law. Principal research interests: empirical methods in law and political science. Papers on SSRN
  • Donald Rubin (Harvard University)
    Donald Rubin is John L. Loeb Professor of Statistics at Harvard University. His work on what today is often called the "Rubin Causal Model" is central to modern understanding of when one can and cannot infer causation from regression. Principal research interests: statistical methods for causal inference; Bayesian statistics. Wikipedia entry

Conference Organizers

  • Bernard Black (Northwestern University Law and Kellogg School of Management)
    Bernie Black is Nicholas J. Chabraja Professor at Northwestern University, with positions in the Law School and Kellogg School of Management, Department of Finance. Principal research interests: law and finance, international corporate governance, health law and policy; empirical legal studies. Papers on SSRN
  • Mathew McCubbins (University of Southern California)
    Mat McCubbins is Provost Professor of Business, Law and Political Economy at University of Southern California, with positions in the Marshall School of Business, the Gould School of Law, and the Department of Political Science. Principal research interests: legislative organization; communication, learning and decisionmaking; research design; network economics. Papers on SSRN

Workshop Outline [Detailed Schedule (pdf)]

  • Day 1: Introduction to Research Design for Causal Inference [Donald Rubin]
    • Introduction [Mathew McCubbins]
    • Morning: brass standard: Ordinary regression; its uses and limits
    • Afternoon: gold standard: randomized controlled trial with predefined goal and endpoint
  • Day 2: Estimating Treatment Effects [Guido Imbens]
    • Morning: Research design to produce "ignorable" assignment to treatment or control group, conditioned on observable controls ("unconfounded" assignment)
    • Afternoon: Addressing non-ignorable assignment to treatment group
  • Day 3: Panel Data [Jonathan Katz]
    • Morning: Basic panel data designs
    • Post-lunch lecture: The importance of choice of control variables in observational studies [Thomas Cook]
    • Afternoon: difference-in-differences; standard errors; wide panels; triple difference designs
  • Day 4: Instrumental Variables and Selection Bias [Nathaniel Beck]
    • Morning: The basics of instrumental variables
    • Afternoon: weak instruments; simultaneous equations; multiple instruments (overidentification); tests for instrument validity. Selection bias and ways to address it.
  • Day 5 morning: Interpreting Treatment Effects [Kevin Quinn]
    • What inferences can one draw from a research design for a heterogeneous population: sample versus population average treatment effects (ATE); local average treatment effects (LATE); average treatment effect on the treated (ATET); intent to treat effect (ITT), etc.
    • Post-lunch lecture: Bloopers [Bernie Black].
      Examples, drawn from different areas, of how to get causal inference wrong.
    • Day 5 afternoon: Feedback on your own research [led by Black and McCubbins]
      present your own research design issues; get group feedback (parallel sessions as needed to meet demand) (15 min to present, 15 min discussion)

Workshop Materials

Workshop materials are available only to participants.

Login

To obtain the username and password, please e-mail causalinference@law.northwestern.edu

[Back to top]


2011 Causal Inference: Bayesian Methods Workshop

Dates and Location: July 11-13, 2011, at Northwestern Law School, Chicago, IL

Overview of the Bayesian Causal Inference Workshop
Research design for causal inference is at the heart of a "credibility revolution" in empirical research. Credible causal inference often requires researchers not to rely on the linearity and normality assumptions underlying classical regression. Bayesian imputation and simulation methods provide many of the analytic tools for doing so. We will cover Bayesian methods for research design and analysis for true randomized experiments, as well as for quasi-experiments where part of the sample is "treated" in some way, the control group is drawn from the rest of the sample, but the researcher controls neither the assignment of units to treatment nor administration of the treatment. These analytic methods include multiple imputation of missing "potential outcomes", using Markov chain Monte Carlo (MCMC) simulations, including Gibbs sample, and other flexible model specifications.

Most empirical methods courses begin with the methods. They survey how each method works, and what assumptions each relies on. We will begin instead with the goal of causal inference and the centrality of research design, and discuss how Bayesian methods allow research designs that better achieve that goal.

The workshop will include an introduction to Winbugs, the principal public domain Bayesian inference software. We will illustrate selected methods with real data and Winbugs code.

Target audience
Quantitative empirical researchers (faculty and graduate students) in social science, including law, political science, economics, many business-school areas (finance, accounting, management, marketing, etc), sociology, education, psychology, etc. - indeed anywhere that causal inference is important.

Minimum prior knowledge
We will assume knowledge, at the level of an upper-level college econometrics or applied statistics course, of how to run multivariate regressions, including OLS, logit, and probit; familiarity with basic probability and statistics including conditional and compound probabilities, confidence intervals, t-statistics, and standard errors; and some understanding of instrumental variables are and how they are used.

Despite its modest prerequisites, this course should be suitable for most researchers with PhD level training and for empirical legal scholars with reasonable even if more limited training. Especially for recent PhD's, there will be overlap with what you already know, but much that you don't know, or don't know as well as you should.

Teaching Faculty
We are fortunate to have recruited outstanding experts in causal research design to teach the workshop sessions.

  • Donald B. Rubin (Harvard University) is John L. Loeb Professor of Statistics, Harvard University. His work on what today is often called the "Rubin Causal Model" is central to modern understanding of when one can and cannot infer causation from regression. Principal research interests: statistical methods for causal inference; Bayesian statistics; analysis of incomplete data. Wikipedia
  • Jeff Gill (Washington University in St. Louis) is Director of the Center for Applied Statistics, Professor of Political Science and Professor of Mathematics at Washington University in St. Louis, and the author of Bayesian Methods: A Social and Behavioral Approach (second edition 2007). Principal research interests: political methodology, Bayesian research methods; statistical computing. Wikipedia

Conference Organizers

  • Bernard Black (Northwestern University, Law and Kellogg School of Management) is Nicholas J. Chabraja Professor at Northwestern University, with positions in the Law School and Kellogg School of Management. Principal research interests: law and finance, international corporate governance, health law and policy; empirical legal studies. Papers on SSRN
  • Mathew McCubbins (University of Southern California) is Provost Professor of Business, Law and Political Economy at University of Southern California, with positions in the Marshall School of Business, the Gould School of Law, and the Department of Political Science. Principal research interests: legislative organization; communication, learning and decisionmaking; research design; network economics. Papers on SSRN

Workshop Schedule

The workshop will run for three full days from 9:00 a.m.-4:30 p.m., plus a reception on Monday from 4:30 p.m.-6:00 p.m. There will be roughly six 50-min lectures each day, plus a lunch break from roughly 12:15 p.m.-1:30 p.m. The starting time is fixed, all else is approximate.

  • Monday July 11 (Don Rubin)
    The "potential outcomes (with assignment mechanism) or "Rubin causal model" approach to causal inference. Understanding the assignment mechanism. Estimating treatment effects using multiple imputation of the missing outcomes for both treated and control units. Comparison of multiple imputation to regression methods. Research design for credible causal inference. Applications to randomized experiments and to "natural" or quasi-experiments with "unconfounded" assignment of units to treatment.
  • Tuesday July 12 (Jeff Gill)
    The basics of Bayesian inference, including the probabilistic treatment of uncertainty, prior and posterior distributions, updating knowledge, and describing results. Comparison of Bayesian to frequentist approaches. Bayesian approaches to construction and assessment of linear and nonlinear regression-style models.
  • Wednesday, July 13 (Jeff Gill)
    Bayesian stochastic simulation. Modern applied Bayesian work is integrally tied to Markov chain Monte Carlo simulation (MCMC) tools. We will develop the theory behind MCMC tools and then learn how to use these tools, using WinBUGS (a well-known, high quality, free program).

Workshop Materials

Workshop materials are available only to participants.

Login

To obtain the username and password, please e-mail causalinference@law.northwestern.edu

[Back to top]