Past Workshops

This page provides information for our past workshops.

You can jump to a specific workshop (within this overall page) using the links below:

• 2012 Main Workshop
• 2011 Frequentist Workshop
• 2010 Frequentist Workshop
• 2011 Bayesian Workshop

2012 Third Annual Main Causal Inference Workshop

Workshop Overview
Research design for causal inference is at the heart of a "credibility revolution" in empirical research. We will cover the design of true randomized experiments and contrast them to simulations and quasi-experiments, where part of the sample is "treated" in some way, and the remainder is a control group, but the researcher controls neither the assignment of cases to treatment and control groups nor administration of the treatment. We will assess the kinds of causal inferences one can and cannot draw from a research design, threats to valid inference, and research designs that can mitigate those threats.

Most empirical methods courses begin with the methods. They survey how each method works, and what assumptions each relies on. We will begin instead with the goal of causal inference, and discuss how to design research to come closer to that goal. The methods reflect the goal and are often adapted to the needs of a particular study. Some of the methods we will discuss are covered in PhD programs, but rarely in depth, and rarely with a focus on causal inference and on which methods to prefer for messy, real-world datasets with limited sample sizes.

Each day will conclude with a Stata "workshop" where we will illustrate selected methods with real data and Stata code.

Target audience
Quantitative empirical researchers (faculty and graduate students) in social science, including law, political science, economics, many business-school areas (finance, accounting, management, marketing, etc), sociology, education, psychology, etc. - indeed anywhere that causal inference is important.

Minimum prior knowledge
We will assume knowledge, at the level of an upper-level college econometrics or applied statistics course, of how to run multivariate regressions, including OLS, logit, and probit; familiarity with basic probability and statistics including conditional and compound probabilities, confidence intervals, t-statistics, and standard errors; and some understanding of instrumental variables are and how they are used.

Despite its modest prerequisites, this course should be suitable for most researchers with PhD level training and for empirical legal scholars with reasonable even if more limited training. Especially for recent PhD's, there will be overlap with what you already know, but much that you don't know, or don't know as well as you should.

Teaching Faculty

We are fortunate to have recruited outstanding experts in causal research design to teach the workshop sessions. Click on the links below and see for yourself.

• Donald B. Rubin (Harvard University) is John L. Loeb Professor of Statistics, Harvard University. His work on what today is often called the "Rubin Causal Model" is central to modern understanding of when one can and cannot infer causation from regression. Principal research interests: statistical methods for causal inference; Bayesian statistics; analysis of incomplete data. Wikipedia
• Justin McCrary (University of California, Berkeley) is Professor of Law, University of California, Berkeley. Principal research interests: crime and urban problems, law and economics, corporations, employment discrimination, and empirical legal studies.
• Alberto Abadie (Harvard University) is Professor of Public Policy at the Kennedy School of Government at Harvard University. Principal research interests: econometrics; program evaluation. Papers on SSRN
• Jens Hainmueller (MIT, Political Science) is Associate Professor at the Massachusetts Institute of Technology. Principal research interests: applied statistics, immigration, political economy, program evaluation. Web page with link to CV and synthetic controls software.

Conference Organizers

• Bernard Black (Northwestern University, Law and Kellogg School of Management) is Nicholas J. Chabraja Professor at Northwestern University, with positions in the Law School and Kellogg School of Management. Principal research interests: law and finance, international corporate governance, health law and policy; empirical legal studies. Papers on SSRN
• Mathew McCubbins (University of Southern California) is Provost Professor of Business, Law and Political Economy at University of Southern California, with positions in the Marshall School of Business, the Gould School of Law, and the Department of Political Science. Principal research interests: legislative organization; communication, learning and decisionmaking; research design; network economics. Papers on SSRN

Workshop Schedule

Detailed Workshop Schedule (pdf) | Workshop Readings (login required)

Monday, August 6 (Don Rubin)
Overview of causal inference and the Rubin "potential outcomes" causal model. The "gold standard" of a randomized experiment. Treatment and control groups, and the core role of the assignment (to treatment) mechanism. Multiple imputation of missing potential outcomes.

Tuesday, August 7 (Justin McCrary)
Instrumental variables (IV), including (i) the core (untestable) need to satisfy the "only through" exclusion restriction, (ii) heterogeneous treatment effects; (iii) randomized trials or quasi-experiments with noncompliance.

(Regression) discontinuity (RD) research designs: sharp and fuzzy designs; bandwidth choice; need to test (not just assume) covariate balance; discontinuities as substitutes for true randomization and as sources of convincing instruments.

Wednesday, August 8 (Alberto Abadie)
Selection (only) on observables, matching and subclassification, comparing matching to regression, propensity score methods, handling poorly matched observations.

Thursday, August 9 (Alberto Abadie)
Matching methods continued, combining matching with DiD, synthetic controls, standard error issues (robust and clustered standard errors, bootstrapping).

Thursday post-lunch talk (Bernie Black): Bloopers
Examples drawn from different areas of how to get causal inference wrong.

Friday morning, August 10 (Jens Hainmueller)
Panel data: Fixed and random effects, and the choice between then. DiD with multiple before and after time periods. Standard errors - what do you cluster on?

Friday afternoon: Feedback on your own research
Attendees will have an opportunity to present their own research questions from current work in breakout sessions and receive group feedback. Session leaders: Bernie Black, Jens Hainmueller, Mat McCubbins,(parallel sessions as needed to meet demand) (15 min to present, 15 min discussion)

[Back to top]

2011 Second Annual Causal Inference: Frequentist Methods Workshop

Dates and Location: Monday - Friday, August 15-19, 2011, at Northwestern Law School, Chicago, IL

Overview of the Causal Inference Workshop
Research design for causal inference is at the heart of a "credibility revolution" in empirical research. We will cover the design of true randomized experiments and contrast them to simulations and quasi-experiments, where part of the sample is "treated" in some way, and the remainder is a control group, but the researcher controls neither the assignment of cases to treatment and control groups nor administration of the treatment. We will assess the kinds of causal inferences one can and cannot draw from a research design, threats to valid inference, and research designs that can mitigate those threats.

Most empirical methods courses begin with the methods. They survey how each method works, and what assumptions each relies on. We will begin instead with the goal of causal inference, and discuss how to design research to come closer to that goal. The methods reflect the goal and are often adapted to the needs of a particular study. Some of the methods we will discuss are covered in PhD programs, but rarely in depth, and rarely with a focus on causal inference and on which methods to prefer for messy, real-world datasets with limited sample sizes.

Each day will conclude with a Stata "workshop" where we will illustrate selected methods with real data and Stata code.

Target audience
Quantitative empirical researchers (faculty and graduate students) in social science, including law, political science, economics, many business-school areas (finance, accounting, management, marketing, etc), sociology, education, psychology, etc. - indeed anywhere that causal inference is important.

Minimum prior knowledge
We will assume knowledge, at the level of an upper-level college econometrics or applied statistics course, of how to run multivariate regressions, including OLS, logit, and probit; familiarity with basic probability and statistics including conditional and compound probabilities, confidence intervals, t-statistics, and standard errors; and some understanding of instrumental variables are and how they are used.

Despite its modest prerequisites, this course should be suitable for most researchers with PhD level training and for empirical legal scholars with reasonable even if more limited training. Especially for recent PhD's, there will be overlap with what you already know, but much that you don't know, or don't know as well as you should.

Teaching Faculty
We are fortunate to have recruited outstanding experts in causal research design to teach the workshop sessions. Click on the links below and see for yourself.

• Alberto Abadie (Harvard University) is Professor of Public Policy at the Kennedy School of Government at Harvard University. Principal research interests: econometrics; program evaluation. Papers on SSRN
• Joshua Angrist (MIT) is Ford Professor of Economics at MIT. Principal research interests: labor economics; econometrics. Author of Joshua Angrist and Jorn-Steffen Pischke, Mostly Harmless Econometrics: An Empiricist's Companion (2009). Mostly Harmless Econometrics blog. Papers on SSRN
• Theodore Eisenberg (Cornell University) is Henry Allen Mark Professor of Law and Adjunct Professor of Statistical Sciences at Cornell University, and the founding editor of the Journal of Empirical Legal Studies. Principal research interest: empirical studies of the legal system. Papers on SSRN
• Daniel E. Ho (Stanford University) is Professor of Law and Robert E. Paradise Faculty Fellow for Excellence in Teaching and Research at Stanford Law School. Principal research interests: empirical legal studies; administrative law; judicial behavior; antidiscrimination law. Papers on SSRN
• Guido Imbens (Harvard University) is Professor of Economics at Harvard. Principal research interests: econometric theory, applied econometrics, labor economics. Coauthor with Donald Rubin of Causal Inference in Statistics and Social Sciences (forthcoming 2011). Papers on SSRN

Conference Organizers

• Bernard Black (Northwestern University, Law and Kellogg School of Management) is Nicholas J. Chabraja Professor at Northwestern University, with positions in the Law School and Kellogg School of Management, Department of Finance. Principal research interests: law and finance, international corporate governance, health law and policy; empirical legal studies. Papers on SSRN
• Mathew McCubbins (University of Southern California) is Provost Professor of Business, Law and Political Economy at University of Southern California, with positions in the Marshall School of Business, the Gould School of Law, and the Department of Political Science. Principal research interests: legislative organization; communication, learning and decisionmaking; research design; network economics. Papers on SSRN

Workshop Schedule [Detailed Schedule (pdf)]

Breakfast will be available each morning from 8:30 a.m.
A registration table will be open Monday beginning 8:00 a.m.

Thursday post-lunch talk (Bernie Black)

Friday post-lunch talk (Ted Eisenberg)

Friday afternoon

Workshop Materials

Workshop materials are available only to participants.

Login

The username and password have been sent to all participants. If you can't locate them, please e-mail causalinference@law.northwestern.edu

[Back to top]

2010 First Annual Causal Inference: Frequentist Methods Workshop

Dates and Location: Monday - Friday, August 16-20, 2010, at Northwestern Law

Overview: Research design for causal inference is at the heart of a "credibility revolution" in empirical research in the last 15 years that spans many fields. We will cover the design of true randomized experiments and contrast them to simulations and to quasi-experiments, where part of the sample is "treated" in some way, and the remainder is a control group, but the researcher controls neither the assignment of cases to treatment and control groups nor administration of the treatment. We will carefully describe the kinds of causal inferences one can and cannot draw from a research design, various threats to valid inference, and research designs commonly used to mitigate those threats.

Most empirical methods courses begin with the methods. They survey how each method works, and what assumptions each relies on. We will begin instead with the goal of causal inference, and discuss how to design research to come closer to that goal. The methods reflect the goal and are often adapted to the needs of a particular study. Some of the methods we will discuss are covered in PhD programs, but rarely in depth, and rarely with a focus on causal inference and on which methods to prefer for messy, real-world datasets with limited sample sizes.

Target audience: Quantitative empirical researchers (faculty and graduate students) in social science, including law, political science, economics, most business-school areas (finance, accounting, marketing, etc), sociology, education, psychology – indeed anywhere that causal inference is important.

Minimum prior knowledge: We will assume knowledge, at the level of an upper-level college econometrics or applied statistics course, of how to run simple regressions, including OLS, logit, and probit; familiarity with basic probability and statistics including conditional and compound probabilities; confidence intervals, t-statistics and standard errors.

Despite its modest prerequisites, this course should be suitable for most researchers with PhD level training and for empirical legal scholars with reasonable even if more limited training. Especially for recent PhDs, there will be overlap with what you already know, but also much that you don't know, or don't know as well as you need to.

Teaching Faculty
We are fortunate to have recruited truly outstanding experts in causal research design to teach the workshop sessions.

• Nathaniel Beck (New York University)
  Nathaniel Beck is Professor of Politics at NYU. Principal research interests: political methodology (empirical methods applied to political science research), especially research design, time-series and time-series cross section analysis; comparative political economy.
• Thomas Cook (Northwestern University)
  Thomas Cook is Joan and Sarepta Harrison Chair in Ethics and Justice, Professor of Sociology, Psychology, Education and Social Policy, and Faculty Fellow, Institute for Policy Research, at Northwestern University. He is a leading scholar on experimental and quasi-experimental design and regression discontinuity and a coauthor of a classic text, Shadish, Cook, and Campbell, Experimental and Quasi-Experimental Designs for Generalized Causal Inference (2002).
• Guido Imbens (Harvard University)
  Guido Imbens is Professor of Economics at Harvard. Principal research interests: econometric theory, applied econometrics, labor economics. Coauthor with Donald Rubin of Causal Inference in Statistics and Social Sciences (forthcoming 2010). Papers on SSRN
• Jonathan N. Katz (California Institute of Technology)
  Jonathan Katz is Professor of Social Sciences and Statistics and Chair of the Division of the Humanities and Social Sciences at Caltech. Principal research interests: American politics, political methodology; formal political theory.
• Kevin Quinn (University of California, Berkeley)
  Kevin Quinn is Professor of Law at University of California, Berkeley, School of Law. Principal research interests: empirical methods in law and political science. Papers on SSRN
• Donald Rubin (Harvard University)
  Donald Rubin is John L. Loeb Professor of Statistics at Harvard University. His work on what today is often called the "Rubin Causal Model" is central to modern understanding of when one can and cannot infer causation from regression. Principal research interests: statistical methods for causal inference; Bayesian statistics. Wikipedia entry

Conference Organizers

• Bernard Black (Northwestern University Law and Kellogg School of Management)
  Bernie Black is Nicholas J. Chabraja Professor at Northwestern University, with positions in the Law School and Kellogg School of Management, Department of Finance. Principal research interests: law and finance, international corporate governance, health law and policy; empirical legal studies. Papers on SSRN
• Mathew McCubbins (University of Southern California)
  Mat McCubbins is Provost Professor of Business, Law and Political Economy at University of Southern California, with positions in the Marshall School of Business, the Gould School of Law, and the Department of Political Science. Principal research interests: legislative organization; communication, learning and decisionmaking; research design; network economics. Papers on SSRN

Workshop Outline [Detailed Schedule (pdf)]

• Day 1: Introduction to Research Design for Causal Inference [Donald Rubin]
  • Introduction [Mathew McCubbins]
  • Morning: brass standard: Ordinary regression; its uses and limits
  • Afternoon: gold standard: randomized controlled trial with predefined goal and endpoint
• Day 2: Estimating Treatment Effects [Guido Imbens]
  • Morning: Research design to produce "ignorable" assignment to treatment or control group, conditioned on observable controls ("unconfounded" assignment)
  • Afternoon: Addressing non-ignorable assignment to treatment group
• Day 3: Panel Data [Jonathan Katz]
  • Morning: Basic panel data designs
  • Post-lunch lecture: The importance of choice of control variables in observational studies [Thomas Cook]
  • Afternoon: difference-in-differences; standard errors; wide panels; triple difference designs
• Day 4: Instrumental Variables and Selection Bias [Nathaniel Beck]
  • Morning: The basics of instrumental variables
  • Afternoon: weak instruments; simultaneous equations; multiple instruments (overidentification); tests for instrument validity. Selection bias and ways to address it.
• Day 5 morning: Interpreting Treatment Effects [Kevin Quinn]
  • What inferences can one draw from a research design for a heterogeneous population: sample versus population average treatment effects (ATE); local average treatment effects (LATE); average treatment effect on the treated (ATET); intent to treat effect (ITT), etc.
  • Post-lunch lecture: Bloopers [Bernie Black].
    Examples, drawn from different areas, of how to get causal inference wrong.
  • Day 5 afternoon: Feedback on your own research [led by Black and McCubbins]
    present your own research design issues; get group feedback (parallel sessions as needed to meet demand) (15 min to present, 15 min discussion)

Workshop Materials

Workshop materials are available only to participants.

Login

To obtain the username and password, please e-mail causalinference@law.northwestern.edu

[Back to top]

2011 Causal Inference: Bayesian Methods Workshop

Dates and Location: July 11-13, 2011, at Northwestern Law School, Chicago, IL

Overview of the Bayesian Causal Inference Workshop
Research design for causal inference is at the heart of a "credibility revolution" in empirical research. Credible causal inference often requires researchers not to rely on the linearity and normality assumptions underlying classical regression. Bayesian imputation and simulation methods provide many of the analytic tools for doing so. We will cover Bayesian methods for research design and analysis for true randomized experiments, as well as for quasi-experiments where part of the sample is "treated" in some way, the control group is drawn from the rest of the sample, but the researcher controls neither the assignment of units to treatment nor administration of the treatment. These analytic methods include multiple imputation of missing "potential outcomes", using Markov chain Monte Carlo (MCMC) simulations, including Gibbs sample, and other flexible model specifications.

Most empirical methods courses begin with the methods. They survey how each method works, and what assumptions each relies on. We will begin instead with the goal of causal inference and the centrality of research design, and discuss how Bayesian methods allow research designs that better achieve that goal.

The workshop will include an introduction to Winbugs, the principal public domain Bayesian inference software. We will illustrate selected methods with real data and Winbugs code.

Target audience
Quantitative empirical researchers (faculty and graduate students) in social science, including law, political science, economics, many business-school areas (finance, accounting, management, marketing, etc), sociology, education, psychology, etc. - indeed anywhere that causal inference is important.

Minimum prior knowledge
We will assume knowledge, at the level of an upper-level college econometrics or applied statistics course, of how to run multivariate regressions, including OLS, logit, and probit; familiarity with basic probability and statistics including conditional and compound probabilities, confidence intervals, t-statistics, and standard errors; and some understanding of instrumental variables are and how they are used.

Despite its modest prerequisites, this course should be suitable for most researchers with PhD level training and for empirical legal scholars with reasonable even if more limited training. Especially for recent PhD's, there will be overlap with what you already know, but much that you don't know, or don't know as well as you should.

Teaching Faculty
We are fortunate to have recruited outstanding experts in causal research design to teach the workshop sessions.

• Donald B. Rubin (Harvard University) is John L. Loeb Professor of Statistics, Harvard University. His work on what today is often called the "Rubin Causal Model" is central to modern understanding of when one can and cannot infer causation from regression. Principal research interests: statistical methods for causal inference; Bayesian statistics; analysis of incomplete data. Wikipedia
• Jeff Gill (Washington University in St. Louis) is Director of the Center for Applied Statistics, Professor of Political Science and Professor of Mathematics at Washington University in St. Louis, and the author of Bayesian Methods: A Social and Behavioral Approach (second edition 2007). Principal research interests: political methodology, Bayesian research methods; statistical computing. Wikipedia

Conference Organizers

• Bernard Black (Northwestern University, Law and Kellogg School of Management) is Nicholas J. Chabraja Professor at Northwestern University, with positions in the Law School and Kellogg School of Management. Principal research interests: law and finance, international corporate governance, health law and policy; empirical legal studies. Papers on SSRN
• Mathew McCubbins (University of Southern California) is Provost Professor of Business, Law and Political Economy at University of Southern California, with positions in the Marshall School of Business, the Gould School of Law, and the Department of Political Science. Principal research interests: legislative organization; communication, learning and decisionmaking; research design; network economics. Papers on SSRN

Workshop Schedule

The workshop will run for three full days from 9:00 a.m.-4:30 p.m., plus a reception on Monday from 4:30 p.m.-6:00 p.m. There will be roughly six 50-min lectures each day, plus a lunch break from roughly 12:15 p.m.-1:30 p.m. The starting time is fixed, all else is approximate.

• Monday July 11 (Don Rubin)
  The "potential outcomes (with assignment mechanism) or "Rubin causal model" approach to causal inference. Understanding the assignment mechanism. Estimating treatment effects using multiple imputation of the missing outcomes for both treated and control units. Comparison of multiple imputation to regression methods. Research design for credible causal inference. Applications to randomized experiments and to "natural" or quasi-experiments with "unconfounded" assignment of units to treatment.
• Tuesday July 12 (Jeff Gill)
  The basics of Bayesian inference, including the probabilistic treatment of uncertainty, prior and posterior distributions, updating knowledge, and describing results. Comparison of Bayesian to frequentist approaches. Bayesian approaches to construction and assessment of linear and nonlinear regression-style models.
• Wednesday, July 13 (Jeff Gill)
  Bayesian stochastic simulation. Modern applied Bayesian work is integrally tied to Markov chain Monte Carlo simulation (MCMC) tools. We will develop the theory behind MCMC tools and then learn how to use these tools, using WinBUGS (a well-known, high quality, free program).

Workshop Materials

Workshop materials are available only to participants.

Login

To obtain the username and password, please e-mail causalinference@law.northwestern.edu

[Back to top]