2016 Main Causal Inference Workshop

Notice (March 31, 2016):  The workshop is at capacity for 2016.  Please email bblack@northwestern.edu to be added to the waiting list, or to receive early notice of next year’s workshop.

We would like to invite you to attend the Seventh Annual Workshop on Research Design for Causal Inference, sponsored by Northwestern University, Duke University, and the Society for Empirical Legal Studies.

Monday-Friday, August 1-5, 2016, at Northwestern Law School, 375 East Chicago Avenue, Chicago, IL

Teaching Faculty and Organizers | Registration | Schedule | Detailed ScheduleHotels | Materials (login required) 

Workshop Overview

Research design for causal inference is at the heart of a “credibility revolution” in empirical research.  We will cover the design of true randomized experiments and contrast them to natural or quasi experiments and to pure observational studies, where part of the sample is treated in some way, the remainder is a control group, but the researcher controls neither the assignment of cases to treatment and control groups nor administration of the treatment.  We will assess the causal inferences one can draw from a research design, threats to valid inference, and research designs that can mitigate those threats.

Most empirical methods courses survey a variety of methods.  We will begin instead with the goal of causal inference, and emphasize how to design research to come closer to that goal.  The methods are often adapted to a particular study.  Some of the methods are covered in PhD programs, but rarely in depth, and rarely with a focus on causal inference and on which methods to use with messy, real-world datasets and limited sample sizes.  Several workshop days will include a Stata “workshop” to illustrate selected methods with real data and Stata code.

Target audience
Quantitative empirical researchers (faculty and graduate students) in social science, including law, political science, economics, many business-school areas (finance, accounting, management, marketing, etc), medicine, sociology, education, psychology, etc. –anywhere that causal inference is important.

Minimum prior knowledge
We will assume knowledge, at the level of an upper-level college econometrics or similar course, of multivariate regression, including OLS, logit, and probit; basic probability and statistics including conditional and compound probabilities, confidence intervals, t-statistics, and standard errors; and some understanding of instrumental variables.

Despite its modest prerequisites, this course should be suitable for most researchers with PhD level training and for empirical legal scholars with reasonable but more limited training. For recent PhD's, there will be overlap with what you already know, but much that you don't know, or don't know as well as you should.

Teaching Faculty

We are fortunate to have recruited outstanding experts in causal research design to teach the workshop sessions. The faculty are listed in order of appearance.

  • Donald Rubin (Harvard University, Department of Statistics)
    Rubin is John L. Loeb Professor of Statistics, Harvard University. His work on the "Rubin Causal Model" is central to modern understanding of when one can and cannot infer causation from regression. Principal research interests: statistical methods for causal inference; Bayesian statistics; analysis of incomplete data. Wikipedia page
  • Alberto Abadie (Harvard University, Kennedy School of Government)
    Alberto Abadie is Professor of Public Policy at the Kennedy School of Government at Harvard University.  Principal research interests: econometrics; program evaluation.  Papers on SSRN 
  • Joshua Angrist (MIT)
    Joshua Angrist is Ford Professor of Economics at MIT.  Principal research interests:  labor economics; econometrics.  Author of Joshua Angrist and Jorn-Steffen Pischke, Mostly Harmless Econometrics:  An Empiricist’s Companion (2009) and Mastering ‘Metrics:  The Path from Cause to Effect (2014). Mostly Harmless Econometrics blog
  • Jens Hainmueller is Associate Professor in the Stanford Political Science Department.  He also holds a courtesy appointment in the Stanford Graduate School of Business.  His research interests include statistical methods, political economy, and political behavior. 

Conference Organizers

  • Bernard Black (Northwestern University)
    Nicholas J. Chabraja Professor at Northwestern University, with positions in the Law School and Kellogg School of Management.  Principal research interests: law and finance, international corporate governance, health law and policy; empirical legal studies. Papers on SSRN
  • Mathew McCubbins (Duke University)
    Professor of Political Science and Law at Duke University, with positions in the Law School and the Political Science Department, and director of the Center for Law and Democracy.  Principal research interests: democratic institutions, legislative organization; behavioral experiments, communication, learning and decisionmaking; statutory interpretation, administrative procedure, research design; network economics. Papers on SSRN

Registration and Workshop Cost

Please click here to register.

Main workshop: Tuition is $850 ($500 for graduate students (PhD, SJD, or law) and post-docs; $350 for Northwestern or Duke-affiliated attendees). 

The workshop fees include all materials, temporary Stata 14 license, breakfast, lunch, snacks, and an evening reception on the first workshop day.

Registration deadline: June 1, 2016 (but the main workshop is likely to fill early). Amounts will increase by $50 on June 1, 2016 (but the workshop is likely to fill up before then). You can cancel by June 17, 2016 for a 75% refund and by July 1, 2016 for a 50% refund (in each case, less credit card processing fee), but there are no refunds after that.

Workshop Schedule

Monday, August 1 (Don Rubin)

Registration and breakfast: 8:00

Introduction to Modern Methods for Causal Inference
Overview of causal inference and the Rubin “potential outcomes” causal model.  The “gold standard” of a randomized experiment.  Treatment and control groups, and the core role of the assignment (to treatment) mechanism.  Causal inference as a missing data problem, and imputation of missing potential outcomes.  Choosing estimands (the science), and how the estimand affects research design. 

Tuesday, August 2 (Alberto Abadie)

Designs for “Pure” Observational Studies
Selection [only] on observables and common support assumptions.  Subclassification, matching, reweighting, and regression estimators of average treatment effects.  Propensity score methods. 

Wednesday August 3 (Josh Angrist)

Instrumental variable methods
Causal inference with instrumental variables (IV), including (i) the core, untestable need to satisfy the “only through” exclusion restriction; (ii) heterogeneous treatment effects; and (iii) intent-to-treat designs for randomized trials (or quasi-experiments) with noncompliance. 

Wednesday afternoon August 3 (Joshua Angrist and Don Rubin)

Uses and limits of regression methods:  A discussion/debate
When should researchers rely primarily on regression, and when should they use “balancing methods” for pure observational studies?  How much difference is there likely to be in estimated outcomes?  We will conduct a discussion/debate between Josh Angrist and Don Rubin, two grandmasters of causal inference who have quite different views on the uses and limits of regression.

Thursday-Friday August 4-5 (Jens Hainmueller)

Panel Data and Difference-in-Differences
Panel data methods:  pooled OLS, random effects, correlated random effects, and fixed effects.  Simple two-period DiD.  The core “parallel changes” assumption.  Testing this assumption.  Leads and lags and distributed lag models.  When does a design with unit fixed effects become DiD?  Accommodating covariates.  Triple differences.  Robust and clustered standard errors.

Regression Discontinuity
(Regression) discontinuity (RD) research designs: sharp and fuzzy designs; bandwidth choice; testing for covariate balance and manipulation of the threshold; discontinuities as substitutes for true randomization and sources of convincing instruments.

Friday afternoon:  Feedback on your own research
Attendees will present their own research design questions from current work in breakout sessions and receive feedback on research design.  Session leaders:  Bernie Black, Mat McCubbins, Jens Hainmueller.  Parallel sessions as needed to meet demand.

Detailed Workshop Schedule and Readings

All readings, including individual book chapters, but not full books, are posted to the course website.

Workshop dates:  August 1-5, 2016

Workshop Location:  Northwestern Law School, 375 East Chicago Ave., Chicago IL 60611, Rubloff 150

The law school is in downtown Chicago.  The “main” Northwestern campus is in Evanston.  All sessions (except Friday afternoon) will be in the “Rubloff” building, in Room 150.  The Rubloff building is the “new” law building, closest to Lake Michigan.  Rubloff 150 is roughly midway between Chicago Ave. on the north side and Superior Ave. on the south side.

The Superior Ave. entrance to Rubloff closest to Lake Michigan is currently under construction, so please enter either from the Chicago Ave side or from the next entrance further away from the lake.

Registration and meals:  Breakfast will be available each morning from 8:30 in Rubloff 155.  A registration table will be open on the first day of each workshop beginning at 8:30.  Lunch will be provided between the morning and afternoon sessions.  Snacks and liquids (coffee, tea, sodas, juice, water) will be available throughout the day in Rubloff 155.

Conference url:  http://www.law.northwestern.edu/research-faculty/conferences/causalinference/

Access to Readings:  You will need a Northwestern Box account.  If you have not received login instructions, please email Cara Peterson (causalinference@law.northwestern.edu).

Wireless:  Northwestern has a “guest” wireless network.  For those of you from universities that participate in the “eduroam” network, please use that network because bandwidth on the guest network is limited.

General schedule:  “Lecture” sessions will run roughly 9:00-12:15, lunch 12:15-1:30, sessions 1:30-4:45.  Please plan to arrive the evening before the workshop.  All times, other than the morning starting time, are approximate.

Return travel:  A return flight on Friday leaving 7:00 or later from O’Hare airport should be reasonably safe; 7:30 is quite safe.

Questions during the workshop:  Please email Bernie Black (bblack@northwestern.edu) and Cara Peterson (cara.peterson@law.northwestern.edu).  The workshop email address --  causalinference@law.northwestern.edu -- will reach both of us.
 

Monday August 1 (Don Rubin)

Introduction to Workshop (9:00-9:30) (Bernie Black):
Overview of some of the main things we hope you will learn during the workshop.

Introduction to Modern Methods for Causal Inference (9:30-12:30, 1:30-3:30)
Overview of causal inference and the Rubin “potential outcomes” causal model.  The “gold standard” of a randomized experiment.  Treatment and control groups, and the core role of the assignment (to treatment) mechanism.  Causal inference as a missing data problem, and imputation of missing potential outcomes.  Choosing estimands (the science), and how the estimand affects research design. 

Reading

  • Imbens, Guido, and Donald Rubin, Causal Inference for Statistics, Social, and Biomedical Sciences (2015), chapters 1-8

Stata-based examples (3:45-4:45) (Bernie Black)
Intended as a gentle introduction, for people who know a little bit about Stata, to how to use Stata to implement some of the methods we will discuss during the week.  If you are a novice Stata user, you may find the introductory materials on the course folder at Northwestern Box\Causal Inference Workshops\Stata and R materials.

R-based examples (Josh Lerner)

Similar, but using R rather than Stata.  Room to be announced.


Monday reception:  5:00-6:30:  In the law school central outdoor courtyard if weather permits, otherwise inside.

Tuesday, August 2 (Alberto Abadie)

Designs for “Pure” Observational Studies (9:00-12:00; 1:15-3:15)
Selection [only] on observables and common support assumptions.  Subclassification, matching, reweighting, and regression estimators of average treatment effects.  Propensity score methods.

Readings 

  • Imbens and Rubin, Causal Inference in Statistics and Social Sciences (2015), chapters 12-22
  • Imbens, Guido W. (2004), “Nonparametric Estimation of Average Treatment Effects under Exogeneity: A Review,” 86 Review of Economics and Statistics    4-29.
  • Rosenbaum, Paul R., and Donald B. Rubin (1983), “The Central Role of the Propensity Score in Observational Studies for Causal Effects,” 70 Biometrika    41-55.
  • Rubin, Donald B. (1977), “Assignment to Treatment Group on the Basis of a Covariate,” 2 Journal of Educational Statistics 1-26.

Stata- and R-based examples (3:45-4:45) (Bernie Black and Josh Lerner)

Continuation of the Monday talks. 

Wednesday, August 3 (Josh Angrist)

Instrumental variable methods (9:00-12:00; 1:15-2:45)
Causal inference with instrumental variables (IV), including (i) the core, untestable need to satisfy the “only through” exclusion restriction; (ii) heterogeneous treatment effects; and (iii) intent-to-treat designs for randomized trials (or quasi-experiments) with noncompliance.

Readings

  • Angrist and Pischke, Mastering ‘Metrics:  The Path from Cause to Effect (2015), chap 3.
  • Angrist, Joshua D., and Jorn-Steffen Pischke, Mostly Harmless Econometrics (2009), chap 4.
  • Angrist, Joshua, Guido Imbens, and Donald Rubin (1996), Identification of Causal Effects Using Instrumental Variables, 91 Journal of the American Statistical Association 444-455
  • Angrist, Joshua (1998), Estimating the Labor Market Impact of Voluntary Military Service Using Social Security Data on Military Applicants, 66 Econometrica 249-288.

Wednesday afternoon August 3 (Joshua Angrist and Don Rubin)

Uses and limits of regression methods:  A discussion/debate
When should researchers rely primarily on regression, and when should they use “balancing methods” for pure observational studies?  How much difference is there likely to be in estimated outcomes?  We will conduct a discussion/debate between Josh Angrist and Don Rubin, two grandmasters of causal inference who have quite different views on the uses and limits of regression.

Readings

  • Angrist, Joshua D., and Jorn-Steffen Pischke, Mostly Harmless Econometrics (2009), chap 3.
  • Angrist and Pischke, Mastering ‘Metrics:  The Path from Cause to Effect 2015), chap 2.
  • Gutman, Roee, and Donald B. Rubin, Estimation of Causal Effects of Binary Treatments in Unconfounded Studies, 34 Statistics in Medicine 3381-3398 (2015)
 
Thursday-Friday, August 4-5 (Jens Hainmueller)

Thursday:  Panel Data and Difference-in-Differences(9:00-12:00; 1:45-4:45)
Panel data methods:  pooled OLS, random effects, correlated random effects, and fixed effects.  Simple two-period DiD.  The core “parallel changes” assumption.  Testing this assumption.  Leads and lags and distributed lag models.  When does a design with unit fixed effects become DiD?  Accommodating covariates.  Triple differences.  Robust and clustered standard errors.

Readings
  • Angrist, Joshua D. and Jorn-Steffen Pischke (2009). Mostly Harmless Econometrics: An Empiricist's Companion, Chapter 5.
  • Card, David, and Alan B. Krueger (1994), Minimum Wages and Employment: A Case Study of the Fast-Food Industry in New Jersey and Pennsylvania ," 84 American Economic Review 772-793.
  • Card, David, and Alan Krueger (2000), Minimum Wages and Employment:  A Case Study of the Fast Food Industry in New Jersey and Pennsylvania:  Reply, 90 American Economic Review 1397-1420.
  • Wooldridge, Jeffrey M. (2010). Econometric analysis of cross section and panel data. MIT press. Second edition, Chapter 10.
  • Autor, David H. (2003). Outsourcing at will: The contribution of unjust dismissal doctrine to the growth of employment outsourcing. 21 Journal of Labor Economics 1-42.
Thursday lunch talk (Bernie Black)

Bloopers:  How Smart People Get Causal Inference Wrong (12:30-1:30)
Examples, drawn from different areas, of how to get causal inference wrong.  Time permitting, I plan to use the following papers as examples, in case you want to look at them before the talk and see if you can figure out what I think is wrong:

  1. Sanjai Bhagat & Bernard Black, The Non-Correlation Between Board Independence and Long-Term Firm Performance, 27 Journal of Corporation Law 231-274 (2002) (http://ssrn.com/abstract=133808)
  2. John Donohue and Daniel Ho, The Impact of Damage Caps on Malpractice Claims:  Randomization Inference with Difference-in-Differences, 4 J Empirical Legal Studies 69-102 (2007)
  3. Daniel Kessler & Mark McClellan (1996), Do Doctors Practice Defensive Medicine?, 111 Quarterly Journal of Economics 353-390 (1996). and Kessler, Daniel, and Mark B. McClellan, (2002), Malpractice Law and Health Care Reform: Optimal Liability Policy in an Era of Managed Care, 84 Journal of Public Economics 175-197.
  4. Gompers, Paul, Joy Ishii & Andrew Metrick, Corporate Governance and Equity Prices, 118 Quarterly Journal of Economics 107-155 (2003). 
  5. Daron Acemoglu & Simon Johnson, Unbundling Institutions, 113 Journal of Political Economy 949-995 (2005).
  6. Kathryn Dewenter, Xi Han and Paul Malatesta, Firm Values and Sovereign Wealth Fund Investments, 98 Journal of Financial Economics 256-278 (2010).
  7. Ran Duchin, Paul Matsusaka and Oguzhan Ozbas, When are Outside Directors Effective?, 96 Journal of Financial Economics 195-214 (2010) and their reply (2015) to Atanasov and Black, The Trouble with Instruments (working paper 2016), at http://ssrn.com/abstract=2697098.
Friday morning:  Regression Discontinuity (9:00-12:00)
(Regression) discontinuity (RD) research designs: sharp and fuzzy designs; bandwidth choice; testing for covariate balance and manipulation of the threshold; discontinuities as substitutes for true randomization and sources of convincing instruments.

Readings
  • Angrist, Joshua D. and Jorn-Steffen Pischke (2009). Mostly Harmless Econometrics: An Empiricist's Companion, Chapter 6.
  • David S. Lee (2008), Randomized Experiments from Non-random Selection in U.S. House Elections, 142 Journal of Econometrics 675-697.
  • Imbens, Guido W., and Thomas Lemieux, 2008, Regression Discontinuity Designs:  A Guide to Practice, 142 Journal of Econometrics 615-635.
Friday afternoon (1:30-5:00):  Feedback on your own research (3 or 4 parallel sessions)
Attendees will have an opportunity to present their own research design questions from current work in breakout sessions.  Goal: obtain feedback on research design; not present results from a complete paper.  [We ask presenters to stay for the full session, and can’t promise an early slot for those who must leave early.]  (15 min to present, 15 min discussion).  Session leaders:  Bernie Black, Jens Hainmueller, Mat McCubbins; we’ll add additional sections as needed.

Hotels

We reserved a block of rooms at the Marriott Resident Inn Chicago, which is walking distance to Northwestern Law. The hotel rate is $199 per night plus tax.  

Please click here to reserve your hotel room.

Questions about the workshops: Please email Bernie Black (bblack@northwestern.edu) or Mat McCubbins (mathew.mccubbins@duke.edu) for substantive questions or fee waiver requests, and Cara Peterson (causalinference@law.northwestern.edu) for logistics and registration.