Search results

This paper considers recent methodological developments in the treatment effects literature, describes their value for applied evaluation work, and suggests next steps. It pays particular attention to documenting the presence of treatment effect heterogeneity, to the quest to attach treatment effect heterogeneity to particular subgroups and other moderators, and to the recent application of machine learning methods in this domain.

Imagine a region suffering from a widening income gap that becomes eligible for a generous transfer programme (the treatment). Imagine difference-in-differences analysis (DD) — a before-and-after comparison of the income-level difference — shows that the handicap has risen. Most observers would conclude to the policy's inefficiency. But second thoughts are needed, because DD rests heavily on the validity of a key assumption: parallel paths in the absence of treatment; an assumption that is often violated. To cope with this problem, economists traditionally include polynomial (linear, quadratic…) trends among the regressors, and estimate the treatment effect as a once-in-a-time trend shift. In practice that strategy does not work very well, because inter alia the estimation of the trend uses post-treatment data. What is needed is a method that i) uses pre-treatment observations to capture linear or non-linear trend differences, and ii) extrapolates these to compute the treatment effect. This paper shows how this can be achieved using a fully-flexible version of the canonical DD equation. It also contains an illustration using data on a 1994–2006 EU programme that was implemented in the Belgian province of Hainaut.

Imagine a region suffering from a widening income gap that becomes eligible for a generous transfer programme (the treatment). Imagine difference-in-differences analysis (DD) — a before-and-after comparison of the income-level difference — shows that the handicap has risen. Most observers would conclude to the policy's inefficiency. But second thoughts are needed, because DD rests heavily on the validity of a key assumption: parallel paths in the absence of treatment; an assumption that is often violated. To cope with this problem, economists traditionally include polynomial (linear, quadratic…) trends among the regressors, and estimate the treatment effect as a once-in-a-time trend shift. In practice that strategy does not work very well, because inter alia the estimation of the trend uses post-treatment data. What is needed is a method that i) uses pre-treatment observations to capture linear or non-linear trend differences, and ii) extrapolates these to compute the treatment effect. This paper shows how this can be achieved using a fully-flexible version of the canonical DD equation. It also contains an illustration using data on a 1994–2006 EU programme that was implemented in the Belgian province of Hainaut.

Imagine a region suffering from a widening income gap that becomes eligible for a generous transfer programme (the treatment). Imagine difference-in-differences analysis (DD) - a before-and-after comparison of the income-level difference - shows that the handicap has risen. Most observers would conclude to the policy's inefficiency. But second thoughts are needed, because DD rests heavily on the validity of a key assumption: parallel paths in the absence of treatment; an assumption that is often violated. To cope with this problem, economists traditionally include polynomial (linear, quadratic .) trends among the regressors, and estimate the treatment effect as a once-in-a-time trend shift. In practice that strategy does not work very well, because inter alia the estimation of the trend uses post-treatment data. What is needed is a method that i) uses pre-treatment observations to capture linear or non-linear trend differences, and ii) extrapolates these to compute the treatment effect. This paper shows how this can be achieved using a fully-flexible version of the canonical DD equation. It also contains an illustration using data on a 1994-2006 EU programme that was implemented in the Belgian province of Hainaut.

It is very important to know how much we can extrapolate from a study's results. This paper examines the issue using data from impact evaluations in development.