The real dependent variable problem: the limitations qualitative analysis in comparative policy studies

Abstract

Comparative public policy relies heavily on processing quantitative data, typically done by looking for the relationship between variables or by grouping empirical data into categories. In methodological terms, comparative data commonly deal with nations as policy units, and observations are liable to be interconnected, rather than independent. In statistical terms, there are problems in the identification of appropriate data, violations of the assumptions are rife, and there are just not enough nations to be able to make comparisons sensibly in these terms. The 'dependent variable' problem relates to the definition, operationalization and measurement of key variables, but the problems of comparison go deeper than that. The real dependent variable problem is whether it makes sense to look for a dependent variable at all. The evidence is always equivocal; a methodology that attempts to bracket off disparate influences cannot be valid; and attempting to apply general principles across different circumstances and conditions is inconsistent with what we know about policy development. Quantitative methods offer ways to sort and systematize information, but they do not provide a basis for generalization.