oa Werkzame principes

Working principles: Building scientific memories in social research

Internal and external validity of experimental and quasi-experimental research are both essential in the evaluation of interventions; this is true in such diverse practices as social policy and pharmacology. However, they may trade-off if the quest for internal validity brings about the treatment of contextual variables – that helped to make or fail the intervention – as competing with the intervention instead of necessary for its success. Investment in internal validity takes the form of many types of correction for bias, often obscuring the role of contextual factors.

As helping factors make or break the intervention, they should rather be treated as causal factors of interest no less than the intervention. I claim that, only then, the conclusions of the (quasi)experiment can serve to gain knowledge of working principles in a variety of contexts.

Correction for bias is useful, but only if the evaluation of the success of an intervention is theory driven. Ex ante explanatory hypotheses must be the starting point of evaluation research and testing them the object of it. Thus, the empirical cycle is covered entirely. If no theoretical hypotheses guide the questions of evaluation research, all we will learn is that ‘most interventions sometimes work’, to speak with Pawson and Tilley (1997). What we are left with is little more than meaningless intervention-impact couples that cover half the empirical cycle lacking any cognitive value.